DEV Community

WAFER
WAFER

Posted on

LangGraph's Routing Is LLM-Guessing. I Wrote 50 Lines of Code to Make It Deterministic.

#langgraph #agents #python #ai

Every time your LangGraph agent sees "check nginx logs", it might call a different tool.

That's not an exaggeration. LangGraph's routing is driven by an LLM prompt — and prompts aren't reproducible. Same input, different day, different LLM mood, different tool selected.

I spent months debugging this. Every wrong route meant tweaking a prompt, hoping the next call would be better. It never was.

So I wrote something different.

The Problem: Routing by LLM is fragile

LangGraph is great at orchestrating multi-step agent workflows. But the first step — "what tool should I use?" — is a black box.

You define a prompt, the LLM decides. If it chooses wrong, you can't debug it. You can only guess: was the prompt not specific enough? Too specific? Wrong example?

This is the routing problem: given user input, which domain or tool should handle it? LangGraph leaves this to the LLM. I think it shouldn't.

The Fix: 50 Lines of YAML + Python 

from decide_router import RouteTable

rt = RouteTable("routes.yaml")
rule, _ = rt.match("check nginx error logs")
# rule.domain → "monitoring" — always, every time
Enter fullscreen mode Exit fullscreen mode

No LLM call. No prompt. Just a YAML file and 50 lines of matching logic.

The YAML defines domains with keywords and regex patterns:

domains:
  monitoring:
    priority: 100
    keywords: [log, error, nginx, monitor, health, status]
    patterns: ["(check|look).*(log|status|error)"]
  coding:
    priority: 100
    keywords: [code, script, deploy, write]
  human:
    priority: 100
    keywords: [delete production, restart cluster]
    require_confirm: true
Enter fullscreen mode Exit fullscreen mode

Plug it into LangGraph as a node:

from langgraph.graph import Graph
from decide_router import RouteTable

rt = RouteTable("routes.yaml")

def routing_node(state):
    rule, _ = rt.match(state["input"])
    state["domain"] = rule.domain if rule else "unknown"
    return state

graph = Graph()
graph.add_node("decide_route", routing_node)
graph.set_entry_point("decide_route")
Enter fullscreen mode Exit fullscreen mode

Your LangGraph agent now routes with rules, not guesses. Same input → same domain. Every time.

The Surprising Part: It Learns From Corrections

The best part wasn't planned — it emerged from using it.

I noticed I kept correcting wrong routes. "That's monitoring, not coding." Every correction was a signal. So I added a feedback cache:

# User corrects once → remembers immediately
rt.record_feedback("check nginx errors", "monitoring")
Enter fullscreen mode Exit fullscreen mode

Same correction 3 times → becomes a permanent routing rule at priority 70. Rules not used in 30 days auto-delete. The system gets smarter with every correction. No fine-tuning. No prompt engineering.

The core module is one file, 200 lines of Python. Read the whole thing in 5 minutes.

Try It 

pip install decide-router
Enter fullscreen mode Exit fullscreen mode

Or browse the code: github.com/chex0210-crypto/decide-router

I built this because I was tired of guessing why my agent picked the wrong tool. If you've had the same frustration, star the repo — it helps other people find it.

Top comments (0)