If you have built a multi-agent AI system over the past two years, you have almost certainly encountered what we call the Supervisor Tax.
The patt...
For further actions, you may consider blocking this person and/or reporting abuse
"Agents should execute, state machines should route" is the sentence I wish I'd had two years ago. We hit the exact same wall — an LLM supervisor asked to "verify all requirements are met" would hallucinate a missing criterion and loop, because natural-language evaluation has no convergence bound. Moving routing to a typed transition table killed the infinite loops outright, because a state machine literally cannot invent a state that isn't in the graph.
The auditable-receipts point is underrated and I'd argue it's the bigger long-term win over the token savings. When a customer asks "why did the agent run this write?", a transition log answers it in one line; parsing thousands of tokens of conversational chatter to reconstruct intent is not a real audit trail.
The one place we kept an LLM in the routing seat was the initial intent classification — the fuzzy front door — then handed off to the deterministic machine. Did you find a clean boundary between "fuzzy enough to need a model" and "route it deterministically," or is your intent classifier itself constrained to a fixed enum of output states?
We strictly constrain the front door to a fixed enum via structured outputs. The model is allowed to interpret the unstructured user intent, but its only output is a validated schema containing an enum intent and extracted parameters.
If the classification fails schema validation or doesn't map cleanly to a known intent enum, the system halts at the ingress boundary and asks for clarification. We never allow the model to dynamically invent next states. Once the ingress step produces that validated intent type, the model is stripped of all routing authority and the deterministic transition table takes over completely.
Our front door model produces structured output constrained to a fixed target workflow enum plus extracted entities, but the boundary is strict: the model is only allowed to classify intent, never execute or dictate the transition sequence. Once an intent is resolved to a workflow ID, the state machine takes over completely.
If the intent classifier returns low confidence or an unrecognized pattern, it transitions to a dedicated
needs_clarificationortriagestate rather than letting the LLM invent intermediate routing steps on the fly.The typed receipt is a strong boundary: the router sees a validated result and transition key instead of inheriting the whole worker transcript. I was curious about the 71.4% token reduction—did you compare task success and human-escalation rates on the same workload alongside the cost numbers? A bounded repair loop is a win, but it would be useful to see whether any former supervisor recoveries shifted into
NEEDS_HUMANafter the change.Yes—across our benchmark runs on multi-step extraction and tool execution workflows, task completion actually rose from ~82% to 94%, while human escalations dropped. In practice, the supervisor LLM rarely produced genuine recoveries; more often it hallucinated success on partial failures or spun in unguided retry loops until hitting context limits.
By moving recovery to the leaf tool—bounded by schema validation and a strict step counter—routine transient errors get resolved locally without re-prompting the whole hierarchy. When a run does emit
NEEDS_HUMAN, it's now exclusively for deterministic blockers (auth challenges, CAPTCHAs, or missing credentials) rather than supervisor thrashing.Typed state machines instead of a supervisor LLM is a good example of moving control out of the model. The 70 percent savings is the visible benefit, the less visible one is that transitions become deterministic and testable instead of depending on how the supervisor read the prompt that day.
Exactly. Once the routing logic lives in code, you can write unit tests for the transition graph and test edge-case failures without spending an API call.
The biggest operational win for us was being able to replay a failed state transition offline with the exact snapshot payload, rather than trying to reproduce whatever subtle prompt drift caused the supervisor LLM to loop or misroute.
How are you currently structuring state boundaries in your own agent workflows?
The rule I'd use is to draw the boundary where a wrong transition is expensive or hard to undo, and give those transitions typed inputs and an explicit allowed-next-state list. The cheap steps can stay loose. Replaying from the exact snapshot payload also makes the boundary auditable later, since the input that caused a bad transition is on record.
That distinction between irreversible mutations vs cheap intermediate reads is spot on. We ended up putting strict schema validation and explicit next-state enums on anything touching external APIs or local persistence, while letting exploratory steps stay fluid.
Having the exact payload captured at those critical boundaries makes post-mortem triage much faster when an external service drifts or rejects a payload.
Your typed receipt boundary also creates a useful replay contract: a verifier can re-run the transition from stepId, nextTrigger, and artifact hashes without loading the worker transcript. I’d be curious how you handle schema evolution when an older receipt is replayed after the state machine changes—do you version the transition graph or migrate receipts at ingestion?
We version the transition graph rather than mutating past receipts.
Because receipts store immutable artifact hashes and transition signatures, migrating them at ingestion breaks verifiable provenance. Instead, the receipt header carries the
graphVersion(or spec hash). The verifier loads the corresponding graph definition for replay. If a transition is modified or deprecated, we keep historic validator definitions active for audit/replay while routing all fresh runs through the new schema.Moving routing out of the model and keeping repairCount as a code-side cap makes a lot of sense to me. Our review loops are only halfway there: after five stuck rounds a person decides, but that stop is a written rule rather than a guard in code, so it still depends on the model following it. Out of curiosity, what share of your tasks actually hit the repairCount >= 3 guard in practice?
In practice, about 5–8% of our runs enter a repair loop, and only around 1–2% actually hit the
repairCount >= 3ceiling.When an agent hits attempt 3, it's almost never a subtle reasoning mistake that another retry would fix—it's usually a hard external condition like an unexpected DOM shift, an expired session, or a rate limit. Letting deterministic code fail fast to a human handoff or a blocked state saves a ton of token burn.
On your review loops, what usually triggers the stuck rounds—model hallucination on ambiguous output, or external tool/environment errors?
This is the kind of optimization that improves reliability as well as cost. A typed state machine makes the handoffs visible instead of asking a supervisor to rediscover workflow state every turn. Did you test recovery from partial tool failure, where the saved state matters more than the happy path?
Partial tool failure is actually where the state machine gave us the biggest win over supervisor loops. Every state transition persists its input, receipt hash, and execution status to disk before moving forward.
If a tool times out or returns a transient network error, the machine transitions to a bounded
retry_pendingorreconcile_requiredstate instead of restarting the prompt chain. When the process resumes, it reads the last verified checkpoint and continues from the failed edge without re-executing expensive upstream steps or having an LLM supervisor hallucinate what already completed.Partial failure is where the state machine really proves its value over supervisor loops. Each tool execution produces an immutable, typed receipt (payload hash, status code, idempotency key, and error category).
When a tool times out or returns a partial error, the state machine transitions to an explicit
reconcileorretry_boundedstate that inspects the last verified receipt rather than re-running from scratch. Because the state is serialized on every step, a crashed worker or network drop resumes by reading the uncompleted step directly instead of asking an LLM to guess where it left off.The supervisor tax framing matches what I see in traces: the coordinator burns most of its tokens re-reading context it just wrote. A typed state machine is also easier to evaluate because every transition is a discrete checkpoint you can score, instead of hoping the supervisor's reasoning matches what actually happened. How do you handle transitions the state machine does not have a type for yet?
When a leaf tool returns an output variant that doesn't map to an existing transition edge, we route it directly to a terminal
unhandled_transitionfallback state rather than trying to recover dynamically.The runner emits a structured receipt containing the raw payload, halts that execution lane, and surfaces an alert. In practice, treating unexpected outputs as hard schema violations forces us to add explicit state transitions in code rather than letting an LLM guess a path forward on invalid assumptions. If a human operator intervenes and resolves the block, the machine resumes from that verified checkpoint.
When a workflow encounters an unhandled transition or unexpected tool response shape, it falls through to a catch-all
escalate_or_quarantinestate. Rather than letting an LLM guess a path forward, the machine records the exact input/output snapshot, halts further side effects on that branch, and emits an unhandled-transition event.We then review those trace events to either add a new typed transition in code or define an explicit fallback policy. It treats unexpected transitions as missing schema definitions rather than runtime reasoning tasks.
Loop termination is the result I'd put first because unpredictable cost spikes are the thing that kills agent adoption, and an LLM retrying 15 times is exactly that. I'm curious about the receipt boundary though: when a subagent returns a receipt that's structurally valid but built on hallucinated data, the graph still advances. Do you have a semantic layer at the boundary, or does the schema contract itself catch that in practice? From my runs, the rare but painful case is the validator saying VALIDATED on a plan that's internally consistent but wrong.
Schema validation only guarantees syntactic shape, so we treat it as the bare minimum entrance gate rather than truth.
To catch hallucinated or internally consistent wrong output, we enforce external deterministic read-backs at the state transition. The receipt can't just be an LLM self-report saying "done"; it has to carry a verifiable handle—like a disk path hash, file stat, database ID, or an exact read-back string from the underlying tool output. The validator then runs a deterministic tool check against that handle before the state machine allows the graph to advance.
If the subagent hallucinated a file write or an API response, the read-back check fails immediately and routes to a targeted retry or failure branch, without relying on an LLM supervisor to guess whether it happened.
The receipt detail is where I'd add a caution, because we tried this.
We gave actions a typed receipt with a
reproducible_byfield — the command that should re-derive the verdict later. First run, 2 of 4 cases failed. The receipt wasn't wrong; verify and re-run just happened in different working directories. A git receipt saved without an explicit workdir “looks” complete and silently re-checks the wrong repo. Making the workdir part of the receipt fixed it: 4/4.So the receipt has to be re-runnable by something with no memory of the original context. If it needs the reader to know where it came from, it's a log line, not a receipt.
Your supervisor has the same problem one level up — it's asked to check work from a summary, with no independent way to re-derive the outcome. The ceiling on your repair loop is the other half of the fix. A counter you can read beats hoping the checker notices it's looping.
How much of your receipt is intended to be sufficient for a completely context-free re-check?
The missing workdir trap is spot on. We ran into the exact same issue with relative paths and ambient git state early on. In our setup, receipts never store a bare shell string: they record the absolute
workdir, the exact argv array, environment fingerprints, and a separate read-only verification probe. That probe lets an independent worker check the post-state without re-running the mutation.The goal is full context-free verification: any process can take the receipt, run the read-back probe in the recorded directory, and confirm whether the external effect landed. We also tie repair ceilings to strict numeric step counters in the state machine instead of letting an LLM judge whether it's looping.
One benefit that doesn't come up here: the state machine logic becomes independently testable from the LLM workers. You can unit test every transition without firing a single API call, swap workers for mocks, and run full workflow coverage in milliseconds. With an LLM supervisor there is no clean seam between routing logic and generation logic to exploit.
The schema validated receipt contract also functions as living documentation. The worker output type is the only thing the transition guard reads, so a worker can be replaced entirely without changing the routing layer. Did you version the AgentExecutionReceipt type separately from the state machine itself, or do they evolve as a unit?
thank you for sharing!