1. The argument is being filed under the wrong heading
Most debates about agent reliability get filed as model problems. The proposed fix is a better model, a lower temperature, a longer system prompt, a different framework, or more evals. All of those can help. None of them change where the decision actually lives.
Our position is that reliability in an agent system is not a property of the model. It is a property of the architecture around it: which decisions the model is allowed to make, which ones are made by ordinary code, and whether the system keeps enough evidence to tell the two apart afterwards.
That is a technical position, not a product claim. The phrase we use for it is deterministic delivery, and it is worth stating precisely, because it is easy to oversell.
2. What the position actually says
Deterministic delivery is controllability, auditability and fallbacks at the architecture level. It is explicitly not a claim that the model will always be right. A model reading an ambiguous email can still get it wrong inside a perfectly built backbone.
Three conditions have to hold before we'd use the phrase:
- Every decision node leaves a replayable, traceable record: inputs, outputs, latency, and the acceptance criterion it was supposed to meet.
- Rule logic is deterministic by construction — same input, same state transition, same audit entry — while semantic nodes stay probabilistic and are measured rather than assumed.
- A failure can be located to a node, not just observed as a wrong final answer.
If any of those is missing, adding tests or tuning prompts treats a symptom. The gap is architectural.
This also fixes a common misreading. "Deterministic" here does not describe the model's output. It describes the parts of the system that are not allowed to improvise.
3. The split, and where the other side is right
The alternative is a probabilistic black box: each step decides what happens next, state lives inside the prose, a retry is a model call that sounds like a retry, authorization is a line in a prompt asking nicely. When the output is wrong, you get a paragraph, not a location.
We'd argue that's the wrong default for production. But it is not wrong everywhere, and a position that cannot say when its opposite is fine is a preference, not a position. For a one-off script, a throwaway prototype, or internal exploration where a wrong answer costs nothing and nobody has to explain it later, chaining model calls is the cheaper and more honest choice. A deterministic backbone earns its keep when failures must be attributed, outputs reach external users, an audit trail matters, or more than one person has to maintain the system past the demo.
So the real disagreement is not about whether to use workflows, or how autonomous an agent should be. It is about a boundary: which decisions require judgement, and which decisions require a diff? Framed that way, the boundary is usually much narrower than the initial design assumed.
4. Three constraints, one property
The position above is not implemented by a single component. Three engineering constraints produce it, and deterministic delivery is the property that emerges when all three hold. They are not a feature list; each has a criterion you can check and a characteristic way of failing.
C1 — hybrid orchestration. A deterministic backbone owns state, retries, authorization, audit trails and rollback; a model is invoked only at nodes requiring semantic judgement.
Criterion: count the nodes that could be written as an if. If the answer is "almost none," there is no orchestration, only a chain of prompts with extra latency.
Failure mode: state hidden in generated text, so a retry silently multiplies the same call and nobody can reconstruct what ran.
C2 — structured prompt contract. A versionable spec of role, hard constraints, I/O contract and fallback rules. Not a persona — persona in English means role-play, and role-play is the part that does not survive contact with production. What matters is the constraint block: missing_context → ask for clarification, never infer; output must be a typed object; if a constraint cannot be satisfied, emit fallback_reason and stop.
Criterion: you can diff the constraint set between two deploys and assert on it without running the whole agent.
Failure mode: the rule exists in a system prompt, but nothing records whether it was ever consulted on this input.
C3 — white-box evaluation. Capture per-node inputs, outputs, latency and acceptance criteria into replayable test sets, instead of judging the final answer by feel. Three sets, kept separate: positive (the happy path with grounded context), boundary (empty retrieval, contradictory sources, out-of-scope questions, malformed tool responses), and follow-up (multi-turn drift and corrections).
Criterion: a failure from last week can be replayed as a single node, with its original input and criterion.
Failure mode: evaluation that scores only the final answer, which tells you something is wrong and nothing about where.
Note the ordering. C1 defines where decisions live, C2 makes the rules at those boundaries explicit and testable, C3 proves they fire. Skipping C1 and going straight to C3 is the most common mistake: you end up with a large test suite over a system whose nodes only exist in the model's output text.
5. What this looks like in one trace
Consider a support agent that tells a customer refunds apply to a product category no policy document covers. Nobody notices until a customer complains, because the only thing the system kept was the final answer. An illustrative trace of that node would look like this:
{
"step": "answer_policy_question",
"input": { "question": "...", "retrieved_chunks": 0 },
"constraint": "missing_context -> ask_for_clarification",
"constraint_evaluated": false,
"fallback_reason": null,
"output": { "answer": "Refunds apply to this category" },
"latency_ms": 1840
}
Read against the three constraints: C2's rule exists but was never evaluated; C1 has no record of the state transition that should have stopped the call; C3 has no boundary case asserting on retrieved_chunks = 0. All three failed at once, and the trace is the only reason we can say so. The model produced a plausible continuation — that is what a language model does with no grounded context. Blaming hallucination ends the investigation; the trace starts it.
One detail worth keeping: log the evaluation, not just the spec. Plenty of systems record the constraint and assume it ran. The gap between "we wrote the rule" and "the rule fired on this input" is where most of these incidents live.
6. Scope and limits
A position is only useful if it says what it does not cover.
- It does not fix semantic errors on genuinely ambiguous input, missing knowledge in the retrieval corpus, or requirements that were wrong to begin with. A perfect backbone around a bad question still produces a bad answer.
- It does not mean fewer model calls, cheaper inference, or no hallucinations. There is a cost consequence — explicit state makes retries visible and spend attributable to a step instead of to "the agent ran long" — but that is downstream of the architecture, not the goal.
- It does not remove the need for model quality. It changes what you can do when model quality is insufficient: add a boundary case, tighten a fallback, move a decision into code, and prove the change with a replay.
- It is not worth the overhead when failures are cheap, invisible and unattributable — the cases in §3 where the other side is right.
What it does buy is a short, boring postmortem. Failures become addressable, "add a boundary case" becomes a normal engineering response instead of "tune the prompt and hope," and the model can be changed later without re-litigating the whole architecture. That is what keeps an agent maintainable past the demo — not a cleverer chain of prompts.
7. What follows
If this position holds, the first thing to build is not an eval suite. It is the definition of where the nodes are and who owns them, because every artefact above — the contract, the trace, the test sets — is indexed by that boundary. Get the boundary wrong and the rest becomes bookkeeping over a system you still cannot inspect.
Two questions follow, and we'd ask them before discussing models at all: where is the line between rule logic and model judgement written down so it can be diffed, and can a wrong answer be replayed to the exact node with its input, its criterion and the fallback that did not fire? In your stack today, does that boundary live in the orchestrator's config — or in whatever the model happens to emit at runtime?
Top comments (2)
"The gap between 'we wrote the rule' and 'the rule fired on this input' is where most of these incidents live." This is the single sharpest systems observation published on agent architecture this year.
Your illustrative trace exposes the fatal flaw of first-generation agent observability: recording a constraint in a system prompt and assuming the model evaluated it during next-token prediction.
To answer your two concluding questions—and to offer a production pattern that closes that exact evaluation gap:
1. Where the line is written down: Compiled Invariant Grammar (ANIR)
When constraints are written in conversational Markdown (
CLAUDE.md,.cursorrules), the boundary is permeable. The model treats prose as negotiable guidance, yielding a 30% to 35% violation rate once the context fills past turn 25.We make constraints versionable and diffable by compiling them into an Agent-Native Intermediate Representation (ANIR):
• Lifecycle Phase Partitioning: Rules are strictly partitioned by state (
[PHASE:BEFORE],[PHASE:DURING],[PHASE:AFTER]) rather than competing in a flat document.• Contrastive Negation Tokens: Explicit negative assertions (e.g.
[GATE: PLANNING_LOCK !EDIT_FILES_BEFORE_APPROVAL !BYPASS_CPLAN]) act as high-salience attention anchors in the self-attention matrix, reducing generative drift to under 4%.Because this is a declarative manifest committed to Git, you can
git diffthe constraint rules between two deploys and assert on their presence without running the model.2. Where the boundary lives: Capability-Based Tool Gating
The boundary cannot live in whatever the model emits at runtime, and it cannot rely on the model self-attesting that a constraint was checked.
We solve your
"constraint_evaluated": falseproblem through Capability-Based Tool Gating (Object Capabilities for Agents):Instead of granting the model ambient authority to call execution tools:
This closes the gap between "the rule was written" and "the rule was evaluated." You do not trust the model to report whether it checked policy; the runtime physically denies execution unless the policy gate issued a capability receipt.
3. The Multi-Turn Entropy Reality
Your trace models a discrete node, but production coding agents loop over 30 to 60 turns.
Here, deterministic delivery faces a second failure mode: Context Entropy. Even with a structured prompt contract (C2), attention over early system invariants degrades non-linearly as the context window fills with intermediate tool diffs and terminal output.
True deterministic delivery requires an out-of-band Lifecycle Health Watchdog: monitoring turn-depth proximity, predicting attention compaction boundaries, and enforcing deterministic re-anchoring before the model suffers attention amnesia.
The Takeaway
The fundamental mistake of first-generation agent tooling was treating prompts as the operating system.
Prompts are untrusted user-space inputs; deterministic middleware is the kernel. You do not ask the application space to enforce process isolation, and you do not ask an LLM's next-token predictor to enforce an authorisation boundary. True deterministic delivery only emerges when the model is restricted to semantic judgement, while an external runtime owns the capability tokens, the execution gates, and the audit trail.
The distinction between where a decision is written and where it runs is where most agent systems fall down.
When teams put operational constraints into markdown prompts, they treat the LLM as both the actor and the policy enforcer. If the context window fills up or the prompt drifts, the model negotiates with its own instructions. The failure looks like a hallucination, but the architectural flaw was asking next-token prediction to act as an access control list.
In my own harnesses, the cleanest boundary has been treating the model strictly as an untrusted generator of proposed edits or tool payloads. Every invariant that matters (filesystem boundaries, API allowances, schema validation, and state progression) lives in the shell wrapper or middleware. If a tool call exceeds scope or misses required fields, the runtime halts it with an exit code before any state mutates. Having a deterministic receipt on disk means a failed run can be replayed and tested locally without paying for twenty conversational turns to hit the same bug.