The reframing from "which model" to "which decisions require judgement vs. a diff" is the right cut, and it's one most teams resist because writing a prompt is cheaper than writing the deterministic path — so the boundary silently creeps toward the model over time. The part I'd underline is the replayable record per node. In multi-agent setups the thing that actually kills you isn't a wrong answer, it's a wrong answer you can't localize: several agents in a chain, each sounding confident, and no way to say which one invented the bad fact. Once every node logs its inputs, output, and the acceptance criterion it was supposed to meet, debugging stops being archaeology. One question — how do you keep that boundary from drifting back over time? We've found it takes an explicit, recurring review ("should this node really be a model call?"), because the default gravity is always toward handing more to the model.
Unified AI gateway — China's best LLMs + OpenAI/Gemini, the OpenAI way. Singapore-hosted (Tencent Cloud SG), PDPA-aligned, up to 90%+ cheaper than OpenAI on China models. Build with AI, Southeast Asia
Agree on the drift direction — writing a prompt is cheaper than writing the deterministic path, so the boundary almost never migrates back on its own. A calendar-based review is better than nothing, but I'd make it trigger on observable signals instead.
Two that seem useful: when a node's acceptance criterion stops being machine-checkable and turns into "a human reads the output and decides", the decision has already moved into the model, regardless of what the config says. And when a structured prompt contract accumulates conditional branches, fallback clauses and "if X, instead do Y" lines across versions, that's the rule layer being rewritten in prose. Both show up in the diff, so the review can be driven by the contract's version history and by the test sets rather than by a quarterly reminder.
The multi-agent point is the sharper one though: localization is what the per-node record buys you, and drift destroys it silently — the chain still produces confident answers, but the boundary between a judgement and a diff is no longer recoverable after the fact.
Is your review cadence fixed, or triggered by changes to the contract and the test sets?
"The gap between 'we wrote the rule' and 'the rule fired on this input' is where most of these incidents live." This is the single sharpest systems observation published on agent architecture this year.
Your illustrative trace exposes the fatal flaw of first-generation agent observability: recording a constraint in a system prompt and assuming the model evaluated it during next-token prediction.
To answer your two concluding questions—and to offer a production pattern that closes that exact evaluation gap:
1. Where the line is written down: Compiled Invariant Grammar (ANIR)
When constraints are written in conversational Markdown (CLAUDE.md, .cursorrules), the boundary is permeable. The model treats prose as negotiable guidance, yielding a 30% to 35% violation rate once the context fills past turn 25.
We make constraints versionable and diffable by compiling them into an Agent-Native Intermediate Representation (ANIR):
• Lifecycle Phase Partitioning: Rules are strictly partitioned by state ([PHASE:BEFORE], [PHASE:DURING], [PHASE:AFTER]) rather than competing in a flat document.
• Contrastive Negation Tokens: Explicit negative assertions (e.g. [GATE: PLANNING_LOCK !EDIT_FILES_BEFORE_APPROVAL !BYPASS_CPLAN]) act as high-salience attention anchors in the self-attention matrix, reducing generative drift to under 4%.
Because this is a declarative manifest committed to Git, you can git diff the constraint rules between two deploys and assert on their presence without running the model.
2. Where the boundary lives: Capability-Based Tool Gating
The boundary cannot live in whatever the model emits at runtime, and it cannot rely on the model self-attesting that a constraint was checked.
We solve your "constraint_evaluated": false problem through Capability-Based Tool Gating (Object Capabilities for Agents):
Instead of granting the model ambient authority to call execution tools:
The tool interface itself (e.g. file mutation, shell execution, database query) is physically locked at the middleware layer.
To invoke a state-changing tool, the model must supply a cryptographically signed state token produced by the pre-execution constraint evaluator.
If the prerequisite condition was not satisfied (e.g. missing context, zero retrieved chunks, or unapproved planning state), no capability token exists. The tool call fails with exit code 1 at the process boundary before any filesystem or network resource is touched.
This closes the gap between "the rule was written" and "the rule was evaluated." You do not trust the model to report whether it checked policy; the runtime physically denies execution unless the policy gate issued a capability receipt.
3. The Multi-Turn Entropy Reality
Your trace models a discrete node, but production coding agents loop over 30 to 60 turns.
Here, deterministic delivery faces a second failure mode: Context Entropy. Even with a structured prompt contract (C2), attention over early system invariants degrades non-linearly as the context window fills with intermediate tool diffs and terminal output.
True deterministic delivery requires an out-of-band Lifecycle Health Watchdog: monitoring turn-depth proximity, predicting attention compaction boundaries, and enforcing deterministic re-anchoring before the model suffers attention amnesia.
The Takeaway
The fundamental mistake of first-generation agent tooling was treating prompts as the operating system.
Prompts are untrusted user-space inputs; deterministic middleware is the kernel. You do not ask the application space to enforce process isolation, and you do not ask an LLM's next-token predictor to enforce an authorisation boundary. True deterministic delivery only emerges when the model is restricted to semantic judgement, while an external runtime owns the capability tokens, the execution gates, and the audit trail.
Unified AI gateway — China's best LLMs + OpenAI/Gemini, the OpenAI way. Singapore-hosted (Tencent Cloud SG), PDPA-aligned, up to 90%+ cheaper than OpenAI on China models. Build with AI, Southeast Asia
The capability receipt is the part I'd underline: it moves "was the constraint evaluated" from a model self-report to a precondition of execution, and a tool that physically cannot be invoked without a signed gate result is a different kind of boundary from prose in a system prompt. That's a stronger answer than the trace alone.
Where I'd still separate the claims is between existence and evaluation. A diffable manifest proves what was deployed; a capability receipt proves the gate issued a token. Neither proves the gate evaluated the right condition on this input. If retrieved_chunks is zero but the gate reads a field an earlier step already populated, it issues a receipt for a condition that never held — and the trace shows a successful, signed, compliant call. That's the same gap moved one layer down: the check becomes deterministic, but its input can still be wrong. Which is why the article records constraint_evaluated as an event with its inputs, rather than inferring it from the presence of either the rule or the receipt.
I'd leave the percentages out of the argument. Without a stated measurement protocol — what counts as a violation, over how many turns, judged by whom — a violation rate isn't comparable across setups, and two approaches measured differently will produce any number you want. The architectural claim doesn't need them.
Does the gate log its own condition evaluation into the trace, or only whether the token was issued?
Fair push—the stale witness problem. If the gate merely inspects an in-memory pipeline variable like step.chunks_count that an earlier turn populated, you've just automated an illusion.
To your question: the gate has to log the raw witness, not just the token.
The pattern that keeps this honest in practice is treating the gate as a pure function over content-addressed inputs rather than ambient pipeline state. The token doesn't simply assert PASS; it cryptographically binds the content digests of the actual retrieved chunks to the execution parameters. If retrieval returned zero chunks, the witness payload in the receipt is literally hash([]), which fails the execution precondition downstream.
That still doesn't solve whether the retrieval query itself was semantically "good"—that is the classical oracle problem, and anyone claiming architecture eliminates that is selling magic. But it does mean that in a post-incident audit you can replay the exact predicate offline against the recorded witness and get byte-for-byte reproducibility.
And I take your point entirely on the percentages. Without a published benchmark harness, turn-depth definition, and reproducible fixture, throwing around violation ratios just reads like hand-waving. Appreciate the pushback—it sharpens the boundary considerably.
Unified AI gateway — China's best LLMs + OpenAI/Gemini, the OpenAI way. Singapore-hosted (Tencent Cloud SG), PDPA-aligned, up to 90%+ cheaper than OpenAI on China models. Build with AI, Southeast Asia
The content-addressed witness is a real step past an ambient pipeline variable: the gate now has to bind what it actually saw, and hash([]) failing the precondition downstream is a check that cannot be talked out of it. I also agree with where you left the oracle problem — whether the retrieval query was any good is a semantic question, and no amount of hashing settles it.
One edge I'd still separate: hash([]) is evidence of absence, not evidence of why the input was empty. Corpus has no such policy, the query string was built wrong, and a scope filter narrowing the result set to zero all produce the same digest. The predicate replays byte-for-byte in all three cases, so the audit proves the gate was honest and still cannot tell you what went wrong upstream. A signed empty set is a stronger failure than an unsigned one, but it is still one failure class with three causes.
The small addition is to content-address the query side alongside the chunks: query text, filters, corpus version, retriever revision. Then the empty witness carries a reproducible reason, and the boundary set can be split into three cases instead of asserting the same thing for all of them — otherwise a regression suite passes while the root cause stays unknown. I'd also keep the witness and the receipt written by different components. A hash computed and consumed inside one process is a self-report with better cryptography; the rule logic stays deterministic by construction, but the evidence source does not become independent.
None of this makes the query semantically better, and it adds fields to every run. It only changes "precondition failed" into "precondition failed for a recorded, replayable reason". If you deliberately corrupt the query string while keeping retrieval empty, is there a field in the witness that changes with it — and if not, how would a post-incident replay distinguish "nothing was found" from "we looked in the wrong place" ?
A node replay needs a clear distinction between supplied input and ambient state. The same request can take a different deterministic branch if the rule reads the current clock, a mutable configuration, or a database row that changed after the original run.
I would capture the exact state revision and any clock or external-read values alongside the acceptance criterion, then replay against those recorded values. The test should also prohibit live side effects during reconstruction. That makes the single-node replay criterion stronger than rerunning the current implementation with the same question and hoping its environment still matches.
Unified AI gateway — China's best LLMs + OpenAI/Gemini, the OpenAI way. Singapore-hosted (Tencent Cloud SG), PDPA-aligned, up to 90%+ cheaper than OpenAI on China models. Build with AI, Southeast Asia
This tightens the replay criterion more than the original framing did. "Same input" quietly expands from the prompt to the entire read surface of the node — and once you accept that, a rule reading the current clock, a config value or a database row is reading an input it never declared.
Two places where the capture tends to be incomplete. External reads are broader than clock and config: feature flags, the model and tool versions actually used, and the routing decision itself all change which branch a deterministic rule takes. If those aren't in the record, replaying against recorded values still reproduces the wrong run. And the prohibition on live side effects only holds if capture happens at the boundary — one direct read inside the rule code bypasses any harness that assumes callers inject state.
That also suggests replay is a property of the interface, not of the logging: the node has to ask a provider for its clock and revision so the harness can substitute recorded values. Otherwise "replay with recorded values" is a convention that a single refactor can break.
In your records, do model version and routing choice count as state, or do you keep them as separate version tags?
For the replay contract I am proposing, both belong in the captured envelope, with different roles. The immutable model and tool revisions identify what executed. A routing choice is an output when replaying the router, but an input when replaying the downstream node that received the request.
I would preserve that boundary explicitly. The router test should recompute its choice from recorded inputs and compare it with the recorded choice; the downstream test should receive that recorded choice without consulting the live router. Merely attaching version tags would not enforce either behavior.
Unified AI gateway — China's best LLMs + OpenAI/Gemini, the OpenAI way. Singapore-hosted (Tencent Cloud SG), PDPA-aligned, up to 90%+ cheaper than OpenAI on China models. Build with AI, Southeast Asia
The output-of-the-router / input-of-the-downstream-node split is the sharper version of what I wrote, and it exposes what my earlier reply left implicit: one captured value has to be used in two incompatible roles depending on which node is under test. Version tags attach metadata; they don't enforce either behaviour.
The part I'd want defined before calling that a contract is the equivalence relation for the router test. Recomputing a recorded choice and comparing it with the recorded one assumes "different" is meaningful — but a tie between two routes, a reordered candidate list, or a threshold read from a value that has since been revised can all produce a legitimate difference. Without a rule for which differences count as a regression, replay degrades into a string comparison against one historical run, and it will fail on exactly the changes you want it to tolerate. I'd put that rule in the white-box evaluation set: positive cases that must reproduce, boundary cases at the switch threshold, and follow-up cases where an upstream change is allowed to alter the choice.
There's also a closure between your "prohibit live side effects" and the read-surface point from earlier. If the downstream test must not consult the live router, and a rule must not read a live clock or config, then the envelope's boundary is precisely the node's declared read surface. Capture isn't a list of convenient fields — it's the complement of that surface. Otherwise you get a contract that passes locally and fails in production on a read nobody declared.
When a recomputed route differs from the recorded one, how do you classify it: regression, environmental drift, or allowed non-determinism — and does that classification live in the trace or in the reviewer's head?
I would store that classification in the trace, together with the comparison rule and its version. The replay specification should define allowed equivalent routes, tie handling, and boundary tolerances before the run.
A difference under the same captured inputs and a violated rule becomes a regression. A changed dependency or undeclared read becomes environmental drift, with its evidence attached. An explicitly permitted alternative records which equivalence rule accepted it. Anything outside those categories stays unresolved instead of being silently excused by the reviewer.
A technologist working in automation. Here I write up what I actually learn building tools, engineering AI agents, and keeping systems running, along with the workflows and habits that stick.
The distinction between where a decision is written and where it runs is where most agent systems fall down.
When teams put operational constraints into markdown prompts, they treat the LLM as both the actor and the policy enforcer. If the context window fills up or the prompt drifts, the model negotiates with its own instructions. The failure looks like a hallucination, but the architectural flaw was asking next-token prediction to act as an access control list.
In my own harnesses, the cleanest boundary has been treating the model strictly as an untrusted generator of proposed edits or tool payloads. Every invariant that matters (filesystem boundaries, API allowances, schema validation, and state progression) lives in the shell wrapper or middleware. If a tool call exceeds scope or misses required fields, the runtime halts it with an exit code before any state mutates. Having a deterministic receipt on disk means a failed run can be replayed and tested locally without paying for twenty conversational turns to hit the same bug.
Unified AI gateway — China's best LLMs + OpenAI/Gemini, the OpenAI way. Singapore-hosted (Tencent Cloud SG), PDPA-aligned, up to 90%+ cheaper than OpenAI on China models. Build with AI, Southeast Asia
The actor / policy-enforcer split is the cleaner way to state the boundary: it makes the question about where a rule is enforced, not how strongly it is worded. That is exactly what a deterministic backbone is for — the model proposes, code decides, and the receipt records both.
One thing the wrapper design tends to leave out: if it only validates the payload's shape, a replay can still diverge. The same tool call can be legal under one state revision and illegal under the next, and schema validation won't see it. So the recorded input has to include the state version the rule read, not just the arguments it was passed.
The other gap is on the rejection path. If the runtime halts an out-of-scope call before any state mutates, the attempt often never reaches the log — which means the audit trail shows a clean run and can't tell you whether the rules held or the model simply never tested them. Recording rejected payloads alongside accepted ones is what turns a receipt into evidence; otherwise you're replaying only the branch that happened to work.
Do you record the payloads your wrapper rejects, or only the calls that went through?
We're a place where coders share, stay up-to-date and grow their careers.
The reframing from "which model" to "which decisions require judgement vs. a diff" is the right cut, and it's one most teams resist because writing a prompt is cheaper than writing the deterministic path — so the boundary silently creeps toward the model over time. The part I'd underline is the replayable record per node. In multi-agent setups the thing that actually kills you isn't a wrong answer, it's a wrong answer you can't localize: several agents in a chain, each sounding confident, and no way to say which one invented the bad fact. Once every node logs its inputs, output, and the acceptance criterion it was supposed to meet, debugging stops being archaeology. One question — how do you keep that boundary from drifting back over time? We've found it takes an explicit, recurring review ("should this node really be a model call?"), because the default gravity is always toward handing more to the model.
Agree on the drift direction — writing a prompt is cheaper than writing the deterministic path, so the boundary almost never migrates back on its own. A calendar-based review is better than nothing, but I'd make it trigger on observable signals instead.
Two that seem useful: when a node's acceptance criterion stops being machine-checkable and turns into "a human reads the output and decides", the decision has already moved into the model, regardless of what the config says. And when a structured prompt contract accumulates conditional branches, fallback clauses and "if X, instead do Y" lines across versions, that's the rule layer being rewritten in prose. Both show up in the diff, so the review can be driven by the contract's version history and by the test sets rather than by a quarterly reminder.
The multi-agent point is the sharper one though: localization is what the per-node record buys you, and drift destroys it silently — the chain still produces confident answers, but the boundary between a judgement and a diff is no longer recoverable after the fact.
Is your review cadence fixed, or triggered by changes to the contract and the test sets?
"The gap between 'we wrote the rule' and 'the rule fired on this input' is where most of these incidents live." This is the single sharpest systems observation published on agent architecture this year.
Your illustrative trace exposes the fatal flaw of first-generation agent observability: recording a constraint in a system prompt and assuming the model evaluated it during next-token prediction.
To answer your two concluding questions—and to offer a production pattern that closes that exact evaluation gap:
1. Where the line is written down: Compiled Invariant Grammar (ANIR)
When constraints are written in conversational Markdown (
CLAUDE.md,.cursorrules), the boundary is permeable. The model treats prose as negotiable guidance, yielding a 30% to 35% violation rate once the context fills past turn 25.We make constraints versionable and diffable by compiling them into an Agent-Native Intermediate Representation (ANIR):
• Lifecycle Phase Partitioning: Rules are strictly partitioned by state (
[PHASE:BEFORE],[PHASE:DURING],[PHASE:AFTER]) rather than competing in a flat document.• Contrastive Negation Tokens: Explicit negative assertions (e.g.
[GATE: PLANNING_LOCK !EDIT_FILES_BEFORE_APPROVAL !BYPASS_CPLAN]) act as high-salience attention anchors in the self-attention matrix, reducing generative drift to under 4%.Because this is a declarative manifest committed to Git, you can
git diffthe constraint rules between two deploys and assert on their presence without running the model.2. Where the boundary lives: Capability-Based Tool Gating
The boundary cannot live in whatever the model emits at runtime, and it cannot rely on the model self-attesting that a constraint was checked.
We solve your
"constraint_evaluated": falseproblem through Capability-Based Tool Gating (Object Capabilities for Agents):Instead of granting the model ambient authority to call execution tools:
This closes the gap between "the rule was written" and "the rule was evaluated." You do not trust the model to report whether it checked policy; the runtime physically denies execution unless the policy gate issued a capability receipt.
3. The Multi-Turn Entropy Reality
Your trace models a discrete node, but production coding agents loop over 30 to 60 turns.
Here, deterministic delivery faces a second failure mode: Context Entropy. Even with a structured prompt contract (C2), attention over early system invariants degrades non-linearly as the context window fills with intermediate tool diffs and terminal output.
True deterministic delivery requires an out-of-band Lifecycle Health Watchdog: monitoring turn-depth proximity, predicting attention compaction boundaries, and enforcing deterministic re-anchoring before the model suffers attention amnesia.
The Takeaway
The fundamental mistake of first-generation agent tooling was treating prompts as the operating system.
Prompts are untrusted user-space inputs; deterministic middleware is the kernel. You do not ask the application space to enforce process isolation, and you do not ask an LLM's next-token predictor to enforce an authorisation boundary. True deterministic delivery only emerges when the model is restricted to semantic judgement, while an external runtime owns the capability tokens, the execution gates, and the audit trail.
The capability receipt is the part I'd underline: it moves "was the constraint evaluated" from a model self-report to a precondition of execution, and a tool that physically cannot be invoked without a signed gate result is a different kind of boundary from prose in a system prompt. That's a stronger answer than the trace alone.
Where I'd still separate the claims is between existence and evaluation. A diffable manifest proves what was deployed; a capability receipt proves the gate issued a token. Neither proves the gate evaluated the right condition on this input. If retrieved_chunks is zero but the gate reads a field an earlier step already populated, it issues a receipt for a condition that never held — and the trace shows a successful, signed, compliant call. That's the same gap moved one layer down: the check becomes deterministic, but its input can still be wrong. Which is why the article records constraint_evaluated as an event with its inputs, rather than inferring it from the presence of either the rule or the receipt.
I'd leave the percentages out of the argument. Without a stated measurement protocol — what counts as a violation, over how many turns, judged by whom — a violation rate isn't comparable across setups, and two approaches measured differently will produce any number you want. The architectural claim doesn't need them.
Does the gate log its own condition evaluation into the trace, or only whether the token was issued?
Fair push—the stale witness problem. If the gate merely inspects an in-memory pipeline variable like
step.chunks_countthat an earlier turn populated, you've just automated an illusion.To your question: the gate has to log the raw witness, not just the token.
The pattern that keeps this honest in practice is treating the gate as a pure function over content-addressed inputs rather than ambient pipeline state. The token doesn't simply assert
PASS; it cryptographically binds the content digests of the actual retrieved chunks to the execution parameters. If retrieval returned zero chunks, the witness payload in the receipt is literallyhash([]), which fails the execution precondition downstream.That still doesn't solve whether the retrieval query itself was semantically "good"—that is the classical oracle problem, and anyone claiming architecture eliminates that is selling magic. But it does mean that in a post-incident audit you can replay the exact predicate offline against the recorded witness and get byte-for-byte reproducibility.
And I take your point entirely on the percentages. Without a published benchmark harness, turn-depth definition, and reproducible fixture, throwing around violation ratios just reads like hand-waving. Appreciate the pushback—it sharpens the boundary considerably.
The content-addressed witness is a real step past an ambient pipeline variable: the gate now has to bind what it actually saw, and hash([]) failing the precondition downstream is a check that cannot be talked out of it. I also agree with where you left the oracle problem — whether the retrieval query was any good is a semantic question, and no amount of hashing settles it.
One edge I'd still separate: hash([]) is evidence of absence, not evidence of why the input was empty. Corpus has no such policy, the query string was built wrong, and a scope filter narrowing the result set to zero all produce the same digest. The predicate replays byte-for-byte in all three cases, so the audit proves the gate was honest and still cannot tell you what went wrong upstream. A signed empty set is a stronger failure than an unsigned one, but it is still one failure class with three causes.
The small addition is to content-address the query side alongside the chunks: query text, filters, corpus version, retriever revision. Then the empty witness carries a reproducible reason, and the boundary set can be split into three cases instead of asserting the same thing for all of them — otherwise a regression suite passes while the root cause stays unknown. I'd also keep the witness and the receipt written by different components. A hash computed and consumed inside one process is a self-report with better cryptography; the rule logic stays deterministic by construction, but the evidence source does not become independent.
None of this makes the query semantically better, and it adds fields to every run. It only changes "precondition failed" into "precondition failed for a recorded, replayable reason". If you deliberately corrupt the query string while keeping retrieval empty, is there a field in the witness that changes with it — and if not, how would a post-incident replay distinguish "nothing was found" from "we looked in the wrong place" ?
A node replay needs a clear distinction between supplied input and ambient state. The same request can take a different deterministic branch if the rule reads the current clock, a mutable configuration, or a database row that changed after the original run.
I would capture the exact state revision and any clock or external-read values alongside the acceptance criterion, then replay against those recorded values. The test should also prohibit live side effects during reconstruction. That makes the single-node replay criterion stronger than rerunning the current implementation with the same question and hoping its environment still matches.
This tightens the replay criterion more than the original framing did. "Same input" quietly expands from the prompt to the entire read surface of the node — and once you accept that, a rule reading the current clock, a config value or a database row is reading an input it never declared.
Two places where the capture tends to be incomplete. External reads are broader than clock and config: feature flags, the model and tool versions actually used, and the routing decision itself all change which branch a deterministic rule takes. If those aren't in the record, replaying against recorded values still reproduces the wrong run. And the prohibition on live side effects only holds if capture happens at the boundary — one direct read inside the rule code bypasses any harness that assumes callers inject state.
That also suggests replay is a property of the interface, not of the logging: the node has to ask a provider for its clock and revision so the harness can substitute recorded values. Otherwise "replay with recorded values" is a convention that a single refactor can break.
In your records, do model version and routing choice count as state, or do you keep them as separate version tags?
For the replay contract I am proposing, both belong in the captured envelope, with different roles. The immutable model and tool revisions identify what executed. A routing choice is an output when replaying the router, but an input when replaying the downstream node that received the request.
I would preserve that boundary explicitly. The router test should recompute its choice from recorded inputs and compare it with the recorded choice; the downstream test should receive that recorded choice without consulting the live router. Merely attaching version tags would not enforce either behavior.
The output-of-the-router / input-of-the-downstream-node split is the sharper version of what I wrote, and it exposes what my earlier reply left implicit: one captured value has to be used in two incompatible roles depending on which node is under test. Version tags attach metadata; they don't enforce either behaviour.
The part I'd want defined before calling that a contract is the equivalence relation for the router test. Recomputing a recorded choice and comparing it with the recorded one assumes "different" is meaningful — but a tie between two routes, a reordered candidate list, or a threshold read from a value that has since been revised can all produce a legitimate difference. Without a rule for which differences count as a regression, replay degrades into a string comparison against one historical run, and it will fail on exactly the changes you want it to tolerate. I'd put that rule in the white-box evaluation set: positive cases that must reproduce, boundary cases at the switch threshold, and follow-up cases where an upstream change is allowed to alter the choice.
There's also a closure between your "prohibit live side effects" and the read-surface point from earlier. If the downstream test must not consult the live router, and a rule must not read a live clock or config, then the envelope's boundary is precisely the node's declared read surface. Capture isn't a list of convenient fields — it's the complement of that surface. Otherwise you get a contract that passes locally and fails in production on a read nobody declared.
When a recomputed route differs from the recorded one, how do you classify it: regression, environmental drift, or allowed non-determinism — and does that classification live in the trace or in the reviewer's head?
I would store that classification in the trace, together with the comparison rule and its version. The replay specification should define allowed equivalent routes, tie handling, and boundary tolerances before the run.
A difference under the same captured inputs and a violated rule becomes a regression. A changed dependency or undeclared read becomes environmental drift, with its evidence attached. An explicitly permitted alternative records which equivalence rule accepted it. Anything outside those categories stays unresolved instead of being silently excused by the reviewer.
The distinction between where a decision is written and where it runs is where most agent systems fall down.
When teams put operational constraints into markdown prompts, they treat the LLM as both the actor and the policy enforcer. If the context window fills up or the prompt drifts, the model negotiates with its own instructions. The failure looks like a hallucination, but the architectural flaw was asking next-token prediction to act as an access control list.
In my own harnesses, the cleanest boundary has been treating the model strictly as an untrusted generator of proposed edits or tool payloads. Every invariant that matters (filesystem boundaries, API allowances, schema validation, and state progression) lives in the shell wrapper or middleware. If a tool call exceeds scope or misses required fields, the runtime halts it with an exit code before any state mutates. Having a deterministic receipt on disk means a failed run can be replayed and tested locally without paying for twenty conversational turns to hit the same bug.
The actor / policy-enforcer split is the cleaner way to state the boundary: it makes the question about where a rule is enforced, not how strongly it is worded. That is exactly what a deterministic backbone is for — the model proposes, code decides, and the receipt records both.
One thing the wrapper design tends to leave out: if it only validates the payload's shape, a replay can still diverge. The same tool call can be legal under one state revision and illegal under the next, and schema validation won't see it. So the recorded input has to include the state version the rule read, not just the arguments it was passed.
The other gap is on the rejection path. If the runtime halts an out-of-scope call before any state mutates, the attempt often never reaches the log — which means the audit trail shows a clean run and can't tell you whether the rules held or the model simply never tested them. Recording rejected payloads alongside accepted ones is what turns a receipt into evidence; otherwise you're replaying only the branch that happened to work.
Do you record the payloads your wrapper rejects, or only the calls that went through?