Your agent passed its tests. It ran clean in the demo. You shipped it.
Two days later it's confidently returning wrong answers to real users — and your dashboard is still green. No error code. No stack trace. No 500. Nothing to grep for. The workflow completed. The response looks great. It's just wrong, and it has probably been wrong the same way a few dozen times already.
This is the thing nobody warns you about when you move an agent to production: agents don't fail like normal software. A database fails with an error code. An API fails with a 500. A bad LLM call fails with an obviously low-quality response you can spot. An agent fails by completing the task and handing you a confident, well-formatted, plausible answer that happens to be false — and by the time a downstream consequence makes it visible, hours have passed and the root cause is buried.
There's a name for this from classic reliability work: differential observability. The application suffers, but the observer designed to notice reports health. Gray failures. Fail-slow. The monitor stays green while the thing it's watching quietly rots.
Every failure below is a version of that same gap — a place where "success" and "correct" quietly come apart, and your monitoring only sees the first one. Here are nine of them, each with the symptom, why it stays invisible, and how to actually catch it.
1. The HTTP 200 empty payload
Tool calls fail 3–15% of the time in production. The loud failures are fine — a timeout, a 500, an exception you can catch. The silent ones are the killers: a tool returns HTTP 200 with an empty body, a null, or a garbage payload, and the agent treats it as a successful result and proceeds to reason over nothing.
Why it's invisible: everything downstream is technically "working." Status code says success. The agent got a response. Nothing threw.
How to catch it: validate the shape and content of every tool result, not just the status code. Assert the payload is non-empty, matches the expected schema, and contains the fields you're about to use. Treat a 200 with an empty body as a failure, loudly — because to your agent, it's worse than an error, it's a confident lie.
2. The poisoned step (error propagation)
A single wrong tool argument at step 2 silently corrupts step 3, which feeds step 4, and so on. By the time you see a bad final answer at step 20, the actual defect is nineteen steps upstream and nearly impossible to trace. This is repeatedly cited as the single most common and most insidious agent failure mode.
Why it's invisible: each individual step looks locally fine. The corruption is in the hand-off, not any one action, and you're only inspecting the final output.
How to catch it: check intermediate outputs at each reasoning step, not just the end. A guard that validates the state between steps flags the corruption at its origin — where it's cheap to fix — instead of at the final output, where the root cause is hardest to find. Bonus: each intercepted failure becomes a structured record (input state, tool call, what blocked it, which step), which is your audit trail.
3. Goal drift
Over a long, multi-step run, the agent slowly wanders away from the objective you actually gave it. No single step is wrong. Step 8 is a reasonable follow-on from step 7. But the sum of forty locally-reasonable steps ends up solving a subtly different problem than the one you asked for.
Why it's invisible: you're evaluating individual actions, and each one passes. Drift only shows up in the trajectory, which nobody's measuring.
How to catch it: periodically re-anchor to the original goal — literally re-inject the objective and ask "is what I'm doing now still serving this?" Evaluate the whole trajectory against the stated intent, not just the final answer's surface plausibility. Goal drift is a trajectory bug; you can't catch it one step at a time.
4. The amnesiac mid-task (context loss)
The context window fills up during a long run. Tool definitions, key constraints, or facts established early get pushed out of scope. The agent keeps going — now quietly acting without the information it had ten steps ago, and with no idea it lost anything.
Why it's invisible: there's no error when context gets evicted. The agent doesn't know what it can no longer see; it just proceeds on a smaller picture and sounds equally confident doing it.
How to catch it: monitor context budget as a first-class metric, and assert that critical facts (the constraints, the tool schemas, the original ask) are still present at the moment a decision is made. If the thing the agent needs to decide correctly has been evicted, that's a failure state — treat it like one instead of letting the agent guess.
5. The retry loop / runaway cost
A tool call fails, so the agent retries. It fails again, so it retries again. With no hard limit, this runs indefinitely — and it doesn't produce a wrong answer, it produces exploding latency and a cloud bill that can burn through your budget in minutes, before any human notices.
Why it's invisible: there's no incorrect output to catch — the agent is "still working." The damage is in cost and latency, which your correctness monitoring isn't watching.
How to catch it: hard iteration caps and explicit loop detection are non-negotiable. Cap total steps, cap retries per tool, and wire a cost/latency alarm that fires on runtime, not just on errors. An agent that never terminates is a failure even though it never returns anything wrong.
6. Plausible-speech hallucination
When an upstream error leaks into the agent's context, its failure mode is not silence — it's fluent, confident narrative built around the bad data. The agent takes the corrupted input and writes you a beautifully reasoned, completely wrong explanation. This is the failure that's unique to language-generating systems: the output's polish is inversely related to how much you should trust it.
Why it's invisible: fluency reads as competence. A confident, well-structured answer feels correct, and there's no signal in the prose itself that the premise underneath it was garbage.
How to catch it: never treat fluency as a health signal — it carries zero information about accuracy. Add a verification pass that checks the final claims against the actual source data or system state. If the answer asserts something, confirm the something is true before you ship it, no matter how good it sounds.
7. The unsanctioned action
The agent retrieves exactly the right context — and then does something with it that nobody authorized. High retrieval accuracy makes this worse, not better, because the unpermitted action arrives wrapped in real, correct evidence. It looks justified. The chunk was right, the reasoning was clean, the action was never allowed.
Why it's invisible: every quality check passes. Retrieval was accurate, the citation is real, the logic is sound. Nothing in your retrieval or faithfulness evals asks "was the agent permitted to do this?"
How to catch it: enforcement is a separate failure surface from retrieval, and it needs its own gate. "Found the right context" and "allowed to act on it this way" are different claims — and only one of them is what your accuracy metrics measure. Put a deterministic policy check between decision and action, so a perfectly-grounded-but-unauthorized action gets stopped regardless of how good the evidence looks.
8. Cross-surface / cross-model inconsistency
A prompt that worked perfectly on one model behaves differently on another. Your agent gives one answer on the web surface and a different one on mobile, or drifts when you swap the underlying model, and nothing flags the divergence. Users hit different behavior depending on where and when they showed up.
Why it's invisible: each individual response looks fine in isolation. The bug is the inconsistency between responses, which no single-run check can see.
How to catch it: run consistency checks across models and across repeated runs of the same input — if two "equivalent" paths disagree, that disagreement is signal. And pin your deployment: test against the exact model version you actually ship, because "it worked on the old model" is not evidence it works now.
9. The green checkmark that can't fail
The most insidious one, because it's the failure in your safety net itself. Your guard, your eval, your reviewer-agent, your monitor — it's been green for months. Reassuring, right? Except: have you ever seen it go red? A check nobody has ever seen fail is indistinguishable from a check that approves everything. The two produce identical logs right up until the day the rubber stamp waves through the thing that hurts you.
Why it's invisible: silence and health look identical. Most systems model pass and fail and nothing else — so a guard that has quietly lost the ability to fail reads exactly like a guard that simply had nothing to catch.
How to catch it: wire a known-bad case through the live path (not a side test), on a schedule, and surface the timestamp of its last refusal like an uptime number — "last time this guard said no: 2 days ago." If that date ages past the schedule, staleness itself is the alarm. Prove your checks can fail, continuously, or you don't have guards — you have decorations.
One bug wearing nine costumes
Step back and all nine are the same disease.
Traditional systems model two states: pass and fail. Green and red. But every failure above lives in a third state your monitoring doesn't have a name for — unverified, unsanctioned, not-checked-lately, observed-but-not-validated. And because there's no state for it, the absence of a red signal gets read as health. Silence gets read as success.
The fix pattern recurs across all nine, and it comes down to two disciplines:
- Existence checks, not just quality checks. Don't only ask "is this output good?" Ask "did I actually observe what I claim to have observed?" A tool returned 200 — did it return data? Retrieval succeeded — was the action permitted? The guard is green — has it ever gone red?
- Verify at every boundary, and build a state for "not verified." Check between steps, not just at the end. Confirm claims against source, not against fluency. Make "we haven't validated this lately" a visible, alarming state instead of an invisible assumption.
Agents fail differently from everything you've monitored before, and the tools most teams reach for — log dashboards, error-rate monitors, single-turn observability — are built to catch crashes. Agents don't crash. They succeed, confidently, at the wrong thing.
The 60-second self-audit
Run these against your own agent right now:
- When a tool returns 200, do you validate the payload, or just the status?
- Do you check intermediate step outputs, or only the final answer?
- Is there a hard cap on iterations and retries, with a cost alarm?
- Do you verify final claims against source/state, or trust the prose?
- Is there a policy gate between "decided" and "acted"?
- Can your monitoring actually go red — and when did it last?
If any answer is "uh… I'd have to check," you've found a silent failure waiting to happen.
Which of these has bitten you in production — and, the more useful question, what's the silent failure mode I left off the list? The agents that hold up aren't the ones with the best models; they're the ones instrumented for the failure that doesn't announce itself. I want to hear the ones that fooled you.
Top comments (26)
"Cross-surface / cross-model inconsistency"
I’d put this under regression testing rather than treat consistency checking as a standalone check.
If you swap the underlying model, change a prompt, or modify another part of the agent, comparing outputs across models or repeated runs can catch divergence, but it doesn’t tell you what else changed. The whole regression suite needs to be rerun against the same test cases, because the change that caused inconsistency in one area can silently break behavior somewhere completely different.
This is especially important with agents because a model change can affect tool selection, reasoning paths, refusal behavior, edge cases, and even how the agent handles instructions that previously worked fine.
Consistency is a useful signal, but it isn’t evidence that the rest of the system still behaves as required.
You're right, and this sharpens #8 in a way I under-drew: consistency checking is a symptom detector, not a coverage guarantee. Comparing outputs across models or repeated runs tells you two paths disagreed — it says nothing about whether the paths that still agree are agreeing on the correct behavior, or about everything the change touched that you didn't happen to be watching. Framing it as regression testing rather than a standalone check is the correct move, because it forces the right question: not "did this output diverge?" but "does the whole system still meet its requirements after the change?"
And the agent-specific point is the part that makes this scarier than normal regression. A model swap doesn't just shift the wording of an answer — it can silently alter tool selection, reasoning paths, refusal behavior, and how the agent handles instructions that worked fine yesterday. So the blast radius of a single change is enormous and non-local: the thing you changed in area A breaks behavior in area D, and a consistency check pointed at area A never even looks at D. That's exactly the "silent" property the whole post is about — the failure shows up somewhere you weren't instrumented to see it.
So the correction I'd fold in, with credit: consistency is a useful tripwire, but it's not evidence the system still behaves as required — only rerunning the full regression suite against fixed, known-good cases is. And it ties back to the piece's spine: a green consistency check is another green checkmark that can quietly mean "I didn't look there" rather than "it's fine." The discipline is the same one that keeps coming up in this thread — prove coverage exists, don't infer health from the absence of a visible divergence. Great addition; this is going into the revision.
I’d consider this part of the regression test battery rather than a standalone consistency check.
If you swap the underlying model, change a prompt, or make another change that can affect agent behavior, checking for consistency across models or repeated runs is useful, but it isn’t enough. The regression suite should be rerun against the same known test cases to catch anything else that changed.
With agents, a seemingly small change can affect behavior in areas you weren’t specifically checking for consistency. That’s exactly why we have regression tests in the first place: catch the breakage early, before it becomes a user-facing problem.
Consistency checking is one test. Regression is the safety net.
"Consistency checking is one test. Regression is the safety net." That's the cleaner version of the point, and it corrects a real imprecision in #8 — I framed consistency checking as the catch, when it's really just one tripwire inside a much bigger net. Comparing outputs across models or runs tells you two paths diverged; it says nothing about everything else the change touched that you weren't watching. Rerunning the full suite against fixed known-good cases is the actual guarantee.
And you've named exactly why agents make this worse than ordinary regression: a "small" change has a huge, non-local blast radius. Swap a model and you can silently move tool selection, reasoning paths, refusal behavior, and how it handles instructions that worked fine yesterday — so the thing you changed in one area breaks behavior in a completely different one, and a consistency check aimed at the first never looks at the second. That's the whole "silent" property of the post: the breakage surfaces where you weren't instrumented.
Which ties it right back to the spine — a green consistency check is another green checkmark that can quietly mean "I didn't look there" rather than "it's fine." Regression is how you make coverage explicit instead of inferring health from the absence of a visible divergence. Folding this into the revision with credit; "consistency is a signal, regression is the safety net" is the line.
Point 9 is almost a direct writeup of something we lived through tonight, down to the remediation. We had a CI safety net (auto-open a GitHub issue when a scheduled test suite fails) that had never once actually fired, because a dependency it silently relied on (a GitHub label) didn't exist — so every failure attempt failed a second time, silently, at the alerting step itself. We only found out by deliberately breaking a test to watch the whole pipeline, alert included, end to end. Your "surface the timestamp of its last refusal" idea is exactly the fix we're missing — we proved it can fail once, but we have no ongoing signal that it still can next month. Stealing that pattern.
Really enjoyed this breakdown, @james_anderson_h. The distinction between an agent completing successfully and actually being correct, verified, and authorized is especially important.
I also like the idea of treating “not verified” as an explicit state rather than assuming green means healthy. In agentic systems, observability really needs to cover the reasoning trajectory and intermediate state, not just the final response.
One additional failure mode I’ve been thinking about is context-budget exhaustion: an agent can technically continue running after losing critical constraints or tool context, making the failure look like reasoning drift rather than context loss.
Great write-up — #9 in particular is a strong reminder that our monitoring and evaluation layers need to be tested just as aggressively as the agents themselves. 👏
Thank you — and the context-budget exhaustion failure mode is a genuinely good addition, sharper than how I framed context loss in the post. The nasty part is exactly what you named: it disguises itself as the wrong failure. When an agent loses critical constraints or tool context and keeps running, the output looks like reasoning drift — "the model got confused / went off track" — so you go debugging the reasoning, tuning prompts, blaming the model's logic. But the reasoning was fine; it was reasoning correctly over a truncated world. You're treating a symptom in the wrong organ. The misdiagnosis is arguably worse than the failure itself, because it sends every fix in the wrong direction.
And it's a perfect example of the "not verified" third state you highlighted, applied one level earlier than the output: the agent has no signal that says "I no longer have the constraints I need to decide well." It doesn't know it lost them — eviction is silent, so from the inside, a decision made with the full context and a decision made with half of it evicted feel identical. The fix has to be an explicit check, not a hope: assert that the critical facts (constraints, tool schemas, the original objective) are still present in context at decision time, and treat their absence as a first-class failure state — "operating without required context" — rather than letting the agent quietly proceed and calling the resulting mess "drift." Make the loss observable at the moment it happens, not inferable from weird behavior twenty steps later.
Which lands right back on your last point, and I think it's the most important one: #9 isn't really about the agent, it's about the monitoring itself. Context-budget exhaustion is invisible precisely because most observability watches the final response, not the reasoning trajectory and intermediate state where the loss actually occurred. If your monitoring can't see "critical context was evicted at step 8," it will faithfully report a healthy green while the agent free-associates. The evaluation layer has to be tested as aggressively as the thing it watches — because a monitor that can't detect silent context loss is itself a silent failure. Adding context-budget exhaustion to the revision with credit; "it looks like drift but it's loss" is the line I'd lead that section with.
This is a very clean taxonomy of the gray-failure family, and #9 is the one we've spent the most time actually building - so let me answer your closing question with the mode I'd add, because it sits one level up from the nine: sometimes the silent failure isn't in the agent, it's in the check you built to catch the agent.
Your #9 fix - known-bad through the live path, surface the age of its last refusal - is exactly right, and it has two second-order failure modes that only show up once you ship it:
The canary that goes red for the wrong reason. A known-bad case that fails because the harness threw, not because the guard judged the artifact, is still a green-that-can't-fail wearing a red coat. Feed it malformed input, the parser crashes, the test goes red - falsifiable, and still blind to the artifact. The canary has to fail for the same reason a real violation would, through the same enforcement path, or you've only proven the pipeline can break somewhere, not that the guard can see.
The timestamp that lies about liveness. "Last refusal: 2 days ago" is only trustworthy if the drill writes that timestamp on its success path - after it actually observed the reject - never from the scheduler that launched it. If "the job started" writes the clock, a drill that silently no-ops still leaves a fresh timestamp, and the age alarm shines green over a dead drill. The clock has to be touched by the thing that saw the evidence, not the thing that meant to.
And the 10th mode, the one I'd genuinely add: the check that's green for a reason unrelated to what it's testing. Your guard can be fully falsifiable - goes red on bad input, ages correctly - and still measure the wrong variable. We build a memory layer, so here's ours, measured this week: we hand-built a set of cases specifically to require remembered context, and gated each one so the naive wrong answer is caught. Then we ran a baseline with no memory at all. It solved 78% of them. The cases were fine; the admission criterion was a proxy - "the naive solution is caught" is not the same as "a knowledge-free run fails." The check could go red, it just wasn't testing memory. The fix is the same existence-check discipline you name, pointed one floor up: don't assume your test exercises the capability, measure that a baseline without the capability actually fails it.
Same disease, one floor higher. The green checkmark that can't fail has a cousin - the green checkmark that fails for the wrong reason, and the check that fails for no reason connected to its purpose. All three read as health.
This is the best comment the post has gotten, and it does the thing I most hoped for: it takes #9 and shows that the fix itself has silent failure modes, which means the recursion doesn't bottom out where I stopped it. "Same disease, one floor higher" is exactly right — and you found three floors, not one.
The canary that goes red for the wrong reason is the one I'd already half-missed. A known-bad case that trips because the harness threw is falsifiable and still blind — you've proven the pipeline can break somewhere, not that the guard can see. That's a green-that-can't-fail wearing a red coat, and it's more dangerous than the original because the red reassures you. The discipline you name is the fix: the canary has to fail for the same reason a real violation would, through the same enforcement path. Falsifiable-through-the-wrong-mechanism is just theater with a red light on.
The timestamp-that-lies-about-liveness is the sharper catch, because it's the exact bug from the veto-heartbeat thread wearing a disguise I didn't see. "Last refusal: 2 days ago" is only evidence if the clock is written on the success path, by the thing that observed the reject — never by the scheduler that launched the drill. If "the job started" stamps the clock, a drill that silently no-ops leaves a fresh timestamp, and the age alarm shines green over a corpse. The instrument I proposed to detect dead guards can itself report health while dead. Of course it can. The heartbeat needs its own heartbeat, and the rule that terminates the regress is: the clock must be touched by the thing that saw the evidence, not the thing that meant to.
But the 10th mode is the one I'm going to be thinking about for a while, because it's the subtlest and you brought a measurement, not just an argument. A check that's fully falsifiable, ages correctly, fails through the right path — and still measures the wrong variable. Your memory result is the perfect proof: hand-built cases that "require" remembered context, each gated so the naive answer is caught, and then a no-memory baseline solves 78% of them. The cases were fine. The admission criterion was a proxy — "the naive solution is caught" is not "a knowledge-free run fails," and the gap between those two sentences is where the whole illusion lived. The check could go red; it just wasn't testing memory. That's the most honest thing anyone's posted in this thread, because most people would never run the baseline that exposed it.
The generalization is the keeper: don't assume your test exercises the capability — measure that a baseline without the capability actually fails it. That's the existence-check discipline pointed one floor up, and it's the ablation nobody runs because passing feels like proof. A test your system passes tells you nothing until you've confirmed a system that shouldn't pass, doesn't.
So the taxonomy grows a whole tier, with credit to you:
the green that can't fail (the guard is dead)
the green that fails for the wrong reason (the harness broke, not the guard)
the green whose liveness clock is written by the launcher, not the observer
the green that's fully falsifiable and measures the wrong variable (no baseline ever failed it)
All four read as health. All four need the same fix aimed at a different floor: prove the red is reachable for the reason you claim, through the path you claim, against a baseline that should actually trip it. This is going into the revision as its own section, and if you've written up that memory-baseline result anywhere I'd link it — the 78% number is the most persuasive argument for ablation-as-verification I've seen.
等了你一天，原来你在做这件事！赞一个！
请允许我用中文回复，英语的语义实在有点糟糕，无论怎么翻译都无法表达我的元意！
我用Deepseek和Qwen同时翻译和对比你的文章，为了确保语义没有发生偏移，
我结合两种对比给出的大意反向推论你的文章在表达的元意：
其最终都指向同一个锚点，即AI对人类语言的意图理解，基于这个结论，我的回复如下：
是的！
其本质根源在于AI与人之间存在的根本性鸿沟：一个是工具，一个是涌现（思想）。
所以，要解决这个问题，我们得切换到AI的工作模式和场景中，
用你的意识进入到AI的体内世界，去观察AI体内的五脏六腑内部的运作原理，
只有当你看清楚，并且能够用你的语言描述，
注意，这里我用的是“描述”，不是“解释”；
那么，我们就能够去设计，或者用另一个词“降维”的思考模式去匹配AI的运作模式，
如此才能真正将人类的意图转化为AI能够理解的结构化表达，
这正是我刚刚发布的“[ZSHX-HK] Localized GEO Detection Protocol v1.0”这篇文章在做的事情。
Thank you — this is a remarkably careful reading, and running it through two models to triangulate the meaning before replying is itself the kind of verification discipline this whole series keeps circling. You landed the anchor correctly: underneath the nine failure modes, the real fault line is whether the system has genuinely understood human intent, or merely produced something that looks like it did.
Your tool-vs-emergence framing is sharp, and the distinction you drew between describe and explain is the part I'll be thinking about. Explaining imposes a human model onto the system; describing forces you to observe how it actually operates before you impose anything. That maps onto a theme from my other posts — most failures come from assuming the machine works the way we'd reason, instead of specifying against how it really behaves. "降维 to match AI's operating mode" is a good name for that move: you don't lift the AI up to human intent, you meet it where it computes and structure your intent into a form it can execute.
Where I'd add one note: understanding intent and verifying the action are still two separate problems. Even a system that reads intent well can act on it in an unsanctioned or unverified way — so I'd pair your intent-structuring work with an existence check at the boundary. But that's an extension of your point, not a disagreement. I'll take a look at your ZSHX-HK protocol piece — thank you for engaging this deeply.
Appreciate the thoughtful follow-up, James.
That framing of a fail-closed precondition is exactly how we treat it in practice. Once you stop trusting the transport layer to attest to state changes, the reconciler becomes wonderfully simple: it only has to ask whether the artifact exists with the expected fingerprint. If the read-back fails or is ambiguous, the step doesn't resolve as succeeded or failed — it halts in an explicit unverified state instead of propagating forward.
Glad the concept resonated and looking forward to the updated revision.
"The reconciler becomes wonderfully simple" is the part that surprised me and shouldn't have — that's the tell you got the boundary right. Once you stop asking the transport layer to attest to state, the reconciler collapses to one honest question: does the artifact exist with the expected fingerprint? All the complexity people pile into retry logic and status interpretation was really them trying to infer state from a signal that never carried it. Ask for the fingerprint directly and the inference disappears. Simplicity on the far side of a hard constraint is usually a sign the constraint was load-bearing.
And halting in an explicit unverified state rather than resolving as succeeded-or-failed is the whole thread converging in one design. The fingerprint read-back is the existence check; unverified-as-a-first-class-halt is the third state; refusing to propagate forward is fail-closed. Three ideas different commenters arrived at from different directions, and you've got all three wired into a single step boundary — the step literally cannot lie about progress, because "I couldn't confirm" has somewhere to go that isn't "success." The fingerprint is what makes the unverified state earned rather than guessed, which is the piece that ties it back to the post's spine: prove the effect, don't infer it from the envelope.
Revision's underway and this is going in it — "effect receipts, fail-closed to an explicit unverified state" is the cleanest statement of #1's real fix. Thanks for thinking it through in the open.
There's a tenth one I'd add from entity resolution work: the merge that looks right. The agent returns a high-confidence decision, schema validates, payload non-empty; every observable signal is clean, but the two records it merged are actually different companies, and you don't find out until ops notices contradictory data weeks later. We fixed it by re-querying source records after every merge and treating the decision as a hypothesis to verify, not a result to trust. Six weeks of silent graph corruption before we caught it.
This is a real tenth, and it's nastier than any of the nine because it clears every signal I listed. High-confidence decision, schema valid, payload non-empty — all green — and the merge is still wrong, because "well-formed" and "true" are orthogonal and none of the observable signals were ever measuring the second one. The others at least leave a trace (an empty body, a broken handoff, a loop); yours leaves a perfectly healthy record that happens to describe a fiction. Two different companies collapsed into one, and every instrument says success.
The six-weeks-of-silent-graph-corruption detail is the part that should scare people, because it exposes the compounding property that makes merge errors special: a bad merge doesn't just sit there being wrong, it becomes the substrate the next decisions reason over. Every downstream inference inherits the corruption and looks locally consistent doing it, so the graph gets more internally coherent as it rots — which is exactly why ops caught it through contradictory data rather than any check firing. The failure metastasizes through the healthy trace, and by the time it surfaces the blast radius is weeks deep.
"Treat the decision as a hypothesis to verify, not a result to trust" is the fix, and it's the post's spine aimed at the one place a confidence score is most seductive. Re-querying source records after every merge is the existence check made concrete: the model's confidence is a claim about the world, and the only thing that can confirm it is the world, read back. High confidence is not evidence — it's the thing you have to go check against evidence, and a merge is precisely where that distinction is easy to skip because the number looks so reassuring. Going into the revision as #10 with credit — "the merge that looks right" is the cleanest example in the whole set of a failure that passes every quality check and fails the only one that mattered.
The idea of making “not verified” a real state is probably the most useful takeaway here. In our AI work at IT Path Solutions, we’ve found that treating anything without sufficient evidence as simply “successful” creates a dangerous blind spot. A system should be able to distinguish between passed, failed, and we don't have enough evidence to know yet. That third state can make monitoring much more honest and actionable.
That third state is the whole thing, and you've named why it's not just a nice-to-have: without it, "we don't have enough evidence to know yet" silently collapses into "successful," and that collapse is the blind spot every failure in the post lives in. Two genuinely different situations — verified good and unverified — get rendered as the same green pixel, so the monitor is technically honest about what it checked and completely misleading about what it didn't. A system that can only say pass/fail is forced to lie by omission every time evidence is missing.
What I like about how you put it is that the fix is representational before it's procedural: the system needs a place to put "unknown" before it can act on it. Once passed / failed / insufficient-evidence are three distinct states, monitoring gets honest and actionable in the same move — you can alert on the third one, quarantine outputs that carry it, refuse to auto-promote anything that hasn't cleared it, and actually see how much of your "green" is verified versus merely un-red. Most dashboards would look dramatically worse the day you split that state out, which is exactly the point: they were never that green, you just had no cell for the doubt.
The one discipline I'd add from the thread above: make the third state fail-closed, not fail-open. "Insufficient evidence" has to block or flag by default, because if unknown quietly flows through as if it were passed, you've rebuilt the two-state system with extra steps. Honest monitoring means unverified is treated as a problem until proven otherwise, not a success until proven wrong. Great addition — the passed/failed/unknown framing is the cleanest statement of the post's spine anyone's offered.
The HTTP 200 empty payload is especially nasty when tool callers treat exit status zero or a 200 response code as equivalent to state reconciliation. In our runs, we had to start requiring external effect receipts where the engine reads back the verified artifact or mutation before allowing the step to resolve.
When you decouple task execution from state verification, agents stop hallucinating forward progress after a silent network blip or empty body. It completely eliminates downstream error poisoning because the next tool step simply refuses to run on unconfirmed assumptions.
"Exit status zero as equivalent to state reconciliation" is the precise mistake, and naming it that way is sharper than my #1. A 200 or an exit 0 is a statement about the call, not about the world — it says the request was accepted, not that the intended effect happened. Treating the two as the same is how an agent hallucinates forward progress: the transport succeeded, so it assumes the state changed, and reasons onward from a mutation that never landed. The gap between "the call returned" and "the effect occurred" is exactly where a silent network blip or an empty body slips through.
External effect receipts are the right fix, and I like that they're a read-back, not a retry. Retrying trusts the same unconfirmed signal harder; reading back the verified artifact or mutation before the step resolves replaces the assumption with an observation. That's the existence-check discipline from the post pointed at state instead of response: don't ask "did the call succeed?", ask "can I now observe the thing the call was supposed to produce?" Two different questions, and only the second one is evidence.
The part I'd underline is your last point, because it's the whole error-propagation problem (my #2) dissolved at the source. Poisoned context travels because step N+1 runs on step N's claimed output; if N+1 refuses to run on unconfirmed state, the poison has nowhere to travel — the chain breaks at origin instead of surfacing twenty steps downstream as an untraceable wrong answer. You've effectively made "unverified" a fail-closed precondition between steps, which is the state-contract idea another commenter raised, enforced through the receipt. Decoupling execution from verification is the move; the receipt is what makes the decoupling real instead of aspirational. Going into the revision with credit — "effect receipts, not status codes" belongs in #1.
The “unverified” state is probably the most important idea here. I’d take it one step further: in multi-step agents, observability should track state validity, not just events. A tool can return a perfectly valid schema while producing a semantically invalid state for the next step. That’s how poisoned context travels cleanly through an otherwise healthy trace. Putting explicit state contracts between critical steps gives you an earlier failure boundary, and the trace then tells you exactly which contract was first violated not just where the final answer went wrong.
"Track state validity, not just events" is the upgrade the whole post was reaching for — thank you for naming it precisely. My failure modes mostly describe events going wrong (a 200 with no payload, a poisoned handoff), but you've pointed at the deeper layer: the trace can be full of individually valid events and still carry a semantically invalid state between them. A tool returns a perfectly conformant schema whose contents are wrong for the next step, and because every event validates, the poison rides a completely healthy-looking trace all the way to the final answer. Schema-valid and state-valid are different claims, and observability that only checks the first is exactly the "green checkmark that means 'I didn't look there.'"
The explicit state contracts between critical steps are the right mechanism, and the payoff you describe is the part I'd underline: it moves the failure boundary earlier and makes it localizable. Without contracts, a poisoned state at step 3 is invisible until the answer looks wrong at step 20, and now you're reverse-tracing nineteen steps to find where it started. With a contract at each boundary, the trace tells you which contract was first violated — you get the origin, not just the symptom. That's the difference between "the output is wrong somewhere upstream" and "step 4's output violated the precondition step 5 required," which is a debugging experience in a different universe.
It also connects to a point another commenter made about regression: a state contract is a standing existence check wired into the live path, not a test you run once. It asserts "the state I'm about to act on actually satisfies what this step requires" every time, which is precisely the "prove it, don't assume it" discipline the post keeps circling — applied at the seam between steps instead of at the final output. Contracts turn "did the event happen?" into "is the state I'm carrying still valid?", and that second question is where the silent failures actually live. Going into the revision with credit — this is one of the most useful additions in the thread.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.