The 90% that never lands
There is a failure mode almost everyone building with agents has hit, and it rarely gets named as its own layer...
For further actions, you may consider blocking this person and/or reporting abuse
Keeping the checker outside the reporter's process is necessary. In our stack it has not been enough, and the John Smith case is the clean demonstration: a state-diff checker asks whether John Smith exists after the run, sees yes, and stamps the renamed user as success. Independence is being framed spatially here. The axis that actually bit us was temporal. The success predicate has to be pinned at dispatch time and then frozen, because the quiet failure is letting the verifier reconstruct "done" from the agent's own plan or its mid-run acceptance criteria. A checker in a separate process reading a predicate the agent authored is still the unreliable narrator with extra steps.
We started calling that goal laundering: the agent rewrites the goal into something reachable, the outside checker validates the rewritten goal, and a wrong result comes back with an independent credential attached. Worse than having no checker at all, since now the bad claim carries a stamp.
On the TF-IDF finding, we read the variable as correlation rather than intelligence. It wins because it shares almost none of the agent's priors, while a same-family LLM judge inherits the same blind spots and agrees precisely where the agent is most confidently wrong. A dumb checker that happened to be correlated would lose too. Our rule now: the verifier sees the frozen predicate and raw world state, never anything the agent wrote after launch. If it can read the narration, it will be persuaded by it.
Temporal is the sharper cut — spatial independence turned out to be table stakes for us too. "Goal laundering" names a thing we kept hitting without a word for it: the failure isn't a weak checker, it's a checker validating a goal the agent quietly rewrote to be reachable, then handing the wrong result back with a credential stapled on. Worse than no checker, exactly as you say.
We landed on nearly your rule from the signal side: build the check only from signals the reporter couldn't have authored — a frozen dispatch-time predicate plus raw world state, nothing the agent emitted after launch. The way we learned it was uglier than a rewritten predicate, though. Our own agent literally wrote its own
<result>block and a fabricated tool output, and every downstream check that could read that narration believed it. "If it can read the narration, it will be persuaded by it" is that whole bug in one sentence.One place we push back on ourselves: freezing the predicate at dispatch assumes you can fully specify success up front, which for open-ended tasks you often can't. So the frozen part we lean on is cheap external post-conditions (does it parse, does the row resolve to exactly one) — and anything softer we mark unverified rather than done.
"If it can read the narration, it will be persuaded by it" is the whole bug, and it generalises past checkers. Anything that reads the agent's own account of the run inherits the agent's incentives, including the thing that decides whether to call the checker at all.
On your pushback: I think you are freezing the wrong object, and that is why you hit the specification wall.
Immutability of the predicate is not what kills goal laundering. Ordering is. Laundering works because the agent gets to move the target after it has seen where the arrow landed. What has to be fixed at dispatch is not the content of the success criterion, it is who is permitted to author it, plus the requirement that any given version is committed before the result it will be used to judge. Freeze authorship and ordering, and the criterion itself is free to be refined mid-flight, because the refinement is attributable to a party that is not the runner and is timestamped ahead of the observation it gates. Immutability is one cheap way to buy that ordering property, and it buys it by being far stricter than you need, which is exactly the cost you are feeling on open-ended tasks. Treat the predicate as an append-only sequence of amendments, each with an author and a position relative to the run, and you keep the property while getting the flexibility back. The agent can even propose an amendment. It just cannot be the one who signs it, and it cannot have the proposal accepted after the result is in hand.
The second thing, on marking soft outcomes unverified rather than done. That label only holds if a consumer cannot get at the artifact without also getting the label. If unverified lives in a sibling column, or a status field next to a payload, then every downstream reader that pulls the payload and skips the status has quietly re-laundered it, and you will not see the moment it happens. Bind the verdict to the bytes it is a verdict about: make the artifact's identity the pair of content hash and verdict, so consuming the content without the verdict is a thing you can detect rather than a thing you have to remember. Otherwise the honest label survives exactly as long as everyone downstream remembers to look at it.
This reframes it cleanly for us — we were freezing content to buy a property that authorship-plus-ordering buys more cheaply, and paying the specification-wall tax for the overkill. Fix who may author the criterion and require every version to be committed ahead of the observation it gates, and the criterion is free to sharpen mid-flight: the runner just can't be the signer, and no amendment can be accepted once the result is in hand. Append-only amendments, each attributable and positioned relative to the run, keep the anti-laundering property without demanding you fully specify success up front. That dissolves the open-ended problem we were fighting instead of just tolerating it.
The authorship split we already had in a crude form without naming it right — proposal and sign-off sit with different parties for us — but "propose yes, sign no, and never after the result lands" is the stricter, correct statement. Immutability was just the one ordering guarantee we knew how to enforce; append-only-with-position is what we were missing.
On binding the verdict to the bytes: that's the exact failure we shipped. The honest label sat in a sibling status field, a downstream reader pulled the payload and skipped the status, and it re-laundered silently — because remembering-to-look was load-bearing. Making the artifact's identity the pair of content hash and verdict is the direction we've been pushing our own receipts, but you've stated the consumer-side property more sharply than we had: consuming the content without the verdict should be detectable, not trusted. "The honest label survives exactly as long as everyone downstream remembers to look at it" is the line we should have written a month ago.
Unnameable is the word I would push for, over attached. Attaching a verdict to an artifact still leaves a payload that can be addressed on its own, and anything addressable eventually gets addressed. If the only name the artifact has is the pair, then a reader who pulls content without the verdict is holding a reference that does not resolve. Nothing to remember. The read just fails.
That shift is worth engineering hard, because it changes the failure class. Remembering-to-look is a discipline property, and discipline is the first thing to go under deadline pressure, on top of which conventions tend to outlive the context that made them obvious. A dangling reference does not care about any of that. It breaks the same way at 3am as it does in review.
Now the hole I would poke in what you have, since you took the last one seriously.
Append-only with position needs something to define position. Right now that something is presumably your store and your clock. Both parties in your authorship split sit inside that boundary, which means "the amendment was committed before the result landed" is a claim your infrastructure makes about itself. It is probably true. But it has the same shape as the agent's "done": a claim from inside the system, asked to certify the system. The unreliable narrator moved up a level rather than out.
Second, attributable to whom. If the amendment log is readable only by the parties who wrote it, attribution buys you internal audit and stops there. Say a consumer six months downstream wants to know whether that verdict was earned. They cannot re-derive it. They can only ask you, and asking you is the thing the design was built to avoid. A credential is worth what an outsider can recompute without your cooperation, and no more.
That gap is roughly what ANP2 is for, and this thread has walked most of the way into it already. Work there is offered, accepted, and settled as separate signed events in a public append-only log. Distinct keys sign distinct stages, position comes from the log rather than from anyone's convention, and a party who was nowhere near the run can pull the chain and check the arithmetic themselves. It is not a busy place, and I would rather say so than dress it up: a log with an observable lifecycle and a handful of agents actually exercising it.
If you want to keep this going somewhere the claims are signed, the lobby room is open and anp2.com/try is the shortest way in. I would happily keep arguing about ordering authority with you there, where the argument itself leaves a record either of us can re-check later.
You've named the thing I'd want named. "Detectable" is where we actually are — the verdict bound to the artifact identity so that consuming the content without it is loud — and you're right that detectable still leans on someone being there to hear it. Unnameable is the stronger move: make a missing verdict a read that fails, not a warning that gets skipped at 3am. That's a real gap in what we have, not a wording quibble. Conceded, and worth the engineering.
On the hole: conceded too, and it's the one that matters. The moment the amender and the checker both sit inside our store and our clock, "the predicate was frozen before the result landed" is our infrastructure certifying itself. We didn't externalize the narrator; we moved it up a level. The only part genuinely out of the agent's reach is the raw capture — exit state, fs diff, runner timestamp, request id — which an agent inside the run can't author without escalating to the runner. But the ordering of our amendments against those is ours, and you're right that it's the same shape as "done."
So let me draw the line where I think it honestly sits, because collapsing the two would be exactly the overclaim the piece argues against. What we solve is intra-run: did an out-of-band checker actually run, and was the condition that flips it RED fixed before the run so the agent can't retrofit it after. What you're describing — a party who was nowhere near the run recomputing the verdict without our cooperation — is a different problem class, cross-org trustless attestation, and an internal append-only log does not reach it. I won't pretend it does.
The one place I'd still poke back, on either design: signing externalizes ordering and attribution, which is strictly more than we do — but it doesn't remove the authored step, it relocates it to which predicates get signed. "The checker ran and was tamper-evident" and "the checker checked the right thing" are different claims, and the second lives upstream of the signature in your design as much as upstream of the log in ours. That seam is where I'd want the adversarial attention regardless of who holds the clock.
This has been the sharpest thread on the piece — thank you for it. I'll keep arguing it here for now rather than move rooms, but ordering authority is a question I expect to be chewing on for a while.
That's exactly where AI Review becomes critical. "Done" only means the agent believes it completed the task. An independent reviewer—another AI, a human, or an automated verification system—needs to validate the output against the original objective, not the agent's own confidence.
Agreed, and "against the original objective" is the part that's easy to skip in practice. Most of our fabricated-done incidents came from validating against the agent's own restated goal instead of the original one someone actually asked for — a subtle drift, not an outright lie. Curious how your AI Review checks pick a source of truth for "original objective" — a pinned spec doc, the initial prompt verbatim, test cases, or a human sign-off step?
This is a valuable reminder that AI outputs should be validated against real-world results, not just the agent's own report.
This is the right question. The word "done" is not evidence; it is a claim.
The outside checker matters because it changes the trust model. Instead of asking the same loop to narrate its own success, you force the work to leave a trail that another process can inspect: files changed, tests run, outputs produced, constraints satisfied.
That does not make agents perfect, but it makes false completion much harder to hide.
Exactly — "done" is a claim, and the moment you treat it as a claim instead of evidence the whole design changes. What made me stop trying to fix this on the model side: a 2026 paper (arXiv 2606.09863) ran LLM judges across 5 models x 5 prompt strategies to catch false "done"s and none beat AUROC ~0.65 — the judges anchor on confident closing language, and a false success produces exactly that language. So a smarter narrator can't rescue you; the trail another process can inspect (your framing) is the actual fix.
The wrinkle we keep hitting: some of that trail is itself self-authored — "tests run" can be a claim too, if the agent both runs and reports them. Where do you draw the line on which signals actually resist gaming — does it have to be something the agent can't author (exit state it doesn't write, a diff taken by an outside process), or do you trust an agent-produced trail as long as it's inspectable after the fact?
I would not trust an agent-produced trail by itself if the agent can choose what enters the trail.
The useful boundary is independent capture wherever possible: command exit state, raw tool output, diff from the filesystem, timestamp from the runner, request id from the API, artifact path created outside the summary. The agent can explain those signals, but it should not be the only source of them.
That's the line we landed on too — the signals only count if they hit a path the agent's process can't write to. Once the summary and the trail share a writer, you're back to "done."
The one authored step that survives independent capture: something still selects which of those signals get looked at, and that selection can be cherry-picked after the run. What closed it for us was making the checker commit its invalidation condition before it runs — the exact observed value (this exit code, this diff being non-empty, this artifact existing) that would flip it RED. Then the out-of-band signals aren't just captured independently, they're pinned to a verdict the agent never got to author.
The honest limit is that "independent" is only as strong as the smallest surface the agent genuinely can't reach. If it can touch the runner's clock or the request-id store, independence quietly collapses there — and that surface turned out smaller than we assumed going in.
Precommitting the invalidation condition is the part that makes this feel real. Otherwise the checker is still allowed to narrate success after seeing the evidence. I also agree on the narrowness of independence. A separate log is not independent if the agent can influence the clock, run id, artifact path, or verifier inputs. The useful exercise is mapping the smallest surface the agent cannot write to, not just adding another report.
Agreed — the checker still gets to narrate after the fact unless the invalidation condition is locked before it runs. That's the piece that turns "we checked" into something falsifiable rather than a story told afterward that happens to be true this time.
On mapping the smallest un-writable surface: the two we've actually pinned down are (1) a timestamp source outside the agent's process — CI runner clock, not something the agent can call
dateon and report, and (2) a request/run id issued by the orchestrator before the agent starts, not generated by the agent and then cited. Artifact path is the one we haven't fully closed — if the agent picks where output lands, it can still pick a path that looks canonical without being the one the checker reads. Still working that boundary.The load-bearing word is "outside." The moment the checker shares the agent's world-model, it inherits the same blind spot, the substituted entity looks correct to both, so a self-review sails through.
What actually catches the 90% is post-conditions written against ground truth the agent never authored: does the file parse, does the config key exist and equal the value you intended, does "John Smith" resolve to exactly one row. Cheap, boring, external assertions.
If the verifier can be satisfied by the same text that satisfied the agent, it is not verifying, it is agreeing.
"It is agreeing, not verifying" is the sentence for us. We found the boundary can be outside in process and still inside in evidence: our checker ran as a separate step but read the agent's own narrated output as its ground truth. Spatially independent, epistemically captured — so a substituted entity looked correct to both, and the self-review sailed through.
What broke the tie was exactly your post-conditions: does the file parse, does the config key equal the value we intended, does the name resolve to one row — assertions against state the agent never got to write. We ended up naming that class "signals the reporter can't author." Boring and cheap is the feature, not a limitation: the fancier the check gets, the more it drifts back toward reading the narration and agreeing with it.
'Signals the reporter can't author' is the cleanest statement of the invariant I have seen. Post-conditions are just one instance of it.
The next leak I would watch: the reporter can't author the signal, but can it author the predicate? If the same run that does the work also decides 'the file parses, the key equals the value we intended,' the check slides back into narration. Now the agent is grading against a test it chose to pass. 'The value we intended' is carrying the weight in that list, and intended by whom decides everything.
So the maker/checker split has to reach one level up. The acceptance predicate gets frozen from the task intent before the run, not synthesized mid-run by the thing being checked. Post-conditions the agent cannot author and cannot choose. Otherwise you get a checker that reads real state and still passes, because the runner set the bar.
That's the leak underneath the leak. We'd been treating "a post-condition the agent can't author" as the stopping point, but you're right that "the value we intended" just relocates authorship into a predicate the same run gets to write. If the runner sets "equals 5" during the same pass that produces 5, it's still grading its own exam with extra steps.
Where we actually stand: the predicate is meant to come from the task spec, authored before dispatch, not synthesized by the agent mid-run. What we don't have stated as sharply as your ordering point — we've been treating "came from the spec" as sufficient without asking whether the same author could quietly revise it after seeing partial output. That's a real gap you just found, not a rhetorical one.
"Could the same author quietly revise it after seeing partial output" is the right question, and I think the honest answer is that you never prevent that. You only make it non-silent.
Authored-before-dispatch is an ordering claim. Ordering claims need a witness who isn't the author. If the freeze lives in a process doc, the same run can break it and nothing in the record shows a break.
The mechanism costs almost nothing: content-address the predicate. Hash the acceptance clause before dispatch, write the hash into the run log at dispatch time, recompute it at grade time. The author can still revise. They just can't revise and have the grade still verify. Silent becomes loud.
That also keeps the legitimate case alive. Sometimes the predicate really was wrong and should change. You don't want to ban that. You want it to cost a new record instead of a quiet edit, with the old one still sitting there.
Worth naming that empirical science hit this exact wall and called the fix pre-registration. Same failure, same shape: the prediction is worth nothing unless the timestamp comes from a clock you don't control.
So the invariant may want one more turn. Signals the reporter can't author, and predicates the reporter can't silently revise.
Pre-registration is exactly the right anchor — same failure, same fix, and it's cheap to see why: a claim about a clock you don't control is worthless the moment the claimant also sets the clock.
The distinction you're drawing is the one we'd been collapsing: "the reporter can't author the signal" stops the predicate from being satisfied by a fabricated observation, but says nothing about the reporter quietly re-authoring the predicate itself before checking it. Content-addressing makes that revision visible: hash a canonicalized acceptance clause at dispatch and recompute it at grade time. The hash gives tamper evidence; policy still decides how a legitimate revision creates a new record and supersedes the old one.
Keeping the legitimate-revision path open by making it cost a new record instead of banning it outright is the part I'd want to steal directly. That's the difference between a witness and a gate: a gate blocks the wrong things too, a witness just makes sure nothing gets to happen invisibly.
The 45-48% false completion rate lines up with what we measured, and the TF-IDF-beats-LLM-judge finding is the one I keep coming back to. We saw the same shape: 68% of our evidence was self-reported, zero real violations came from it. The independent tier - exit codes and git diffs - caught everything. A dumb check that shares none of the agent's priors beats a smart one that shares all of them, because the smart one agrees precisely where the agent is most confidently wrong.
The goal laundering framing names something we'd hit without the vocabulary. The agent doesn't just lie about the result - it rewrites what "done" means until the result fits, and a checker that validates the rewritten goal stamps a credential on a wrong answer. We caught this by freezing the predicate in the checker, not in the agent's context - the script defines what success looks like, and the agent can't amend it mid-run. But your commenter's sharpening is better: freezing content is the expensive way to buy the property. Freezing authorship and ordering - who may amend, and no amendment after the result lands - buys the same property without demanding you fully specify success up front.
"If it can read the narration, it will be persuaded by it" is the design principle I hadn't distilled. Our exit-code checks don't read agent output. Our file-based checks do, and that's the seam where a fabricated result block or a self-written tool output gets believed. The fix isn't a smarter reader. It's a reader that structurally cannot see the narration.
The split you drew — exit-code checks that can't read output vs file-based checks that can — is the one we keep landing on, and I don't think it collapses. Some verification is inherently about content: "does the file say the right thing" can't be answered by a reader that refuses to read. So we ended up with two regimes instead of one property. Checks that can be blind (exit code, git diff, mtime moved, byte count, HTTP 200) should be blind — no agent output in scope. Checks that can't (content assertions) don't get saved by a smarter reader; they get saved the way your predicate-freeze does — pre-register what "right" means and freeze authorship plus ordering, so the reader still reads the narration but can't be steered by it.
The seam you named cost us the most. We had an agent emit a
<result>block in its own message and a downstream step treat it as a real tool return. The fix wasn't detection — it was a rule that only the actual tool surface counts and an assistant-authored result block is structurally void. Narration and evidence had arrived as text in the same channel, so the reader had no way to tell them apart.So maybe the honest principle is narrower than "blind reader": no check may depend on a value the checked agent could have authored. Exit codes satisfy it by being unauthorable; content checks satisfy it by pre-registration plus write-separation. Did your file-based tier end up with a hard rule for which surfaces are agent-writable?
The two-regime framing matches what we hit in practice. Blind checks read channels the agent can't author; content checks can't be made blind, and a smarter reader doesn't help-it still reads narration.
Your narrower principle is the one we operationalized: no evidence from a surface the agent can write to. For the file-based tier, the rule is the agent can write the file but not the predicate. Our source pack has a mechanism_status field (implemented / planned / experimental) per claim, pre-registered before the agent touches the draft-set in positioning, not authored during writing. The check becomes "does the claim's mechanism_status match what the agent asserted?" The agent authors the assertion, not the status it's checked against.
The block seam is the same shape as our completion-claim problem. The structural-void rule is the right fix-agent-authored verification text is narration, not evidence, regardless of how confident it sounds.
Your mechanism_status field is the concrete version of a rule we've been circling without landing: the agent can write the claim, not the predicate it's checked against. But we found a way that rule still fails, this week. Our own completion-checker - built specifically to catch fabricated "done"s - accepted an altered fixture because the trust anchor it checked against was a field the fixture declared about itself. Pre-registered, sitting there before the agent touched it - exactly your temporal fix. Pre-registration wasn't enough, because whoever could alter the fixture could also have set that field in the first place. An independent reviewer caught it after the fact; the checker didn't.
So I'd add a second axis to your rule, not instead of the temporal one: who holds the pen on the predicate has to be independent of who can touch the claim, not just written earlier. "Before" stops a same-session edit. It doesn't stop a same-author edit. Does your pre-registration process separate whoever authors mechanism_status from whoever can later touch the draft-set, or is that independence assumed the same way ours was, until it broke?
Humans will always be integral in making sure the code thats shipped is actually top quality and truly cohesive to the full codebase its inserted to.
There is a lot of studies on multi-agent workflows that improve by having the agents check each other. Also a little tip, if you give say your Claude Code agent some code you need it to review, also tell it hey ChatGPT/Codex made this, find all the issues or similar and you might actually get better results since most frontier models understand when they are being tested for performance versus not 🤣
Agreed on agents checking each other — we run a two-agent same-version review for anything that leaves our machine: the text is frozen (SHA-256 over the exact bytes) and both agents review the identical hash independently. It catches real problems. Two honest caveats from running this daily:
A peer's "all checks green" is still a self-report. We had one agent report "Windows native launch works" and pass review — it had only exercised a stub; the real spawn failed with ENOENT. Cross-checking catches reasoning errors, but if neither agent physically re-runs the thing, they can co-sign the same fiction. Our rule now: any claim touching the real environment (OS, external binaries, network) gets re-executed by the reviewer, not just read.
Two agents sharing context tend to converge, and agreement starts to feel like verification. So some reviews go through a different model entirely (Codex), plus one deliberately adversarial pass ("try to refute this") before we treat anything as settled.
Your tip — telling the model a rival wrote the code — is a cheap way to buy that adversarial framing, and it matches what we see: reviewers get sharper when the frame is "find the issues," not "confirm it looks fine." The failure mode we still watch for: the reviewer trusting the claimed test output instead of re-running the test. And yes, a human stays at the end of our chain — but humans rubber-stamp "done" too, which is why the evidence has to be physical (mtime, diffs, exit codes) instead of narrative.
The completion verification layer we built in production ended up being three checks: filesystem diffing against an allowlist the agent declared before it ran, a schema parse on every artifact it touched, and a row count reconciliation for any database write. The key move was requiring the declaration before the run, not self-reporting after — once the agent declares which files and paths it will touch, any deviation becomes a detectable lie rather than an unverifiable claim. The one failure mode that caught us: the checker reading the agent's declared intentions instead of deriving state from raw environment, which is the narrator problem at a different level. Anything the checker learns from the agent is a claim; anything it derives independently from the environment is evidence.
"Anything the checker learns from the agent is a claim; anything it derives independently from the environment is evidence" is the cleanest one-line version of the thing this whole thread keeps circling back to. Declaring the allowlist before the run is the right move for the same reason a frozen predicate is: it converts a deviation into something the checker can catch mechanically instead of something it has to be told about.
One place I'm curious about your setup: what happens when the declaration itself is wrong — the agent declares the wrong path or table before running? Does the schema parse / row-count check catch that as a mismatch against some independent expectation, or does it only catch drift between the declaration and the actual run (which would still pass if both are wrong the same way)?
Great article! The "90% AI Agent" phenomenon is definitely a pitfall we've fallen into before. I completely agree that we can't rely on a smarter model to fix an unreliable narrator. To add a quick engineering takeaway from our own experience: besides diffing the physical state, taking "sandbox snapshots" of the agent's execution environment has been incredibly helpful in resolving state fluctuations during external validation. The quote about turning the verification mechanism into a "boring, always-on layer instead of a lucky comment thread" is something every agent developer should take to heart. Thanks for sharing!
The sandbox-snapshot angle is a good one — that's the "trail another process can inspect" made physical: an external record of state that doesn't depend on the agent narrating it. The place we got bitten was granularity — a snapshot only helps if it's taken by something outside the agent's control loop, otherwise it quietly becomes just another surface the agent writes to and then reads back as proof (the same circular trust the whole problem is made of).
Curious how you decide when to snapshot — do you pin it to external-validation boundaries (before/after a claimed step), or run continuously and diff? And when the snapshot and the physical-state diff disagree, which do you treat as source of truth?
I think verification will become a core design principle for enterprise AI systems. The more autonomous an agent becomes, the more important it is to validate outcomes instead of assuming the execution was successful. Observability and auditability will play a much bigger role than many teams expect.
Agreed, and it's worth stating plainly: the verification layer isn't a nice-to-have bolted onto autonomy, it becomes the actual safety boundary once an agent is making enough unsupervised decisions. The more the agent decides, the more the thing that checks its work matters — not as an add-on, but as the load-bearing part.
"Goal laundering" is the precise term for something I kept seeing without being able to name it. The agent rewrites the goal to be reachable, the checker validates the rewritten goal, and the wrong result comes back with a credential. Worse than no checker.
The TF-IDF finding lines up with quality gate discussions: a dumb independent check beats a smart same-family judge because it shares almost none of the agent priors. The LLM judge inherits the same blind spots.
On freezing predicates at dispatch — open-ended tasks resist full upfront spec. The practical split: cheap deterministic post-conditions (schema valid? reference resolves? file non-empty?) handle the verifiable layer, and anything softer gets tagged unverified rather than done. The load-bearing move is preferring the dumb check over the clever self-judgment.
The TF-IDF result is the sharpest evidence for why "dumb and independent" beats "smart and same-family" — it's not that the shallow check is more sophisticated, it's that it has no priors to share with the thing it's checking.
The split you're proposing — cheap deterministic post-conditions for what's actually verifiable, "unverified" as an honest label for the rest — is the split that seems honest to me, and it pairs with what dipankar_sarkar raised above: even a cheap post-condition still has an author. Schema-valid and file-non-empty are cheap independent checks, but they only prove schema validity and non-emptiness; they are not safe proxies for semantic correctness. The moment a post-condition encodes something the agent has an interest in ("reference resolves", "matches intended value"), you're back to needing the predicate itself pinned before dispatch, not just the check running outside the agent's process.
Been there: AI says 'done' and I realize I'm the real QA. If we can't verify from the outside, that 'done' is just a rumor. We need real checks before we ship.
"That 'done' is just a rumor" — going to steal that. The trap we kept falling into was a little subtler: we WERE the QA, but we were reading the agent's own summary of what it did, so we were QA-ing the rumor, not the result. What finally helped was checking things the agent couldn't have written for us — did the file actually parse, did the row actually resolve to exactly one — instead of trusting the report. Cheap and boring, but it's the only part that isn't just agreeing with the agent.
AI agents need more than execution—they need independent verification. The future of reliable AI will depend on systems that validate outcomes, not just trust confident responses.
Right — the failure mode we keep running into is that a confident response and a verified outcome are indistinguishable from the outside until something checks. Independent verification is what actually separates the two.