A real-world production bug, a protocol-design insight from the community, and why cryptographic signatures alone can't save you.
Two weeks ago, I posted on LinkedIn about OpenWorkProof, a verification protocol for AI agent work. I expected maybe a dozen likes. I did not expect a global community of engineers, protocol designers, and security researchers to spend 72 hours dismantling our assumptions in public — and then rebuilding them into something better.
One comment, from a senior engineer at a large fintech company, stands out as the single most valuable piece of feedback we've ever received. He described a production bug that had survived for months:
The Verifier That Couldn't Fail
Their gateway ran a verification check on every API response. The check was signed, audited, and reported verified: true on every run. For months, nobody questioned it.
Then someone wrote a negative control: a deliberately broken input designed to provoke a failure. The check should have returned verified: false. It didn't.
The bug was a single line: ln.strip(). A stray newline character had pushed an assert statement after a return — nested inside a function body where it would never execute. Exit code: 0. Gateway verdict: verified.
Eight different caller-test shapes ran against this check. Five of them produced false passes.
The numbers he shared are sobering:
| Status | Count |
|---|---|
| Guards total | 40 |
| Proven (can detect failure) | 7 |
| Broken | 0 |
| Unproven | 33 |
No agent lied. No log was tampered with. Every signature would have verified. The checker was structurally incapable of failing.
That's not a bug. That's a category error.
What Signatures Can't Prove
This is the gap we've been wrestling with for weeks — and the community has now named it with precision.
A cryptographic signature can prove three things:
- Who signed the claim
- When the claim was signed
- That the claim hasn't been tampered with since signing
It cannot prove that the claim was correct.
The ln.strip() bug illustrates this perfectly. The signature verified that the check ran. It verified that the exit code was 0. It verified that the result hadn't been altered. Every cryptographic guarantee held. The answer was still wrong.
This is what ANP2, another community contributor, called the difference between reproducibility and falsifiability. Reproducibility proves the same bytes ran in the same environment and produced the same output. It does not prove that the output was meaningful. A test suite that always exits 0 — frozen by digest, authored by an independent source, executed in verifier mode — will produce perfectly valid signatures in perpetuity. Every digest will match. Every chain will verify. Nothing will be tested.
Mikhail, who has contributed some of the most precise protocol-design insights in the entire thread, captured this with an aphorism that has become our design principle: immutable evidence ≠ immutable truth. A receipt can prove a thing happened. It cannot prove the thing was right.
The Missing Layer: Negative Controls
The fintech engineer's response to his own bug was elegant in its simplicity: every guard must declare a negative control that must exit non-zero.
This is not a nice-to-have. It is a prerequisite for meaningful verification. A check that cannot demonstrate it can fail is a check whose passing results tell you nothing.
In protocol terms, this translates to what we're now calling dual-arm verification. Any claim-bearing receipt must reference two things:
- Arm 1: A pinned test suite that passes (proves the system works on the happy path)
- Arm 2: A pinned mutant or negative control that fails (proves the instrumentation has discriminative power)
The verifier checks both arms independently. Arm 1 passing proves the claim is consistent. Arm 2 failing proves the claim is discriminating. Either arm failing its expected outcome invalidates the receipt.
This isn't theoretical. It's been battle-tested. The engineer's team now runs 40 guards with 7 fully proven, 0 broken, and 33 still unproven. The 33 is the honest number — a constant reminder of how much verification infrastructure still cannot demonstrate it works on anything but the happy path.
The Four Layers of Verification Trust
After integrating feedback from the entire thread, here's how we've come to understand the verification stack for AI agent work:
| Problem | Solution Layer | What It Proves |
|---|---|---|
| "Does this check have discriminative power?" | Negative controls | The checker can actually fail |
| "Was this the check that was actually run?" | Receipt chain (digest binding) | The claimed execution matches the actual execution |
| "Was the result altered after execution?" | Cryptographic signatures | Integrity of the evidence chain |
| "Who authorized this check to run in the first place?" | PolicyDecision (capability grant) | Authority, scope, and time window |
Each layer solves a failure mode that the layer below it cannot see. A system with negative controls but no signatures is vulnerable to tampering. A system with signatures but no negative controls is vulnerable to — well, to ln.strip().
What This Means for AI Agent Verification
The AI agent ecosystem is rapidly building infrastructure for connecting agents to tools (MCP) and agents to agents (A2A). But neither protocol addresses the verification gap. They answer what an agent can do. They don't answer whether what it did was correct, authorized, and auditable.
This is where verification protocols like OpenWorkProof enter the picture. But the key insight from this community discussion is that verification without falsifiability is theater. A receipt chain that can't distinguish between "the check ran" and "the check worked" is just expensive logging.
The minimum viable verification stack for any AI agent producing claims is:
- Negative controls that prove the verification instrumentation itself can detect failure
- Receipt chains that bind every execution step to a specific config, environment, and authorization
- Independent recomposition that allows any third party to reconstruct the execution environment and re-verify without trusting the original executor
- Retraction receipts that allow previously accepted claims to be marked as refuted — because even correctly verified claims can become wrong as context shifts
Thanks to Tom Jones, ANP2, Mikhail, Giulio D'Erme, Zira, Brian Jin, Puneet, and the rest of the LinkedIn thread for the feedback that drove these insights.
Top comments (4)
The "40 guards, 7 proven, 33 unproven" table is the whole article for me. A test that has never once returned false isn't passing — it's untested, and you have no way to tell the two apart from the green dashboard. That
ln.strip()bug (anassertshoved after areturnwhere it can never run) is almost the platonic example: exit 0, signature valid, verdict meaningful-looking, answer wrong.The negative-control discipline you land on is really mutation testing pointed at your verifiers instead of your code: deliberately break the input, and any guard that still says "verified" just outed itself as decorative. I'd go one step further and make it continuous — feed a known-bad fixture through every guard on a schedule, not just once by hand, because a guard that's proven today silently rots into unproven the moment someone refactors the thing it depended on.
Your "immutable evidence ≠ immutable truth" line is the right hill. A signature answers who/when/untampered; it says nothing about correct. Most agent-verification stacks I've seen spend all their rigor on the first three and quietly assume the fourth.
Max — this is exactly the kind of comment that makes an article worth writing.
Your reframe — "a test that has never returned false isn't passing, it's untested" — crystallizes the entire problem in a way I wish I'd led with. The "40 guards, 7 proven, 33 unproven" table is the article. Everything else is just elaboration.
Your observation that this is mutation testing aimed at verifiers instead of code is the single most precise description I've seen of what negative-control discipline actually does. It's not a bug-hunting technique; it's an integrity audit on the verification layer itself. And your .lstrip() example — the assert after return — is almost the platonic ideal of the problem: every green checkmark is technically correct, but the verdict is structurally meaningless.
On continuous vs. one-shot negative control: This is where I think you've pushed the idea further than we did in the article. The distinction between "proven" and "unproven" guards is not static — it's a decaying function over time. A guard that was proven today becomes unproven the moment someone refactors the dependency it relied on. Your suggestion to feed known-bad fixtures through every guard on a schedule is essentially treating verification integrity as a continuous service, not a one-time event.
This maps directly to something we're experimenting with in OWP: the Known-Broken Control Test (a permanently-failing control that must be caught by every verifier as a baseline). But your "scheduled rot" framing is better — it suggests we should be measuring not just whether a guard catches the bad input, but when it last proved it did. The "33 unproven" guards aren't just unproven — they're stale.
On "immutable evidence ≠ immutable truth": This is the hill the entire verification industry needs to die on. Most agent-verification stacks I've seen spend all their rigor on who/when/untampered, and quietly assume correctness. The cryptographic signature becomes a security blanket — it proves chain-of-custody, not chain-of-logic. Your comment captures exactly why this is dangerous: the signature is necessary but not sufficient for trust.
One question back to you: If we treat negative-control as a continuous, scheduled process, how do you think about the cost of this discipline? The mutation-testing analogy is apt because mutation testing in CI is notoriously expensive. Do you see a path to making "continuous verifier mutation" lightweight enough to run on every commit, or does it remain a periodic deep-audit that complements (rather than replaces) fast-path verification?
I'm starting to think the right model might be something like: fast verification on every commit (pass/fail) + continuous negative-control audit (proven/unproven decay tracking) + periodic full verifier mutation (comprehensive integrity check). Each layer catches a different class of "silently wrong" problems.
Your framing has already influenced the next draft. Thank you for this.
GitHub: github.com/dengyier/OpenWorkProof
Dengyier, Max,
Great article — it synthesizes the whole discussion brilliantly. Making "immutable evidence ≠ immutable truth" a core design principle is spot on.
The Four Layers nail execution integrity (checker can fail, execution matches claim, result not tampered, proper authorization). But for LLM agents, there's a missing Layer 0: Semantic Correctness.
In traditional software,
exit code 0means success under deterministic logic. In LLM agents,exit code 0just means execution finished. If an agent writesdef add(a, b): return a + b + 1along with tests that share its own blind spots, every cryptographic and execution check passes. You end up with a cryptographically perfect hallucination.Instead of trying to replace existing supply-chain standards (Sigstore, SLSA, in-toto), OWP’s real power is acting as a Semantic Safety Overlay on top of them — validating not just that the code executed securely, but whether the semantic intent actually holds up.
Happy to open a GitHub Discussion to flesh out Layer 0 specs if you want to explore this direction.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.