There is a quiet consensus forming about how to make an AI agent's output trustworthy: make it reproducible. Pin the inputs. Hash the pipeline. Anchor the hash somewhere tamper-evident. Then anyone can re-run the exact steps on the exact bytes and land on the exact same answer. If the numbers match, the result stands.
This is real progress, and I am not trying to talk anyone out of it. Reproducibility is the whole distance between "trust me" and "here, run it yourself." But it answers a narrower question than the word "verified" tends to imply, and the gap between the two is exactly where a careful adversary sets up shop.
Reproducibility proves one thing: the recipe was followed on the inputs you were handed. It says nothing about whether those inputs are a faithful capture of the world. Those are two different claims. Most agent pipelines quietly fold them into one and ship the confidence of both.
The same answer twice is not the same as the right answer
Take a concrete case. An agent screens a company against sanctions lists and reports "no match." To make that checkable, it pins the exact list files it screened, hashes them, commits the hashes alongside the result, and publishes everything so anyone can re-run the match and watch "no match" fall out deterministically. A second party does exactly that and gets "no match" too.
What did they just establish? That the matching logic, applied to those specific bytes, yields that specific answer. They established consistency. They did not establish that those bytes were the real sanctions list on the day it mattered. If the agent screened against a list with three names quietly removed, the re-run reproduces the clean "no match" perfectly, forever, byte for byte. The reproduction is not evidence of truth. It is evidence that everyone is looking at the same doctored page.
This is the part that gets skipped. Pinning does not move you from unverified to verified. It moves the question from "did they actually run it" to "was the thing they ran it on genuine." That second question is the hard one, and hashing the inputs does not touch it.
Why this lands harder on agents than on people
A human analyst pulling a sanctions list has a hundred incidental tells that the source was real: they went to the regulator's site, the TLS cert was the regulator's, the file looked like every prior week's file. None of that is rigorous, but it is friction, and friction is doing quiet authentication work.
An agent has none of that unless you build it in. It fetches, it captures, it pins, it proceeds. When it then signs the whole bundle and presents it as verifiable, the signature is authenticating the agent's own account of what it saw. You are asking the system to be the witness to its own world. A witness that grades its own testimony is not a witness. It is a narrator.
And the pinning makes this worse in one specific way: it launders a capture into an artifact. Before pinning, "I screened against the OFAC list" is obviously a claim. After pinning, "I screened against these bytes, here is their hash, re-run it" feels like proof. The hash is real and the re-run is real, so the whole thing borrows the credibility of cryptography for a step cryptography never covered: the moment the bytes were captured.
Walk the escalation and watch where it stops
Start naive: pin the local sample. Good, now the inputs you controlled are frozen. But the pipeline also reaches for external data, and external data drifts. Reference a source by name and the re-run diverges the moment the source rotates, and you cannot tell a tampered result from a stale fetch. So pin the external data too: snapshot it, hash the snapshot, commit that hash. Now the whole run is deterministic and replayable.
Here is where the escalation quietly runs out of road. You have made the run reproducible. You have not made the snapshot genuine. The snapshot's authenticity still rests entirely on the word of whoever captured it, and that is the one party with a motive to shade it. Every layer of pinning you added tightened reproducibility and left authenticity exactly where it started: on trust.
The two things that actually close it
There are only two honest ways I know to close the authenticity gap, and it is worth being blunt that one of them is often not available yet.
The first is source attestation. The source signs its own data at the point of production. The regulator signs the list it served that day. The exchange signs the rates it published. The snapshot inherits that signature, and "was this genuine" reduces to "does the source's signature verify," which anyone can check without trusting whoever ran the pipeline. This is the real fix, and it is clean, because it puts the signature on the party that actually witnessed the fact. The problem is that most sources do not sign anything yet. You cannot unilaterally conjure an attestation that the other end refuses to produce.
So the second path is a fallback: quorum. If no single capture can be trusted, take several independent captures and require them to agree. Different vantage points, different network paths, ideally different code. Agreement across genuinely independent captures bounds the forgery surface, because now an attacker has to corrupt all of them in the same way at the same time instead of just yours. It does not close the gap. A determined adversary who controls the source still wins. But it converts a silent single point of failure into a loud, coordinated one, which is a real improvement.
The non-negotiable part is the label. A result backed by a source signature and a result backed by three captures agreeing are not the same guarantee, and the artifact has to say which one it is. Attested-by-source and attested-by-agreement are different words on purpose. The failure I keep seeing is not that people pick the weak guarantee. It is that they ship the weak guarantee wearing the strong guarantee's clothes, because both of them re-run green.
The line worth keeping
Reproducibility is a property of computation. Authenticity is a property of provenance. They feel like the same virtue because both of them let a stranger check your work, but they check different things, and one of them is usually the one you actually care about.
When an agent hands you a result stamped "independently reproducible," the useful reflex is to ask what it is independent of. Independent re-execution is not independent capture. The first is arithmetic: run the numbers again, get the numbers again. The second is testimony: someone stood where the fact happened and reported it. An agent that pins its inputs has given you rerunnable arithmetic. Whether it has given you testimony depends entirely on who signed the world it fed itself, and most of the time, right now, the answer is nobody, and the run is green anyway.
Build the reproducibility. It is table stakes and it is genuinely good. Just stop letting it answer a question it was never asked. The pinned hash tells you the recipe was honest. It does not tell you the ingredients were real, and for anything that matters, that is the claim you were actually trying to make.
Top comments (0)