DEV Community

Cover image for 5 Questions to Ask Any Agent Verification Vendor
rambo
rambo

Posted on

5 Questions to Ask Any Agent Verification Vendor

5 questions to ask any agent verification vendor

Every vendor now sells "verifiable agents." The phrase shows up in press releases, on pricing pages, in sales decks. It sounds rigorous. Most of the time it means whatever the vendor wants it to mean.

That is a problem for buyers, because verification is one of those words with a precise technical meaning. Either a third party can check the record without trusting anyone involved, or they cannot. There is no middle ground, and marketing copy lives in the middle ground.

Here are five questions that cut through it. Ask them in order. A vendor with real verification will answer all five crisply. A vendor with a dashboard will start hedging around question two.

1. Can a third party verify the record with zero trust in you?

This is the whole game. Hand the execution record to a program written by someone who has never heard of the vendor. That program has the record, the claimed result bytes, and a hash function. Nothing else. No network access to the vendor's systems. No API keys. No login.

Can that program answer one question: was this record changed since it was created?

A good answer sounds like this: "Yes. Verification is pure recomputation. Hash the exact input and output bytes, compare to the receipt, walk the hash chain. Anyone can write a verifier from the spec in an afternoon."

A bad answer sounds like this: "Use our verification dashboard." Or: "Call our API to check." Or: "Our verifier app confirms it." Every one of those smuggles trust back in through a side door. If verification needs the vendor's infrastructure, keys, or software, you are not verifying. You are asking the vendor to vouch for itself.

2. What exactly does verification prove?

Precision here separates the serious from the salesy. Ask the vendor to state the claim in one sentence, with no adjectives.

A good answer: "A verified receipt proves the saved result was not changed since it was saved." That is the entire claim. It is narrow on purpose. Narrow claims are checkable. Checkable claims are the only ones that survive an audit.

A bad answer is any version of "it proves the agent did the right thing." Verification of the record is not verification of correctness. A receipt cannot tell you the tool was called with sensible inputs, that the answer was right, or that anything happened in the real world. If the agent called a payments API, the receipt proves the call record is intact. It does not prove money moved. That requires checking the bank.

Watch for scope creep in the answer. "Proves execution integrity" sounds strong and means nothing. "Proves the output is trustworthy" is a feeling, not a check. Vendors who overclaim here are telling you they do not understand their own boundary, which means you cannot trust them to hold it.

3. Is the format open or proprietary?

A receipt format only works as infrastructure if anyone can implement it. Ask: "Can an engineer I have never met write a verifier from your documentation alone, with no access to your systems?"

A good answer points to a published, versioned specification developed in the open, with public review and a real revision history. The gold standard is a standards-track document, like an IETF Internet-Draft, where strangers with no stake in the vendor's success read the text and file issues. Fifteen rounds of that kind of review will find gaps the author never saw.

A bad answer is "our proprietary format," or documentation that lives behind a login, or a spec that says "contact sales for the verification API." Proprietary receipts are lock-in with cryptographic garnish. The auditor needs accounts on twelve platforms. The evidence dies when a startup does. And every vendor with a proprietary format has a quiet incentive to keep verification just hard enough that you never leave.

Open formats also let you compare vendors on equal terms. When every receipt follows the same spec, you can benchmark who mints the best receipts instead of who has the glossiest dashboard.

4. Does verification survive you disappearing?

Ask what happens to five years of receipts if the vendor shuts down next Tuesday.

A good answer: "Nothing. Verification is hash recomputation against the receipt and the saved bytes. It needs no network, no vendor API, no license server. Your evidence is a file and a hash function. Both will still exist."

A bad answer involves any live dependency: an API endpoint, a certificate service, a hosted verifier, a chain the vendor operates. Evidence with an expiration date is not evidence. It is a subscription.

This question also catches the anchored variants. An anchored hash proves the hash existed at a time. It says nothing about whether the hashed content was true, and if the anchoring scheme depends on the vendor's relayer or indexer, you are back to trusting the vendor. Ask what you hold in your hands when they are gone. If the answer is "our API," walk away.

5. Who captures the record: the agent or the execution layer?

This is the question vendors hope you never ask, because it exposes the most common failure mode in the space.

A good answer: "The execution layer captures the record at the moment the tool runs, before the result returns to the agent. The agent never touches the receipt. It cannot forge one, edit one, or suppress one without breaking the hash chain."

A bad answer is anything where the agent reports on itself: logs written by the application, transcripts, "the agent confirms completion," summaries generated after the fact. The agent is the least trustworthy witness to its own behavior. It has every incentive to report success. A verification story built on self-reporting is a story, not a check.

Press on the timing. "Captured at execution" means the record is written by the layer that ran the tool, with the exact input and output bytes, at the moment of execution. "Logged after the run" means someone reconstructed it later. Reconstruction is where fabrication lives.

The short version

If you only remember one thing: verification means a stranger with no trust in anyone can recompute the math and confirm the record is intact. Everything else is marketing.

There is an open Internet-Draft that defines exactly this: AER-1, the verifiable execution receipt format, currently draft-zambo-aer1-15 on the IETF datatracker (https://datatracker.ietf.org/doc/draft-zambo-aer1/15/). It specifies what gets captured, how inputs and outputs get hashed, how receipts chain, and how a third party verifies with nothing but the record and a hash function. It is a draft, not a finalized standard, and it is implemented in multiple languages.

Zambo is the reference implementation. It works with any AI, mints a verifiable receipt for every tool call, and the receipts verify independently of Zambo itself.

Do not take anyone's word for any of this, including mine. Try it: https://zambo.dev

Then verify a receipt yourself: https://zambo.dev/verify/

Your AI said "done." For $0.99, prove it: https://zambo.dev/day-pass/

Top comments (0)