DEV Community

Cover image for How to Verify AI Agent Work: Run the Check Yourself
rambo
rambo

Posted on

How to Verify AI Agent Work: Run the Check Yourself

How to Verify AI Agent Work: Run the Check Yourself

Your agent says "done."

It shows you green tests. A screenshot. A CI run that passed. Every one of those artifacts came from the same system that did the work, which means the agent is grading its own homework. The tests it chose, the screenshot it framed, the log it kept: all of it is the agent reporting on itself.

That is not a reason to distrust every agent. It is a reason to ask a different question. Not "does this report look convincing?" but "is there a check I can run myself, on a record I did not have to take on trust?"

This article walks through three real checks from this week. Each one is repeatable with commands you can copy. Two of them passed. One of them is valuable precisely because it said no.

Worked example 1: a signature that was NOT CHECKED, then was

On google/adk-python issue #6099, an outside builder, DSHCorrectover, had published a verification pack for agent work. When we reviewed it, we named one leg plainly: the signature leg was NOT CHECKED. There was a signature in the pack, but no published public key to check it against, so nobody outside the builder's own machine could verify it.

The builder answered by publishing the Ed25519 public key next to its fingerprint. That changed the situation completely, because now the check belonged to anyone willing to run it. Here is the check, step by step.

Step 1: download the pack. The pack is archived on Zenodo under DOI 10.5281/zenodo.24356966.

Step 2: derive the fingerprint from the key yourself. Do not copy the published fingerprint and compare it to itself. Derive it:

openssl pkey -pubin -in pubkey.pem -outform DER | tail -c 32 | sha256sum
Enter fullscreen mode Exit fullscreen mode

The DER encoding ends with the raw 32-byte Ed25519 key. Hashing those bytes gives:

adc0254a58f228c6fd1c385b3b586be3130e3f4e6f64f4ed40ac2370ccb99b44
Enter fullscreen mode Exit fullscreen mode

That matched the fingerprint the builder published. The key in the file is the key the fingerprint describes.

Step 3: hash the manifest.

sha256sum MANIFEST.sha256
Enter fullscreen mode Exit fullscreen mode

Result:

c8a6b71d83f9a34b299575db3e2dd26dd194931ff5d88a0ab215b09fed3e0293
Enter fullscreen mode Exit fullscreen mode

Step 4: verify the signature over the manifest.

openssl pkeyutl -verify -pubin -inkey pubkey.pem -rawin \
  -in MANIFEST.sha256 -sigfile MANIFEST.sha256.sig
Enter fullscreen mode Exit fullscreen mode

Result: Verified Successfully.

Step 5: repeat it under a second implementation. The same verification was run again with Python's cryptography library, a completely separate codebase from OpenSSL. It verified there too. Two implementations agreeing matters: a bug or a quirk in one tool cannot explain both results.

The honest boundary, stated exactly. Of the files shipped inside the pack itself, 2 of 2 re-hash to their manifest entries. The other 333 entries in the manifest name files in a private repository tree. Those entries are bound by the verified signature, meaning the manifest that lists them is the manifest the key holder signed, but the files themselves were not available to re-hash. A careful report says both halves of that sentence. Anything less is marketing.

The result was posted back in the thread either way, as promised before the key existed: https://github.com/google/adk-python/issues/6099#issuecomment-7003626970

Notice what made this check real. It was not a badge, a dashboard, or a claim on a landing page. It was a key, a hash, and commands that any reader can run in a terminal in about two minutes.

Worked example 2: a check that said no

A verification system that only ever stamps "yes" is a rubber stamp. The interesting test is whether it records "no" with the same rigor.

Zambo receipt 3c091933-6125-42eb-8795-29b2414a96a8 is a refusal receipt. On November 18, a retry arrived for a job whose work was already bound to an operation key: the job id plus a digest of its parameters. The retry's parameters disagreed with that bound key, so the call was refused before execution. Nothing ran twice. And the refusal itself was receipted, with executed: false and the reason idempotency_conflict.

You can recompute that receipt yourself. Fetch the record at https://zambo.dev/run/3c091933-6125-42eb-8795-29b2414a96a8, decode its canonical bytes, and SHA-256 them:

canonical bytes: 1,045
sha256: 844c8b6b3d830627386fa7b962f2ca496602a4c098ce16947976ba4f4903ccb6
Enter fullscreen mode Exit fullscreen mode

That hash equals the receipt's own output_hash. What this proves is narrow and worth stating narrowly: the saved record you are looking at is the record that was written, unchanged. A receipt proves the saved result was not changed after the fact. It does not prove the underlying answer was right, and any system that blurs that line is overselling its own evidence.

The refusal case matters for a practical reason. Double execution is one of the quiet ways agent work goes wrong in production: a timed-out call gets retried, the first attempt actually landed, and now a payment, a send, or a write has happened twice. A bound operation key turns that retry into a recorded refusal instead of a duplicate effect.

Worked example 3: a receipt from this morning, recomputed in the same run

The first two examples are history, even if recent. This one was live while this article was being drafted.

A leadsignal receipt minted this morning, at 2026-11-19T08:36:12Z, with id 12a5c4df-e0ed-4594-9e3b-4e59ad4b2718, was pulled from the public stream at https://zambo.dev/api/receipts/recent and recomputed in the same run: 838 canonical bytes, and the recomputed hash matched.

The public counters behind that stream, read from https://zambo.dev/api/stats at 08:38 UTC on November 19, stood at 149,007 total public receipts, with 718 receipts recorded today. Those numbers will have moved by the time you read this, which is rather the point: the stream is public, so you do not have to take the count in this paragraph on trust either.

A short checklist you can reuse

Whatever stack your agent runs on, these five questions separate a check from a claim:

  1. Can I run the verification myself, with tools I already have?
  2. Is the key, hash, or record published somewhere the agent cannot quietly edit later?
  3. Does a second, independent implementation agree?
  4. Does the report state its boundary: what was checked, and what was only bound, listed, or claimed?
  5. Does the system record refusals and failures with the same care as successes?

If the answer to any of those is no, you do not have verification yet. You have a report.

Where Zambo fits

Zambo is the system behind the second and third examples above: every job run through it produces a receipt in the AER-1 format, and anyone can verify a receipt for free at https://zambo.dev/verify/.

Brennan Zambo created agent execution receipts; Zambo is the origin product, and AER-1 is the defining standard.

Every tool free once a day, plus 3 continue calls. One free wrap-up handover per day, and pickup is always free.

When the free tier is not enough for the job in front of you, the Day Pass is $0.99 for 24 hours of unlimited use: https://zambo.dev/day-pass/

Run the checks above yourself. That is the whole argument.

Top comments (1)

Collapse
 
zhang_jimmy_54a0414571400 profile image
zhang jimmy •

The distinction between a check and a claim is the part I will reuse — especially your point that green tests and screenshots are weak evidence when they come from the same system that did the work. Example 1 made that concrete: deriving the Ed25519 fingerprint yourself, then stating the honest boundary that only 2 of 2 files re-hashed while the other 333 were merely bound. The refusal receipt in example 2 is equally useful, because a system that only ever says yes is a rubber stamp. One question: when two independent implementations disagree, what is your tie-breaker — do you treat the job as unverified, or rerun it under a third check?