DEV Community: Ventse

How to verify what an AI agent actually did

Ventse — Tue, 21 Jul 2026 08:46:24 +0000

Your agent says it finished the job. Its log file agrees. Neither of those is evidence: the log was written by the same process it describes, and it can be rewritten afterwards by anyone with disk access. If you need to show a customer, an auditor or a court what an agent did, you need a record that fails loudly when altered and that a stranger can check without trusting you. This is a practical walkthrough of doing exactly that with an open verifier and a signed execution receipt.

Step 1: install the verifier

The verifier is a small open-source Python package with a published receipt specification:

$ pip install traceseal-verify
$ traceseal-verify receipt.json
[OK] receipt.json — operator signature verified

The check runs offline. It needs no account, no API call and no access to the operator's infrastructure — which is the point. Verification that requires the operator's cooperation is not independent verification.

Step 2: read the receipt — the anatomy

A receipt is a single JSON document with three blocks. Here is a real one, abridged:

{
  "attestation": {
    "attested_at": "2026-04-15T05:29:04Z",
    "operator_fingerprint": "ed25519:d8d13a6f...",
    "operator_public_key": "3ba02728...",
    "signature": "4678a52c..."
  },
  "execution": {
    "skill_name": "yoast-seo-audit",
    "skill_version": "0.1.0",
    "inputs_hash":  "sha256:6ac78ba8...",
    "outputs_hash": "sha256:66858bb9...",
    "sandbox_profile_hash": "sha256:e7a3e6b8...",
    "skill_manifest_hash":  "sha256:79c14974...",
    "exit_code": 0,
    "ok": "true",
    "wall_time_ms": 47
  },
  "provenance": {
    "manifest_hash": "sha256:79c14974...",
    "publisher_fingerprint": "ed25519:d8d13a6f...",
    "transparency_log_seq": 5
  },
  "receipt_version": "1.0"
}

Each block answers a different question:

provenance — which code ran. The publisher signs a content-addressed manifest: every artefact (the skill definition, its capability declaration, the code itself) is listed by its SHA-256 hash (FIPS 180-4). Change one byte of the code and the manifest hash no longer matches.
execution — what happened. Inputs, outputs and the sandbox policy are recorded as hashes of their canonical JSON form. That proves integrity without exposing the data — you can demonstrate the output is unchanged without publishing the output.
attestation — who vouches for it. The operator signs the whole record with an Ed25519 key (RFC 8032). The signature covers a canonical JSON encoding — sorted keys, fixed separators — so there is exactly one valid byte sequence for a given receipt. Any edit anywhere breaks the seal.

Step 3: understand what the check proves

When traceseal-verify prints [OK], four things have been established:

The receipt is byte-for-byte what the operator signed — nothing was edited afterwards, by the agent, the operator's tooling, or anyone who handled the file since.
The code that ran is exactly the code the publisher signed, down to the hash of each file.
The recorded inputs, outputs and sandbox policy are the ones present at execution time — anything you are later shown can be checked against the hashes.
The signing key is identified by fingerprint, so repeated receipts from the same operator are linkable, and the receipt can be anchored in a public transparency log (the transparency_log_seq field), which prevents the operator quietly maintaining two versions of history.

What a receipt does not prove

Honest tools state their limits, and this matters if you ever rely on a receipt in a dispute:

It does not prove the work was good. A receipt proves what ran and what it produced — not that the output was correct. Checking outcomes against ground truth is a separate, complementary verification step.
It does not prove events it never covered. A receipt seals one execution. Actions taken outside sealed executions are simply absent — which is why instrumentation has to start before the incident, not after.
It binds a key, not a person. The signature proves the holder of the operator key vouched for the record. Tying that key to a legal identity is a key-management question, the same as with any signing scheme.

The trust inversion: with logs, the burden is on the reader to trust the writer. With signed receipts, the burden is on the record to survive verification. That is the difference between "our system says it behaved" and "check it yourself".

Why this beats screenshots and log exports

The evidence teams typically produce today — screenshots, log exports, dashboard PDFs — shares one flaw: it is all produced by, or under the control of, the party whose behaviour is in question. Computer-security guidance on log management has warned for years that logs require protection precisely because they are alterable by whoever controls the system (NIST SP 800-92). A detached signature over a canonical record removes the alterability, and an open verifier removes the need to take anyone's word for it.

Everything shown here is open: the receipt spec, the verifier on PyPI, and the transparency log. You can implement the format yourself; the spec even includes a reference canonical-JSON encoder.

Originally published on traceseal.io.

EU AI Act Article 50: what it means for teams running AI agents

Ventse — Sun, 19 Jul 2026 10:14:59 +0000

Cross-posted from traceseal.io.

On 2 August 2026, the transparency obligations in Article 50 of the EU AI Act (Regulation (EU) 2024/1689) become applicable. If your organisation deploys AI systems that interact with people or produce content — and AI agents do both — this deadline matters wherever you are based: the Act applies wherever a system's output is used in the Union.

What Article 50 actually requires

The full text of Article 50 sets out four transparency duties:

Disclosure of AI interaction. People must know they are interacting with an AI system, unless obvious from context.
Marking of synthetic content. Generated text, audio, image or video must be marked as artificially generated, machine-readably where feasible.
Deployer disclosures. Emotion recognition, biometric categorisation and deepfake-style content require informing the people affected.
Published text disclosures. AI-generated text published on matters of public interest must be disclosed, unless a human editor takes responsibility.

Non-compliance carries penalties up to €15 million or 3% of worldwide turnover (Article 99(4)).

Why agents make this harder than chatbots

For a chat window, disclosure is a banner. For an agent — a system that plans, calls tools, edits files, sends messages and commits code — the question is not "did we tell the user it's AI?" but "can we show what the system actually did?"

Application logs are written by the same system they describe, can be edited after the fact, and prove nothing to a third party. When an agent's action is challenged, "our logs say it behaved" is an assertion, not evidence.

From documentation to verification

The robust way to demonstrate what an AI system did is to make the record tamper-evident and independently verifiable:

Sign the code. The publisher signs the agent skill with an ed25519 key over a content-addressed manifest — provable which code ran.
Seal the execution. The operator runs it in a kernel-namespace sandbox and signs a record of inputs, outputs, timing and sandbox policy — as SHA-256 hashes, proving integrity without exposing data.
Let anyone verify — one command, offline, no access to the operator's systems:

$ pip install traceseal-verify
$ traceseal-verify receipt.json
[OK] receipt.json — operator signature verified

Because the receipt is canonical JSON under a signature, any tampering breaks the seal. That converts an Article 50 conversation from a documentation exercise into a one-command check.

The practical test: if a regulator asked you tomorrow to prove what your agent did last Tuesday, would you hand them a log file they must trust — or a receipt they can verify?

What to do before 2 August 2026

Inventory your agents. Anything interacting with people or generating content is in scope for at least one duty.
Decide your evidence standard. Logs are trust-me evidence; signed receipts are check-it-yourself evidence.
Instrument now, not after an incident. Receipts only cover executions sealed at the time.

The receipt format, verifier and transparency log are open: the spec, the verifier on PyPI, and the public log. You can adopt the format without adopting us.

Proving What Your AI Agent Did: Introducing Traceseal

Ventse — Thu, 16 Apr 2026 14:01:09 +0000

Every AI agent framework can make agents do things. None of them
can prove what the agent did.

When a client asks "was your AI agent running authorized code when
it published that article?" the answer today is "trust me." When a
regulator asks "what capabilities did the automated system have?"
the answer is "trust me." When something goes wrong and someone
asks "what exactly happened?" the answer is still "trust me."

We built Traceseal to replace "trust me" with math.

What Traceseal does

Traceseal produces an Execution Receipt — a signed JSON document
that proves:

What code ran (manifest hash over every source file)
Who authorized it (publisher's ed25519 signature)
What sandbox it ran in (hash of the kernel-namespace configuration)
What it produced (SHA-256 of inputs and outputs)
Who vouches for it (operator's ed25519 signature over all of the above)

A third party verifies the receipt with one command:

pip install traceseal-verify
traceseal-verify receipt.json

[OK] receipt.json
  skill:     agentmail v1.2.0
  operator:  ed25519:f19bc125...
  publisher: ed25519:a07c7eb5...

No access to the operator's machine. No trust assumptions. Just math.

What we built

8 weeks. 217 tests. 27-entry audit chain on real Debian 13 hardware.
Three signed production skills with verified receipts:

A multi-entry-point skill bundle — signed, sandboxed with
kernel-namespace isolation, receipt verified by a third party. 116ms.
A real production skill with SDK dependencies — the agentmail
SDK was mounted read-only into the sandbox. The execution failed
(bad API key) and the receipt honestly records the failure.
The system doesn't hide bad outcomes — it seals them.
A web scraping skill — signed, sandboxed, 47ms, receipt verified.

The full pipeline — sign, sandbox, execute, receipt, verify — runs
in under 200ms.

Why this matters

AI agents are moving from demos to production. When they get there,
someone will ask for proof. Proof that the code was authorized.
Proof that the sandbox was real. Proof that the outputs weren't
tampered with. Proof that a third party can check independently.

Every other trust system in the AI agent space offers dashboards,
not proofs. They log what happened. We seal it. The difference:
a log says "the operator claims this happened." A receipt says
"anyone can independently verify this happened."

Open spec, open verifier

The Execution Receipt Specification is Apache 2.0:
→ RECEIPT-SPEC.md

The standalone verifier is Apache 2.0:
→ pip install traceseal-verify
→ GitHub

The transparency log is live:
→ log.traceseal.io

Anyone can verify receipts. Anyone can implement the spec.
The hard part — signing, sandboxing, audit logging, the full stack
that generates receipts worth verifying — is Traceseal.

What's next

Traceseal currently produces receipts for skill execution. We're
extending the receipt format to cover:

Model call receipts — prove which model processed your data
Orchestration receipts — prove a multi-step workflow executed in order with declared inputs at each step
Data flow receipts — prove what data was sent where

Same format. Same verifier. Same transparency log.

The goal: a cryptographic audit trail for every action your AI agent
takes, verifiable by anyone, without trusting the operator's machine.

→ traceseal.io