DEV Community: Ventse

in-toto and SLSA for AI agents: what build provenance teaches us about agent records

Ventse — Tue, 20 Oct 2026 08:46:18 +0000

Software supply-chain security has spent the last few years answering a question that AI agents now raise again: given an artifact, how does someone who was not in the room find out how it was produced, and how far should they believe the answer? For builds, the answer the industry has converged on is an attestation, usually an in-toto attestation carrying SLSA provenance. This piece reads those two specifications as a design guide for recording what an agent did, and is explicit about where the analogy holds and where it does not.

What an in-toto attestation is

The in-toto attestation framework defines an attestation as "authenticated metadata about one or more software artifacts". The current specification (v1.2) splits it into layers, each with one job:

Predicate: "arbitrary metadata about a subject artifact, with a type-specific schema". Provenance, a test result or a vulnerability scan are all predicates.
Statement: "binds the attestation to a particular subject and unambiguously identifies the types of the predicate".
Envelope: "handles authentication and serialization", in other words the signature.
Bundle: a way of grouping several attestations together.

A v1 Statement has four fields: _type, always https://in-toto.io/Statement/v1; subject, a list of artifacts, each of which "MUST have digest set"; predicateType, a URI naming the schema of the predicate; and predicate itself. Two rules in that document matter for anyone thinking about agents. Subjects "are assumed to be immutable", and they "are matched purely by digest, regardless of content type". The statement is about specific bytes, not about a file name or a version label.

The envelope: DSSE and why it refuses canonical JSON

in-toto attestations are normally signed with DSSE, the Dead Simple Signing Envelope. The protocol signs a pre-authentication encoding of the payload rather than the payload alone:

PAE(type, body) = "DSSEv1" + SP + LEN(type) + SP + type + SP + LEN(body) + SP + body
SIGNATURE = Sign(PAE(UTF8(PAYLOAD_TYPE), SERIALIZED_BODY))

Two design choices are worth understanding. First, the payload type is inside the signature. The design notes explain that without an authenticated type, a signer and a verifier may interpret the same bytes differently, and give the example of a message produced as CBOR being read as protobuf. Second, DSSE deliberately avoids canonicalisation. The same document says a signing scheme "SHOULD avoid depending on canonicalization for security", warns that "two semantically different payloads could have the same canonical encoding", and prefers to "transmit the encoded byte stream exactly as it was signed" so that verification happens before any parsing.

That second point is a direct critique of the approach our own receipts take. The Traceseal receipt specification signs the canonical JSON encoding of the execution and provenance objects, which means a verifier has to parse the receipt and re-encode it before checking the signature. We mitigate this by keeping every field a string (the spec encodes ok as "true" rather than a JSON boolean for that reason) and by publishing a reference encoder, but the DSSE authors' objection is a fair one and it applies to us. If you are designing an agent record format from scratch today, signing the exact bytes with a typed envelope is the safer default.

What SLSA provenance records

SLSA describes provenance as "the verifiable information about software artifacts describing where, when and how something was produced", and models it as "an attestation that a particular build platform produced a set of software artifacts". It is an in-toto predicate with type https://slsa.dev/provenance/v1, and it has two halves:

buildDefinition, the input to the build: a buildType, the externalParameters "under external control, such as those set by a user or tenant", the internalParameters under the platform's control, and the resolvedDependencies pulled in at build time.
runDetails, this particular execution: a builder.id, metadata including invocationId, startedOn and finishedOn, and any byproducts.

Map that onto an agent run and most of it carries over with little strain. The task prompt and the user's instructions are external parameters. The model identifier, system prompt and tool configuration chosen by the operator are internal parameters. Retrieved documents, tool definitions and the code of any skill the agent loaded are resolved dependencies. The harness that ran the agent is the builder. The files it wrote, the messages it sent and the commits it made are subjects, identified by digest.

The useful idea: who is trusted for which field

The most transferable part of SLSA is not the schema but the way it assigns trust to each field. The provenance specification says external parameters "are untrusted; they MUST be included in the provenance and MUST be verified downstream", while internal parameters "are trusted because the platform is trusted". It also states that "the accuracy and completeness" of the build definition "are implied by runDetails.builder.id", and that consumers "MUST accept only specific signer-builder pairs".

Put plainly: a provenance document is only as good as the thing that wrote it, and the format makes that dependency explicit rather than hiding it. Agent logs rarely do this. A transcript mixes what the user asked for, what the model decided, what a tool returned and what the harness observed, all written by the same process and all presented with the same apparent authority. Separating "values supplied from outside, to be checked" from "values recorded by the platform, trusted if the platform is" is a small change in a record format, and it tells a reader exactly what they are being asked to believe.

Build levels, read as agent levels

The SLSA v1.0 build track grades how hard the provenance is to fake:

Build L1: provenance exists. It "can be used to prevent mistakes but is trivial to bypass or forge", and it may be "incomplete and/or unsigned".
Build L2: the build runs on a hosted platform, "not an individual's workstation", and the provenance is signed by it. Forging it "requires an explicit 'attack', though this may be easy to perform".
Build L3: hardened builds. The platform must "prevent secret material used to sign the provenance from being accessible to the user-defined build steps", and runs must not be able to influence one another.

The L3 requirement is the one agent systems most often get wrong. In a build, the user-defined steps are the untrusted part. In an agent run, the untrusted part is the agent itself: its behaviour depends on a model and on whatever content it reads, and indirect prompt injection means that content can steer it. If the agent process can reach the key that signs its own record, the record is at L1 in SLSA's terms no matter how good the cryptography is, because the thing being described can write the description. The OWASP agentic guidance reaches the same conclusion from the other direction.

Our own tooling follows this rule. Alibi runs each agent inside a bubblewrap sandbox and signs receipts on the host after the contained processes have exited; the key is never mounted inside the sandbox. Our sandboxing article covers the profile, including a DNS leak we found when we tested it.

A quick test for any agent record: could the agent, if it misbehaved, have produced a record that verifies but says something false? If the answer is yes, the signature is protecting the record from everyone except the party you most need it to be protected from.

Where the analogy breaks

Build provenance can rely on three properties that agent runs do not have, and it is worth being exact about them.

Builds aim to be repeatable; agent runs are not. A verifier who distrusts build provenance can, in principle, rebuild from the declared inputs and compare digests. An agent run against a hosted model, with live tool results, cannot be replayed to the same outcome. The record is the only evidence there will ever be, which raises the stakes on getting the signer right.
Build inputs are artifacts; agent inputs are often conversations. A resolved dependency has a digest. A web page the agent read at 14:02, or the response of an API that has since changed, can be hashed, but the hash only helps if someone kept the bytes.
The subject of a build is the output; the subject of an agent run is often the action. An email sent, a payment initiated or a ticket closed is not an immutable artifact with a digest in the sense the in-toto Statement assumes. You can hash a description of the action, but then you are attesting to the description, and the question becomes who wrote it.

None of this makes the build world's tools wrong for agents. It means the trust model has to be carried over, not just the field names. And SLSA provenance, like our receipts, is an attestation and not a proof. Our specification says it directly: a receipt "proves that the operator's key signed the attestation. It does NOT prove the operator is honest." The same is true of any signed provenance. What the format buys you is that the claim is attributable, tamper-evident and checkable by a third party, which a log file is not.

What to take from this

Identify subjects by digest. Record the hash of every file an agent wrote and every payload it sent, not only a path or a name.
Split external from internal parameters. Mark what came from the user or from content the agent read separately from what the operator configured.
Keep the signing key out of the agent's reach. This is the single requirement that decides whether a record is evidence or self-description.
Sign exact bytes and the payload type if you are designing a format now. DSSE's reasons are sound.
Get an outside party involved in time and ordering. A signature does not fix when something happened; see our piece on trusted timestamps and on transparency log witnesses.

The receipt format, verifier and log are open: the spec, the verifier on PyPI, and the public log.

Originally published at traceseal.io.

EU AI Act Article 14: human oversight for AI agents, and what an oversight record should prove

Ventse — Tue, 13 Oct 2026 08:48:31 +0000

"Human in the loop" is the usual answer to the question of who is in charge of an AI agent. Article 14 of the EU AI Act (Regulation (EU) 2024/1689) turns that answer into a legal requirement for high-risk systems, and it is more specific than the slogan. This piece covers what the article says, who carries the duty, when it applies now that the deadlines have moved, and the question most teams skip: if someone later asks whether a person really oversaw a particular agent action, what record could you show them?

When it applies: later than first planned

Article 14 sits in Chapter III, Section 2, the requirements for high-risk AI systems. Following the 2026 Digital Omnibus amendments, the consolidated text of Article 113 now applies those requirements from 2 December 2027 for systems that are high-risk under Article 6(2) and Annex III, and from 2 August 2028 for systems that are high-risk under Article 6(1) and Annex I. The original date for most of them was 2 August 2026.

The delay changes the timetable, not the obligation. It also does not move the transparency duties covered in our Article 50 piece.

Whether an agent is high-risk depends on what it is used for, not on the fact that it is an agent. A coding agent that edits your repository is unlikely to be in scope. An agent that filters job applications or evaluates candidates is, because recruitment and selection are listed in Annex III, point 4(a). So is one that evaluates the creditworthiness of natural persons, under point 5(b).

What Article 14 actually says

The core requirement is in Article 14(1):

High-risk AI systems shall be designed and developed in such a way, including with appropriate human-machine interface tools, that they can be effectively overseen by natural persons during the period in which they are in use.

Article 14(3) says the oversight measures "shall be commensurate with the risks, level of autonomy and context of use". That is the clause that matters most for agents. A system that recommends is one thing. A system that plans and then acts on its own plan has a higher level of autonomy, and the oversight is expected to scale with it.

Article 14(4) then lists what the people assigned to oversight must be enabled to do, "as appropriate and proportionate":

(a) Understand and monitor. Properly understand the system's capacities and limitations, and monitor its operation, including detecting and addressing anomalies, dysfunctions and unexpected performance.
(b) Resist automation bias. Stay aware of the tendency to rely, or over-rely, on the system's output. The Act names this "automation bias".
(c) Interpret the output. Correctly interpret what the system produced.
(d) Decide not to use it. In any particular situation, decide not to use the system, or "disregard, override or reverse" its output.
(e) Stop it. Intervene in its operation or interrupt it "through a 'stop' button or a similar procedure that allows the system to come to a halt in a safe state."

Article 14(5) adds a stricter rule for remote biometric identification under Annex III point 1(a). No action may be taken on an identification unless it has been "separately verified and confirmed by at least two natural persons". Few agent deployments will fall under it. It is still the one place where the Act spells out a two-person check.

Who carries the duty

The work is split. Under Article 14(3), the provider builds oversight measures into the system, or identifies measures the deployer should implement. The deployer then has to staff them. Article 26(2) is short:

Deployers shall assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support.

Breaching the deployer obligations in Article 26 falls under Article 99(4)(e). Fines there go up to €15 million or 3% of total worldwide annual turnover, whichever is higher.

Why agents strain the model

Article 14(4)(d) speaks of overriding or reversing the system's output. For a model that recommends, the output is a suggestion, and a person can ignore it. For an agent, the output is often an action: an email sent, a record updated, a payment initiated. Some actions can be reversed afterwards. Many cannot.

In our reading, that leaves agent deployments with two practical places to put oversight. The first is a gate before any action that cannot be undone, where a named person sees what the agent proposes and approves or rejects it. The second is a stop mechanism that works while the agent runs, as Article 14(4)(e) describes. Reviewing a dashboard after the run meets neither. It is monitoring, which point (a) asks for as well, but it does not let anyone override an email that has already gone.

Automation bias, point (b), gets worse with agents for a simple reason: volume. A reviewer approving the fortieth tool call of the morning is not overseeing it in any sense a regulator would recognise. That argues for gating fewer, higher-consequence actions rather than every step.

What an oversight record should show

Article 14 states what oversight must make possible. It does not prescribe a record of each oversight decision. Logging duties for high-risk systems sit separately in Articles 12, 19 and 26(6), which we covered in AI agent audit trails. In a dispute, though, "a person approved this" is a factual claim, and someone will ask you to back it up. Our view is that a useful record of an oversight decision answers five questions:

Who decided? A named person, identified by something other than the agent's own credentials.
What did they see? The exact proposal put in front of them, or a hash of it, so a later reader can tell they were not shown something different.
What did they decide? Approve, reject, modify, or stop.
Did the decision come before the action? An approval logged after the email went out is not oversight. That needs a time someone other than the agent's host vouches for. Our piece on trusted timestamps covers what that takes.
Does the action match what was approved? The inputs the agent finally executed with should be the inputs the person approved, not a later variant.

The common failure is a record that cannot answer the first question honestly. If the approval is a row written by the agent platform, signed (if at all) with the platform's key, then it shows the platform says a person approved. That may be true. It is still the system under review describing itself.

Where signed receipts help, and where they stop

Traceseal receipts address question five directly, and they say plainly what they cannot do. A receipt binds the code that ran to hashes of its inputs and outputs, signed with the operator's ed25519 key, and anyone can check it offline:

$ pip install traceseal-verify
$ traceseal-verify receipt.json

If the approver's decision covers the same inputs hash that appears in the receipt, a third party can confirm that what ran is what was approved. Change one byte of the inputs and the hashes no longer match.

The published receipt format has no field for a human approval. A receipt shows who signed the execution record, not that a person looked at it first. The receipt specification is explicit about the limit that matters here:

The receipt proves that the operator's key signed the attestation. It does NOT prove the operator is honest.

An oversight record therefore needs its own signature from a key the approving person controls and the agent cannot reach, made before the action and over the same inputs hash. It is the same separation the OWASP agentic guidance insists on for signing keys, which we covered in our OWASP Top 10 piece. A receipt on its own does not give you that. A receipt beside an approver-signed decision lets a third party check the chain from "this person approved these inputs" to "these inputs are what ran".

The practical test: pick one consequential action your agent took last month. Can you show, without trusting the agent platform's own database, who approved it, what they were shown, and that the approval came first?

What to do before December 2027

Classify by use, not by technology. Map each agent against Annex III. Most will fall outside it. For those that fall inside, Article 14 is a design requirement, and it is easier to build in than to retrofit.
Find the irreversible actions. List the actions each in-scope agent can take that cannot be undone. Those are where pre-action approval belongs.
Test the stop. Article 14(4)(e) asks for a halt "in a safe state". Interrupt a real run part-way through and see what state it leaves behind.
Name the overseers. Article 26(2) asks for people with competence, training and authority. Write down who they are and what authority they actually hold.
Keep approvals separate from the agent. The approver's key should not be one the agent or its host can use. Otherwise the record is the system vouching for itself.

The receipt format, verifier and transparency log are open: the spec, the verifier on PyPI, and the public log. You can adopt the format without adopting us.

Originally published at traceseal.io.

Trusted timestamps for AI agent logs: what RFC 3161 proves, and what it does not

Ventse — Tue, 06 Oct 2026 08:47:56 +0000

Every record an AI agent leaves behind carries a time. A tool call was logged at 14:02:11, a file was written at 14:02:13, a receipt was signed at 14:02:14. When an action is disputed, the time is often the first thing that matters: was the agent's change made before or after the incident, before or after the approval, before or after the policy changed? Most of those times were written by the system under question, using its own clock. This piece looks at what it takes to make a time on an agent record something a third party can rely on, starting with the standard most people search for, RFC 3161.

A signature tells you who, not when

A digital signature binds a key to a sequence of bytes. If the bytes include a time, the signature covers that time, but it says nothing about whether the time was true when the signer wrote it. An operator who wants to backdate a record simply writes an earlier time and signs it. The result verifies perfectly.

Our own receipts have exactly this property, and the receipt specification says so. A receipt carries execution.timestamp and attestation.attested_at, both ISO 8601 UTC strings supplied by the operator. Section 6.3 notes that a verifier with access to the operator's hash-chained audit log can detect replayed or out-of-order receipts, while "a verifier without audit log access cannot detect replays; they can only verify that the receipt is internally consistent." A valid signature on a receipt proves which key signed it and that nothing has changed since. It does not prove the moment of signing to anyone who does not already trust the operator's clock.

Closing that gap needs a party other than the signer to say something about time. There are two well-established ways to get one: a time-stamping authority, or a public append-only log that someone else is watching.

What RFC 3161 actually specifies

RFC 3161, published in August 2001, defines the Time-Stamp Protocol. In its words, a time-stamping authority (TSA) "is a TTP that creates time-stamp tokens in order to indicate that a datum existed at a particular point in time." The client sends a hash, the TSA returns a signed token over that hash and the time. Section 2.1 lists what the TSA is required to do, and several of those requirements matter for agent records:

It sees only a hash. The TSA must "only time-stamp a hash representation of the datum" and must not examine the imprint beyond checking its length. You can timestamp a receipt that contains sensitive inputs without sending the inputs anywhere.
It does not record who asked. Tokens must not include "any identification of the requesting entity". The token says that this hash existed at this time, not who held it.
It must use a trustworthy time source and include a unique serial number in every token, a property the RFC says must survive a crash of the service.
It names a policy. Each token carries an identifier for the security policy it was issued under. The RFC deliberately leaves operational security to that policy rather than to the protocol, so what a token is worth depends on which TSA issued it.

Inside the token, the genTime field holds the time in UTC, and an optional accuracy field gives the margin either side. RFC 5816 updated the protocol in 2010 so that the TSA's certificate can be identified with a hash other than SHA-1.

Producing a request

OpenSSL ships a client and a minimal TSA in openssl ts. Building a request is local and sends nothing over the network. This is the output on our build box with OpenSSL 3.5.7, with the ASCII column trimmed:

$ openssl ts -query -data receipt.json -sha256 -cert -no_nonce -out receipt.tsq
$ openssl ts -query -in receipt.tsq -text
Version: 1
Hash Algorithm: sha256
Message data:
    0000 - 03 07 47 87 d1 95 c2 ff-03 66 c8 a4 5f 44 90 e6
    0010 - 83 86 00 6b 26 dc 60 2a-0e 2a 3d ef a1 06 00 43
Policy OID: unspecified
Nonce: unspecified
Certificate required: yes

The request is posted to a TSA, which returns a response file. Verification takes the original data, the response and the TSA's CA certificate. The manual requires at least one of -CAfile, -CApath or -CAstore:

$ openssl ts -verify -data receipt.json -in receipt.tsr -CAfile tsa-ca.pem

We have not sent a request to a public TSA for this article, so the response side is shown as the documented command rather than captured output. The -no_nonce option we used for the local example removes the nonce; against a live service you would normally keep it, because the nonce is how a client with no reliable clock of its own checks that the response is fresh.

What a token proves, and what it leaves open

A valid token establishes one thing: the hashed data existed no later than genTime, give or take the stated accuracy. That is a useful property, and it is narrower than it first looks.

It is an upper bound only. The token says the receipt existed by 14:02:15. It does not say the agent acted at 14:02:11, or that the receipt was not prepared hours earlier and stamped later. If you also need a lower bound, the record has to contain something that could not have been known before a given moment, such as a value published by an outside party at a known time.
It says nothing about the content. The TSA never sees the receipt. A false record stamped promptly is still false; the token only fixes when it existed.
Ordering is limited by accuracy. Section 2.4.2 states that, unless the TSA sets the ordering flag, two tokens can only be ordered when the gap between their times is greater than the sum of their accuracies. Two agent actions a few hundred milliseconds apart may not be orderable from their tokens alone.
It depends on one key. The security considerations warn that if a TSA's private key is compromised, "any token signed by the TSA using that private key cannot be trusted anymore", and backdated tokens become possible. They suggest tokens from two different TSAs as one defence.
It ages. The same section notes that a TSA key "will have a finite lifetime", so tokens that must outlast it should be time-stamped again later. Agent records kept for regulatory retention periods fall into this category.

Where the law gives timestamps weight

In the EU, the eIDAS Regulation (Regulation (EU) No 910/2014) gives electronic time stamps a defined legal status. In the text as adopted, Article 41(1) says an electronic time stamp "shall not be denied legal effect and admissibility as evidence in legal proceedings solely on the grounds that it is in an electronic form or that it does not meet the requirements of the qualified electronic time stamp." Article 41(2) goes further for qualified stamps, which "shall enjoy the presumption of the accuracy of the date and the time it indicates and the integrity of the data to which the date and time are bound."

A qualified stamp has to meet the requirements in Article 42(1): it binds date and time to the data so as to "reasonably preclude the possibility of the data being changed undetectably", it is "based on an accurate time source linked to Coordinated Universal Time", and it is signed or sealed by a qualified trust service provider. The practical difference is the burden of proof. A self-asserted time can be argued about. A qualified time stamp is presumed correct unless someone shows otherwise.

The other route: a log someone else is watching

A transparency log gets a time bound differently. Records are appended to a Merkle tree, and the log periodically publishes a signed checkpoint: its size and root hash. On its own, the log operator's checkpoint is another self-asserted statement. It becomes evidence about time when an independent witness cosigns it.

The C2SP cosignature format puts a timestamp inside the witness's signature and defines what it means: "as of the specified time, the consistent tree head with the largest size the cosigner has observed for the log identified by the origin line has the specified root hash." Any entry included in that tree therefore existed no later than the cosigned time, on the word of a party that is not the log operator.

Our own log works this way. At the time of writing, the checkpoint at log.traceseal.io/checkpoint is at tree size 24, and the copy held by the Markovian Protocol witness carries a cosignature with the timestamp 1787078705, which is 18 August 2026, 18:45:05 UTC. We checked the log signature and the cosignature against the published keys before publishing this article. So any of the first 24 entries existed by that moment, whatever times their own records claim. Our piece on transparency log witnesses explains the checks a witness performs.

The two routes are not equivalent. A TSA commits to a time-source policy, and a qualified one carries a legal presumption. A witness makes no claim about the quality of its clock beyond what its operator publishes; what it adds is that it refuses to cosign a tree that is inconsistent with one it has already seen, which defends against an operator showing different histories to different people. A TSA token does not give you that.

A workable setup for agent records

For a team that needs agent actions to be datable by someone outside the team, the pieces fit together in a fairly direct way:

Sign each record at the point of action, with a key the agent cannot reach, so the content and the signer are fixed. Our primer on signed receipts covers why the key placement matters.
Get an independent upper bound on time. Either request an RFC 3161 token over the record's hash, or append the hash to a log whose checkpoints an independent witness cosigns. Where records may be used in EU proceedings, consider a qualified TSA.
Add a lower bound if the order of events matters, by including in the record a value that only became public at a known time.
Plan for renewal if retention runs for years, by re-stamping before the TSA key or certificate expires.

Traceseal currently covers the first step with signed receipts and the second through log inclusion and witness cosigning. It does not attach RFC 3161 tokens to receipts, and the receipt's own time fields remain operator-asserted. Anyone who needs a qualified timestamp today can request one over a receipt's hash, since the receipt is a stable file and the TSA never needs to see its content.

The question to ask of any agent log: if you removed the times the system wrote about itself, what would still pin each action to a moment? If the answer is nothing, the log's times are claims, not evidence.

Originally published at traceseal.io. Traceseal issues signed execution receipts for AI agents: an open spec and an open verifier, one command to check.

OWASP Top 10 for Agentic Applications: what it asks you to record, and who should sign it

Ventse — Tue, 29 Sep 2026 08:47:50 +0000

In December 2025 the OWASP GenAI Security Project's Agentic Security Initiative published the Top 10 for Agentic Applications. Most write-ups walk through the ten risks, and this one does too, briefly. It then reads the prevention sections for a narrower question: what the list asks you to record about your agents, and what properties it expects those records to have. The answer is more demanding than "keep logs", and the most useful sentence on the subject comes in the last entry.

The ten risks in one pass

The entries are numbered ASI01 to ASI10. Condensed from the document's own descriptions:

ASI01 Agent Goal Hijack. An attacker redirects the agent's objectives or decision pathways, because agents "cannot reliably distinguish instructions from related content." The mechanics are in our piece on indirect prompt injection.
ASI02 Tool Misuse and Exploitation. Legitimate tools used harmfully through injection, misalignment, unsafe delegation or ambiguous instructions.
ASI03 Identity and Privilege Abuse. Escalation through delegation chains, role inheritance and cached credentials.
ASI04 Agentic Supply Chain Vulnerabilities. Third-party models, tools, plug-ins, MCP and A2A components or registries that are malicious or tampered with.
ASI05 Unexpected Code Execution. Code the agent generates, turned into remote code execution or local misuse.
ASI06 Memory & Context Poisoning. Persistent corruption of stored memory, summaries, embeddings and retrieval stores.
ASI07 Insecure Inter-Agent Communication. Weak authentication, integrity or authorisation between agents that coordinate.
ASI08 Cascading Failures. A single fault propagating across agents into system-wide harm.
ASI09 Human-Agent Trust Exploitation. An agent exploiting a person's trust so that they approve something they should not.
ASI10 Rogue Agents. Agents that deviate from their intended function or authorised scope, where each action "may individually appear legitimate".

The introductory letter frames all ten with a principle it calls Least-Agency: avoid autonomy you do not need, because "deploying agentic behavior where it is not needed expands the attack surface without adding value." In the same passage it says that "strong observability becomes non-negotiable". This piece picks up from that word, observability.

How the logging requirement escalates

Read the prevention sections in order and the logging advice does not stay at one level. It tightens.

ASI01: on an unexpected goal shift, "surface the deviation for review, and record it for audit." A record, of unspecified quality.
ASI02: "Maintain immutable logs of all tool invocations and parameter changes." Now the record must not change.
ASI08: "Record all inter-agent messages, policy decisions, and execution outcomes in tamper-evident, time-stamped logs bound to cryptographic agent identities." Now any change must be detectable, and each entry is tied to a key.
ASI09: "Keep tamper-proof records of user queries and agent actions for audit and forensics."
ASI10: "Maintain comprehensive, immutable and signed audit logs of all agent actions, tool calls, and inter-agent communication".

ASI08 gives the reason. It files the requirement under "logging and non-repudiation" and points back to threat T8, Repudiation and Untraceability, in OWASP's Agentic AI Threats and Mitigations guide, which it summarises as "the ability to trace, attribute, and audit cascading behaviors through resilient logging and non-repudiation mechanisms that prevent silent propagation."

Non-repudiation is the word doing the work. A log supports it when the party that produced an entry cannot credibly deny producing it later, and nobody else can pass an entry off as theirs. An ordinary application log has neither property. Anyone with write access to the store, including a compromised process, can change it without leaving a trace. We set out the fields an agent log needs in the audit trail piece. OWASP's list is concerned with whether anyone other than the author can believe those fields.

Why "immutable" is not enough

The list says less about who writes the record, and ASI09 shows why that matters. Among its examples is "Explainability Fabrications: The agent fabricates plausible audit rationales to justify a risky configuration change." The same entry describes an agent that "acts as an untraceable 'bad influence,' manipulating the human into performing the final, audited action, making the agent's role in the compromise invisible to forensics."

Take those two together. In the first, the agent writes a false entry into the audit trail. In the second, the audit trail is accurate: it records the human's action and misses the agent's part in it. Neither is fixed by immutability. Making a fabricated rationale immutable only preserves the fabrication. What helps is a record the agent did not write. It should describe what actually ran and what it produced, and be written at a boundary the agent cannot reach. That is the same objection to agent-written transcripts we raised in the SOC 2 piece, and here OWASP makes it for us.

The distinction the list implies but does not state: immutability protects a record after it is written. It says nothing about whether the writer was honest at the time. A log counts as evidence only if both hold.

The key belongs outside the agent

ASI10 contains the sentence that says how to get there. After recommending signed behavioural manifests and per-run ephemeral credentials, it adds:

All signing and attestation mechanisms assume hardened cryptographic key management (e.g., HSM/KMS-backed keys, least-privilege access, rotation and revocation). Keys must never be directly available to agents; instead, orchestrators should mediate signing operations so that a compromised agent cannot simply exfiltrate or misuse long-lived keys.

This constraint is what gives "signed logs" any value. If the agent process can reach the signing key, a hijacked agent (ASI01) or a rogue one (ASI10) can sign whatever it likes, and the signature proves only that the key was used. The signer has to sit on the far side of a boundary the agent cannot cross. In practice that means the orchestrator or host, with the key in a separate process, a keystore or an HSM.

ASI04 applies the same logic to what the agent runs. Its advice is to "Sign and attest manifests, prompts, and tool definitions", to "verify provenance before install or activation" and to "auto-reject unsigned or unverified." ASI01 goes further and suggests evaluating an "intent capsule", which it calls "an emerging pattern to bind the declared goal, constraints, and context to each execution cycle in a signed envelope". The direction is the same across the list: bind what was meant to run, what did run and what came out, then sign it with a key the agent cannot use.

Where a signed execution receipt fits

That is the problem a Traceseal execution receipt is built for, so here is the mapping, limits included. The receipt specification defines a signed JSON document in three parts. The execution section holds hashes of the signed skill manifest, the sandbox configuration, the inputs and the outputs, plus the exit code, the timing, and a hash of the matching entry in the operator's hash-chained audit log. The provenance section holds the publisher's key, per-file content hashes and a transparency log reference. The attestation is an ed25519 signature by the operator over both.

ASI02 and ASI05: the input, output and sandbox profile hashes record what a call received, what it returned and what configuration it ran under.
ASI04: the manifest hash and publisher fingerprint tie the execution to one signed artefact. The transparency log reference lets a third party check that the manifest was published rather than swapped in.
ASI08 and ASI10: the operator signature gives each entry the non-repudiation property the list asks for. In our runner the operator key never enters the sandbox, which is the arrangement ASI10 describes. A third party can check the signature with one command and nothing but the receipt file.

Three limits, stated plainly. First, the specification says outright that a receipt "does NOT prove the operator is honest"; it is "an attestation, not a proof of execution". A receipt moves trust from the agent to the operator. That is the point, but the trust has moved rather than disappeared. Publishing to a witnessed transparency log narrows what the operator can rewrite afterwards. Second, receipts detect nothing. A hijacked goal (ASI01) or a manipulated human (ASI09) still produces well-formed receipts. What they give you is a reliable account of what happened when you reconstruct it afterwards. Third, a receipt covers only what passes through the receipting boundary. An agent with a second route to a tool that bypasses it leaves no receipt for that route, as our sandbox write-up discusses.

What the list leaves open

The document is guidance, not a standard. It does not define a log format, say which component should hold the keys beyond "orchestrators", or explain how a third party should check a log that someone else kept. It uses "immutable", "tamper-evident" and "tamper-proof" in different entries without separating them, although they are different properties. The first prevents change. The second makes change detectable. The third is something no software log fully achieves. A team that cites the list in a security review will have to decide which one it means.

The useful reading of the list is not that agentic systems need more logging, since they already produce plenty. It is that the record has to be one the agent could not have written differently, signed with a key the agent could not reach. Every other prevention section assumes that record exists when something goes wrong.

Originally published at traceseal.io. Traceseal issues signed execution receipts for AI agents: an open spec and an open verifier, one command to check.

MCP security: what to log when an AI agent calls tools, and what the log can prove

Ventse — Tue, 22 Sep 2026 08:48:11 +0000

The Model Context Protocol has become the default way an AI agent reaches a tool. That is convenient for the people wiring agents up and awkward for the people who later have to answer for what those agents did, because the protocol standardises the call and says almost nothing about the record. This piece goes through what the current spec actually asks you to log, where a tool call gets written down today, and what each of those records can and cannot establish when someone who does not trust you asks.

Consent is not a record

The 2025-11-25 revision of the specification is blunt about what a tool is. Under its key principles: "Tools represent arbitrary code execution and must be treated with appropriate caution," and "Hosts must obtain explicit user consent before invoking any tool." The tools section adds that clients should prompt for confirmation on sensitive operations and show tool inputs to the user before sending them to the server.

All of that is about permission. None of it is about evidence. A consent dialog establishes that a person was shown a tool name and an argument block and clicked approve. It does not establish what the server did with those arguments, whether the result it returned reflected what changed, or whether the dialog the person saw was the one the model intended. Most hosts keep the consent decision in the same transcript store as everything else the model said, which puts it in the same position as any other line of that transcript: written by the runtime under review, editable by whoever administers the store. We went through why that fails an auditor's test in the SOC 2 piece.

Where the spec does mention logging

The protocol documents proper contain no logging requirement for tool calls. What exists lives in the security best practices document, which sits alongside the authorisation spec rather than inside it, and it appears in three places.

The first is the section on token passthrough, the anti-pattern where an MCP server forwards a client's upstream token to a downstream API without checking it was issued to the server. The document lists the consequences under a heading it calls "Accountability and Audit Trail Issues": the server cannot tell which client is calling, and "the downstream Resource Server's logs may show requests that appear to come from a different source with a different identity, rather than the MCP server that is actually forwarding the tokens." The spec's own words for the result are that "incident investigation, controls, and auditing" become more difficult. It is one of the clearer statements anywhere in the MCP corpus that a log written by the wrong party is worse than useful.

The second is scope minimisation, where broad omnibus scopes are said to obscure audit trails because "a single omnibus scope masks user intent per operation." The mitigation includes a concrete instruction to servers: log elevation events, meaning the scope requested and the subset granted, with correlation IDs.

The third is in the proxy guidance, where proxy services that spawn local servers over stdio are told to "log all stdio transport usage for security monitoring." That is a should, not a must, and it applies only to proxy architectures.

So the sum of the spec's position is: log identity so the downstream system does not attribute your agent's actions to someone else, log scope changes so intent per operation is recoverable, and log process spawns if you proxy. Sensible, and thin. Nothing here says what a record of a tool call should contain, who should write it, or how anyone would check it later.

An annotation is a claim, not a fact

There is a detail in the tools spec that matters more for logging than it first appears. Tools can carry annotations, of which four are behavioural: readOnlyHint, destructiveHint, idempotentHint and openWorldHint. A host that wants to decide which calls need a human in the loop, or which calls to record in detail, will naturally reach for these. The schema says, in a note that is easy to skim past:

All properties in ToolAnnotations are hints. They are not guaranteed to provide a faithful description of tool behavior (including descriptive properties like title). Clients should never make tool use decisions based on ToolAnnotations received from untrusted servers.

The tools section restates it as a requirement: clients must consider annotations untrusted unless they come from trusted servers. Which means a log line reading read-only tool invoked, no confirmation required is not recording what the tool did. It is recording what the tool's author said the tool does, and the author is a party whose code the agent is about to execute. If the server was swapped, updated or compromised through the content it serves, the annotation is exactly the field an attacker would set to true. A record that inherits its risk classification from the server being classified has the same structural flaw as a transcript written by the agent being audited.

The four places a tool call gets written down

Follow one call through a typical deployment and it leaves traces in four places, each written by a different party with a different interest.

The host's transcript. The model's request, the arguments it chose, the consent decision, and whatever the server returned. Written by the agent runtime. Complete in the sense of containing everything the model saw, and worthless as independent evidence for the same reason.
The MCP server's own log. Whatever the server operator chose to record. If you run the server, it is under your control and therefore not independent. If a vendor runs it, it is under theirs, and you will get it in an incident only if they choose to give it to you.
The downstream system. The repository, the database, the ticketing API. This is the closest thing to ground truth about effect, and the token passthrough section above explains the catch: unless the server authenticated correctly, this log may attribute the action to the wrong identity, or to a shared service account that a dozen agents use.
The user's approval. A click, or a standing allowlist. Usually stored in the host, so it collapses into the first bucket.

None of the four is signed by anyone. None is bound to the code that ran: the host records a tool name, and tool names are only required to be unique within a server, so the same name can point at different code tomorrow. And none of them can be handed to a third party without also handing over the trust you place in whoever wrote it. That is the gap between having logs and having evidence, and it is the gap that the receipts primer is about.

The test worth applying: for each record of a tool call, ask who wrote it and whether they had the ability, and any reason, to write something else. If the answer to the first is "the agent" or "the tool", the record is testimony, not evidence.

What a record would need

If the aim is a record that survives that test, the shape is not exotic. Per call, capture the server's identity as a hash of its code or manifest rather than its display name, the tool name and a hash of the exact arguments, a hash of the result, the identity that authorised the call and how (interactive consent, allowlist, or scope elevation with its correlation ID), and the constraints the call ran under, meaning the sandbox profile if there was one. Then sign the whole thing with a key the agent process cannot reach, and put the signature somewhere append-only that a third party can check.

That is deliberately the shape of a Traceseal execution receipt. The specification binds a skill manifest hash, a sandbox profile hash, input and output hashes, exit code and timing under an ed25519 signature from an operator key that never enters the sandbox, and the verifier is a single command that needs nothing but the receipt. The walkthrough shows what an [OK] does and does not establish.

Two limits, stated plainly. A receipt covers what passes through the receipting boundary. A local MCP server launched over stdio runs with the client's privileges, and the security document is explicit that without sandboxing it can read anything the client can; if the agent can reach a tool through a path that produces no receipt, the receipt for the path it did use proves nothing about the other one. And a receipt records that a hashed input produced a hashed output under a hashed policy. It does not say the output was correct or the policy adequate. Our sandbox write-up covers where a containment boundary stops being demonstrable, and the same line applies here.

What is still open

The spec does not define a log format for tool calls, does not require servers to identify their code by hash, and leaves annotation trust to each client's judgement of which servers are "trusted". Every host currently records tool use in its own shape, which means two organisations that both "log all MCP calls" have records that cannot be compared, let alone verified by the same tool. Whether that changes depends on the protocol's maintainers and the vendors shipping hosts, not on anything a single deployment can do.

What a single deployment can do is stop treating the transcript as the record. The spec has already conceded that tools are arbitrary code execution and that a tool's own description of itself cannot be trusted. The consistent next step is to conclude that the tool's, and the agent's, description of what happened cannot be trusted either, and to keep a record they did not write.

Originally published at traceseal.io. Traceseal issues signed execution receipts for AI agents: an open spec and an open verifier, one command to check.

SOC 2 for AI agents: what evidence an auditor can actually accept

Ventse — Tue, 15 Sep 2026 08:47:10 +0000

At some point in the next twelve months, an auditor is going to ask what your AI agents did. Not what they were allowed to do — what they did. If you sell software to companies that care about SOC 2, the question arrives through your own audit; if you buy it, the question arrives through a vendor questionnaire. Either way, the answer "we have the chat logs" is about to stop being good enough, and it is worth understanding why before the fieldwork starts.

Where agents land in the criteria

SOC 2 does not have an AI section. A report is issued against the AICPA's 2017 Trust Services Criteria, with the points of focus revised in 2022, and nothing in that document mentions agents, models or prompts. That is not a loophole. The criteria are written around systems and system components, and an agent that reads a ticket, edits a repository and opens a pull request is a system component in every sense the criteria care about. It holds credentials, it changes production, and it does so without a human approving each step.

Three criteria do most of the work. CC6 covers logical access: what the agent can reach, with which credentials, and who granted them. CC8 covers change management: whether changes the agent makes to infrastructure, data or code go through the same authorisation, testing and approval controls as changes a person makes. And CC7 covers system operations. The wording of CC7.2 is the one to read twice:

The entity monitors system components and the operation of those components for anomalies that are indicative of malicious acts, natural disasters, and errors affecting the entity's ability to meet its objectives; anomalies are analyzed to determine whether they represent security events.

To monitor the operation of a component for anomalies, you need a record of its operation that you can trust more than the component itself. For a database or a load balancer that is routine. For an agent, whose "operation" is a sequence of tool calls chosen at runtime by a model, it is the whole problem.

ISO/IEC 42001 says the same thing more directly

If you are also looking at ISO/IEC 42001:2023, the AI management system standard, the requirement is explicit rather than implied. Annex A control A.6.2.8, AI system recording of event logs, requires the organisation to determine at which phases of the AI system's life cycle event logs are recorded, and sets a floor: at minimum, while the system is in use. The standard's text is paywalled, so we will not quote further, but the shape is clear. Logging the agent while it is operating is not an optional hardening step under 42001; it is a control you either have or do not.

Both frameworks converge on one demand. You need a record of what the agent did, produced in a way that the auditor can rely on. Which raises the question of what "rely on" means.

Why a transcript is not evidence

Most teams' first answer is the conversation log: the prompts, the model's responses, the tool calls and their results, stored by whatever agent framework they run. It feels complete, and for debugging it often is. For an auditor it has three problems.

It is written by the thing being audited. The agent runtime produces the transcript, and the agent runtime is the component whose behaviour is in question. A model that has been hijacked by the content it read will still produce a tidy transcript; it will just be a transcript of the wrong actions. An auditor testing CC7.2 wants a monitoring record that would catch the anomaly, not one that the anomaly gets to write.

It describes intent, not effect. A tool-call record says the agent asked to run git push. It does not say the push succeeded, what commit went where, or whether the working tree it pushed matched the diff shown to the reviewer. The distinction between what an agent reported and what happened on disk is the entire premise of grading agents on ground truth, and it is also the distinction an auditor is trained to draw.

It can be edited after the fact by anyone with write access to the store. A Type 2 report covers a period, commonly six to twelve months, and the auditor samples from it. If the transcript store is a database your platform team administers, the auditor has to test the controls over that before the transcripts are worth anything. We covered why a hash chain alone does not close this gap in the receipts primer: a chain proves order and continuity, but whoever holds the chain can regenerate it.

The test an auditor actually applies: can I obtain this evidence, or something that corroborates it, from a source other than the system under review? A transcript fails that test by construction. It is the system under review describing itself.

What passes

Audit evidence has to be reperformable. The auditor picks a sample — twenty agent runs from March, say — and needs to establish, for each one, what ran, under what constraints, and with what result, in a way that does not depend on taking your word for it. That maps onto four properties a record of agent activity needs:

Bound to the code that ran. A hash of the exact skill or tool code, so the auditor can confirm the sampled run used the approved version and not something patched in between. This is what makes the record useful under CC8 as well as CC7.
Bound to the constraints it ran under. A hash of the sandbox profile or execution policy, so "the agent could not reach the network" is a checkable claim rather than an assertion. What a sandbox can and cannot prove on its own is in our sandbox write-up.
Bound to inputs and outputs. Hashes rather than contents, so the auditor can confirm a specific input produced a specific output without the record itself leaking customer data. This matters for the confidentiality criteria: an evidence trail that stores raw prompts and outputs creates a second copy of every sensitive thing the agent touched.
Signed by a key the agent cannot use. The attestation has to come from outside the agent's reach, otherwise you are back to the transcript problem. And the auditor has to be able to check the signature with a public tool, on their own machine, without asking you to run anything.

That is, deliberately, the structure of a Traceseal execution receipt. The specification is short enough to hand to an auditor: an execution block with the skill manifest hash, sandbox profile hash, input and output hashes, exit code and timing; a provenance block that names who signed the code and where it sits in the transparency log; and an attestation block with an ed25519 signature over both from an operator key that never enters the sandbox. The verifier is on PyPI, needs only the receipt file, and prints what the signature does and does not establish. The walkthrough is in How to verify what an AI agent actually did.

Two honest limits. A receipt proves the operator attested that this code ran under this policy and produced this output hash; it does not prove the output was correct or the policy was sensible. Those remain judgement calls, which is what the auditor is for. And a receipt only covers what was executed through the receipting boundary. If the agent can also act through a path that does not produce receipts, the auditor will find it, and should.

What to do before fieldwork

Put agents on the system description. If an agent holds credentials or changes production, it belongs in the scoped system, named as a component, with its access documented under CC6. Auditors find undisclosed components far less forgivable than disclosed ones with gaps.
Route every consequential action through one execution boundary that produces a signed record, and make it the only path. One receipted path and one unreceipted path is worse than either alone, because it invites the question of which one was used.
Keep the records for the whole audit period, and then some. A Type 2 sample can reach back to the first day of the period. The EU AI Act's six-month log retention floor for high-risk systems, covered in our audit-trail piece, is a reasonable minimum even where the Act does not apply.
Test your own evidence before the auditor does. Pick twenty runs at random, verify each receipt with the public tool, and confirm the manifest hash matches an approved release. If you cannot do that in an afternoon, the auditor cannot do it in a week.

SOC 2 has never asked whether your systems are clever. It asks whether you can show, to someone who has no reason to believe you, that they did what you say they did. Agents make that harder, because for the first time the component in question decides its own actions. The answer is not a better transcript. It is a record the agent did not write.

The receipt format, the verifier and the transparency log are open: the spec, the verifier on PyPI and the public log. Originally published at traceseal.io.

The EU AI Liability Directive is dead: what actually governs AI agent harm now

Ventse — Tue, 01 Sep 2026 08:49:12 +0000

The EU AI Liability Directive is dead: what actually governs AI agent harm now

On 6 October 2025, a short notice appeared in the EU's Official Journal confirming what the European Commission had already signalled eight months earlier: the AI Liability Directive was not coming. The withdrawal followed a Commission decision at its 2533rd meeting on 16 July 2025, and closed a proposal that had been on the table since 2022 — the one piece of EU law written specifically to make it easier to prove an AI system caused you harm.

If you run agents that touch EU users, or sell into the EU, the practical question is not "what did we lose." It is: what governs an AI agent causing harm now, and does it cover the harm your agent could actually cause?

What the directive was meant to do

The proposed AI Liability Directive would have complemented the AI Act rather than duplicated it, easing a claimant's path to compensation for AI-caused harm along two specific mechanisms: a court's power to order disclosure of evidence from a provider or deployer of a high-risk AI system, and a rebuttable presumption of a causal link between an AI system's non-compliance and the resulting harm, where the claimant could show fault was likely but proving causation was disproportionately hard given how the system worked. It was designed for exactly the case that makes agents hard to litigate: the claimant cannot see inside the system, and the operator holds all the evidence.

Why it was dropped

The Commission's 2025 work programme, adopted 11 February 2025, listed the directive for withdrawal with a one-line justification: "no foreseeable agreement." Member states and industry groups had spent two years unable to converge on the scope and mechanics, and the withdrawal landed alongside a broader simplification push that also dropped the ePrivacy Regulation. The Commission's own text left the door open rather than closing it: it "will assess whether another proposal should be tabled or another type of approach should be chosen." As of this piece, no replacement has been tabled.

What already applies instead

The gap is not empty. The EU's overhauled Product Liability Directive, Directive (EU) 2024/2853, was adopted separately and was never at risk — it entered into force in the weeks after its publication on 18 November 2024, and member states have until 9 December 2026 to transpose it. Unlike the withdrawn directive, it is not AI-specific, but it was substantially rewritten with software in mind: the definition of a covered "product" now explicitly extends to software, firmware and AI systems, closing a long-standing ambiguity about whether a defective model was a "product" at all.

The mechanics carry over some of what the withdrawn directive was trying to do. A court can compel a defendant to disclose relevant evidence once a claimant has made a sufficiently plausible case. And where the claimant can show a defect is likely but proving it is excessively difficult because of the product's technical complexity — or where the defendant fails to comply with a disclosure order — the product is presumed defective. Practically, that is close to what the AI Liability Directive would have delivered for high-risk AI: the burden moves toward whoever holds the evidence.

The gap it doesn't close

The catch is scope. The Product Liability Directive exists to protect natural persons, and the damage it compensates is death, personal injury, damage to property kept for private use, and destruction or corruption of data not used for professional purposes. It expressly excludes pure economic loss and property "used exclusively for professional purposes." A consumer whose smart device is bricked by a faulty AI update is covered. A business whose AI agent misfires a transaction, corrupts a business dataset, or ships a bad deployment is not — that is a commercial, economic loss, and it stays exactly where the withdrawn directive would have reached: general national contract and tort law, unharmonised across 27 member states.

That is the case most companies running agents actually have. Agents deployed in a business context, causing business harm, to another business. The one EU proposal aimed squarely at that gap is the one that got withdrawn.

What this means if you run an agent

Two different regimes now apply depending on who got hurt, and neither hands a B2B agent operator's counterparty an easy route:

Consumer-facing harm — falls under the revised Product Liability Directive from December 2026 for newly placed products. Courts there can already order disclosure and will presume a defect against whoever cannot produce a credible answer.
Business-to-business, economic harm — the case a coding agent, a data pipeline agent, or an internal automation agent is most likely to produce — stays on national contract and tort law, with no EU-wide presumption to lean on either way.

In both cases, the practical position is the same: nobody is going to take a log file's word for what an agent did. Where the Product Liability Directive applies, the mechanism rewards whoever can actually satisfy a disclosure order with something a court finds credible, not just present. Where it doesn't apply, and the dispute runs on ordinary national rules of evidence, the party who can independently prove what happened is arguing from a position the other side cannot easily contest. A signed execution receipt — a canonical record of inputs, outputs, timing and sandbox policy, signed with a key the agent never touches — is built to be exactly that kind of artefact: something you hand over under a disclosure order, or into evidence, rather than something a claimant successfully argues you should be presumed to be hiding. What such a receipt does and does not establish, and the fields an audit trail needs regardless of which regime ends up applying, are covered in How to verify what an AI agent actually did and AI agent audit trails.

The uncomfortable read: the one EU proposal written specifically for AI harm was withdrawn for lack of agreement. What replaced it either doesn't reach the B2B case most agents create, or rewards whoever can actually produce evidence once a court asks for it. Either way, "trust our logs" was never the safer bet.

What to do before December 2026

Map your exposure. Separate agent deployments that touch consumers and personal data from ones that only produce business-to-business, economic outcomes — they now sit under genuinely different legal regimes.
Don't wait for the transposition deadline. The Product Liability Directive only covers products placed on the market after 9 December 2026; anything shipped earlier, and anything B2B, is already governed by whatever your national law and your contracts say today.
Build the evidence trail regardless of which regime applies. Disclosure orders and presumptions of defect both cut against whoever cannot produce a credible record. That is true whether a court is applying the new EU rules or your counterparty's home jurisdiction's ordinary contract law.

The receipt format, the verifier and the transparency log are open: the spec, the verifier on PyPI and the public log. Regulation in this area is still being written and rewritten around you. The evidence you can produce when someone finally asks is not something you can backdate.

Originally published on traceseal.io.

Indirect prompt injection: how an AI agent gets hijacked by the data it reads

Ventse — Tue, 25 Aug 2026 08:50:13 +0000

An agent that reads a web page, an issue tracker, an inbox or a pull-request description is reading text that somebody else wrote. Indirect prompt injection is what happens when that text contains instructions addressed to the model rather than to the reader, and the model carries them out using the agent's tools and the agent's permissions.

The uncomfortable part is not that the attack exists. It is that the people who study it hardest do not claim to have solved it. This piece covers the mechanism, what the standard mitigations genuinely buy you, and what a team can do about the residue: making what the agent actually did provable after the fact.

Direct and indirect

OWASP's LLM01:2025 Prompt Injection entry splits the class in two. A direct injection is one where a user's prompt input "directly alters the behavior of the model in unintended or unexpected ways". An indirect injection occurs "when an LLM accepts input from external sources, such as websites or files" whose content alters the model's behaviour.

The threat model changes completely between the two. A direct injection requires the attacker to be talking to your system. An indirect one does not. The attacker writes something once, somewhere your agent will eventually read, and waits for the agent to come to them.

OWASP also records that these inputs "do not need to be human-visible/readable, as long as the content is parsed by the model". An HTML comment, pale text on a pale background, a document's metadata or wording rendered inside an image all qualify. A human reviewing the same source may see nothing at all.

The technique was named by Simon Willison in September 2022. Its indirect form was set out systematically in Not what you've signed up for (Greshake et al., February 2023), which argued that LLM-integrated applications "blur the line between data and instructions" and demonstrated working attacks against production systems, including Bing's GPT-4-powered chat and code-completion engines. The authors showed that "processing retrieved prompts can act as arbitrary code execution".

Why there is no boundary to enforce

Classical injection bugs have a structural fix. SQL injection ends when queries are parameterised, because the database is then told, in a channel the attacker cannot reach, which bytes are code and which are data. Prompt injection has no equivalent. Instructions and content arrive at the model as one undifferentiated token sequence. As Willison puts it:

LLMs are unable to reliably distinguish the importance of instructions based on where they came from. Everything eventually gets glued together into a sequence of tokens and fed to the model.

Delimiters, tagging of untrusted regions and system prompts that say "ignore any instructions in the content below" all raise the cost of an attack, sometimes considerably. What none of them provide is a rule the model is architecturally obliged to obey. They are strong suggestions to a system that treats every token as a suggestion.

The lethal trifecta

Willison's lethal trifecta framing (June 2025) is the clearest way to work out whether a given agent is exposed. Three capabilities together create the risk:

Access to private data — which is usually the entire point of giving the agent tools.
Exposure to untrusted content — any route by which attacker-controlled text or images reach the model.
The ability to communicate externally in a way that could carry data back out.

"If your agent combines these three features," Willison writes, "an attacker can easily trick it into accessing your private data and sending it to that attacker." The awkward observation is that most agents worth deploying hold all three by design. A coding agent that reads a private repository, browses documentation and opens a pull request has the full set. Dropping one leg is a real mitigation and frequently an unacceptable product decision.

What the mitigations actually buy you

OWASP lists seven countermeasures: constraining model behaviour, defining and validating output formats, input and output filtering, least-privilege access, human approval for high-risk actions, segregating external content, and adversarial testing. Each is worth doing. The entry prefaces all of them with a sentence that deserves quoting exactly:

Given the stochastic influence at the heart of the way models work, it is unclear if there are fool-proof methods of prevention for prompt injection.

NIST arrives somewhere similar from another direction. AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (March 2025), describes its own purpose as identifying "current challenges in the life cycle of AI systems" and describing "corresponding methods for mitigating and managing the consequences of those attacks". Managing consequences is the language of an attack that landed.

So the design question is not only how to stop injection. It is what your system can still tell you once one gets through.

What changes when the target is an agent

Against a chatbot, a successful injection produces wrong text and a bad afternoon. Against an agent it produces actions: a file written, a secret read, an API called, a branch pushed, a message sent on your behalf. OWASP's impact list includes "executing arbitrary commands in connected systems" and "providing unauthorized access to functions available to the LLM".

Two things follow. First, blast radius is set by the permissions you granted, not by the model's judgement, because the model's judgement is the component under attack. Second, and less widely appreciated: the agent's own account of events becomes attacker-influenced output. If the injected text says "do X, then report that you did Y", the transcript says Y. Reasoning traces, tool-call summaries and self-reports are all generated downstream of the compromise. They are the last artefacts that should be treated as evidence, and often the only ones a team has kept.

What the regulation already assumes

Article 15 of the EU AI Act requires that high-risk AI systems "shall be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities". The technical measures it calls for must, where appropriate, "prevent, detect, respond to, resolve and control for" AI-specific attacks, expressly including "inputs designed to cause the AI model to make a mistake".

Article 15 binds high-risk systems, so it will not reach every agent deployment. The drafting is instructive regardless: prevention is one verb out of five. Detection, response and resolution each presuppose an attack that succeeded, and each requires a record that stays trustworthy after the system it describes has been compromised.

Two controls that survive a failed prevention

Containment. A kernel-namespace sandbox bounds what an injected instruction can reach, whatever the model decides to do. We published a working bubblewrap profile in How to sandbox an AI coding agent, along with the DNS leak we only found by running it. A sandbox limits consequences. It will not tell you that an injection arrived.

A record made outside the agent. A signed execution receipt is a canonical record of inputs, outputs, timing and sandbox policy, stored as SHA-256 hashes and signed with an ed25519 key the sandboxed process never sees. The agent does not write it and cannot amend it. A third party checks it offline, with no access to your systems:

$ pip install traceseal-verify
$ traceseal-verify receipt.json
[OK] receipt.json — operator signature verified

None of this prevents injection. What it does is make the aftermath answerable. Given a suspected compromise last Tuesday, what did the agent read, what did it touch, and can you demonstrate that to somebody who has no reason to take your word for it? The walkthrough of what such a check does and does not establish is in How to verify what an AI agent actually did.

What to record so an injection stays provable

The exact content the agent read, hashed at fetch time. If the poisoned page is quietly edited afterwards, the hash still pins what was actually served to you.
Every tool call and its real arguments, captured at the tool boundary rather than from the model's description of what it intended.
The sandbox policy in force during the run, so containment becomes something a reader can check rather than something you assert.
Observed side effects — files written, network destinations reached, commits created — recorded from outside the agent process.
A signature over all of it, made with a key the agent cannot reach, so the record's integrity does not depend on the agent having behaved.

The field-level detail, including the EU AI Act's log-retention floor, is in AI agent audit trails.

The question to sit with: if one of your agents read a poisoned document last week, would you learn about it from your own records, or from the person its data ended up with?

The receipt format, the verifier and the transparency log are open: the spec, the verifier on PyPI and the public log. Prevention research will keep moving, and it should. In the meantime, the attacks that get through are the ones your evidence has to account for.

Transparency log witnesses: how cosigned checkpoints make a log tamper-evident

Ventse — Tue, 18 Aug 2026 08:49:30 +0000

On 17 August 2026, at 14:17:40 UTC, an independent witness operated by Markovian Protocol cosigned the checkpoint of our public transparency log. Nothing about the log changed that afternoon. What changed is what we can prove about it: from that moment, if we ever rewrote the log's history, a third party outside our infrastructure would be holding cryptographic evidence of the version we abandoned.

This post explains what a transparency log witness is, what a cosigned checkpoint actually proves, and the obligations you take on when one starts watching your log. Most of it we learned by going through the process rather than by reading about it.

The gap witnesses close: a log can lie to you specifically

An append-only log is the standard answer to "how do I show my records existed before the dispute did". You publish entries into a Merkle tree, the tree has a root hash, and anyone can check that a given entry is included. Certificate Transparency (RFC 6962) runs the web's certificate issuance through this structure; the Go checksum database does it for module hashes; Sigstore's Rekor does it for software signatures.

But a Merkle tree on its own has a hole, and RFC 6962 was upfront about it: detecting a misbehaving log requires people to compare notes on what the log showed them. A log operator who controls the serving infrastructure can present one version of the tree to the auditor and a different version to everyone else. Each view is internally consistent, each proof checks out, and the two views never meet. This is the split-view attack, and no amount of hashing inside the log prevents it, because the operator is the one doing the hashing.

For our purposes the operator's honesty is exactly what is in question. Traceseal exists because logs written by the party under scrutiny are not evidence. Our transparency log anchors signed agent receipts, and until last week it had the same weakness as every self-hosted log: you had to trust us not to serve you a private version of history.

What a checkpoint is

A checkpoint is a small signed statement of the log's current state, in the format specified by c2sp.org/tlog-checkpoint: the log's name (its origin), the number of entries, and the root hash of the tree at that size, wrapped in a signed note. Ours currently reads:

log.traceseal.io
19
OhQzzSdQHnpofrO/F2reZkiIDDPjUEM+wuGNc4mlD84=

— log.traceseal.io vLzsyMryP2doyfOtISb5...

Nineteen entries. Certificate Transparency logs hold billions, and the format is the same, which is rather the point: the protocol does not care whether it is guarding the web PKI or a small receipts log. A checkpoint commits the operator to one specific history up to one specific size. What it cannot do alone is stop the operator issuing two different commitments to two different audiences.

What a witness actually does

A witness is a service, run by someone other than the log operator, that keeps the latest checkpoint it has seen for each log it follows. The protocol, c2sp.org/tlog-witness (v1.0.0, March 2026), is short enough to read over coffee. The core of it:

Witnesses verify that the checkpoint is consistent with their previously recorded state of the log (if any), and return a timestamped cosignature.

"Consistent" carries the whole load. When our log grows and asks the witness to cosign the new checkpoint, it must supply a consistency proof: a Merkle proof that the tree at the new size contains the tree the witness already recorded as a prefix. If the proof fails — if any old entry was altered or removed — the witness refuses. The cosignature it returns on success embeds its own timestamp, so the statement a verifier ends up with is precise: at this time, this witness saw this exact view of the log, and that view extended every view the witness had seen before.

A split view now requires deceiving the witness as well, and the witness's records are not ours to edit. That is the entire trick. No new cryptography, just a signature from someone whose incentives are not our incentives.

One implementation detail that cost us an afternoon: the log's own /checkpoint endpoint serves only the log's signature. The cosignature lives on the witness's copy, at /<sha256(origin)>/checkpoint on the witness's host. If you fetch your own checkpoint and wonder where the witness signature went, it was never going to be there.

What a cosignature does not prove

Witnessing is content-blind, and it is worth being blunt about the edges:

It says nothing about what the entries mean. A witness checks append-only consistency, not truth. A log of fabricated receipts, faithfully appended and never rewritten, will be cosigned without complaint. Content verification is a separate job — ours is done by signature checks on each receipt, and the honest-limits caveats in that post still apply.
One witness is one extra party to corrupt. If the log operator and the witness collude, the split view returns. The mitigation is a quorum of independent witnesses, which is why a shared witness network is being built around the C2SP protocols rather than each log recruiting its own friendly observer.
A cosigned checkpoint covers the log up to that size. Entries appended since the last cosignature are committed by the log's own signature only, until the next witnessing round picks them up.

The part nobody advertises: witnessing binds you

Before the pin, our append-only claim was policy. We could have quietly regenerated the log, re-signed it, and nobody outside would have been the wiser. After the pin it is a commitment with teeth, and the teeth point at us.

The log must now stay strictly append-only forever, because the witness's copy of every cosigned checkpoint is permanent evidence against any rewrite. And the log's signing key can no longer rotate casually: the witness pins the key it verified, and a checkpoint signed by an unknown key gets refused. We put both obligations in writing to the witness's operator — no history rewrites, and no key rotation without coordinating first — and they were confirmed back to us as the terms of witnessing.

Worth sitting with: a tamper-evident log is not a feature you add. It is a constraint you accept. If you are not prepared to be caught by your own infrastructure, external witnessing is not for you — which is precisely what makes it persuasive when you do accept it.

Check our log yourself

None of this asks for trust in the telling. Both views are public:

$ curl https://log.traceseal.io/checkpoint
$ curl https://witness.markovianprotocol.com/$(printf 'log.traceseal.io' | sha256sum | cut -d' ' -f1)/checkpoint

The bodies should match byte for byte, with the witness's cosignature as an extra line on the second. We have published the verifier we run daily — it checks the log's Ed25519 signature, the witness cosignature against the witness's public key, and that live and pinned bodies agree — as lib/tlog_witness.py in the Alibi repository.

Where this fits in an agent accountability stack

A signed receipt proves an execution record has not changed since it was sealed. An audit trail tells you what to put in that record. The transparency log proves the receipt existed before anyone had a reason to dispute it. The witness closes the last gap in that chain: it stops the log itself being quietly replaced. Each layer covers a failure the previous one cannot.

One witness is not the end state; a quorum of independent witnesses is. What ours bought is the step from "trust us" to "trust us, or catch us out through a party we do not control". How many witnesses a receipts log needs before a court or an auditor treats its timestamps as settled is a question nobody has an answer to yet. We would rather be early to finding out.

Originally published on the Traceseal blog. The receipt format, verifier and transparency log are open: spec, verifier on PyPI, public log.

AI agent audit trails: what to record, and what your records can prove

Ventse — Tue, 11 Aug 2026 08:51:37 +0000

An AI agent does not just answer questions. It calls tools, edits files, opens pull requests, moves money, sends messages. Sooner or later one of those actions gets questioned, by a customer, an auditor, a regulator, or your own incident review, and what you can say at that point depends entirely on what you recorded at the time.

Building a useful audit trail for an agent is really two problems, and teams usually only notice the first one. Completeness: did you capture the right events? Credibility: can anyone other than you believe the record? This post covers both, including the record-keeping duties the EU AI Act now attaches to high-risk systems.

What an AI agent audit trail should record

Chat transcripts are not an audit trail. The transcript records what the model said; the questions that matter later are about what the system did. A trail worth keeping records, for every run:

Which code and model ran. Agent and skill versions, the model identifier, and content hashes of the code that executed, not just a version string someone can retag.
The inputs. The task or prompt, the configuration, and the policy in force. Store sensitive payloads as SHA-256 digests rather than plaintext: a hash proves the input was what you say it was without retaining the data itself.
Every tool call. Name, arguments, result and exit status. Tool calls are where an agent touches the world; a trail that omits them records the commentary and drops the actions.
What changed. Files written, diffs applied, messages sent, records created, again as digests, so the artefact can be checked against the record later.
Timing. Start and end of the run and of each step. This is not gold-plating: for biometric systems, Article 12(3) of the EU AI Act requires logging "the period of each use of the system (start date and time and end date and time)" explicitly.
Human involvement. Which steps ran under an approval, and who gave it. When something goes wrong, the first question after "what happened" is "who signed it off".

The regulatory floor: EU AI Act record-keeping

If your agent falls in the Act's high-risk category, record-keeping stops being a best practice and becomes a duty with numbers attached. Three articles of Regulation (EU) 2024/1689 do the work:

Article 12 requires high-risk AI systems to "technically allow for the automatic recording of events (logs) over the lifetime of the system", covering events relevant to identifying risk, post-market monitoring, and monitoring of operation by deployers.
Article 19 obliges providers to keep those automatically generated logs, to the extent they are under their control, for a period appropriate to the system's purpose and at least six months.
Article 26(6) places the mirror-image duty on deployers: keep the logs your high-risk system generates, again for at least six months, longer where other EU or national law (data protection in particular) requires it.

On timing: under Article 113 the Act applies generally from 2 August 2026, which brings the obligations for high-risk systems listed in Annex III with it; systems that are high-risk because they are safety components of regulated products (Article 6(1)) follow from 2 August 2027.

Most coding and operations agents are not high-risk systems under the Act. But the transparency duties in Article 50 already apply to systems that interact with people or generate content, and if you are going to keep records at all, the Act's six-month retention floor is the obvious number to copy rather than invent your own.

A complete log can still prove nothing

Suppose you record all of the above faithfully. You now have a trail that is excellent for debugging and incident response — and still worthless the moment someone outside your organisation needs convincing.

The problem is authorship. Application logs are written by the same system they describe, stored on infrastructure the operator controls, and editable by anyone with write access to the store. When the record's author is the party whose conduct is in question, the record cannot clear them. Security practice has long treated missing or unprotected logs as a top-tier weakness (it is A09 in the OWASP Top 10), but integrity is a separate property from existence. A log that exists, is complete, and could have been rewritten yesterday answers "what happened?" only for people who already trust you.

We have written up the full argument in why logs are not evidence; the short version is that WORM storage, centralised log collectors and hash chains each move the trust boundary without removing it.

Making the trail verifiable

The fix is mechanical, and none of it is exotic cryptography:

Hash every artefact the run touched, so the record commits to specific bytes rather than descriptions.
Serialise the record as canonical JSON, so there is exactly one byte sequence a signature can cover.
Sign it at execution time with an ed25519 key held outside the agent's reach, so neither the agent nor a later intruder can rewrite history without breaking the seal.
Anchor it in a transparency log, so the record provably existed before the dispute did.

That turns your audit trail into a receipt anyone can check offline, with no access to your systems:

$ pip install traceseal-verify
$ traceseal-verify receipt.json
[OK] receipt.json — operator signature verified

We walk through what that verification actually proves in an earlier post, and the honest limits matter here too. A signed receipt proves the record has not changed since it was sealed and who sealed it. It does not prove the record is complete: an action taken outside the instrumented path leaves no receipt at all. That is why receipts pair naturally with sandboxing the agent — the sandbox narrows what can happen off the record, and the receipt seals what happened on it.

The test worth applying: pick one agent action from last month and try to show a third party what happened, without asking them to trust your database. If the exercise ends at "here is a row in our logging system", you have an audit trail. You do not yet have evidence.

A checklist you can apply this week

Record tool calls and artefacts, not just conversation.
Hash sensitive inputs and outputs instead of hoarding them.
Retain records for at least six months, the floor set by Article 19 and Article 26(6).
Sign records when they are made. A trail you start keeping after the dispute begins says nothing about what came before.
Rehearse retrieval: if you have never pulled a specific run's record under time pressure, you do not know whether you can.

The receipt format, verifier and transparency log are open — the spec, the verifier on PyPI, and the public log. You can adopt the format without adopting us.

Originally published on the Traceseal blog.

How to sandbox an AI coding agent (and what a sandbox cannot prove)

Ventse — Tue, 04 Aug 2026 08:50:37 +0000

The question usually arrives after the run rather than before it. An agent has spent forty minutes in a repository, the diff is larger than anyone expected, and somebody asks whether it touched anything outside the working directory. The instinct is to search the transcript. The trouble with that instinct is the one this blog keeps returning to: the transcript is written by the process you are asking about, so a clean transcript is exactly what a misbehaving run produces.

The durable answer is to make the action impossible rather than to watch for it. This post is a working profile for doing that on Linux, an honest account of where it leaked when we tested it, and the point at which a sandbox stops helping.

What a sandbox is, underneath the word

"Sandbox" is used loosely enough to mean anything from a Docker container to a permissions prompt. On Linux it has a specific meaning, assembled from three kernel facilities:

Namespaces give a process a private view of a global resource: its own mount table, process tree, or network stack (namespaces(7)). User namespaces are the piece that matters for developer tooling, because they let an ordinary account create the others without root.
Seccomp filters system calls, so a process can be denied whole classes of kernel interface rather than denied a file (seccomp(2)).
Landlock lets an unprivileged process restrict its own filesystem access and never get it back, which is the right shape for a wrapper that launches something it does not trust (kernel documentation).

Bubblewrap is a small setuid-or-userns tool that composes the first two into a single command. It is what Flatpak uses underneath, it needs no daemon, and it starts in milliseconds, which matters when the thing you are wrapping is invoked hundreds of times a day. Everything below was run against bubblewrap 0.11.0 on Debian.

A profile that actually runs

The shape you want for a coding agent is narrow: it may read the system, it may write to one directory, and it may not reach the network.

$ bwrap \
    --ro-bind / / \
    --bind "$PWD" "$PWD" \
    --dev /dev --proc /proc \
    --unshare-net \
    --unshare-pid \
    --die-with-parent \
    --chdir "$PWD" \
    -- your-agent-cli run

Line by line: --ro-bind / / mounts the entire host filesystem read-only, so the agent keeps the toolchain, libraries and language runtimes it needs and can modify none of them. --bind "$PWD" "$PWD" punches one writable hole at the working directory. --unshare-net puts the process in a fresh network namespace with no interface but loopback. --unshare-pid stops it seeing or signalling other processes on the box, and --die-with-parent means an orphaned agent is killed rather than left running.

Two classes of action stop being things you monitor and become things that cannot happen. A write outside the working directory fails at the kernel:

$ touch /home/tim/SHOULD_NOT_EXIST
touch: cannot touch '/home/tim/SHOULD_NOT_EXIST': Read-only file system

And there is no route off the machine, so a connection to a raw address fails to establish. That is a better guarantee than any amount of transcript review, because it does not depend on the agent's cooperation, its honesty, or your diligence in reading the output.

Where it leaked

Testing the profile above produced a result we did not expect. Egress was properly gone: ip route was empty inside the namespace and curl to a bare IP address failed to connect. But name resolution still worked.

$ getent hosts example.com
2606:4700:10::6814:179a  example.com
2606:4700:10::ac42:93f3  example.com

The explanation is in /etc/nsswitch.conf. Debian's hosts line includes the resolve module, which does not send a DNS packet at all: it talks to systemd-resolved over a Unix socket at /run/systemd/resolve/io.systemd.Resolve. That socket arrived inside the sandbox as an ordinary file, carried in by the read-only bind of /. Masking the directory confirms it, and the resolution stops:

$ bwrap ... --unshare-net --tmpfs /run/systemd/resolve -- getent hosts example.com
$   # no output

The general lesson is worth more than the specific fix. A network namespace governs interfaces, routes and sockets in the network stack. It has nothing to say about a Unix socket to a host daemon that still has full network access, and a filesystem bind will hand you one of those without mentioning it. Anyone auditing an agent sandbox should assume the same is true of the container runtime socket, the SSH agent socket, and the D-Bus session bus.

Worth stating plainly: we wrote the confident version of the paragraph above first, then ran it, and the run disagreed. A sandbox profile that has not been driven end to end is a hypothesis about isolation, not a control.

Three further limits are structural rather than fixable. The working directory is writable by design, and for most repositories that is where the interesting secrets live: .env files, and .git/config remotes with an access token embedded in the URL. Environment variables cross the boundary untouched unless you add --clearenv, so an API key exported in the parent shell is available inside. And the kernel is shared, which is the standing caveat on all container-style isolation and the reason NIST SP 800-190 treats it as weaker than a virtual machine boundary. Docker's none network driver is the same idea with the same caveats, if a container is already in your stack.

The part nobody budgets for

Isolation changes what the agent does, not only what it is able to do, and this is a genuine operational cost rather than a footnote. An agent that cannot reach the network will still try to install a package. When that fails, it improvises: it writes a stub, or vendors something from a cache, or quietly adjusts a test so the missing dependency stops mattering. The run does not stop; it goes sideways.

The second effect is on diagnosis. A run that failed because the sandbox blocked it and a run that failed because the task was wrong produce output that looks much the same, and the agent's own explanation of which one happened is not evidence either. Teams adopting sandboxing tend to lose a week to this before they start logging the policy denials separately from the task failures. It is worth doing on day one. This is also why grading agents against ground truth on disk beats grading them on what they said they did.

From control to evidence

Here is the limit of everything above. A sandbox constrains; it does not communicate. You now know the agent could not write outside the repository. A customer, an auditor or a regulator asking the same question next quarter still has nothing but your description of a profile you say you applied.

Closing that gap is what an execution receipt is for. The receipt specification carries a sandbox_profile_hash field: a SHA-256 hash of the sandbox configuration, meaning the bubblewrap argument list plus the environment. A verifier holding a known-good profile can hash it and check whether the run used that profile or a different one, offline, with one command and no access to your systems. Combined with the signature over the whole record, an altered profile cannot be presented as the original. For what the signature covers and what an [OK] means, see the walkthrough and the primer on why logs are not evidence.

What that does not do is more interesting, and the specification says so itself rather than leaving a reader to discover it. Among the things a valid receipt does not prove:

That the sandbox actually enforced the declared profile (the receipt records what the operator claims the sandbox was; a compromised operator could lie).

Section 6.1 of the same document puts it more bluntly still: the receipt is "an attestation, not a proof of execution in the zero-knowledge sense". An operator who never ran bubblewrap at all can sign a receipt naming a profile hash they took from the documentation.

What is still open

That last gap does not close by thinking harder about receipt design, and we are not going to pretend a signature solves it. It closes only by moving the seal outside the party being trusted, and each route out has a real cost. A second party can run or observe the sandbox and cosign the record, which works and requires someone willing to be that party. Hardware-rooted attestation can bind the measurement to a chip rather than an assertion, at the price of a much heavier deployment. A third-party execution environment removes the operator's discretion entirely, along with the operator's control of their own infrastructure.

Which of those becomes normal is going to be settled by procurement departments and insurers rather than by cryptographers, and probably not this year. In the meantime the honest position is that a signed profile hash moves an unverifiable claim to a checkable one and stops short of proof, which is still a considerable distance from a log file nobody can check at all.

The regulatory direction is at least consistent with the effort. Article 15 of Regulation (EU) 2024/1689 requires high-risk systems to be designed for an appropriate level of accuracy, robustness and cybersecurity across their lifecycle. A demonstrable containment boundary is easier to argue than a policy document describing one.

Questions to ask of your own setup

Has the profile been run, or only written? Try to write outside the workdir and try to reach the network from inside it. Ours resolved DNS through a socket we had not thought about.
What sockets came in with the filesystem? Enumerate what is under /run and /var/run in the mount. Each one is a channel your network namespace does not cover.
What is in the environment? If you have not used --clearenv, every secret in the parent shell is inside the sandbox.
Are policy denials logged separately from task failures? If not, you cannot tell a blocked agent from a confused one.
Can anyone outside your team check which profile ran? If the answer is that you would tell them, that is a description, not a record.

The receipt format, the verifier and the transparency log are open. You can adopt the format without adopting us.

Alibi: grade coding agents on what actually happened on disk

Ventse — Thu, 30 Jul 2026 08:36:27 +0000

Ask a coding agent whether it finished the job and it will usually say yes. The closing summary may be fluent, itemised and confident. None of that proves what happened.

We have written before about why logs are not evidence: a record produced by the system it describes inherits that system's honesty. An agent's account of its own run has the same defect one layer up.

So we built Alibi, an MIT-licensed harness that ignores the account entirely. Alibi runs the same missions through agentic CLIs and grades each run against ground truth on disk: the files that exist and the changes that were made. It never grades what the agent says it did.

The grading rule

Each configured CLI, currently Claude Code and OpenAI Codex, receives an identical mission brief inside its own bubblewrap sandbox. Each lane gets a private working tree, an isolated home and an isolated /tmp.

When the process exits, Alibi inspects the tree:

Does the requested file exist?
Does it contain what the mission required?
Did the specified tests pass?
Did anything change outside the permitted area?

The agent's exit message is not consulted. A mission passes if the disk says it passed.

The sandbox also makes containment measurable. A write outside the allowed tree is a recorded violation, not a judgement about style. Alibi retains the working directory, sandbox home, temporary files and logs after the run, so someone can derive the grade again and dispute it.

What Alibi refuses to claim

Alibi is not a leaderboard. The mission set is small and the runs happened on our machines. A handful of missions is a probe, not a census, so we are not publishing rankings from it.

It also says nothing about consent behaviour. Lanes run with approvals granted by design because the object of study is execution, not permission seeking. How a CLI behaves when it has to ask first is a separate question.

The grades remain Alibi's judgement. Every assertion is ordinary code in the mission definition. You can read a check, run it again and dispute it.

The grader cannot be trusted either

A grading harness has the same structural problem as the agents it grades: it produces the record of its own run. If our scores mattered to anyone, we could quietly edit them.

Alibi now supports signed Traceseal receipts. Point one environment variable at an Ed25519 key and each CLI and mission result is sealed into a receipt containing hashes of:

the mission definition
the result record
the retained evidence tree

A run-level receipt chains those results into one verifiable unit. Signing happens on the host after the contained processes exit, and the key is never mounted inside the sandbox.

A second party can act as a witness. The witness tool derives every hash again from the retained evidence, checks each signature and the chain, and cosigns only if everything matches. It refuses to cosign with the original signer's key.

There is an important limit. Two keys on one machine prove key separation, not operator separation. Full independence requires the witness to verify on hardware the original signer cannot write to.

The signature means "this run occurred as described." The scores remain Alibi's judgement. Signing makes them tamper-evident, not true.

What to ask of any agent benchmark

First, what record was it graded against? If the answer is the agent's final message, the benchmark measured prose.

Second, could the operator rewrite the results afterwards? A results table in a repository the grader controls is still a log.

Third, can you run it yourself? The missions, assertions and harness should be readable and executable without the author's cooperation.

Finally, what was the consent model? Auto-approved runs measure capability. They do not measure conduct, and a result that blurs the two flatters everyone.

Alibi is our attempt to answer those questions cleanly for our own testing. Whether its missions generalise beyond the behaviours we care about remains open. The repository is small enough to read in an afternoon.

Read the missions, dispute the assertions and grade your own agent on GitHub.