Help Net Security covered a new open-source tool this week that writes tamper-evident audit logs for AI agents. It stores recorded agent activity in a file, each record carries a hash of the record before it, and edits to old records break the later fingerprints. The author is upfront about what it can't do, which I respect. That's rarer in this space than it should be.
The story points at a wider problem with this wave of tools. "Tamper-evident" is starting to read as a synonym for proof. A log that's awkward to edit and evidence that holds up when money or blame is on the line are different things. Two questions expose the difference, and neither one needs a security background.
Question one: who holds the key?
A hash chain with no signature can be rewritten by anyone who holds the file, right up until its head gets checkpointed somewhere outside the operator's reach. Anyone who can alter an old record can recompute the later hashes, and the chain checks out clean again. A bare hash chain only proves the file agrees with itself, and a rewritten file agrees with itself too. An outside checkpoint makes rewrites of already-checkpointed history visible, and that's real progress, but it still can't show which actions the recorder never captured. The docs for the tool in the story say "no secret required; that is the point." For making verification easy, fair enough. For evidence, that's the limit: if no secret was needed to write the record, no secret is needed to rewrite anything that hasn't been checkpointed yet. An unsigned chain is a notebook with numbered pages, not a notarized record.
Question two: where does the recorder sit?
When the recorder only runs inside the agent process, the agent decides what gets recorded. A prompt-injected agent can skip the logger before it exfiltrates your data, and malware doesn't file a report on itself. The record is complete only when the thing being audited chooses to make it complete. An audit trail needs a boundary outside the process being recorded, where the agent can't skip it, or the trail documents the agent's good days and goes quiet on the bad ones.
Some tools add a witness service: an outside party that periodically stores the chain's length and fingerprint. Witnesses are a good idea, and more of them should appear. A witness can only vouch that the chain hasn't changed since it last looked. It can't vouch for what the recorder left out when the recorder lives inside the process it's reporting on. A timestamp on an incomplete record is still an incomplete record.
The same test, pointed at Pipelock
Apply the same two questions to Pipelock, because a test that exempts its own author isn't a test. Pipelock mediates traffic at a separate proxy boundary. With enforced containment, meaning the network only lets agent traffic out through the proxy, recording happens at a boundary the agent can't skip. Containment is part of the deployment, and skipping it leaves a weaker guarantee, which the docs say plainly. Receipts are signed, and the operator who deploys Pipelock holds the key. I never see it. You can verify a receipt offline against a public key the operator provides, with no account and no server. The limit needs stating plainly: operator-signed means the operator could omit records before signing. The receipts prove what the proxy saw and that nobody changed it afterward. They don't yet prove independence from the operator who ran it. Outside anchoring is where this whole category needs to go. The math doesn't deliver more than that.
A shared vocabulary would help, because "audit trail" covers everything from a text file to signed, replayable evidence, and buyers can't price the difference. That's why I published Agent Evidence Levels, an open spec and checker for testing what an evidence artifact can and can't establish. A producer or operator can't award a grade through it. A public grade takes a verifier who's independent of both, and that rule applies to Pipelock the same as everyone else. If you build one of these tools, publish an artifact and a capability declaration an independent verifier can check before someone with a subpoena does the checking for you.
Top comments (2)
The recorder-inside-the-process problem is the one that keeps coming back in agent work. I run agents with persistent filesystem access between sessions, and the first thing I learned is that an agent that can write its own logs can also not write them. My workaround has been crude but functional: every agent writes to a directory it can append to but not delete from, and a separate process checksums the directory on a fixed interval that the agent does not control. Not tamper-proof by any real standard, but it catches the gap between "the agent reported this" and "something outside the agent saw this."
Your point about unsigned chains being a notebook with numbered pages landed well. The numbered-pages metaphor gets at why people over-trust these systems. A notebook is useful, but nobody enters a notebook into evidence without asking who wrote in it. The same question should apply to agent logs, and it usually does not.
Your workaround is more right than you're giving it credit for. An append-only directory the agent can't delete from, checksummed on an interval by a process the agent doesn't control, is a witness. That's the same shape the fancy tools are selling, you just built it out of directory permissions. The interval is your exposure window: anything written and rewritten between two checksum passes looks clean, and the checksummer still can't see what the agent never wrote down. But you've already crossed the line most setups never cross, which is getting the observer outside the thing being observed.
And yeah, that's exactly why I keep coming back to the notebook framing. The question "who wrote in it" has a boring, well-understood answer in every other evidence context. Agent logs just haven't been asked it yet.