I built a hostile AI agent receipt that passed the cryptographic checks.
The signature was valid. Sequence coverage was complete. The evidence still failed to prove the claim.
The correct answer was undecidable.
That failure exposes a gap in how AI agent receipts are being compared. An authentic record is valuable, but authenticity alone does not tell another system whether the evidence is strong enough to authorize what happens next.
I think receipt systems have three separate jobs.
Attestation strength asks whether the record is authentic and who controlled the signing key.
Evidence sufficiency asks whether the attached evidence actually proves the claim.
Post-handoff control asks what another system can safely do with that proof: continue, reject, review, quarantine, or roll back.
These jobs overlap, but they are not interchangeable.
What the experiment found
The OpenLine Audit Debt Harness ran 648 matched workflows and produced 1,944 observations.
Each workflow was rendered three ways:
- a vendor-native trace;
- a normalized unsigned export;
- a signed OLP receipt bundle with evidence hashes and explicit sequence coverage.
Against the normalized unsigned track, the signed OLP track reached exactly the same audit decisions:
- 422 correct;
- 0 incorrect;
- 226 undecidable.
It reduced median inspected bytes by 31.1% and median lexical tokens by 51.8%.
It reduced evidence reads by 0%.
That last result matters. The receipt compressed the path to the evidence. It did not remove the need to inspect the evidence itself.
All six hostile controls returned undecidable, including the most important one: a perfectly valid signature attached to evidence that failed to prove the claim.
Where the current systems differ
Agent Receipts is stronger than this harness at the signing boundary. Its reference tooling supports an out-of-process daemon that owns the signing key and chain, plus separate Go, Python, and TypeScript implementations that verify one another.
Sello moves the signer to the receiving service, creating a different and potentially stronger witness boundary when that service is genuinely independent.
OLP is testing a different layer: whether a portable receipt carries enough evidence for another system to reproduce the decision and act safely after the handoff.
None of those strengths erase the others.
A strong signature can protect an unsupported claim.
A semantically useful receipt can still come from a weak signing boundary.
A policy hook can block an action while leaving the later recipient unable to reproduce why.
That is why I built a source-backed comparison around three axes:
attestation strength → evidence sufficiency → post-handoff control
The comparison includes version pins, primary sources, limitations, reproduction paths, and a machine-readable JSON matrix. It does not assign vague high, medium, or low grades.
I built OLP and this harness, so that affiliation is disclosed directly on the page.
The open challenge
Run the same audit with real vendor traces.
Keep the evidence availability, policy question, normalization schema, auditor, hostile cases, and materiality thresholds fixed.
Then publish every decision, abstention, byte count, evidence read, retrieval failure, and correction.
An authentic receipt can still be unfit to authorize the next thing.
Read the source-backed AI agent receipt landscape
Run the OpenLine Audit Debt Harness
terryncew
/
openline-audit-debt-harness
Falsifiable harness comparing the audit work required for vendor traces, normalized exports, and signed OLP receipts under matched evidence conditions.
OpenLine Audit Debt Harness
A falsifiable synthetic experiment for one narrow question:
Can a portable, evidence-bound receipt reduce the work required for an isolated auditor to reach a correct decision?
The policy is intentionally simple: sending data to an unapproved destination is a violation.
Each identical workflow is rendered as:
- a vendor-native trace;
- a normalized unsigned export;
- a signed OLP receipt bundle with evidence hashes and explicit sequence coverage.
Evidence availability is matched before rendering. The experiment reports correctness, abstention, time, bytes, lexical tokens, evidence-object reads, and retrieval failures separately. It produces no combined Audit Debt score.
Quick start
python -m pip install -e .
python -m pytest -q
openline-audit-debt run --output results
openline-audit-debt verify --output results
Or run the complete release gate:
python scripts/release_check.py
Outputs
-
results/observations.csv— every matched case, decision, and work measurement. -
results/summary.json— track-level results, paired differences, and separate claim tests. -
results/run_manifest.json— parameters, seed, environment…
Top comments (0)