agent-trace-witness v0.1.0: signed readiness seal, post-execution capture, and PROV-DM causal graphs for autonomous AI agents

An external witness library + CLI for autonomous multi-agent AI systems. Implements mechanisms 1, 2, and 3 of the HANSARD framework (arXiv:2608.22512) as offline, deterministic, CPU-only Python. Post-execution forensics — not prevention, not live observability.

Why an external witness

When an autonomous agent runs, three things need to be observable independently of the agent itself: what it was allowed to do (the seal), what it did (the capture), and what would have happened if some action had been different (the counterfactual replay). The agent can lie about any of these, so the witness has to live outside the agent process and see events at the protocol boundary, not in the agent's memory.

agent-trace-witness is a small, auditable Python library + CLI that does exactly that. It signs readiness seals with HMAC-SHA256, captures events at five protocol choke points via a real MCP client (cassette or live stdio), and emits a PROV-DM JSON-LD causal graph that you can replay counterfactually to answer "what if this tool call hadn't happened". No network in tests, no LLM calls, no kernel tracing.

What the library does

Mechanism	What it does	CLI	Status
Seal	Signed readiness profile before the agent runs: SHA-256 of the system prompt, tool list with scopes, timestamp, witness identity, HMAC-SHA256 signature. Any byte changed in the body invalidates the signature.	witness seal	stable since 001
Capture	Five choke points observed outside the agent: (a) tool call before MCP, (b) MCP response, (c) message to model, (d) model response, (e) external effect (file write, delete, side-effect).	witness capture	stable since 002 (5/5 choke points)
Capture transport	RealMCPClient.from_cassette(path) reads a frozen JSONL file with zero network and zero credentials. RealMCPClient.from_stdio(cmd, args, timeout=) spawns a real MCP binary and speaks JSON-RPC 2.0 per the 2025-03-26 spec, with ATW_RECORD=1 to record live cassettes.	library only	stable since 003
Graph	Causal graph as PROV-DM JSON-LD (Entity / Activity / Agent + wasGeneratedBy / used / wasAssociatedWith). Canonical, deterministic, interoperable with other PROV tools.	witness graph	stable since 002
Verify	Check a graph against a seal: every observed tool must be in the seal, every seal's tool must be in the observed set, scopes match, no seal tampering.	witness verify	stable since 002
Replay	Counterfactual replay — replay(graph, {remove: URI}) removes a node and returns the transitive compensation set, a synergy_residual proxy for HANSARD mechanism 5, and a not_replayable flag for actions that can't be undone. Deterministic: 10 runs produce byte-identical JSON.	witness replay	stable since 002
Key management (Q1)	Generate, store, rotate, and revoke HMAC keys via witness keygen/rotate-key/revoke-key/list-keys. Default keyring file is 0600 on POSIX. Ed25519 (multi-witness distributed verification) is feature 005, not in 0.1.0.	witness keygen etc.	stable since 004 (merged 2026-09-01)

Architecture in one diagram

  agent spec (JSON)  ────►  witness seal  ────►  signed seal (JSON)
                                                  │
                                                  ▼
  MCP server  ──►  RealMCPClient  ──►  witness capture  ──►  events.jsonl
  (cassette or                                                       │
   live stdio)                                                        ▼
                                                       witness graph  ──►  graph.jsonld (PROV-DM)
                                                                          │
                                            ┌─────────────────────────────┤
                                            ▼                             ▼
                                  witness verify (anomalies)    witness replay (counterfactual)

The capture transport is intentionally pluggable. The same record_tool_call, record_tool_response, record_model_input, record_model_output, and record_external_effect functions work with both a from_cassette and a from_stdio client, so you can develop against a frozen JSONL file and switch to a live MCP server without changing the capture layer.

How the seal actually works

The seal is a flat JSON object with a canonical-body HMAC-SHA256 signature. The body is exactly four fields — created_at, system_prompt_sha256, tools, witness_id — sorted, separator-stripped, UTF-8 encoded, then HMAC'd. The signature field is "hmac-sha256:<hex>". Any change to any of the four body fields invalidates the signature.

Why a canonical encoding? Because JSON has multiple ways to represent the same value, and any ambiguity in canonicalization is a place for an attacker to slip a "semantically identical, byte-different" payload past a naive verifier. The test suite contains a regression test that recomputes the HMAC byte-by-byte against a known-fixture seal (tests/fixtures/seal_without_damaging_tool.json, signature dc91ea105843ada26bedb76388116820c12ff4e81f276328ab20a691a182996a) and asserts equality with hmac.compare_digest for constant-time comparison.

The keyring sits in front of the seal. witness keygen produces a 32-byte HMAC key, writes it to keys.json with mode 0600 on POSIX, and registers it as the active entry. The verifier uses the key in keys.json to check the signature. Keys can be rotated via witness rotate-key (old key kept for v1-fixture backward compatibility) and revoked via witness revoke-key (kept in the file but excluded from verification).

The HMAC body does not include the key_id. A SealedSeal written before feature 004 has no key_id; a verifier with a keyring tries every non-revoked key until one verifies (try-all v1). A SealedSeal written after feature 004 has a key_id; the verifier looks up that exact entry (exact-match v2). This keeps the 001-era fixtures and pipelines working without any migration, while letting new seals carry an explicit key selector. Ed25519 will replace HMAC in feature 005 to enable the multi-witness quorum that HMAC's symmetric-key design can't do.

How the live MCP capture works

The live stdio transport is a strict subset of the MCP spec (2025-03-26). RealMCPClient.from_stdio(cmd, args, timeout=2.0) spawns the child process with subprocess.Popen(..., shell=False, ...), sets explicit timeouts, and on close sends requests.CancelNotification first, then notifications/cancelled if no ack, and finally Process.terminate followed by Process.wait(timeout). The protocol state machine implements the four mandatory handshake methods (initialize, notifications/initialized, tools/list, tools/call) and rejects unknown methods with JSON-RPC error -32601. The ATW_RECORD=1 env var records every line in and out of the child to a JSONL file that's loadable with from_cassette, so a test recorded once on a developer's machine is reproducible byte-for-byte by every CI run.

A test (test_live_stdio_determinism.py) makes 10 sequential calls and asserts byte-identical event tuples. A separate test runs the spec-conformance handshake against a self-built MCP stub server and checks method-by-method against the 2025-03-26 spec text. These are not vacuous tests — they were the catch for a real bug (an unhandled notification that hung the client indefinitely; the timeout finally fired but the child had no graceful-shutdown path).

Replay: counterfactual analysis

HANSARD's mechanism 4 is "what if". Given a graph and a node you want to remove (a tool call that happened but shouldn't have, an external effect that was wrong), the replay returns:

• compensation_set: the transitive closure of every other action that depended on the removed one (BFS over used / wasGeneratedBy edges).
• synergy_residual: a proxy for HANSARD mechanism 5 — the ratio of the compensation set to the original action set. A small synergy_residual means the system absorbed the wrong action cleanly; a large one means it compounded.
• not_replayable: a list of actions that can't be undone (e.g. an external_effect of type delete is technically reversible, but the witness refuses to claim it can re-create what was deleted).

The replay is pure: it doesn't touch the agent or the MCP server, it operates on the graph. That means the same graph can be replayed under many counterfactuals to compare outcomes, without re-running the agent. Determinism is enforced by 10-run byte-equality tests.

What's honestly open

The repo is alpha (Development Status :: 3 - Alpha in the pyproject.toml classifiers). Concretely:

• No Ed25519 yet. Distributed verification with independent verifiers (HANSARD multi-witness quorum) requires a public-key signature scheme, so HMAC is structurally insufficient. Ed25519 is feature 005, planned.
• No live observability. The witness only sees events after the protocol boundary; it can't see what the agent was thinking. For that you'd want something like BekchiAI (arXiv:2608.26867) or a similar runtime-introspection tool. The witness's output (the PROV-DM graph) is exactly what those systems would consume.
• No contract-level prevention. The witness doesn't refuse tool calls; it just records them and verifies after the fact. If you need call-time policy enforcement, TraceGrant (arXiv:2608.21126) is the complementary tool.
• No Streamable HTTP transport yet. The MCP transport is stdio only. The HTTP transport is feature 005, planned.
• 164 tests pass, 1 skipped, on a single Python 3.12 + Linux box. Test coverage is 69.67% (the gap is cli.py at 14%, which is exercised via subprocess.run and not captured by line-level coverage of the import graph). No CI runs the suite automatically yet; pyproject.toml declares a fail_under = 80 gate that the local run does not pass.

The repo is a working 0.1.0, not a finished product. Treat the seal as a real primitive (the HMAC code is small and easy to audit) and the rest as in-progress scaffolding that you should read before relying on.

Auditability

Three real bugs were caught by reviewers before the relevant code merged to main, and the test suite contains regression tests for each:

• Atomicity of rotate_key: the first version deactivated the old key before generating the new one. If the new-key generation failed (timestamp collision, in this case) the keyring was left with zero active keys, and witness seal would have been unable to sign anything until manual intervention. Fixed by moving the deactivation inside the if block, so a failed rotation leaves the keyring unchanged. Test: test_rotate_failure_is_atomic forces a guaranteed collision and asserts e1.active is True and kr.active_key() is e1 after the RuntimeError.
• AssertionError leaking from sign_seal: passing an empty keyring to sign_seal(keyring=...) raised a raw AssertionError from Keyring.active_key(), which the rest of the module never raises — the rest uses WitnessKeyError and WitnessSealError. Fixed by translating the AssertionError to a WitnessKeyError with a hint that says witness keygen to generate one. Test: test_empty_keyring_raises_witness_key_error asserts the type and the message content.
• key_id timestamp collision: the first version used a timestamp at second granularity, which collides under stress (e.g. two consecutive rotate-key calls within the same second produce the same key_id). Fixed by switching to microsecond precision, with a 3-attempt retry as a residual network. Test: 10 consecutive rotate-key calls in a tight loop all produce unique key_ids.

The chmod-on-save issue (keys.json was created with mode 0644 instead of the 0600 the docstring promised) was caught by an independent audit after the merge and is fixed in fix/keyring-permissions-and-docs (PR #2). The audit's review process is documented in the repo's commit history.

Try it

python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
witness --help

# One-time: generate an HMAC key and lock the file to 0600.
witness keygen -o keys.json

# Seal an agent spec, then verify the signature, capture events,
# build a graph, and replay a counterfactual.
witness seal    --spec examples/agent_spec.json --out /tmp/seal.json
witness capture --scenario examples/scenario.json --seal /tmp/seal.json --out /tmp/events.jsonl
witness graph   --events /tmp/events.jsonl --seal /tmp/seal.json --out /tmp/graph.jsonld
witness verify  --graph /tmp/graph.jsonld --seal /tmp/seal.json
witness replay  --graph /tmp/graph.jsonld --seal /tmp/seal.json --counterfactual '{"remove":"atw:activity/tool_call_1"}' --out /tmp/replay.json

The full repository, including the constitution (C1–C8), the per-feature specs in the Obsidian vault, and the audit history (KNOWN_ISSUES.md), is at the link below.

References

• HANSARD: A Framework for Post-Execution Forensics in Autonomous AI Systems (arXiv:2608.22512) — the framework this repo implements mechanisms 1–3 of.
• MCP 2025-03-26 specification — the transport protocol the stdio client implements.
• PROV-DM: The PROV Data Model (W3C Recommendation) — the format of the causal graph output.
• BekchiAI: Live Observability and Remote Termination for Agentic Systems (arXiv:2608.26867) — complementary, not implemented here.
• TraceGrant: Contract-Governed Prevention for Autonomous Agents (arXiv:2608.21126) — complementary, not implemented here.
• CTF-ABACUS: Solve Profiles for Multi-Agent Forensics (arXiv:2608.26237) — complementary, not implemented here.

Links

• Repository: https://github.com/amurlaniakea/agent-trace-witness
• License: AGPL-3.0-or-later

---

Author: Pedro Sordo Martínez ([e-mail]). The repo is alpha; the seal primitive is stable enough to audit, the rest is in-progress.