DEV Community

willamhou
willamhou

Posted on

5 things missing from your AI agent audit logs (and how we fixed them in Signet v0.10)

#ai #security #agents #compliance

TL;DR — If your AI agent audit log only signs the intent (tool name + args), you're shipping demo-ware. Real audit needs 5 things most projects skip: outcome binding inside the signature scope, durable nonce stores, persistent server identity, portable forensic bundles, and encrypted-but-verifiable payloads. v0.10 of Signet ships all five today.

The gap between "demo passes" and "compliance team approves"

I've spent the last few months building Signet — an Ed25519-signed cryptographic audit SDK for AI agents. Along the way I noticed a pattern across every "let's add cryptographic audit to AI agents" project I looked at:

  1. Generate a keypair.
  2. Agent calls a tool.
  3. Sign {tool, params} with Ed25519.
  4. Stuff the signature into the audit log.
  5. npm test passes. Demo done.

That's a working prototype. It is not what a compliance team can sign off on.

Below are the five things v0.9 of Signet did not have, that v0.10 (shipped today) does. If you're building anything in this space, I think you have to answer these before claiming production-ready.

1. Outcome binding — sign what happened, not just what was requested

Most agent receipts sign the request: {tool: "delete_user", params: {id: 42}}. The receipt becomes a record that the agent intended to call that tool.

But here's the audit question that actually matters: what did the server actually do? Did it succeed? Did it reject the call (policy violation)? Did it crash mid-flight? An intent-only receipt can't tell you.

In v0.10, Signet's bilateral receipt (v3) embeds an Outcome field with one of four states — verified, rejected, executed, failed — inside the signature scope. If a server claims it logged "executed" but the agent observed "failed", the receipt's signature is invalid.

// Rust: signing a bilateral receipt with outcome
let receipt = sign_bilateral_with_outcome(
    &server_key,
    &agent_receipt,
    &response,
    Outcome {
        status: OutcomeStatus::Executed,
        reason: None,
        error: None,
    },
)?;

// The status field is now under the signature.
// Tampering with it after the fact breaks verification.
Enter fullscreen mode Exit fullscreen mode 
# Python — same shape
agent.sign_bilateral_with_outcome(
    agent_receipt=receipt,
    response={...},
    outcome={"status": "executed"},
)
Enter fullscreen mode Exit fullscreen mode

This is small in code but big in semantics. The receipt becomes a record of what actually happened, not just what was requested.

2. Durable nonce store — replay protection that survives a restart

Replay attacks against signed receipts are well-known. The standard mitigation: track nonces, reject any receipt whose nonce you've seen before.

The catch: most implementations use an in-memory hash set. Process restarts? Set is empty. Replay protection: gone for the duration of the warm-up window.

v0.10 ships FileNonceChecker — a JSON-file-backed nonce store, single-host pilot grade. It survives process restarts:

let checker = FileNonceChecker::new(Path::new("/var/lib/signet/nonces.json"))?;
let opts = BilateralVerifyOptions::default()  // now includes replay check by default
    .with_nonce_checker(Box::new(checker));

verify_bilateral(&receipt, &server_pubkey, &opts)?;
Enter fullscreen mode Exit fullscreen mode 
# Python equivalent
agent.verify_bilateral(
    receipt=receipt,
    server_pubkey=pubkey,
    nonce_store="/var/lib/signet/nonces.json",
)
Enter fullscreen mode Exit fullscreen mode 
// TypeScript — packages/signet-mcp-server FileNonceCache
import { FileNonceCache } from '@signet-auth/mcp-server';
const cache = new FileNonceCache('/var/lib/signet/nonces.json');
verifyRequest(req, { nonceCache: cache });
Enter fullscreen mode Exit fullscreen mode

Single-host only — distributed nonce stores (Redis-backed, etc.) are post-1.0. But "single-host pilot" is what most teams need first, and v0.9 didn't even have that.

Behavioral break to note: BilateralVerifyOptions::default() now enables in-memory replay protection by default. Use BilateralVerifyOptions::insecure_no_replay_check() for forensic replay flows where nonce reuse is expected.

3. Persistent server identity — your trust bundle finally has something to anchor

Bilateral signing means both the agent and the server sign the receipt. The server's public key needs to be stable — otherwise nothing can pin it. Trust bundles, allowlists, audit replay — all break the moment the server pubkey rotates.

In v0.9, the server keypair was ephemeral: generated on signet proxy startup. New process, new pubkey. Compliance teams loved that.

In v0.10:

# Generate identities — agent and server must be different keys
signet identity generate --name agent-prod
signet identity generate --name openclaw-gateway

# Use them persistently — same pubkeys across every restart
signet proxy \
  --target ./my_mcp_server \
  --key agent-prod \
  --server-key openclaw-gateway

# Compose the trust bundle by hand — CLI bundle creation is on the roadmap;
# `signet trust` today exposes inspect / list / disable / revoke / rotate for
# editing existing bundles. Full schema lives in docs/guides/team-deployment.md.
cat > trust.json <<JSON
{
  "version": 1,
  "bundle_id": "pilot-2026-Q2",
  "org": "your-org",
  "env": "pilot",
  "generated_at": "2026-05-11T00:00:00Z",
  "agents": [{
    "id": "agent-prod-2026-05",
    "name": "agent-prod",
    "owner": "you",
    "pubkey": "$(signet identity export --name agent-prod)",
    "status": "active",
    "created_at": "2026-05-11T00:00:00Z"
  }],
  "servers": [{
    "id": "openclaw-gateway-2026-05",
    "name": "openclaw-gateway",
    "owner": "you",
    "pubkey": "$(signet identity export --name openclaw-gateway)",
    "status": "active",
    "created_at": "2026-05-11T00:00:00Z"
  }],
  "roots": []
}
JSON

# Sanity-check the bundle parses
signet trust inspect ./trust.json
Enter fullscreen mode Exit fullscreen mode

The CLI also refuses to use the same key for both agent and server roles — a common foot-gun that silently invalidates the entire bilateral protocol.

4. Forensic bundle / restore — the artifact compliance teams actually want

A compliance team's question is rarely "is this audit log valid right now on this machine?" It's usually:

"Six months from now, on a different machine, with no access to your keystore, can you prove that this set of audit records was signed by [these specific keys] and that nothing was tampered with?"

That requires the audit data + the keys + the hash chain proof + version metadata, packaged together, self-verifying.

v0.10 ships:

# Producer side — package up an evidence bundle
signet audit --bundle ./evidence-2026-Q2 \
  --include-trust-bundle ./trust.json

# Output:
#   evidence-2026-Q2/
#   ├── records.jsonl       # the audit records
#   ├── manifest.json       # version + chain root + signer pubkeys
#   ├── hash-summary.txt    # human-readable chain summary
#   └── trust-bundle.json   # (optional) signer key set

# Verifier side — re-verify on ANY machine, no keystore required
signet audit --restore ./evidence-2026-Q2
# Verifies:
#   - every receipt's signature
#   - hash chain integrity
#   - timestamps in expected window
#   - trust bundle attestation chain
Enter fullscreen mode Exit fullscreen mode

The --restore flow is replay-tolerant by design: it uses BilateralVerifyOptions::forensic() so re-verifying the same receipt twice doesn't fail nonce-replay checks. Forensic verification is a different mode from live verification, and v0.10 makes that explicit instead of confusing the two.

5. Encrypted audit envelope — verifiable AND confidential

Audit logs that are cryptographically verifiable are great for forensics. Audit logs that contain customer data in plaintext are great for incident response. These two goods are usually in tension.

v0.10 wraps the params field of audit records in an XChaCha20-Poly1305 encrypted envelope:

# When signing, opt the audit record into encrypted params
signet sign \
  --tool send_email \
  --params '{"to": "alice@example.com"}' \
  --key agent-prod \
  --encrypt-params

# Forensic decrypt during export — uses the matching local identity from
# the keystore (no separate "encryption-only" key flag today)
signet audit --export ./audit-decoded.jsonl --decrypt-params
Enter fullscreen mode Exit fullscreen mode

The signature still covers the encrypted ciphertext — meaning verifiability is preserved without ever exposing the plaintext at rest. Anyone holding the audit log without the keystore sees only ciphertext; the --decrypt-params export materialises plaintext into the export file, scoped to that one operation.

v0.10 limitation: --encrypt-params reuses the signing agent's identity to derive the envelope key, so today "decrypt access" and "sign access" share one key. A dedicated encryption-only identity is on the roadmap; if you need that separation today, please open an issue.

This is the thing your security team has been asking for since you turned audit on.

Putting it together — a pilot deployment

Here's roughly what a single-host Signet pilot looks like with all 5 in play:

# 1. Generate identities (agent and server must be different keys)
signet identity generate --name agent-prod
signet identity generate --name openclaw-gateway   # stable server key

# 2. Compose the trust bundle JSON by hand
#    (paste pubkeys from `signet identity export --name <NAME>` — see Section 3)
$EDITOR ./trust.json
signet trust inspect ./trust.json   # verify it parses

# 3. Run the proxy with persistent agent + server identities + a policy
signet proxy \
  --target ./my_mcp_server \
  --key agent-prod \
  --server-key openclaw-gateway \
  --policy /etc/signet/policy.yaml

# 4. On the verifier side (same host or compliance host) — durable nonce store
#    is a `signet verify` flag, not a proxy flag in v0.10
signet verify <receipt-path> --nonce-store /var/lib/signet/nonces.json

# 5. Periodically bundle audit evidence for off-host handoff
signet audit --bundle ./evidence-2026-05-14 --include-trust-bundle ./trust.json

# 6. Compliance team can verify on a different machine
scp -r ./evidence-2026-05-14 compliance-host:
ssh compliance-host
signet audit --restore ./evidence-2026-05-14
# ✓ all signatures valid
# ✓ hash chain intact
# ✓ trust bundle attestation chain valid
Enter fullscreen mode Exit fullscreen mode

Full operator runbook: docs/guides/team-deployment.md.

What's not in v0.10 (and where the work is going)

Honest disclaimers:

  • Single-host only. Distributed nonce stores (Redis, FoundationDB-backed) are post-1.0.
  • No managed key rotation. You can rotate manually with new identities and bundle merges, but there's no operator-friendly flow yet.
  • No multi-tenant isolation. Each pilot is a single-tenant deployment.
  • BilateralVerifyOptions::default() is a behavioral break. It now defaults to in-memory replay protection. If you were calling verify_bilateral() repeatedly on the same receipt for forensic replay, you need insecure_no_replay_check().
  • OpenClaw plugin is fail-closed by default. That means a misconfigured Signet will block all tool calls, not silently allow them. This is the right default but trips people up — see the pilot runbook for the readiness signal flow.

Try it 

# Python
pip install signet-auth==0.10.0

# Rust
cargo install signet-cli

# OpenClaw gateway plugin
openclaw plugins install @signet-auth/openclaw-plugin
Enter fullscreen mode Exit fullscreen mode

Repo: github.com/Prismer-AI/signet
v0.10 release notes: v0.10.0 release
Compliance mapping (SOC 2 / ISO 27001 / EU AI Act / NIST AI RMF): COMPLIANCE.md

If your AI agent audit log is missing any of these five things, I'd love to hear which one bites first in your environment. The fastest way to shape v0.11 is to tell me what doesn't work for you.

Top comments (0)