I built an open-source audit trail for AI agents (after mine silently failed for hours)

The problem

I was running a multi-agent pipeline and one of my agents silently failed. The only alert I got said "daily loss limit reached" — completely misleading. The real cause was a missing file the agent never reported.

I had zero visibility into what any agent had actually done.

What I built

AgentLens — a Python SDK for AI agent governance. Three modules:

• Audit trail — every LLM call and tool use logged to SQLite automatically
• Authorization — policy-based gates so agents can only call what you've approved
• Anomaly detection — baseline + threshold config, alerts when behavior drifts

One-line integration

Drop-in for Anthropic:

python
from agentlens.integrations.anthropic import TracedAnthropic

client = TracedAnthropic(agent_id="my-agent")
response = client.messages.create(...)  # auto-traced
## Install

pip install llmsentinel

Comments

[Zira]

This is the right failure to instrument, but I’d make the audit record a replayable event rather than just a trace line. At minimum I’d persist run_id and parent_run_id, agent/runtime identity, model and tool name, normalized arguments or an args hash, policy version, approval decision, start/end timestamps, and an explicit outcome such as SUCCESS, FAILED, NOT_SENT, or UNKNOWN. That makes “the agent stopped reporting” distinguishable from “the tool side effect happened but the journal write was lost.”

The failure fixtures I’d add are: process kill after a tool side effect but before the SQLite commit, SQLite lock or disk-full during flush, a retry after an UNKNOWN result, clock skew between workers, and a policy change between planning and execution. Then test whether restart reconciliation uses the run ID or idempotency key instead of blindly replaying the last call. For anomaly detection, separate behavior drift from infrastructure noise such as model/API errors and missing telemetry, or the detector may alert on the observer failing rather than the agent.

One small reproducibility check: the article names the project AgentLens but the install example says pip install llmsentinel; aligning the package/import names, or documenting the relationship, would make the one-line integration easier to verify.