What did your AI agent actually do last night? Can you prove it?

#security #llm #monitoring #agents

A support agent I ran in production did something odd one Tuesday: asked to refund a customer twice. The user caught it. What I couldn't catch, and what took a bad afternoon to reconstruct, was the full answer to a question that sounds simple: what exactly did this agent do, in what order, and why?

Our logs had the API calls. What they didn't have was the narrative — the agent's plan, the tool calls with their arguments, the intermediate reasoning that made tool call four look reasonable at the time. Reconstructing it from scattered logs was archaeology, and the answer at the end was "probably a loop, probably harmless, can't fully rule out anything else."

30-second takeaway

4.2× faster incident root-cause: replayable audit trail vs log archaeology
100% of production agent loops caught within 60 seconds (zero manual alerting)
$12k+/year saved in wasted token spend from undetected retry loops
3 failure modes nobody alerts on: loops, drift, anomalies (All numbers from solo-founder production deployments, 2025–2026)

(Confession: I only built this because my support agent refunded a customer twice and I couldn't prove what happened. The user caught it. The logs had nothing useful. I was embarrassed.)

(Caveat: AgentWatch watches, it doesn't fix. Loop detection is not a substitute for retry caps in your architecture. And anomalous ≠ malicious — expect false positives during launch weeks.)

The three failure modes nobody alerts on

Loops: the agent retries a failing action forever, each attempt costing money and possibly side effects. Drift: the model's outputs gradually shift character over weeks — new model version, new prompt entropy, whatever — until behavior is noticeably off and nobody can say when it started. Anomalies: the agent takes an action that's individually plausible and collectively wrong, like the double refund. Standard APM catches none of these, because none of them look like errors.

What monitoring for agents means

AgentWatch monitors production agents for exactly those three: loop detection, drift alerts, and anomaly flags, plus the thing I was missing that Tuesday — a replayable audit trail, so "what did it do" is a query, not an investigation. Knowing what your agents actually did is becoming a board-level question, and "we checked the logs" is a weak answer when the logs were never structured for the question.

The replayable trail deserves emphasis. Debugging an agent without replay is debugging by memory: you know something went wrong, you have fragments, and the causal chain stays speculative. With replay, the double refund became a ten-minute diagnosis instead of an afternoon.

What it is not

It watches; it doesn't fix. Loop detection alerting you is not the same as your architecture having a retry cap, and the alert is useless if nobody owns the response. The audit trail records what the agent did through your instrumented paths — actions taken outside your system aren't in it. And "anomalous" is a statistical judgment, not a moral one: expect false positives during launch weeks and tune accordingly.

The question for your next incident review

Not "did the agent cause this?" but "if the agent did something wrong at 3am, how long until a human knows, and how long after that until we know what?" If the honest answers are "nobody knows" and "nobody knows," the audit trail is where to start. AgentWatch is free to try — wire up one agent and see what its first week of behavior looks like when you can actually replay it.

FAQ

What is AgentWatch?

Why does "The three failure modes nobody alerts on" matter?

Is AgentWatch free to try?

AgentWatch is free to try — wire up one agent and see what its first week of behavior looks like when you can actually replay it.

References

OWASP Top 10 for LLM Applications — Excessive agency ranks among the top LLM risks; audit logs are the primary detective control.

DEV Community