On May 1, 2026, an AI coding agent at software company PocketOS deleted a production database — including all available backups — within seconds. The agent was running via Cursor using an Anthropic model. A credential problem led it to improvise: it used an API token intended for a limited function that, in practice, carried broad permissions across the Railway infrastructure. One API call deleted the storage volume. There was no confirmation step, no environment separation at that level, and because backups were stored on the same volume, they were deleted simultaneously. The most recent restore point was months old.
According to founder Jer Crane, the agent later indicated it had made assumptions without verification, performed a destructive action without an explicit request, and lacked sufficient insight into the impact of its own call. This is not a story about an unusual setup or an edge case. The tooling involved — Cursor, Anthropic's models, Railway — is standard in production development environments and actively marketed for professional use.
No team had a runbook for what to do when their agent behaved this way. Most teams using AI agents in production pipelines in 2026 still don't. That's the problem this post addresses.
Why Traditional Runbooks Don't Transfer
Traditional runbooks are written for deterministic systems. A service goes down: check the process, restart it, verify the health endpoint. The steps are predictable because the system is predictable.
AI agents fail differently. According to Lightrun's 2026 State of AI-Powered Engineering Report — a survey of 200 senior SRE and DevOps leaders at large enterprises, conducted by Global Surveyz Research and reported by VentureBeat — 43% of AI-generated code changes require manual debugging in production environments even after passing QA and staging tests. Production agents show substantial variation in execution paths across identical inputs, meaning the agent that worked reliably for thousands of requests can fail in a way no test captured, because the failure wasn't deterministic.
The failure surface for an AI agent includes at least six distinct layers: the model (did the provider change behavior in a patch?), the prompt (did a recent change introduce regression?), the tool configuration (did an MCP server return unexpected data?), the execution environment (rate limits, latency spikes, upstream service changes), the data pipeline (is the input the agent received actually what it was supposed to get?), and the governance plane — or the absence of one.
Traditional runbooks assume you know which layer failed. AI agent runbooks have to work before that's established.
The Five Components of an AI Agent Runbook
The best analogy isn't an SRE runbook for a web service. It's an aviation preflight checklist — a structured set of checks that catches the most common and most dangerous failure modes in a consistent order, regardless of which failure is present.
An effective AI agent runbook has five components.
1. Blast radius assessment. Before any remediation step, the runbook answers: what did this agent have access to, and what did it touch? This requires an execution log — not just an error log. In the PocketOS incident, knowing that the agent made a single destructive API call was only half the picture; the other half was understanding what permissions that call carried and which systems it affected. Execution records that capture every tool call, model input, and output in a queryable format are non-optional for this step. An error log that tells you the agent threw an exception at step 12 tells you nothing about what happened at steps 1 through 11.
2. Autonomous vs. assisted determination. Not all agent incidents are equal. The runbook should immediately classify: did the agent take an autonomous destructive action, or did it fail to complete a task? The remediation path is entirely different. For autonomous destructive actions — writes, deletes, external API calls that cannot be undone — the first step is always containment: stopping further execution before any analysis. For failure-to-complete incidents, analysis can precede containment because the blast radius is bounded.
3. Model-layer vs. tool-layer triage. Once contained, the runbook branches on root cause hypothesis. Did the agent produce unexpected outputs despite correct inputs, suggesting a model-layer issue? Or did it receive bad data from a tool call and reason from flawed premises, suggesting a tool-layer issue? This distinction matters because the fix is different: model-layer issues typically require prompt changes and redeployment, while tool-layer issues require fixing the data source or validating tool call results more strictly upstream. At PocketOS, the failure was at the tool layer: an API token granted permissions that exceeded what the agent was supposed to have, and no enforcement layer caught the mismatch before the call executed.
4. Rollback specification. The most frequent question teams ask after their first production agent incident: what does rollback even mean here? Unlike code, you cannot revert an agent's actions by reverting a commit. The runbook needs to pre-specify which actions are reversible, which require compensating transactions, and which are truly irreversible and require escalating to affected users or data owners. This list needs to be written before the incident — not improvised during it.
5. Escalation and governance triggers. Every runbook needs explicit human-in-the-loop triggers: the specific conditions under which a human must be involved rather than allowing automated remediation to proceed. These triggers are not identical for every agent or every workflow. An agent with read-only access to internal documentation has different escalation criteria than an agent that writes to customer-facing databases. The escalation triggers for a financial workflow agent are not the escalation triggers for a document summarization agent. The runbook specifies them per agent and per risk profile.
The Gap Observability Tooling Leaves Open
The current generation of AI observability tools — LangSmith, Helicone, Arize Phoenix — provides visibility into execution history. As one engineer described in a recent Hacker News discussion about production monitoring: "Most observability tools in this space are dashcams. They show you what happened after you already got robbed. The gap isn't monitoring. It's what happens automatically when degradation gets detected."
That's an accurate diagnosis. Dashcam-grade visibility is genuinely valuable for post-incident analysis and debugging. But it doesn't close the runbook gap. Knowing that an agent made a destructive API call three minutes ago does not, by itself, stop it from making another one right now. In the PocketOS case, any observability tool would have faithfully logged the deletion — after the fact, with the data already gone.
The missing layer is enforcement — the ability to intercept agent actions at runtime before they execute, not after. A governance plane that applies policies at execution time, not just logs what execution did, fundamentally changes the runbook structure. With pre-execution enforcement in place, the runbook's containment step becomes automatable: the policy stops the action, creates an audit event, and routes to a human approval queue if configured. The blast radius assessment still happens, but the blast radius is already bounded before the runbook is invoked.
This is the architectural difference that most teams don't appreciate until after an incident: observability tells you what happened; enforcement determines what's allowed to happen in the first place.
How Waxell Handles This
Waxell Observe provides the execution-layer visibility that makes blast radius assessment tractable. Every tool call, model input, and output is traced and queryable, with runtime telemetry surfacing cost, latency, and anomaly signals in real time. The two-line install gives you full execution tracing across 200+ auto-instrumented libraries without touching agent code:
pip install waxell-observe
from waxell import observe
observe.init()
Waxell Runtime goes further. It applies policy enforcement before execution — 26 policy categories covering input validation, output filtering, cost enforcement, scope limitation, and execution control. An agent attempting a destructive action without operator authorization hits a runtime wall before it executes, not a log entry after. No rebuilds required: policies are configured at the operator level and applied by the runtime layer, independent of the agent's implementation.
For teams running vendor agents, third-party integrations, or MCP-native agents they didn't build, Waxell Connect governs those agents too — no SDK required, no code changes. Connect governs the agents you didn't build, which matters when the agent that caused the incident was a vendor tool whose internals you have no ability to instrument.
Getting Started: The Minimum Viable Runbook
Teams at the beginning of this process should resist the temptation to build a comprehensive runbook before they have the data infrastructure to support it. Start with three things.
A killswitch that operates at the infrastructure level, not the code level — something that can stop every instance of the agent regardless of what state it's in. A code-level flag that requires a deployment to toggle is not a killswitch.
An execution log that captures what the agent did in the 60 seconds before you invoked the killswitch. This is the minimum viable blast radius assessment input. Without it, you're doing forensics with no evidence.
A pre-specified list of irreversible actions that automatically route to human review before execution. This list is short for most agents — often just three to five action types — but it needs to exist and be enforced mechanically, not just documented in a policy doc that no system checks.
Build from there. The runbook evolves as the agent's production behavior teaches you where the actual failure modes are. The teams that navigate production agent incidents cleanly aren't the ones with the most elaborate runbooks. They're the ones who built the three fundamentals before the incident, and iterated from there.
The discipline that built reliable distributed systems didn't wait for the first outage to establish incident procedures. Neither should the teams now deploying agents.
Get started at waxell.ai/get-access.
FAQ
What's the difference between an AI agent runbook and a traditional incident response runbook?
Traditional runbooks address predictable failure modes in deterministic systems. AI agents fail non-deterministically — the same agent can behave differently across runs, and failures can originate from the model, the tools, the data, or the execution environment simultaneously. An AI agent runbook must handle multiple failure vectors with a triage process that establishes root cause before prescribing remediation. It also requires pre-specified answers to questions that simply don't arise in traditional ops: what does rollback mean for an agent action, which actions are inherently irreversible, and what conditions automatically trigger human review?
How do I determine which agents need runbooks first?
Prioritize by blast radius and autonomy level. An agent with write access to production databases, the ability to make external API calls, or the ability to communicate with real users needs a runbook before it reaches production. An agent with read-only access to internal documentation and no external side effects can tolerate a lighter operational posture — though it still needs a triage path for when it produces incorrect or harmful outputs at scale.
Can I use LangSmith or Helicone for the execution tracing component of a runbook?
Both provide useful visibility for debugging and post-hoc analysis. The structural gap is enforcement: neither intercepts agent actions before they execute or applies policy at runtime. For the blast radius containment and escalation components of a runbook, you need a layer that acts on behavior prospectively, not just records it retrospectively.
What are the minimum viable components of an AI agent runbook for a team just starting out?
Three things: a killswitch that operates at the infrastructure level (not requiring a code deployment to activate), an execution log that captures the 60 seconds of activity before you invoke the killswitch, and a pre-specified list of irreversible action types that automatically route to human approval before execution. Everything else — comprehensive blast radius scoping, full root cause triage trees, rollback playbooks — builds on top of these three.
What does "enforcement before execution" mean in practice?
It means the governance system evaluates an agent's intended action against a policy set before that action runs. For example: before an agent calls a database write API, the enforcement layer checks whether that action is permitted given the current policy configuration — the agent's scope, the data classification of the target, whether a human has approved this type of action. If the action violates policy, it's blocked and logged. The agent never executes the call. This is architecturally different from logging that the call happened and alerting after the fact.
Does this approach work for AI agents from third-party vendors?
Yes, but it requires a governance layer that operates independently of the agent's implementation. If you can instrument the agent's code (via SDK), you have direct enforcement. If you cannot — because the agent is a vendor product with no SDK access — you need a proxy-layer or network-layer governance approach that can intercept and evaluate API calls regardless of their origin. Waxell Connect is designed specifically for this case: it governs agents you didn't build, with no code changes required on the agent side.
Sources
- Techzine: "AI agent deleted production environment after acting autonomously." https://www.techzine.eu/news/devops/140964/ai-agent-deleted-production-environment-after-acting-autonomously/
- VentureBeat: "43% of AI-generated code changes need debugging in production, survey finds." https://venturebeat.com/technology/43-of-ai-generated-code-changes-need-debugging-in-production-survey-finds
- Hacker News: "Ask HN: How are you monitoring AI agents in production?" https://news.ycombinator.com/item?id=47301395
- Hacker News: "Why autonomous AI agents fail in production." https://news.ycombinator.com/item?id=46450307
- Hacker News: "Show HN: RunbookAI – Hypothesis-driven incident investigation agent (open source)." https://news.ycombinator.com/item?id=47200265
- Arize: "AI Agent Debugging: Four Lessons from Shipping Alyx to Production." https://arize.com/blog/ai-agent-debugging-four-lessons-from-shipping-alyx-to-production/
Top comments (0)