When an AI Agent Goes Wrong: A 16-Case First-Hour Response Runbook

An AI agent has made a real change, and the change is wrong, unsafe, or impossible to explain. It may have opened a customer ticket with invented facts, edited the wrong repository, sent a message after a human took over, or continued retrying after an operator pressed stop. The immediate task is not to decide whether “the AI” is good or bad. It is to stop further effects, establish what happened, and recover the business process without erasing evidence or repeating the action.

This runbook provides an incident lead with a first-hour sequence, a sixteen-case tabletop matrix, a minimal evidence packet, and a runnable triage helper. It applies to coding, support, operations, sales, finance, and internal workflow agents that can call tools or change external state. For background on business agent use, see this AI agents for business guide; the procedures here focus on response after an action becomes uncertain.

The decision is simple: freeze the smallest unit that can still cause harm, preserve the execution trace, inspect the external system of record, and resume only from a verified state with a named human owner. Do not assume a green model response means the side effect succeeded, that a “cancel” button ended a tool call, or that replaying a request is safe. The agent’s narrative is one artifact; the destination system’s event record is another. Keep both, and do not merge them until their relationship is established.

First, define the incident boundary

For this runbook, an incident is an unexpected or unauthorized change, attempted change, disclosure, or continued execution involving an agent and one or more tools. A harmless wrong answer can be a quality defect. It becomes an operational incident when it affects a customer, a record, a production system, a payment, access, data confidentiality, or a workflow that other people depend on.

Not every agent defect is a security incident, and not every operational incident is a personal-data breach. The European Data Protection Board defines a personal data breach around accidental or unlawful destruction, loss, alteration, unauthorized disclosure of, or access to personal data. Its guidance explains that some qualifying breaches must be notified to a supervisory authority within 72 hours, while others require communication to affected people; the obligation depends on facts and risk. Preserve that decision for the privacy or legal owner. Do not call every bot outage a reportable breach, and do not let uncertainty about classification delay containment. EDPB: personal-data breaches

An agent chain creates more moving parts than a single request: model and prompt version, retrieved context, memory, delegated agents, credentials, tool adapters, retries, queues, and the application receiving the action. A 2026 research paper on AI-agent incident reporting argues that reports need autonomy level, memory and memory access, tools, messages, versions, and intermediate states. It also highlights privacy and reporting-system risks. The paper is a proposal based on expert input, not an adopted universal reporting standard. This article’s evidence model follows that practical lesson: document observable execution details and restrict access to sensitive material.

A useful initial scope statement has four parts: the business outcome that may have changed; the identities and tools that could still change it; the records or people that may be in scope; and the evidence needed to decide whether an effect happened. “The AI made a mistake” is too broad to guide containment. “The customer-renewal worker may have sent a message after ownership transferred to an agent” identifies an outcome, pathway, and question to check.

First-hour sequence: contain before investigating

The first hour is a coordination window, not a promise every incident can be repaired in sixty minutes. Name an incident lead and a person who can authorize changes to the affected system. If the affected tool can still write, pause or revoke that tool path first. If the agent writes to several systems, pause its queue or identity rather than shutting down unrelated services. Preserve read-only monitoring where possible.

A single user described a cloud coding agent continuing after a stop request for hours and consuming additional tokens. That forum report is one person’s account, not a measurement of how often this occurs. It is a useful rehearsal prompt: can your team stop a worker independently from the interface used to start it, and can the team prove the stop took effect? Cursor forum report

Use these steps in order, adapting scope to risk. A financial transfer, public disclosure, or production deletion needs immediate escalation to the accountable owner and possibly external response teams.

1. Name the business outcome. Describe the side effect plainly: “the agent may have sent two renewal notices,” not “the LLM hallucinated.”
2. Stop new work at the narrowest reliable boundary. Pause the run, queue, scheduled trigger, agent identity, or tool permission. If a credential grants broad write access, revoke it through the approved break-glass procedure.
3. Prove the stop. Check execution state, queue depth, active workers, credential validity, and destination system. Capture a timestamped API response or screenshot. Cancellation acknowledgement does not prove an already-issued request was rolled back.
4. Preserve evidence. Export run IDs, parent and child IDs, tool calls, arguments, outputs, timestamps, model and prompt versions, retrieved documents, retries, webhook events, and operator actions. Keep originals read-only and record their source.
5. Inspect the system of record. Query the destination directly. Determine whether the action was accepted, rejected, duplicated, partially applied, or pending. Use the receiving system’s transaction or event ID.
6. Contain downstream effects. Pause only workflows that could amplify the change. Notify the business owner and incident channel with a factual summary and next update time.
7. Choose a repair path. Prefer a compensating action that preserves legitimate changes. Do not replay the agent’s whole plan because an earlier call timed out.
8. Require a human release decision. Resume only after an owner names a verified checkpoint, restricted permission set, monitored first transaction, and stop route.

The lead should say which state is known and which remains unknown. “The worker is paused” needs evidence: queue state, revoked token, disabled integration, or tool-side cancellation receipt. A user-interface toast alone is insufficient. Record times in UTC and the local business timezone because events may cross service regions and daylight-saving transitions.

First-hour response flow

flowchart TD
  A[Unexpected agent side effect] --> B[Pause smallest write-capable scope]
  B --> C{Independent stop proof?}
  C -- No --> D[Revoke tool identity or isolate worker]
  D --> C
  C -- Yes --> E[Preserve execution trace]
  E --> F[Query destination system of record]
  F --> G{Effect state known?}
  G -- No --> H[Keep capability contained; escalate query]
  H --> F
  G -- Yes --> I[Choose compensating repair]
  I --> J[Owner verifies restored state]
  J --> K[Restricted monitored resumption]

This flow has two loops. If the stop cannot be proven, containment is incomplete. If the destination cannot resolve whether the action happened, keep the write path restricted and escalate the query instead of retrying. The diagram describes a recommended decision path; it does not prove that a particular product implements these signals.

Use two timelines, not one

A model transcript and an external side effect are different evidence. Build an execution timeline with user request, agent run ID, retrieved context, tool intent, arguments, child-run creation, retry, timeout, stop request, and operator intervention. Build an effect timeline with request received by the external system, authorization decision, transaction or record ID, accepted state, downstream event, notification, and compensating action.

Join them with stable identifiers: idempotency key, request ID, trace ID, record ID, message ID, or payment reference. If there is no join key, mark the association as an inference. Do not treat “same minute” as proof of causation. A reliable timeline distinguishes event time from the time an operator observed or exported the event.

This separation exposes common failures. An agent may call a tool successfully but the target rejects the change. A client may time out after the target committed. A model can say “done” while a downstream queue is processing. A retry may create a second action if the integration lacked an idempotency key. The destination’s event log resolves the effect question; the orchestration trace explains how the request was produced.

Do not merge both timelines into a confident narrative before reconciling them. Otherwise an investigator may treat a plan as a completed operation. Mark each event as observed, inferred, or reported by a person, and preserve the source that supports the classification. If records disagree, keep both versions and assign an owner to resolve the discrepancy.

The minimum evidence packet

Capture enough to reconstruct the action while avoiding unnecessary personal data. Include incident ID, lead, business owner, affected service, start and detection time, and current state. Include agent, model, system prompt, policy, tool adapter, and connector versions. Capture parent and child run identifiers, including which child had write capability.

For each tool call, record tool name, requested operation, normalized target, arguments, authorization result, and raw response. Record retrieved source identifiers and hashes rather than broad copies of confidential documents unless necessary. Preserve retry and timeout policy, queue status, webhook IDs, stop/cancel acknowledgements, destination state before and after containment, identity changes, approver, assumptions, unresolved questions, and next update time.

Do not preserve hidden reasoning as a default substitute for an execution trace. It may be unavailable, sensitive, or unreliable as an explanation. Prefer observable evidence: tool inputs and outputs, versions, permissions, retrieved-document identifiers, and destination records. Follow retention and access rules. If a provider cannot export full history, record that limitation and retain the fields it does expose.

A lightweight evidence record

These fields are a starting contract. They make uncertainty explicit. Store secrets as vault references, never directly in the incident record.

Field	Example	Why it matters
incident_id	INC-2026-1042	Stable reference across teams
detected_at	2026-11-08T08:42:00Z	UTC chronology
business_timezone	Europe/Paris	Human-facing timestamps
agent_run_id	run-7f21	Find orchestration trace
child_runs	run-7f22	Detect delegated work
tool_identity	agent-prod-support-writer	Revoke correct principal
destination_ref	ticket:5841	Query system of record
effect_state	unknown	Prevent guess becoming fact
stop_proof	queue_paused_at=...	Record containment evidence
owner	support-incident-lead	Establish decision authority
next_review_at	ISO timestamp	Make next action observable

Keep records append-only where practical. Correct an entry by adding a new event pointing to the earlier event; do not rewrite the original timeline. For a provider export, record who exported it, when, from which account, and whether the export is complete. Store a digest for large bundles so a reviewer can detect accidental alteration. Do not put access tokens, personal data, or full prompts in a general incident channel.

Original asset: sixteen synthetic incident rehearsals

This article’s reusable asset is a set of sixteen hand-authored hypothetical cases designed on 8 November 2026. They were not drawn from customer incidents, production telemetry, vendor testing, a survey, or a user study. No incident-rate, response-time, or effectiveness statistic was measured. Use these scenarios to rehearse decisions and test whether the named owner, stop mechanism, and destination proof exist.

ID	Scenario	First containment check	Evidence to preserve	Release gate
01	Agent sends duplicate renewal email	Disable outbound send; inspect provider message IDs	Run trace, idempotency key, both message IDs	Customer owner confirms authoritative message
02	Coding agent edits protected branch	Revoke branch token; preserve working tree and audit log	Commit, diff, token identity, stop request	Repo owner reviews diff and branch state
03	Support agent updates wrong ticket	Pause ticket writes for selection rule	Query, target selection, ticket history	Support lead reconciles affected tickets
04	Payment call times out	Do not retry; query processor by idempotency key	Request key, response, ledger and processor event	Finance confirms settled, rejected, or pending
05	Agent deletes records after stale instruction	Freeze delete permission and retention jobs	Query result, delete events, source version	Data owner approves restore or non-action
06	Child agent continues after parent stop	Pause parent queue; isolate child identities	Parent-child graph, child status, credential map	Every child has verified terminal state
07	Internal note posted to public channel	Disable connector route; preserve audit log	Message ID, audience, config, source data	Communications/security owner classifies exposure
08	Tool returns 500 after committing change	Check destination event log before replay	Request ID, response, destination transaction	Target confirms final state and replay safety
09	Retrieval includes obsolete policy	Pause decisions depending on policy	Document ID, version, index time, answer trace	Policy owner approves current source/correction
10	Human updates case while retry is queued	Disable stale retry; compare versions	Human revision, retry payload, lock value	Destination rejects old version; owner approves
11	Agent grants wrong role	Revoke grant; inspect related principals	Grant event, role mapping, principal, approver	IAM owner confirms least-privilege state
12	Agent continues after cancel toast	Verify worker, queue, and token separately	UI response, heartbeat, queue item, token status	Independent signal confirms no writer remains
13	Agent sends after human handoff	Disable bot send; inspect webhook order	Handoff event, message IDs, delivery receipts	Human owner sees control; stale sends suppressed
14	Prompt or tool config changes mid-response	Freeze rollout; pin known version	Config diff, deployment event, run version	Change owner approves pinned configuration
15	Broad shutdown blocks unrelated critical work	Isolate connector or tenant instead	Dependency map, service impact, isolation events	Service owner confirms scoped recovery
16	No one can verify repair worked	Keep write capability paused	Missing records, query attempts, escalation	Named owner signs reproducible verification

How to run the rehearsal

Give responders only the scenario text. Ask the on-call person to state the first control, evidence to capture, and system to query to determine whether the action happened. Ask another responder to challenge each statement: what if the control never reached the worker, the remote system committed before timing out, or a delegated child remains active?

Score observable readiness: a documented owner exists; the stop path is independent from the agent; destination state is queryable; identifiers join the two timelines; a repair can preserve legitimate changes; and release authority is explicit. Do not score whether someone guessed the “right” root cause from a short scenario. Root cause belongs to the later investigation.

Record each gap as work with an owner and due date. “Test token revocation against an active worker” is actionable. “Improve AI safety” is not. Repeat a scenario after a control change, retaining old and new rehearsal records so the team can see whether the specific gap closed. The exercise is a readiness check, not proof of production effectiveness.

Sixteen case notes for facilitators

01 — Duplicate renewal email. Identify the provider-side message ID, not just the agent’s send attempt. Disable the outbound path for this run or identity while leaving unrelated mail available if the scope is narrow. Query provider and CRM by recipient, campaign, and idempotency key. Preserve both message bodies and delivery statuses under restricted access. If the second notice reached the customer, the service owner decides whether to correct it. Do not send an apology through the same unverified automation. Resume only after a human confirms the authoritative message and duplicate suppression is tested.

02 — Protected-branch code edit. Revoke the agent’s branch token and stop queued jobs. Do not reset the branch immediately: a human may have committed a valid fix after the agent’s action. Preserve commit ID, patch, branch protection event, token identity, and timestamps. The repository owner compares the current tree with the last known good commit and subsequent human work. If the change is isolated, create a reviewed corrective commit instead of rewriting history. Before resuming, use a limited branch, one reviewed pull request, and an identity that cannot merge or bypass protections.

03 — Wrong support ticket. Pause writes for the selection rule or connector if the agent could touch adjacent cases. Inspect the query that selected the customer, matching fields, and audit history for every target. Avoid exporting a full customer list into a public incident room. The support lead decides which tickets need correction and which note must remain for audit. If a stale customer identifier caused the event, fixing only the visible ticket leaves the selector defect in place. Resume after the owner confirms scope and a test proves the selector cannot cross the account boundary.

04 — Payment request timed out. Treat the outcome as unknown until the processor or ledger says otherwise. Repeating a charge because the client received no response can create a duplicate debit. Query using the idempotency key, processor reference, merchant account, and a time window wide enough for delayed settlement. Preserve client request and processor event. Finance determines whether the transaction is pending, settled, rejected, or reversed, and owns any correction. Resume with a verified idempotency policy and a small monitored action. A missing response is not evidence the processor did nothing.

05 — Deletion after stale instruction. Revoke delete permission first and preserve retention or backup jobs that might overwrite recovery points. Identify which instruction and policy version the agent used, then compare the deletion audit log with current records. A restore can overwrite lawful changes made after deletion, so the data owner selects the recovery point and reconciles later updates. Keep original deletion events in evidence. Resume only after the current retention policy is loaded, the data owner approves recovered state, and write access is scoped to the records and operations required.

06 — Child continues after parent stop. Do not accept the parent’s “stopped” status as proof all delegated work ended. Enumerate child run IDs, service identities, queues, scheduled tasks, and credentials. Pause or revoke each child’s write path and query its state independently. Preserve the parent-child graph, handoff messages, and tool events. The release gate is unmet until each child is terminal or isolated and no retry remains queued. A future design should propagate cancellation and expose child inventory; this rehearsal checks whether that behavior exists in the deployed system.

07 — Internal note posted publicly. Disable the connector route and confirm the message audience from the destination, not the agent’s channel label. Preserve message ID, channel configuration, recipient list, source data identifiers, and delivery timestamps. Notify communications and security owners; involve privacy counsel if personal data may have been exposed. Do not delete the artifact before capturing its state and evaluating correction options. Deletion may not retract notifications or copies. Resume only after the correct destination is configured, the route is tested with a harmless draft, and the owner approves the communication plan.

08 — Server error after a committed change. A status code describes the response, not necessarily destination state. Query the destination event log by request ID and inspect resource version. If the change committed, do not replay. If the destination proves it did not commit and the call is idempotent, the owner may authorize retry. Preserve request, server response, target event, and downstream automation triggered by commit. The fix may belong in adapter timeout handling or idempotency, not the prompt. Test ambiguous responses before broad rollout.

09 — Obsolete retrieved policy. Pause decisions that depend on stale policy and pin the current authoritative document. Record old and new document IDs, version dates, index timestamps, and the answer or action that used stale retrieval. The policy owner decides whether completed cases need correction. Do not delete the entire retrieval corpus: unrelated valid sources may be lost. Resume after confirming the indexed version, rebuilding the affected index if needed, and testing a query that distinguishes the current rule from the archived one.

10 — Human update races with retry. Freeze the retry path and compare record versions. Preserve human edit, agent payload, retry ID, and optimistic-lock token. If the destination allows stale writes, prefer adding a version precondition before restoring automation. The service owner decides whether the human edit is authoritative; do not overwrite it with the agent’s earlier snapshot. Resume after a test demonstrates stale writes are rejected or routed for review. This case reveals when a handoff changes UI state but does not invalidate already queued work.

11 — Wrong access role granted. Revoke the grant and inspect sibling principals, group inheritance, and active sessions. Preserving only the agent’s own action misses role propagation. Keep grant event, target identity, role mapping version, approver, and revocation evidence. The IAM owner confirms least-privilege state and whether credentials or sessions need rotation. Resume with an allowlist of permitted roles and a human approval gate for new privileges. Verify authorization using the target’s access log, not only a successful revoke response.

12 — Cancellation toast, work continues. Treat the toast as a UI event, not worker evidence. Check orchestration record, worker heartbeat, queue, tool identity, and destination separately. If the write path remains active, revoke it at connector or identity layer. Preserve timestamps for the cancellation request and each independent stop signal. Do not mark the incident contained until the independent check completes. After the exercise, assign an owner to make the stop response identify which components received it and which remain active.

13 — Message after human takeover. Pause the bot sender and inspect ownership event, message queue, delivery receipt, and webhook ordering. There may be an in-flight send that cannot be recalled, even if the handoff flag is already set. Preserve event sequence and identify which component made the final send decision. The human owner determines whether customer correction is needed. Resume only after human state suppresses pending bot messages, an integration test covers delayed webhooks, and the team can prove the bot stopped sending on that conversation.

14 — Configuration changes mid-response. Freeze rollout and pin prompt, tool schema, policy, retrieval index, and model version used by active runs. A run may start under one configuration and issue a call after deployment changes its meaning. Preserve deployment logs and configuration diffs. The change owner chooses rollback, not the agent. Resume after identifying a stable version and checking queued requests will not execute with mixed assumptions. Attach version identifiers to tool events so later investigators can determine which behavior was in force.

15 — Broad shutdown creates another outage. Isolate the affected tenant, connector, identity, or operation rather than disabling a shared platform where possible. Map dependencies and identify critical processes a global stop would interrupt. Record what is isolated, what remains active, and who accepts residual risk. A narrow containment still needs proof: the affected tool must no longer write while unrelated workflows remain healthy. Resume after the service owner confirms dependencies and the lead documents why the chosen boundary is proportionate.

16 — Repair cannot be verified. Keep the capability paused or restricted. Ask what observable record would demonstrate repair, who can query it, and what result is acceptable. Preserve failed query attempts and missing evidence; do not fill the gap with an agent-generated summary. Escalate to the destination owner or provider if the system of record is unavailable. The release gate is a reproducible check signed by a named person. If verification remains impossible, a leader must explicitly accept the uncertainty before resuming action.

Safe replay and repair

A timeout is not a rollback. Before retrying, ask whether the destination supports an idempotency key and whether this run used it; whether the destination can be queried by that key or a stable transaction ID; whether the action is reversible without overwriting legitimate later changes; and whether replay could trigger notifications, fees, external calls, or downstream automations.

If any answer is unknown, pause replay and ask the system owner. For a record update, compare the current value with both the expected pre-incident value and the agent’s attempted value. For a payment or external message, reconcile the provider’s event record rather than trusting the client response. For a code change, inspect the diff and commit graph before reverting; a blanket reset can erase a developer’s valid work.

Trade-press analysis of disaster recovery in AI-centric environments makes a related point: restoring data and services may not be enough when autonomous actions must be undone without reversing legitimate user work. Models, retrieval data, prompts, configurations, identities, and tokens may all be operational dependencies. This is industry analysis, not a tested recovery benchmark. ITPro: disaster recovery in an AI-centric world

A runnable reference triage helper

This Python 3 script reads one JSON object per line and flags records that should remain contained. It does not call an API, revoke credentials, classify legal obligations, or decide that an incident is safe. Its purpose is to expose missing evidence before a human authorizes resumption.

Save as triage_agent_incident.py:

#!/usr/bin/env python3
"""Fail-closed checklist linter for synthetic agent incident records."""
import json
import sys
from pathlib import Path

REQUIRED = (
    "incident_id", "detected_at", "owner", "agent_run_id",
    "tool_identity", "destination_ref", "effect_state",
)
SAFE_EFFECTS = {"confirmed_none", "reconciled"}
PROOF_FIELDS = {"stop_proof", "destination_query", "next_review_at"}

def inspect(record):
    problems = []
    for field in REQUIRED:
        if not record.get(field):
            problems.append(f"missing:{field}")
    if record.get("effect_state") not in SAFE_EFFECTS:
        problems.append("effect_not_reconciled")
    for field in PROOF_FIELDS:
        if not record.get(field):
            problems.append(f"missing:{field}")
    if record.get("active_child_runs"):
        problems.append("active_child_runs")
    if record.get("write_permission_active") is True:
        problems.append("write_permission_active")
    return problems

def main(filename):
    path = Path(filename)
    for line_no, raw in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
        if not raw.strip():
            continue
        try:
            record = json.loads(raw)
        except json.JSONDecodeError as exc:
            print(f"line={line_no} HOLD invalid_json={exc.msg}")
            continue
        problems = inspect(record)
        status = "HOLD" if problems else "READY_FOR_HUMAN_REVIEW"
        checks = ",".join(problems) if problems else "passed"
        print(f"line={line_no} id={record.get('incident_id','?')} {status} checks={checks}")

if __name__ == "__main__":
    if len(sys.argv) != 2:
        raise SystemExit("usage: python triage_agent_incident.py incidents.jsonl")
    main(sys.argv[1])

Example incidents.jsonl row:

{"incident_id":"INC-2026-1042","detected_at":"2026-11-08T08:42:00Z","owner":"support-incident-lead","agent_run_id":"run-7f21","tool_identity":"agent-prod-support-writer","destination_ref":"ticket:5841","effect_state":"unknown","stop_proof":null,"destination_query":null,"next_review_at":"2026-11-08T09:15:00Z","active_child_runs":["run-7f22"],"write_permission_active":true}

Run it with:

python triage_agent_incident.py incidents.jsonl

The sample should print HOLD because the effect remains unknown, stop and destination proof are absent, a child run is active, and write permission remains enabled. READY_FOR_HUMAN_REVIEW means only the specified fields and conservative checks passed. It does not authorize production restart. Add checks for your identity provider, queue system, and data sensitivity; keep this helper as a review aid, not a control-plane command.

Communication and personal-data questions

Communicate in layers. The incident channel needs current operational state, affected service, owner, containment proof, and next update time. Customer or employee notification needs verified impact and clear instructions, not internal speculation. Legal, privacy, security, and regulatory communications should follow the established escalation path.

For personal data, EDPB guidance says notification decisions depend on whether the event meets the breach definition and risk to individuals. A controller may need to notify a supervisory authority within 72 hours of becoming aware of a qualifying breach, unless the breach is unlikely to result in risk; where risk is high, affected individuals may also need communication. Exact obligations and clock depend on jurisdiction, role, and facts, so route analysis to the privacy lead. EDPB guidance

Do not overstate certainty. “We stopped new outbound messages at 09:12 UTC; we are checking whether the earlier request was accepted” is better than “the bot is fixed.” Give a time for the next update even if the answer is unknown. Keep the detailed internal timeline separate from a customer statement containing only verified and necessary facts.

Resume without creating a second incident

Resumption is a controlled change. The owner should approve a specific capability, not “turn the agent back on.” Start with the smallest safe scope: one tenant, one record, read-only mode, draft-only output, or a restricted credential. Add human confirmation to high-impact actions until evidence supports broader permission.

Before release, verify prompt and tool configuration are pinned to the intended version. Confirm retrieval sources are current, the destination uses version checks or idempotency where possible, and stop/revoke mechanisms still work. Keep the prior configuration available for rollback. If the incident involved an identity or connector, create or rotate credentials through the normal security process rather than copying a temporary secret into a prompt or incident note.

For the first resumed action, have a human observe the destination system, not just the agent transcript. Set a narrow limit on action count or scope. Define a threshold that returns the workflow to paused state, such as duplicate key, unknown target, stale version, unexpected child run, or missing destination receipt. A monitored first action is not statistical validation; it is a chance to catch an obvious regression before re-expanding access.

A KPMG advisory page includes AI asset mapping, non-human identity governance, and incident response planning in its AI-security service scope. That describes one provider’s service framing, not proof that a specific control prevents incidents. The recommendation here is narrower: inventory agent identities and write-capable tools so responders can revoke the correct access. KPMG AI security services

What the evidence does and does not say

The U.S. Census Bureau’s April 2026 working paper, based on the Business Trends and Outlook Survey AI supplement from November 2025 through January 2026, estimates that 18% of firms used AI in a business function during that reference period. This describes AI diffusion; it does not measure autonomous-agent incidents or prove that a sector uses tool-calling agents. Use the estimate only as context for why ownership and recovery planning deserve attention as business use expands. U.S. Census Bureau working paper CES-26-25

A practitioner roundtable published by InfoQ discusses AI support for SRE work. Participants describe using AI to correlate telemetry and draft changes for human review, while warning that incomplete or stale infrastructure context can lead to poor recommendations. This is practitioner discussion, not a controlled effectiveness study. It supports rehearsing context and review boundaries; it does not prove agentic response is safe. InfoQ, AI-powered SRE roundtable

The sources represent different evidence types: public-sector working paper about adoption, official regulator guidance about personal-data breaches, practitioner discussion, a single forum report, trade-press analysis, and a consultancy service description. They do not establish a measured incident rate, validated recovery procedure, or universal legal obligation. Separating containment, effect verification, repair, and release is an operational synthesis. The sixteen-case matrix has not been validated against real incident outcomes. Replace assumptions with system evidence, test stop and idempotency controls, and update the runbook after an exercise or real event.

Assign responsibility before launch

Every production agent should have a named business owner, technical owner, and reachable incident contact. In a small team, one person may cover several roles; the responsibilities should still be stated. The business owner decides what outcome is acceptable and who may authorize customer correction. The technical owner knows how to pause workers, rotate credentials, query queues, and inspect the destination. Privacy, legal, security, finance, or safety specialists join when facts touch their domain.

This division avoids delays. An engineer may stop a connector but lack authority to authorize customer communication. A business manager may know the intended outcome but lack access to prove whether the action committed. Record escalation routes in deployment documentation and test them outside normal office hours. The incident channel should have someone maintaining a timeline so responders do not have to repeat the same facts.

Do not assign root cause before evidence collection. The cause may be stale retrieved data, retry policy, broad permission, integration defect, a race with human update, or malicious input. Containment depends on remaining ability to cause harm; corrective action depends on evidence gathered later. Keeping these decisions separate reduces blame-driven changes that fail to close the control gap.

Lightweight measures for learning, not ranking

You do not need a composite “agent safety score” to learn from incidents. Start with observable measures and keep each denominator visible: time from detection to verified stop and its definition; share of records with a destination event ID linking attempt to outcome; active delegated runs found after parent cancellation; retries without a confirmed idempotency key; time to identify a release owner; and rehearsal cases where responders could not find a stop path.

These are internal measures, not industry benchmarks. A short period with zero incidents does not prove controls are effective; a rise in reports can mean reporting improved. Compare the same scope and definitions, annotate system changes, and never use one metric to rank responders. If the report count changes after new logging, separate a measurement change from any change in risk.

A first dashboard can contain incident_id, service, detected_at, verified_stop_at, effect_reconciled_at, release_owner, replay_used, and evidence_gap. Add no customer content unless necessary and access-controlled. Review trends with the service owner and change one control at a time where feasible. Record definition changes so a chart does not compare unlike events. A metric should prompt a question, not declare a system safe.

Failure modes that make an incident look contained

• The interface says stopped, the worker is still running. Check the queue and worker independently; if the tool identity can still write, containment is not established.
• A token is revoked, but a request was already accepted. Query the external system before declaring that no effect occurred.
• A timeout is mistaken for a failed transaction. The destination may have committed before the response was lost; inspect its event ID before retrying.
• Only the parent run is paused. Enumerate delegated runs, scheduled tasks, and queued retries because they may have separate identities.
• The whole service is shut down without a dependency check. A broad stop can create a second incident; isolate the tenant, connector, or operation if that boundary is reliable.
• The first visible customer is treated as the complete scope. Check the selector, shared prompt, and credential use across adjacent records.
• A blanket rollback overwrites human work. Compare versions and use a compensating change that preserves later legitimate updates.
• The incident summary replaces the source records. Keep original logs and destination events read-only; store the narrative as a separate interpretation.
• Sensitive evidence is copied to a broad channel. Restrict the source bundle and share a minimized status update.
• The stop remains in place forever with no owner. Assign a review time and release authority so containment does not become an untracked outage.

These are rehearsal prompts, not an empirically ranked list. Add failures specific to your queueing system, identity provider, customer channels, and destination APIs. For each one, write down the evidence that would show the failure has occurred and the person who can authorize the next step.

Evidence and source-control checklist

Before using this runbook, verify the evidence and scope:

• [ ] EDPB guidance is used only for qualifying personal-data-breach decisions, not all model failures.
• [ ] The Census estimate is firm use during its survey reference period, not a count of agents or incidents.
• [ ] The forum post is labeled as one user report, not a prevalence estimate.
• [ ] The InfoQ session is practitioner discussion, not independent proof of effectiveness.
• [ ] The KPMG page describes service positioning, not outcome evaluation.
• [ ] ITPro is trade-press analysis, not a formal standard.
• [ ] The synthetic matrix is hand-authored and unmeasured.
• [ ] Original evidence is retained separately from a public incident summary.
• [ ] The response owner can name real system-of-record queries and stop signals.
• [ ] Each recommended control has a local owner, test, and escalation path.

Printable response checklist

• [ ] Incident lead and business owner named.
• [ ] Affected outcome stated in plain language.
• [ ] New agent work paused at a verified boundary.
• [ ] Active children, retries, queues, and scheduled jobs checked.
• [ ] Write-capable identities and connectors enumerated.
• [ ] Stop proof recorded with timestamp and source.
• [ ] Execution and effect timelines started separately.
• [ ] Stable identifiers join requests to destination events.
• [ ] Destination state checked directly; timeout not treated as failure.
• [ ] Evidence exported read-only and access restricted.
• [ ] Customer, privacy, security, legal, and finance escalation considered.
• [ ] Scope beyond the first visible record checked.
• [ ] Retry and compensation safety reviewed by destination owner.
• [ ] Temporary controls have owners and review times.
• [ ] Release decision names scope, permissions, monitor, and stop route.
• [ ] First resumed transaction observed at the destination.
• [ ] Follow-up owner and due date assigned for each control gap.

A practical closure standard

Close the response phase only when further unsafe action is contained, system-of-record state is reconciled or explicitly marked unresolved, the business owner accepts the repair or non-action, required notifications are routed, and a named owner has the next task. Root-cause analysis may continue after restoration, but restoration must not hide an unresolved side effect.

If the destination cannot establish whether an action happened, the correct state is “unknown,” not “probably fine.” Keep the affected write capability paused or restricted while the system owner resolves uncertainty. If that creates another material operational risk, escalate the trade-off to the accountable leader and document the chosen scope. A runbook is useful because it makes uncertainty visible and assigns the decision to a person with authority.

The purpose of first-hour response is not to demonstrate that an AI system can repair itself. It is to preserve people’s ability to make a correct decision when the agent’s account of events is incomplete. Strong operational design keeps the stop path independent, destination state queryable, evidence restricted but durable, and release human-owned. Rehearse those capabilities before relying on them under pressure.