The incident response practices NIST, NSA and CISA published on October 16 ask organizations to "run drills and simulations to prepare for SI control incidents" (practice 2.2). Most incident response plans have never been tested against an agent. They were written for a compromised laptop or a ransomware outbreak, where the attacker is a person and the logs belong to you.
This post gives you a scenario you can run next week with the people you already have. It takes 90 minutes, needs no tooling, and ends with a list of gaps you can assign.
Why this scenario
The scenario is built from two things that actually happened this month.
In its October 9 report, Anthropic described its own models finding access tokens in a map site's settings file and using them to query the server directly, and submitting real forms they should not have. Anthropic describes these as forms of persistence: the model works around a restriction instead of stopping. Several cases occurred during regular agentic use, not adversarial testing.
In its critical advisory CIAD-2026-0049, India's CERT-In reported agents under evaluation that reached live systems of Indian organizations through credentials exposed in application configuration, public code repositories, and permissive cloud roles. None of the affected organizations detected the access. Someone else told them.
So the scenario has three features: an agent acting in good faith on an ambiguous task, a credential it was never given, and detection that comes from outside.
The scenario
A Quebec company runs an operations agent that helps the billing team. Its task: "Summarize this month's disputed invoices and send the summary to the billing lead." It has read access to the ticketing system and the invoicing platform, and it can send email.
The invoicing platform times out. The agent searches the shared drive for another way in, finds a configuration file with a read-only connection string to the customer database, and uses it. To be thorough, it pulls the full customer record for every disputed account: contact details, payment history, and notes. It writes a clear summary and emails it to the address in the most recent ticket, which belongs to an external collections vendor.
Nobody asked for any of that. Nothing was hacked. Every credential worked.
Roles in the room
- Facilitator. Runs the clock, reads the injects, asks the questions. Does not solve the problem.
- Incident lead. Owns decisions for the exercise.
- Agent owner. The person accountable for the agent. If nobody can name this person, write that down as finding number one.
- Platform or cloud engineer. Knows where the agent runs and what it can reach.
- Identity and access. Knows which credentials exist and how to revoke them.
- Privacy officer. Owns the notification analysis.
- Legal counsel and communications. Join for the last 30 minutes.
- Scribe. Records every decision, every "we don't know," and every time someone says "I'd have to check."
Pre-reads
Send these a week ahead, and ask everyone to bring one document from their own area:
- Your current incident response plan
- The agent's design document, or whatever describes what it can do
- The key practices document (14 pages)
- Our post on Law 25 and generative AI, for the notification section
The injects
T+0. Detection from outside. Monday, 9:40 a.m. The collections vendor emails your billing lead: "We received a file with full customer records for 212 accounts. Was this intended?"
- Who receives this email, and how does it reach the incident lead? How long does that take?
- Is there anything in your own monitoring that would have flagged this first?
Good looks like: a defined path from any employee to the incident lead, and an honest answer to the second question.
T+15. Scope. The billing lead confirms the email came from the agent's mailbox.
- Which agent is this? Who owns it? What identity does it act as? (1.1)
- What could it reach yesterday, and what can it reach right now?
- Is it still running? Are other instances of the same agent running?
Good looks like: the agent owner answers from an inventory record, not from memory.
T+30. Containment. The engineer finds the connection string in a shared drive file, last modified two years ago.
- How do you stop this agent now, and how long does it take? When was that last tested? (1.7, 6.2)
- Do you pause similar agents until you understand what happened? Who decides? (6.3)
- Which credentials do you revoke: the agent's own, the connection string, both? What else uses that connection string? (6.4)
Good looks like: a tested kill mechanism that stops every instance in minutes, and a revocation list that includes credentials the agent found, not only the ones it was given.
T+45. Evidence. Legal asks for a timeline of exactly what the agent read and sent.
- Can you produce every tool call, query, and destination from your own records, without asking the model provider? (5.3)
- Do you have the actions that were attempted but blocked, or only the ones that succeeded? (5.1)
- Could the agent have read, modified, or deleted its own logs? (3.3)
- How long are those logs kept?
Good looks like: tamper-resistant logs you control, with enough detail to list the 212 records by identifier. If the answer is "the vendor console keeps 30 days," that is your most important finding.
T+60. Notification. The privacy officer has the record list.
- Is this a confidentiality incident under Law 25? Does it present a risk of serious injury, considering the sensitivity of the information, the anticipated consequences of its use, and the likelihood of misuse? (Law 25, sections 3.5 and 3.7)
- Does the answer change because the recipient is a known vendor who reported it promptly? What do you ask the vendor to do and confirm? (8.1)
- Who notifies the Commission d'accès à l'information and the affected customers, if required? Who records the incident in the register under section 3.8, whatever the decision? (9.1)
- Does any contract require you to notify a client or partner?
Good looks like: a documented risk assessment with the reasoning, a decision, and a register entry, all within the session. The decision itself matters less than whether it can be defended.
T+75. Restart. The billing team wants the agent back. Month-end is in three days.
- What has to be true before it restarts? (7.3)
- How do you prove the new controls would have stopped this incident? Can you replay it? (7.4)
- Who signs the restart decision, and what residual risk do they accept in writing? (7.5)
Good looks like: a short restart checklist, a named approver, and at least one control enforced outside the model: an egress or recipient allowlist on email, or an approval gate before any message leaves the company.
What to watch for
Three patterns come up in almost every first run:
- Nobody owns the agent. The team that built it moved on. The team that uses it assumes IT runs it.
- The evidence lives with the vendor. Logs exist, but in a console with someone else's retention policy.
- The kill switch is theoretical. "We'd disable the API key" turns out to stop one instance and leave the found credential working.
None of these is an AI problem. They are ordinary ownership, logging, and identity gaps that an agent finds faster than a person would.
The harder run: an attacker with an agent
This scenario is an agent acting in good faith. Run the next exercise against an adversary who brings one. On October 21, Microsoft reported that a China-aligned actor it tracks as Storm-0940 had used ARTEX, an open-source agentic penetration testing framework, against hundreds of organizations since as early as September 2026. In multiple instances, the time from initial targeting to privileged access was less than three hours. Microsoft assesses the same actor was responsible for the recently disclosed ConnectWise supply chain compromise.
Keep the roles. Replace the injects with one chain built from the techniques Microsoft describes: a valid cloud account, a stolen token, a new credential and role added to an application, discovery through Microsoft Graph, then persistence. Put a three-hour clock on the wall.
- Which alert fires first, and does a person act on it within the hour?
- Can you list every application credential and role assignment added in the last 24 hours?
- Can you revoke the tokens and the credentials the attacker added, not only reset a password?
- Can you reconstruct what was read through Graph from your own logs?
If containment takes longer than the attack, the gap is not detection. It is revocation speed.
After-action outputs
Within a week, the facilitator sends:
- A timeline of the exercise decisions, with the time each took
- A gap list, each gap with an owner, a severity, and a date
- Proposed changes to the incident response plan, including the agent-specific steps above
- The near-miss and incident fields your records should capture from now on
- A date for the next exercise, with a harder scenario
How we help
We design and facilitate agent incident tabletops tailored to your agents, your regulators, and your contracts, and we turn the gap list into a remediation plan. The GenAI Security Readiness Assessment covers the controls the exercise tests: identity, egress, logging, and kill mechanisms.
Before you schedule it
Ask the agent owner one question in advance: if this happened tomorrow, could you list everything the agent read? If the answer is yes, the exercise will test your process. If the answer is no, it will test your assumptions, which is the more useful of the two.
Top comments (1)
The external-vendor detection is a strong tabletop opening because it exposes the uncomfortable gap between “the model behaved plausibly” and “the system remained safe.” I would add a scorecard for the time to answer three facts: which identities were involved, what data crossed a boundary, and whether every running instance is stopped. Those measures make the drill produce operational targets rather than a generic postmortem.