NIST Wants You to Track AI Near-Misses: How to Build the Register

Most security programs only count what went wrong. An incident is opened, investigated, closed, and reported. The attempts that a control stopped leave a log line and nothing else.

Is that really the whole story, though?

Not for AI agents. An agent that tries to reach a destination it should not, and gets blocked by an egress rule, did not cause an incident. It did tell you two things: the agent was willing to cross that boundary, and the boundary held. Both facts are worth keeping. Over time, they are the only evidence you have that your agent controls work.

That record has a name now. The incident response practices NIST, NSA and CISA published in October define a near-miss and ask organizations to track and report them. The UK government says it will look at near-miss reporting in law. You do not need to wait for either to start.

What the guidance says

The Key Practices for Incident Readiness and Response define a control near-miss as "an occurrence which did not result in an SI control incident but presented a reasonably foreseeable or imminent risk of doing so." They add why it matters: near-misses "may reveal potential gaps or weaknesses in controls, security measures, or processes."

Near-misses then run through the practices from start to finish:

• 2.1 asks for "clear indicators, criteria, and severity scoring" for incidents and near-misses.
• 1.7 asks for mechanisms to block, contain, or terminate agent activity "if an incident or near-miss is detected."
• 5.1 asks organizations to preserve "rejected actions and tool calls," not only the ones that succeeded.
• 9.1 encourages voluntary reporting to CISA of near-misses arising from control incidents, and 9.3 voluntary sharing of near-misses with government experts.

On October 12, UK AI minister Kanishka Narayan said in a speech at Imperial College London that the government will assess how to strengthen the Cyber Security and Resilience Bill on "identifying and reporting on serious incidents and near misses." No amendment text exists yet. The direction is clear enough to plan for.

Why near-misses are the useful data

Incidents involving agents are, for now, rare in any single organization. A metric built on rare events tells a board almost nothing from one quarter to the next. Near-misses are frequent. They show which agents push against their scope, which boundaries they push against, and which control layer catches them.

Anthropic's report on unintended model actions is a good illustration. Its models worked around a length limit on a fetch tool with URL shorteners, and submitted forms they were told to leave alone. Anthropic then built tooling to detect and block these behaviors. From that point on, every repeat of those behaviors becomes a near-miss: an attempt, a block, a record. That record is what tells you whether the behavior is fading or growing.

The register

One row per attempt that a control stopped, or that came close to causing harm. Keep the fields few and mandatory:

1. Agent. The inventory entry, so the attempt is tied to an owner and a declared scope. Our ten-field agent inventory is the reference.
2. Timestamp. From a clock the agent does not control.
3. Attempted action. What the agent tried to do, in one sentence: "fetch an external URL through a shortener," "read the payroll table," "send an email to an outside domain."
4. Boundary. Which rule it hit: egress allowlist, data access policy, approval requirement, credential scope.
5. Control that caught it. Name the layer, or write "none." "None" means the attempt was noticed after the fact, by a person or a review.
6. Outcome. Blocked, escalated to a human, or allowed after review.
7. Severity. Using the scale practice 2.1 calls for: what would have happened if the control had failed?
8. Owner and follow-up. Who reviews it, and whether it changes a control, a prompt, a permission, or nothing.

The two fields that matter most are severity and the control that caught it. Severity tells you how bad the miss would have been. The control column tells you where your real boundary is.

Where the data already lives

You do not need a new tool. Most of this data is already being written somewhere and ignored:

• Egress gateway or proxy denies for traffic from agent identities.
• Policy engine denials on tool calls, where your agent platform has one.
• Approval-gate rejections: a human said no to a consequential action the agent requested.
• Identity provider and cloud access-denied events for agent identities.
• Canary hits: decoy credentials or records that only an out-of-scope agent would touch, as practice 3.5 recommends.
• Harness logs of refused tool calls, kept as practice 5.1 asks.

The work is attribution. Each event needs to map to one agent in the inventory. That only works if each agent has its own identity, which is why the inventory comes first.

Wire it into what you already run

Do not build a separate AI process. Route these events into the SIEM and ticketing system your security operations center already uses:

• Tag every event from an agent identity, so agent activity can be filtered from human activity.
• Create a "near-miss" ticket type with the fields above. Most entries can be created and closed automatically. Severity above a threshold goes to a human.
• Add two fields to your existing incident template, so real incidents feed the same dataset: agent attempted an unauthorized action (yes or no) and control that prevented impact (which one, or none).
• Review the register in the same weekly meeting that reviews other security events, with the agent owners present.

What the board sees each quarter

Directors do not need the rows. Three numbers, per quarter, with the trend:

• Attempts per agent, normalized by activity. A rising rate on one agent is a change in behavior, scope, or exposure worth explaining.
• Catch rate by control layer. If the network catches everything and the approval gate never fires, either your gate is in the wrong place or your agents never ask. Both are findings.
• Count of "control that prevented impact: none." Every one of these is a boundary that did not hold and was found by luck or by review. The target is zero, and the trend matters more than the number.

The format follows our post on board questions before an AI coding tool rollout: a page, not a dashboard.

Two ways this goes wrong

Counting noise. A misconfigured allowlist can generate thousands of denies a day from a legitimate task. If every deny is a near-miss, the register becomes a log, and nobody reads logs. Deduplicate by agent, boundary, and attempted action, and fix the scope before you count it.

Blame. A near-miss register only works if people report into it. Safety-critical industries learned long ago that reporting dries up when a report leads to punishment. Agent owners must be able to log "my agent tried to do this" without that being held against them. The question is always which control failed, not who failed.

How we help

Our GenAI Security Readiness Assessment builds the agent inventory, maps where your deny and rejection events already land, and defines the near-miss criteria and severity scale practice 2.1 calls for. You leave with a register wired into your existing operations and a one-page quarterly format for the board.

Four questions to test yourself

1. If an agent was blocked from an unapproved destination yesterday, would anyone know by today?
2. Can you attribute every deny event to a single agent, or do agents share identities with people and other systems?
3. Do your agent logs keep refused tool calls, or only the ones that ran?
4. What would your board learn from your last quarter of agent near-misses, and does that data exist?

If the answer to the last one is "nothing," start with the two fields on your incident template. They cost nothing, and in three months they become the first trend line your agent controls have ever had.