Same attack, two companies. One never responded.
CISA ran the same red team playbook against two organizations at the same time and published both outcomes. One SOC quarantined the machines. The other received the alerts and never responded — and the reason it didn't is the cheapest problem in this whole story to fix.
In August, CISA published something unusual: the results of two red team assessments run at the same time, against two different critical-infrastructure organizations, using similar tradecraft. Advisory AA26-237A, titled "A Tale of Two SOCs."
It is unusual because it is the experiment nobody gets to run on themselves. Same attackers, same playbook, same week, two environments — and the results printed side by side.
Both organizations were compromised at domain level. In both, the red team reached sensitive business systems and cloud resources. So the interesting variable is not whether the attack worked. It worked twice.
The interesting variable is what happened in the minutes after.
The one that worked
Organization B, a water and wastewater entity, was phished. Three users clicked, and the red team landed on three workstations.
The SOC caught the payload on each one. Every execution generated a medium-severity alert; staff quarantined the workstation, analyzed it, and reimaged it before putting it back online. Three times. Command-and-control never held.
Then something worth noticing happened. Because the defenders removed that initial foothold, the red team could not proceed — so the engagement was restructured. Organization B's own trusted agents — IT staff who knew about the assessment — executed a red-team-provided payload on a designated internal host tied to a standard, non-administrative account, replicating the access the team would have had if they hadn't been thrown out.
Assume-breach is a standard phase of these assessments, so this isn't a trophy on its own. What it does tell you is the direction the pressure ran. The team didn't move to assume-breach on schedule; they moved to it because containment worked.
Later, when the red team reached a bastion host in the OT DMZ, that host blocked outbound connections and the payload execution triggered an alert. Staff quarantined it. When the team logged into the Azure portal with a compromised account, an automated alert fired and staff blocked the account.
None of that is exotic. It is ordinary tooling, plus somebody reading the output.
The one that didn't
Organization A, in government services, did not respond.
Not to the phishing. Not to the privilege escalation — a default Machine Account Quota and a misconfigured AD CS certificate template. Not to credentials sitting in cleartext, or static cloud access keys set never to expire. At one point the red team used an over-permissioned application to review SOC staff email, specifically to see whether staff were aware of the compromise. The SOC never responded.
Here is the part that should change how you think about your own program.
The alerts fired.
The advisory is explicit: the SOC received medium- and low-severity EDR alerts related to red team activity "but did not respond to them." Thousands of false positive alerts corresponding to normal business operations — many at higher severity — obscured the real ones.
That is not a detection failure. The tools detected it. It is a response failure, and it has a completely different fix, at a completely different price.
The detail worth an afternoon
One alert in that environment was real, and a human looked at it.
Red team members observed defenders' chat about an alert on an SCCM server. The defenders tried to determine who owned that system, what it did, and how it was normally used. They could not. The SOC eventually flagged it a false positive.
Read that again slowly. The detection worked. The alert was true. It died in triage because the asset inventory could not name a person.
Which means your CMDB is doing detection work, whether or not anyone staffed it that way. An alert arrives carrying a hostname. Whether it survives the next four minutes depends entirely on whether someone can attach meaning to that hostname at two in the morning.
CISA also found Organization A running multiple SOCs and multiple EDR products whose staff neither communicated with each other nor had visibility into each other's detection tools — and SOC staff and system owners did not communicate either. More consoles is not more coverage. Each additional console is another place a signal can arrive and stop.
The advisory's closing line is blunt, and worth quoting exactly:
"Detection tools are only as effective as the people, processes, and procedures supporting them."
It is not one unlucky organization
You could read all of that as a story about one badly-run SOC. The numbers say otherwise.
The Blue Report 2026, published in August by Picus Labs, measured more than 338 million attack simulations run in real production environments. Two of its scores matter here:
- Logging: 58%
- Alerting: 14% — flat year over year
Fewer than one simulated attack in seven produced an alert at all.
Worth naming the obvious: Picus sells breach-and-attack-simulation tooling, and a report finding that untested detection underperforms is a report that sells more testing. Read it as vendor research. It is still worth citing, because the methodology is stated, the sample is enormous, and the finding is independently corroborated by a government advisory with nothing to sell — one describing a SOC that received real alerts and did not act on them.
Put the two numbers side by side, because the gap is the entire argument. More than half the telemetry is arriving. The event gets written down. What almost never happens is a rule that reads it and routes it to a human.
That is not a collection problem, and it will not be fixed by more retention or a bigger ingest bill. It is a detection-engineering problem. One of those costs money. The other starts with an afternoon.
The afternoon
Five events. Not fifty.
Fifty detections is a project, a budget line, and a thing you never start. Five is an afternoon. Pick five you would be embarrassed to miss, and leave the rest for next quarter:
- A new global or domain administrator
- A log source going dark
- An impossible-travel or anomalous-location login
- A mass export or bulk download
- A new mailbox forwarding or inbox rule
Pick by consequence, not by ease of detection. The temptation is to choose the five you already know alert — which measures your confidence, not your coverage.
Then, for each one, three questions:
- Is it logged? Usually yes. Name the source and the retention window, not just "yes."
- Does it raise an alert? Now you're guessing. A rule in a vendor template is not a rule that is enabled and scoped to your tenant.
- Who receives it, at 2am? This is the one that decides everything.
Question three is where most programs quietly die. "Does it alert" is answerable on paper. "Who receives it at 2am" is answerable only by looking, and the answer is startlingly often a distribution list with one person on it who left, or a channel nobody has opened since the integration was built.
Then stop guessing
Everything above is still paperwork. This part isn't.
Generate the event. Get approval, work inside a change window, and tell the people who would otherwise page out that it's you. Create a throwaway administrator. Trigger a deliberate bulk download. Add a test forwarding rule.
Then watch three places: the console that should show it, the channel that should carry it, and the human who should receive it. Record the wall-clock deltas — trigger to alert, alert to acknowledgement. Those numbers are the deliverable.
And record the silences. An event that produced nothing is the most valuable row on the page, because it converts "would we catch that?" from an assumption into a finding — one you made on your own schedule instead of during an incident.
Then remove what you created, and log the removal.
The artifact is the point
The rule carries over from every review in this series: every claim gets a number and a date.
"We'd catch that" is a mood. Something shaped like "five events tested Sep 10; three alerted within four minutes; one alerted to a dead channel; one produced nothing — two tickets filed, retest Sep 24" is an artifact. It is also the exact thing that answers an assessor asking whether your monitoring is real or aspirational, and the kind of evidence assessors and insurers increasingly ask about.
Fix the routing before you fix the rule, incidentally. An alert nobody receives and an alert that doesn't exist cost precisely the same during an incident, and the routing is usually the cheaper repair.
Both organizations in that advisory had tools. Both had logs. One of them knew what it would see.
Your logging is probably fine. Your alerting is the open question.
The worksheets are free. The Alert Coverage Kit (Gatorbyte #012) is a self-contained offline HTML file — the five-event picker, the three-question test, the generate-and-watch protocol, a coverage log with time columns, the asset-ownership check, and an evidence register — plus CSV and Excel export of every worksheet. No account required — thesecuritygator.gumroad.com/l/gb012-alert-coverage-kit
This is a review format and a practical starting point to adapt for your organization. It supports detection-coverage documentation and audit-preparation workflows; it is not legal, compliance, or audit advice, and not a substitute for managed security or incident-response services. Generating test events changes state in a live environment — get approval and work inside a change window.
Sources: CISA AA26-237A — "A Tale of Two SOCs" · CISA KEV alert, Sep 2 · Blue Report 2026, Picus Labs
Top comments (1)
The comparison between the two organizations highlights a crucial aspect of incident response: the importance of having a well-maintained asset inventory. It's alarming how a genuine alert can be dismissed due to a lack of understanding of system ownership and usage. Strengthening asset management practices could significantly enhance response times and effectiveness. If you're looking to bolster that area or need additional engineering support for any future assessments, I’d be glad to explore a paid collaboration. What strategies do you think are most effective for improving asset visibility in SOCs?