DEV Community: Trust Boundary

The Capital One breach was not an SSRF story

Trust Boundary — Wed, 16 Sep 2026 00:40:52 +0000

Originally published at trustboundarystudio.com. The video version, with diagrams, is on YouTube.

Key facts

When: 22 to 23 March 2019. Discovered 17 July 2019, after an outside party emailed Capital One's responsible disclosure address.
Scale: Personal data of 106 million people.
Entry: Server-side request forgery through a misconfigured web application firewall, returning IMDSv1 credentials.
What made it catastrophic: The IAM role attached to the firewall instance could read S3 buckets across the account.
Penalty: $80 million OCC civil money penalty, August 2020. $190 million class action settlement.
Primary sources: OCC Consent Order 2020-036. FBI criminal complaint. MIT Sloan case study.

In March 2019 someone obtained the personal data of 106 million people from
Capital One. No malware. No zero-day. No stolen employee password. They asked a
web application firewall to make a request on their behalf, and it did.

That much is well known. The part that gets lost is what happened next, and
what the regulator actually penalised eighteen months later. Because the
Office of the Comptroller of the Currency's consent order does not mention
server-side request forgery at all.

The system

Capital One had moved a large part of its IT operations into AWS starting
around 2015, further and faster than most banks its size. That is not the
failure. That is ordinary modernisation, and it mostly went well.

Credit card applications going back to 2005 sat in S3. In front of the
application layer sat a web application firewall running on an EC2 instance.
Its job was to inspect incoming requests and block malicious ones. It was a
security control, and it is the thing that got used to break in.

To do that job, the firewall instance had an IAM role attached.

An IAM role is a set of permissions. Attach one to an EC2 instance and anything
running on that instance can use those permissions. No password, no key on
disk. It is how almost every workload in AWS talks to other AWS services, and
it is a genuinely good design. It is also one of the most consequential
configuration decisions most teams make once and never look at again.

Why the metadata service answered

The firewall was misconfigured in a way that allowed server-side request
forgery. Normally you send a request to a server and it answers. In an SSRF you
send a request that convinces the server to fetch something for you and hand
you the response. That matters because of where the server is standing. You are
outside. It is inside. Anything it can reach, you can now reach through it.

On an EC2 instance there is one address that is always reachable and always
interesting: 169.254.169.254. That is the instance metadata service. It is
link-local, so it exists only from the perspective of the instance itself, and
you cannot route to it from the internet. What it returns includes temporary
credentials for whatever role is attached.

This is not a flaw. It is the mechanism that means you do not hardcode access
keys into your application, which is one of the better patterns AWS ever
shipped. But in 2019 that service, now called IMDSv1, answered any plain HTTP
GET originating from the instance. No token. No authentication. If you could
make the instance issue a request, you got credentials back.

Three commands

The FBI complaint describes three.

The first obtained security credentials. The role appears in the indictment
only as *****-WAF-Role; the rest is redacted, and anyone quoting a full role
name is guessing. The firewall handed over its own credentials, working as
designed at every individual step.

The second listed the names of folders and buckets in Capital One's storage.
The third copied data out of them.

This is where an interesting incident becomes a catastrophic one. Ask what a
web application firewall actually needs. It inspects traffic, matches patterns,
blocks or forwards requests. There is a plausible reason for it to reach S3:
rule sets, configuration, logging. There is no reason for it to enumerate
storage across the account and read the contents.

So the third command was not an exploit. It was a copy. Ordinary S3 operations,
correctly authenticated, fully permitted, and at the API level
indistinguishable from legitimate traffic.

The encryption did not help

Capital One's own statement says they encrypt as standard, and then says this:

Due to the particular circumstances of this incident, the unauthorized access
also enabled the decrypting of data.

The credentials that could read the data could also decrypt it. That is what
encryption at rest is for: a stolen disk, not a valid caller. It is worth being
precise about this, because "the data was encrypted" is repeated constantly as
though it were mitigation, and here it was not.

The intrusion was not a chain of escalating exploits. It was one boundary
crossing followed by entirely legitimate use of over-granted permissions.

117 days, and it was an email

The intrusion happened on 22 and 23 March 2019. Capital One found out on 17
July.

It was not detection tooling that ended it. Someone noticed the data described
on a public GitHub page and wrote to Capital One's responsible disclosure
address. An outside party, reading a public post, told the bank it had been
breached.

What the regulator actually found

The financial consequences were an $80 million civil money penalty from the OCC
and a $190 million class action settlement. The consent order is the part worth
reading, and it is not about the SSRF.

The OCC found that Capital One failed to establish effective risk assessment
processes before migrating its IT operations to the cloud. That internal audit
failed to identify the control gaps. And that the board failed to hold
management accountable.

Not "you got hacked". You did not know what your own environment allowed.

That distinction is the reason this incident is still worth studying. A
vulnerability is a thing you fix. Not knowing what your permissions grant is a
condition you live in, and it is invisible right up until the moment it is not.

Which control would have held

Three candidates, and the order matters.

Fix the firewall misconfiguration. True, and the weakest of the three,
because it assumes you never ship a vulnerability. You will.

The metadata service. On 19 November 2019, four months after this became
public, AWS shipped IMDSv2. It requires a session token obtained by an HTTP PUT
before it will answer, and the choice of PUT is deliberate: most misconfigured
firewalls and reverse proxies do not forward PUT at all. AWS was explicit that
this is defence in depth against exactly this class of problem. If you are
running EC2 today with v1 still enabled, that is the actionable item here.

The IAM role. This is the real one. The first two stop this attack.
Scoping the role limits every attack.

Had that role carried read access to the buckets a firewall actually needs, the
same SSRF, the same stolen credentials and the same three commands would have
returned firewall configuration. Still an incident. Still an investigation. Not
106 million people.

That is the difference between a vulnerability and a catastrophe: not whether
someone gets in, but how far the credentials they find will carry them. Least
privilege is not a compliance checkbox. It decides the size of your worst day.

The pattern underneath

It is not the SSRF that recurs. It is the assumption beneath it, that a service
inside your perimeter is trustworthy because it is inside your perimeter.

The firewall was trusted because it was internal. The metadata service answered
because the request came from the instance. The role was broad because scoping
it properly is tedious and nothing had gone wrong yet.

Every one of those decisions was locally reasonable. That is what a trust
boundary failure looks like in practice. Not a dramatic break-in, but a series
of sensible choices, one of which granted far more than anybody checked.

Go and look at your instance roles this week. Not the ones you wrote recently.
The ones attached to something that has been running since before you arrived.

Sources

Every claim above comes from primary documents rather than coverage of them.

OCC Consent Order #2020-036 (AA-EC-20-51), 6 August 2020. https://www.occ.gov/static/enforcement-actions/ea2020-036.pdf
FBI criminal complaint, US District Court at Seattle, 2019.
Neto & Madnick, "A Case Study of the Capital One Data Breach", MIT Sloan,
1. https://cams.mit.edu/wp-content/uploads/capitalonedatapaper.pdf
Capital One incident disclosure and FAQ, July 2019.
AWS, "Defense in depth: open firewalls, reverse proxies and SSRF vulnerabilities with EC2 IMDS", November 2019.

Corrections are welcome, and any made are listed, dated, at the end of this article.

Questions this answers

Was the Capital One breach caused by SSRF?

The entry point was server-side request forgery through a misconfigured web application firewall. What made it a 106-million-record breach was the IAM role attached to that instance, which could read S3 buckets across the account. The regulator's consent order does not mention SSRF at all.

Why did encryption not protect the data?

The data was encrypted at rest, but the stolen credentials were valid, and credentials that can read the data can also decrypt it. Encryption at rest protects against a stolen disk, not a legitimate caller. Capital One's own statement says the unauthorised access also enabled the decrypting of data.

What did the OCC actually penalise Capital One for?

Failing to establish effective risk assessment processes before migrating IT operations to the cloud, an internal audit that did not identify the control gaps, and a board that did not hold management accountable. The penalty was $80 million, in August 2020.

What is IMDSv2 and would it have stopped this?

IMDSv2 is the second version of the EC2 instance metadata service, released 19 November 2019. It requires a session token obtained by an HTTP PUT before answering, and most misconfigured firewalls do not forward PUT. It would have blocked this particular path. Scoping the IAM role would have limited every path.

The video version

Your change process governs code. This was not code.

Trust Boundary — Wed, 16 Sep 2026 00:39:58 +0000

Originally published at trustboundarystudio.com. The video version, with diagrams, is on YouTube.

Key facts

When: 19 July 2024. Roughly 99% of Windows sensors back online by 29 July.
Scale: 8.5 million Windows devices, under one percent of all Windows machines. Microsoft's estimate.
Cause: The IPC Template Type defined 21 input fields. The integration code supplied 20. No build step compared them.
Why it slept four months: Every earlier instance matched the 21st field with a wildcard, so the value was never read. A 19 July instance used a real match.
Effect: An out-of-bounds memory read in a kernel driver, a blue screen, and a boot loop because the file was already on disk.
Not an attack: CrowdStrike's analysis, backed by an independent review, confirmed the defect was not exploitable.
Primary sources: CrowdStrike External Technical Root Cause Analysis, 6 August 2024. Microsoft, 20 July 2024.

On 19 July 2024, 8.5 million Windows machines stopped booting. Airlines,
hospitals, banks and broadcasters, at the same time, in the same hour.

It was not an attack. CrowdStrike's own analysis, backed by an independent
review, confirmed the defect was not exploitable. What stopped those machines
was a configuration file, and the reason it reached them without anyone
approving it is worth more of your attention than the bug itself.

Two lanes into the same kernel

CrowdStrike Falcon is an endpoint detection agent. To see process creation,
file access and network calls, part of it runs in the Windows kernel. That is
not a design flaw. It is what the job requires, and most endpoint security
products do the same thing.

Falcon takes two different kinds of update.

Sensor Content is the agent itself. Code. It ships on the sensor's release
cycle, with the testing and staged rollout you would expect of something running
in a kernel.

Rapid Response Content is configuration. Data telling the existing sensor
what to look for. It ships in minutes, because the entire value of the product
is reacting to new threats in hours rather than months.

Two lanes, different speeds, different levels of scrutiny, and good reasons on
both sides. Everything that follows lives in the gap between them.

One number, two beliefs

In February 2024, with sensor version 7.11, CrowdStrike introduced a new
Template Type giving visibility into attacks abusing named pipes and other
Windows interprocess communication.

That template defined 21 input fields.

The integration code that actually invoked the Content Interpreter supplied
20.

Two components, each internally consistent, each written carefully, and no
build step compared one to the other. CrowdStrike's own words: the number of
fields in the IPC Template Type was not validated at sensor compile time.

A note on a detail that trips up most write-ups. CrowdStrike's executive
summary says the sensor "expected 20 input fields, while the update provided
21." That is true, and it reads as though the July update was malformed. It
was not. It was valid against the Template Type it was written for. The full
RCA explains why the interpreter expected 20 in the first place, and that
explanation is five months older than the outage.

Four months of evidence, all of it wrong

The mismatch shipped in February and did nothing at all.

On 5 March, after a successful stress test, the first Rapid Response Content for
Channel File 291 went to production. It worked. Three more updates went out
between 8 and 24 April. CrowdStrike's words again: they performed as expected in
production.

Four months. Four successful deployments.

Here is why. Every one of those instances matched the twenty-first field with a
wildcard. A wildcard matches anything, so the interpreter never had to go
and read that value. The field was declared. It was never requested.

This is the part worth sitting with, because it is the part that generalises.
The defect was present the whole time. Every successful deployment was treated
as evidence, and the evidence was worthless. It did not show the code was
correct. It showed that nobody had yet asked the one question that would break
it.

Testing had the same blind spot for the same reason. The channel file used in
development and release testing carried a wildcard in the twenty-first field
too, so the tests exercised everything except the path that mattered.

19 July

Two more Template Instances were deployed. One introduced a real matching
criterion on the twenty-first field. Not a wildcard, an actual value to compare
against.

So for the first time, the Content Interpreter went to read input number 21.

There were 20.

Why a bug became a global event

Reading past the end of an array is an out-of-bounds memory read. In user space
that is a crash of one process: Windows kills it, you get a dialog, you move on.

In kernel space there is nothing above you to catch it. The kernel cannot safely
continue when it does not know what it just read, so it stops. Blue screen.

And because the sensor loads at boot, and the file was already on disk, the
machine crashed again on restart. And again.

That is what turned a defect into a global event. Not that machines crashed, but
that they could not come back on their own.

The update reached hosts in minutes, because reaching hosts in minutes is what
Rapid Response Content is for. The capability that makes the product valuable is
the same capability that made the blast radius global.

Microsoft estimated 8.5 million Windows devices, under one percent of all
Windows machines. That number tells you something about which machines run
endpoint security agents. Not the laptops. The check-in terminals, the
scheduling systems, the payment infrastructure.

Ten days, because the fix needed hands

The fix was trivial: boot into safe mode, delete one file, restart.

Now do that 8.5 million times, individually, on machines in locked server rooms,
on aircraft and in hospitals. Many were encrypted with BitLocker, needing a
recovery key that was often stored in a system which was itself down. There was
no remote fix, because remote management requires a machine that boots.

CrowdStrike reported roughly 99% of Windows sensors back online by the evening
of 29 July. Ten days.

Where the boundary was

Between code and configuration, and it was assumed rather than enforced.

Configuration is treated as safer than code. It ships faster, with lighter
review, and that is a deliberate and defensible trade. You cannot fight fast
threats on a slow release cycle.

But configuration is only safe if the thing interpreting it is defensive. The
moment a config file can steer a kernel-mode parser that does not check its
bounds, that file is code. It has the same power to halt the machine. It just
travels through a pipeline built for something less dangerous.

Three controls, and the order matters.

Bounds checking in the interpreter. CrowdStrike added it on 25 July. This is
the real fix, because it makes the whole class of defect survivable regardless
of what content arrives.

Staged rollout for content, not just code. CrowdStrike now runs deployment
rings with canary testing and bake-in time for Rapid Response Content. That does
not prevent the defect. It caps the blast radius, and the difference between the
first ring failing and 8.5 million machines failing is entirely rollout policy.

Customer control over deployment. Before July 2024 you could not stage this
content yourself. It arrived. CrowdStrike has since shipped controls for it, and
if you run Falcon and have not configured them, that is the action item here.

The uncomfortable part, which is not about CrowdStrike

Count the vendors that can push content into your environment without your
approval.

Endpoint agents. Antivirus definitions. Browser policies. Managed device
configuration. Cloud agents. Most of them auto-update, most run privileged, and
most of that content never passes through your change process, because it is not
code and your change process governs code.

Every one of those is a channel where somebody else's data becomes execution on
your machines.

That is what a trust boundary failure looks like when nobody attacks you at all.
Not a break-in. A pipeline built for one level of risk, quietly carrying
something more dangerous, working perfectly for four months.

This week, go and find out which of your vendors can reach your kernel without
asking you first, and whether any of them will let you stage it.

Credit where it is due

CrowdStrike published a real root cause analysis, with dates, mechanisms and
mistakes in it. Most of this piece is built from their document. That is the
behaviour you want from vendors when something goes wrong, and it is rare enough
to be worth saying out loud.

Sources

CrowdStrike, External Technical Root Cause Analysis: Channel File 291, 6 August 2024. https://www.crowdstrike.com/wp-content/uploads/2024/08/Channel-File-291-Incident-Root-Cause-Analysis-08.06.2024.pdf
CrowdStrike, Falcon Content Update Preliminary Post Incident Report, 24 July 2024. https://www.crowdstrike.com/en-us/blog/falcon-content-update-preliminary-post-incident-report/
Microsoft, Helping our customers through the CrowdStrike outage, 20 July 2024, for the 8.5 million figure. https://blogs.microsoft.com/blog/2024/07/20/helping-our-customers-through-the-crowdstrike-outage/

Corrections are welcome, and any made are listed, dated, at the end of this article.

Questions this answers

Was the CrowdStrike outage a cyberattack?

No. CrowdStrike's root cause analysis, backed by an independent third-party review, confirmed the defect was not exploitable. It was a configuration content update that triggered an out-of-bounds memory read in a kernel driver. Nobody got in. This was an availability failure from start to finish.

What was Channel File 291?

A Rapid Response Content file for the Falcon sensor's IPC Template Type, introduced with sensor 7.11 in February 2024 to detect attacks abusing Windows named pipes. The Template Type defined 21 input fields but the integration code supplied 20. A 19 July update with a non-wildcard match on the 21st field caused the sensor to read past the end of its input array.

How many machines did the CrowdStrike outage affect, and how long did recovery take?

Microsoft estimated 8.5 million Windows devices, under one percent of Windows machines. CrowdStrike reported roughly 99% of Windows sensors back online by 29 July, ten days later, because the fix required booting each machine into safe mode and deleting a file by hand. There was no remote fix, because remote management requires a machine that boots.

Why is the CrowdStrike outage described as a change control failure?

Rapid Response Content is configuration, not code, so it shipped in minutes with lighter review and never passed through customer change processes. But a configuration file that can steer a kernel-mode parser with no bounds check has the same power as code. The pipeline was built for one level of risk and carried something more dangerous.