✓ Human-authored analysis; AI used for formatting and proofreading.
The UK AI Safety Institute published a case study in July 2026. They used frontier AI models to find misconfigurations in their own AWS cloud environment. Over two weeks, using three different approaches, they found and fixed several real issues including a previously undiscovered 5-step attack chain that allowed an ordinary user to escalate privileges, move laterally, and access other users' data.
The cost was under £1,000 in LLM tokens. The most capable model found the critical chain for under £150.
It demonstrates that frontier models can discover real compound attack paths in production cloud environments. It also, unintentionally, demonstrates something else: the class of findings they discovered is the exact class that deterministic configuration evaluation produces without AI, tokens, spending two weeks or the noise.
What AISI found
Three approaches, three different results:
Source code analysis produced the largest number of findings. It was also the noisiest. Many reported issues "were less serious than the model suggested, or not exploitable in practice. Some were design trade-offs we had made deliberately; others were already mitigated elsewhere in the codebase in ways the model had not cross-checked."
The human cost was misleading: easy setup, but the most time-intensive approach because separating important findings from noise "took human judgement, from the most experienced engineers."
Agentic probing and human-in-the-loop red-teaming found the critical 5-step chain: a misconfiguration in environment isolation that allowed privilege escalation and lateral movement. "Exploiting this misconfiguration required five independent steps chained together. Our engineers found the path comprehensible but unintuitive, and it would have been unlikely to surface in routine review."
The meaningful findings such as the compound chain, privilege escalation and lateral movement came only from the approaches that required more human investment to set up. The approach with the least human effor produced the most noise.
The noise is the signal problem
AISI's experience with source code analysis is the false positive problem in miniature. The model scanned the codebase and produced findings. Most findings were technically true. The code does have that pattern. But most weren't exploitable in context. Mitigations existed elsewhere. Design trade-offs had been made deliberately. The model evaluated individual code patterns (nodes) without evaluating how those patterns interact with the rest of the system (edges).
This is the same structural limitation that makes CSPM findings noisy. A scanner evaluates one resource at a time. It reports "this IAM role has broad permissions." True. But the role is constrained by a permission boundary, limited by an SCP, and only assumable from a specific trusted account. The finding is accurate. The risk is zero.
AISI's source code analysis had the same problem at a different layer: true findings without the context that determines whether they matter.
The compound chain is the real finding
The 5-step attack chain was the exercise's most significant discovery. Five independent steps, each individually benign, that together create a complete attack path. The chain was "comprehensible but unintuitive". No individual misconfiguration was obviously wrong. The risk existed only in the relationship between them.
This is a graph-level finding. It requires evaluating not just what each resource's configuration looks like (node properties) but how configurations interact across resources (edge properties). A scanner checking each resource independently sees five compliant resources. The compound chain consisting of an identity with an escalation path, an environment isolation gap, lateral movement and unauthorized data access remains invisible unless you evaluate the underlying graph.
AISI's frontier model found this chain through autonomous exploration. The model probed, escalated, moved laterally, and reached unauthorized data through trial and error. It discovered the chain by exploiting it.
But the chain's preconditions are configuration state. The identity's permissions are in IAM policies. The environment isolation gap is in the resource configuration. The lateral movement path is in trust relationships. Every precondition exists as a static property of the deployed infrastructure, which means it can be read from a configuration snapshot and evaluated by a deterministic predicate.
The model discovered the chain by running. The same chain is discoverable by reading.
Discoverable by reading
A deterministic configuration evaluator takes a snapshot of the cloud infrastructure, including the IAM policies, resource configurations, trust relationships, organizational policies to evaluate compound predicates against it.
The chain AISI found maps to a compound predicate: does an identity exist with an escalation path AND does an isolation boundary have a gap AND does a lateral movement path connect them AND does the destination contain unauthorized data?
Each precondition is a fact extracted from the snapshot. The chain fires when all preconditions are true simultaneously. The output is a deterministic finding: "this 5-step attack path exists in your configuration. Here are the five preconditions, the identity and the path."
Providing identical findings for the same input every time, this determinstic approach is independent of models, token consumed or trial-and-error exploration. The chain is discovered by evaluation the graph instead of a live exploit.
The cost comparison
| Frontier AI (AISI) | Configuration Evaluation | |
|---|---|---|
| Cost per exercise | £150–1,000 in tokens | $0 |
| Time | Two weeks | Seconds |
| Determinism | Non-deterministic | Deterministic |
| Frequency | Periodic | Continuous (CI, scheduled) |
| Infrastructure | Frontier model access | Runs on a laptop |
| Triage required | Experienced engineers | Exploitability is in the output |
| Auditability | Model reasoning is opaque | Evaluation trace is verifiable |
The cost difference is categorical. One approach consumes frontier model compute for every exercise. The other runs locally, offline, with no external dependencies, on every commit.
This compounding divergence in time illustrates the gap between periodic assessment and continuous validation. Rather than dedicating two weeks to an isolated evaluation cycle that requires regular repetition as the capabilities of the model changes, a configuration evaluator runs against every snapshot to provide continuous feedback at the frequency of every commit.
The determinism difference matters for auditors. AISI's finding depends on what the model happened to explore. A different run might find a different chain or miss it entirely. A deterministic evaluator finds the same chains from the same snapshot every time. The findings are reproducible. The evaluation trace is verifiable. An auditor can re-run the assessment and get the same results.
Where AI wins
AISI's exercise demonstrates value in two places a deterministic evaluator cannot reach:
Application logic bugs. The model can reason about what the code DOES, not just how the infrastructure is configured. A misconfiguration in application-level access control. The code doesn't check a permission it should. This is outside the scope of configuration evaluation. The model found these by reading the code.
Creative exploration. The model's agentic probing discovered an attack chain that was "unintuitive". The path wouldn't occur to a human reviewer, and it isn't in any standard misconfiguration checklist. A deterministic evaluator only finds chains that are enumerated in its catalog. A model can discover chains that nobody thought to check for.
These are real capabilities. For organizations that need to find application logic bugs and novel attack patterns, frontier AI is a valuable tool.
In the specific context of infrastructure misconfigurations, which represent the stated target of AISI's exercise, the model executes expensive and non-deterministic work to discover findings that are deterministically discoverable from the configuration state.
The compound chain test
Here's a test for whether a finding needs AI:
Can every precondition of the finding be expressed as a property of the configuration snapshot?
If yes, if the identity's permissions, the trust relationship, the network path, the SCP, the resource policy are all static configuration state. The finding is deterministically discoverable. Using a frontier model to find it is a token cost with no information-theoretic justification. The model isn't adding information. It's expensively reconstructing information that's already in the snapshot.
If no, if the finding requires understanding application logic, runtime behavior, or creative reasoning about uncharted attack surfaces. The model is adding information the snapshot doesn't contain. That's where AI earns its token cost.
AISI's 5-step chain passes the test: every precondition is configuration state. The identity's permissions are in IAM. The isolation gap is in the resource configuration. The lateral movement path is in trust relationships. Snapshot. Predicate. Finding. No tokens required.
What this means for defenders
AISI's case study will be cited by enterprises justifying AI security budgets. "A government lab found critical misconfigurations using frontier AI" is a compelling case for procurement.
The more precise claim: a government lab found critical configuration-graph misconfigurations that are deterministically discoverable from a static snapshot, using a non-deterministic method that costs £1,000 per exercise and requires two weeks of engineer time to set up and triage.
Defenders should use AI for the things only AI can do: application logic review, creative attack surface exploration, novel vulnerability discovery. For configuration misconfigurations, the largest and most common attack surface in cloud environments. The tools already exist to find them exhaustively, deterministically, continuously, and for free.
AISI proved AI can find cloud misconfigurations. The question is whether AI is the right tool for findings that are deterministic properties of the configuration graph. For those, the answer has been available since before frontier models existed. The answer is a predicate.
Stave evaluates configuration snapshots against compound chain rules deterministically. The 5-step attack chain AISI discovered — privilege escalation + isolation gap + lateral movement + unauthorized data access is the class of finding Stave's Soufflé engine produces from a static snapshot. No tokens. No frontier model. No two-week exercise. Same findings, every commit. stave apply --observations ./your-snapshot/
Top comments (0)