DEV Community: Bala Paranj

Your Cloud Has 10^100 Possible States. Stop Trying to Predict Which Ones are Safe. Start Defining Which Ones are Forbidden.

Bala Paranj — Thu, 27 Aug 2026 12:43:17 +0000

A cloud account with 1,000 resources, each with 10 configurable attributes, has roughly 10^1,000 possible configuration states. More states than atoms in the observable universe by a factor of 10^920.

Prediction-based security asks: "which of these 10^1,000 states are safe and which are unsafe?" Then classifies each observed state against the model.

That question is mathematically not answerable. The state space is too large to enumerate, sample representatively and model. No ML system can learn it. No rule engine can traverse it. No human can reason about it.

There's a different question that can be answered: "which PATTERNS of state must never exist?"

The pattern count isn't 10^1,000. It's hundreds to thousands. A pattern like "no S3 bucket shall be publicly readable" covers an INFINITE number of specific forbidden states using a SINGLE statement. The pattern set grows linearly with the catalog. The state space grows exponentially with the cloud.

The first question is prediction. The second is elimination. The entire cloud security industry invested in prediction. The mathematics says only elimination works at scale.

Two modes of reasoning

Mode	Question	How it scales
Prediction	What COULD happen? Classify each possibility.	Exponential in state space — intractable
Elimination	What must NEVER happen? Check current state against forbidden patterns.	Linear in pattern count — tractable

The two modes have different computational properties:

Property	Prediction	Elimination
Setup	Enumerate possible states (infeasible)	Enumerate forbidden patterns (feasible)
Per-evaluation	Search through possible states	Apply patterns to current state
Complexity	Exponential	Polynomial
Failure mode	Misses novel states outside training distribution	Misses forbidden patterns not yet cataloged
Output	Probability ("78% likely unsafe")	Category ("COMPLIANT or NON_COMPLIANT")

Prediction has BOTH infeasible setup AND infeasible evaluation. Elimination has BOTH feasible setup AND feasible evaluation. The difference is categorical. One works at cloud scale. The other mathematically cannot.

Why prediction-based tools fail at scale

Most cloud security tools implicitly operate in prediction mode:

Tool	Prediction behavior	Why it fails at scale
ML anomaly detection	Models "normal" behavior; predicts deviations	Training distribution doesn't match production; novel states escape
Threat intel correlation	Predicts which patterns indicate attack	New patterns aren't in intel feeds; always behind
Behavioral baselines	Models expected behavior; flags anomalies	"Normal" changes faster than baselines update
Risk scoring	Predicts which findings are critical	Severity rankings don't match business risk
Attack-path analysis	Predicts paths attackers might take	Real attackers find paths the model didn't anticipate

Each tool's value depends on PREDICTION ACCURACY. The accuracy depends on COVERAGE of the possible-state space. The coverage is STRUCTURALLY BOUNDED by combinatorial explosion.

The failure modes are INTRINSIC to prediction-mode reasoning:

ML drift:              Training distribution ≠ production (always, by construction)
Threat intel lag:      New patterns arrive after exploitation (always, by definition)
Baseline obsolescence: "Normal" evolves faster than baselines (always, in dynamic systems)
Score miscalibration:  Severity ≠ business impact (always, without business context)
Path incompleteness:   Real attackers find unmodeled paths (always, in complex systems)

Each failure is a CONSEQUENCE of attempting to predict in a space too large to traverse. Making the tools smarter or faster doesn't overcome the combinatorial barrier. The barrier is mathematical, not engineering.

How elimination scales

Elimination scales because of four properties:

1. Forbidden patterns are sparse

Cloud systems have MANY possible states but only SOME are unsafe. The unsafe set is a small fraction. An invariant like "no public bucket" eliminates a specific PATTERN — not a specific state. The pattern covers ALL states where ANY bucket has public_read=true, regardless of every other attribute.

One pattern. Infinite specific forbidden states covered.

2. Patterns generalize

"No S3 bucket may be publicly readable" covers:

→ All current S3 buckets in the account
→ All future S3 buckets (not yet created)
→ Buckets in every region
→ Buckets with any other attribute combination
→ Buckets regardless of who created them or when

One statement. Universal coverage for that pattern. No ML model achieves this generalization because ML models learn from SPECIFIC EXAMPLES, not from UNIVERSAL PATTERNS. The pattern's explicitness makes generalization work.

3. Evaluation is local

Checking whether a specific asset matches a forbidden pattern doesn't require knowing the rest of the state space. The check is LOCAL to the asset being evaluated:

For each asset in snapshot:
    For each control in catalog:
        Does this asset match this forbidden pattern?
        Yes → mark violation
        No  → continue

Complexity: O(assets × controls)
NOT:        O(possible-states)  ← which would be O(10^1000)

The evaluation parallelizes well, produces interpretable per-asset results, and doesn't require re-evaluating old assets when new ones are added.

4. Patterns compose

Multiple patterns apply to the same state. The verdict is the CONJUNCTION: if pattern A holds AND pattern B holds AND pattern C holds → all three safety properties are satisfied. The composition is straightforward and formally sound (Alpern-Schneider: safety properties compose by conjunction).

Prediction-mode tools DON'T compose this easily. Two ML models' predictions can conflict. Two risk scores can disagree. Two behavioral baselines can contradict. The composition problem is unsolved in prediction mode and trivially solved in elimination mode.

The type-system analogy

The elimination approach has a precise analogy in programming languages: TYPE SYSTEMS.

A type system doesn't predict which programs will crash. It ELIMINATES a CLASS of programs, those that don't typecheck. The ill-typed programs are FORBIDDEN. They don't exist as valid programs.

Type system:     "This function requires an integer. You passed a string. REJECTED."
Invariant system: "This bucket requires private access. You configured public. REJECTED."

Both are ELIMINATION reasoning. Both catch CLASSES of errors, not specific instances. Both operate at the STRUCTURAL level, not the behavioral level.

The type-system revolution in programming caught classes of bugs that runtime checks couldn't such as null pointer dereferences, type mismatches, uninitialized variables. The invariant system catches classes of misconfigurations that runtime monitoring can't such as public resources, overpermissive roles, expired credentials, toxic combinations.

The parallel is structural:

Property	Type system	Invariant system
What it eliminates	Ill-typed programs	Unsafe cloud states
How it specifies	Type declarations	Catalog entries (CEL predicates)
When it checks	Compile time	Pre-deployment + periodic evaluation
What passes	Programs that typecheck	States that satisfy all invariants
What's rejected	Programs that don't typecheck	States that violate any invariant
Failure mode	Overly restrictive types → valid programs rejected	Overly restrictive invariants → valid configs rejected

The theoretical foundations are mature

Elimination-mode reasoning has deep foundations across multiple engineering disciplines:

Discipline	Elimination mechanism	Maturity
Formal methods	Specifications as constraints verified by model checkers	40+ years
Type systems	Forbidden programs eliminated at compile time	50+ years
Database integrity	Constraints as forbidden states (CHECK, FOREIGN KEY)	40+ years
Hardware verification	Safety properties verified by SMT solvers	30+ years
Control theory	Safety regions defined; controller prevents departure	70+ years
Cloud security	???	Least mature

Every other engineering discipline adopted elimination-mode reasoning decades ago. Cloud security is the LAST major discipline to make the move. The theory exists. The engineering patterns exist. The tools exist (SMT solvers, model checkers, constraint engines). The application to cloud security is the missing step.

The practical-theoretical synthesis

Pure elimination (theoretical only) produces specifications nobody implements. Pure prediction (practical only) produces tools that fail at scale. The system that works operates in the INTERSECTION:

Pure theoretical:     Formal specifications → nobody implements them
Pure practical:       ML/behavioral tools → fail at combinatorial scale
Intersection:         Formally specified invariants evaluated against cloud state

The intersection means:

Practically anchored. Every catalog control corresponds to a REAL misconfiguration pattern observed in the field. The catalog is empirically driven. Each control exists because someone observed a real breach or misconfiguration it would catch.

Theoretically disciplined. Every catalog control is a PRECISE specification, not a probabilistic detector. The control either fires or doesn't. The evaluation is deterministic. The verdict is categorical.

The practical anchoring ensures RELEVANCE (the catalog forbids states that cause breaches). The theoretical discipline ensures COVERAGE CLARITY (the catalog names categories, not just instances — "any public storage" covers both observed and not-yet-observed instances of the pattern).

What elimination doesn't cover

Novel attacks. New attack patterns require new forbidden states. The catalog must keep pace with attacker innovation. Until a new pattern is cataloged, elimination can't catch it. This is the same limitation type systems have: they catch known bug classes but not logic errors.

Probabilistic safety. Some safety concerns are genuinely probabilistic such as anomaly detection of UNKNOWN patterns, user-behavior analytics, zero-day detection. These don't reduce to elimination. They remain in prediction territory.

Context-dependent states. "Public S3 bucket" is forbidden in most contexts but ALLOWED for specific public-asset hosting. Elimination handles this via profile-based catalog selection, but the context-dependence adds catalog complexity.

The bounded claim: elimination covers the CATALOG-COVERABLE subset of cloud security where 95%+ of misconfiguration breaches live. The remaining 5% (novel attacks, genuinely probabilistic threats) still needs prediction-mode tools. The two approaches are complementary.

The investment question

The cloud security industry's investment is overwhelmingly in prediction:

PREDICTION INVESTMENT (current):           ELIMINATION INVESTMENT (needed):
    ML model training                          Catalog authoring
    Threat intelligence feeds                  Pattern discovery  
    Behavioral baseline tuning                 Catalog review and testing
    Severity score calibration                 Invariant design
    Attack-path simulation                     Catalog coverage expansion

The investment is categorically misaligned with the problem's mathematical structure. Prediction investment hits diminishing returns because the combinatorial barrier is STRUCTURAL. Elimination investment has INCREASING returns because each new pattern covers an infinite class of specific states.

Should we stop investing in prediction? Some prediction capability is valuable for the genuinely novel. The question is: "should 90% of our investment go to prediction when elimination covers 95%+ of the problem more effectively?"

For founders: prediction vs elimination checklist

Test your system against the distinction:

1. Does your system enumerate POSSIBLE states or FORBIDDEN states? If possible → prediction mode (combinatorial barrier). If forbidden → elimination mode (tractable).

2. Is your output PROBABILISTIC or CATEGORICAL? If probability → prediction. If binary/categorical → elimination. Categorical outputs drive pipeline action without interpretation.

3. Does adding cloud complexity INCREASE or NOT AFFECT your evaluation cost? If increase → prediction (evaluation cost grows with state space). If not affected → elimination (evaluation cost depends on catalog size, not cloud size).

4. Does one specification cover MANY specific states? If yes → elimination (pattern generalization). If no → prediction (instance-by-instance classification).

5. Do your evaluations COMPOSE formally? If two evaluations' results combine cleanly (conjunction) → elimination. If they conflict or require reconciliation → prediction.

Your cloud has 10^1,000 possible states. Prediction tries to classify them. Elimination defines the forbidden patterns and checks current state against them. One is mathematically impossible at scale. The other is linear in catalog size. The mathematics isn't a suggestion.

Elimination-mode reasoning — 3,000+ forbidden-pattern controls evaluated against air-gapped snapshots via CEL predicates. Linear in catalog size, not exponential in state space. Each pattern covers infinite specific states. Categorical verdicts, not probabilities. Standardized facts (JSONL, SMT-LIB) exported for external reasoning engines. Stave, an open-source risk reasoning engine. Define the forbidden. Eliminate it. Try it: bash examples/demo-ai-security/run.sh

Five Mental Models From the Data Center that are Breaking Your Cloud Security

Bala Paranj — Wed, 26 Aug 2026 12:33:47 +0000

We kept our titles when we moved to the cloud. Security Analyst. Network Engineer. SysAdmin. The titles stayed. The physics changed. Every assumption baked into those roles, where trust comes from, how you fix things, what you harden, where truth lives was a property of physical machines in physical rooms. The cloud has none of those properties.

Five mental models from the data center era are still in the heads of the people who secure it. Each one creates a gap. The attacker doesn't have these mental models. The attacker sees the cloud as it is: software-defined everything, where the API is the control plane, identity is the perimeter, and the observed state is the only state that matters.

1. Trust is based on location

Data center model: If I know your IP is 10.0.1.5, I know who you are and which rack you're in. Trust comes from the network segment. Inside the perimeter is trusted. Outside is untrusted.

Cloud reality: IPs are ephemeral. A Lambda function might have a different IP every invocation. An ECS task gets a new ENI when it starts. Trust comes from identity such as IAM roles, service accounts, OIDC tokens. The network segment tells you nothing about the caller.

The gap: Security teams spend 80% of their effort on security groups, NACLs (stateless network filters at the subnet level), and firewall rules. Location-based controls. Meanwhile, the attacker ignores the network entirely. They steal an IAM credential often through the Instance Metadata Service (IMDS), the endpoint every EC2 instance exposes at 169.254.169.254. An SSRF vulnerability in the application code lets the attacker reach IMDS, extract the instance role's temporary credentials, and use them from anywhere.

AWS has mitigated the simplest form of this by defaulting newer instances to IMDSv2, which requires a PUT request to obtain a session token before credential retrieval. That blocks trivial SSRF. But the structural argument remains: more complex SSRF (where the attacker controls request headers) can still obtain credentials, and credentials stolen through any path such as SSRF, leaked environment variables, exposed .aws/credentials files, social engineering can work from any network location the IAM policy permits. The network boundary is irrelevant to the credential's scope.

What it produces: Organizations build Zero Trust networks and leave Infinite Trust IAM roles attached to the resources inside them. The network is segmented. The identity is not segmented. An IAM role with s3:* on * renders every network boundary irrelevant. The role can reach every bucket regardless of which subnet the compute sits in.

Network segmentation still provides defense-in-depth such as VPC endpoints, security groups, and NACLs constrain the blast radius of a compromised workload at the network layer. But teams systematically over-invest in network controls while under-investing in IAM. The current industry imbalance is the gap, not the existence of network controls.

What security reasoning requires: Evaluate identity paths, not just network paths. The question isn't only "can traffic reach this subnet?". It's "can this identity reach this resource through any combination of role assumptions, policy attachments, and trust relationships?" That question is about IAM. The privilege escalation controls in tools like Stave exist because the identity path is the primary attack surface. The network path is defense-in-depth, not the perimeter.

2. Fixing means patching

Data center model: Fixing a vulnerability means logging into a server and running yum update. The server is a pet that is long-lived, manually maintained and unique.

Cloud reality: Fixing a vulnerability means updating a Dockerfile and triggering a redeploy. The server is cattle that is short-lived, immutable and replaced not repaired. If you patch a running EC2 instance, that patch disappears the moment the Auto Scaling Group replaces the instance with a fresh image.

The gap: Vulnerability management teams scan running instances and send spreadsheets of missing patches to developers. The developers ignore them because they don't patch servers. They update a base image, push to the registry, and redeploy. The scan reports on hosts that will be replaced in hours. The images remain vulnerable in the registry where nobody scans them.

What it produces: A Scan-and-Patch cycle that's too slow for the Build-and-Deploy cycle. The organization has passing scans on hosts that are already dead while the images that will create the NEXT generation of hosts carry the same vulnerabilities.

What security reasoning requires: Shift the scan target from the running instance to the image in the registry. The image is the source of truth. The running instance is a derivative. Patching the derivative is wasted effort. Patch the source.

3. Hardening targets the operating system

Data center model: Hardening means disabling unused services on Linux, tuning the kernel, locking down SSH, running CIS benchmarks against the OS.

Cloud reality: The attacker doesn't exploit your Linux kernel. They phish a console password. They find a leaked ACCESS_KEY in a public GitHub repo. They call the AWS API to snapshot a database disk, copy the snapshot to their own account, and restore it. The OS never saw a login. The EDR agent never fired. The data is gone.

The gap: Companies spend millions on Endpoint Detection and Response (EDR) and OS hardening. They have zero monitoring on their CloudTrail logs. They watch the front door (the OS) while the attacker walks through the industrial control room (the Cloud API console).

What it produces: An attacker who uses aws rds create-db-snapshot followed by aws rds modify-db-snapshot-attribute to share the snapshot with an external account. The database is exfiltrated without touching the OS, the network, or the application. Every OS-level security control is irrelevant. The attack happened entirely through the cloud API.

What security reasoning requires: Treat the cloud API as the primary attack surface. IAM is the new kernel. Just as we used to audit /etc/sudoers to understand who could run privileged commands on a server, we now need to audit AssumeRolePolicyDocument to understand who can assume privileged roles in the account. Monitor CloudTrail the way you monitor SSH logs. Evaluate IAM policies the way you evaluate OS permissions. The rds:CreateDBSnapshot permission on an IAM role is the cloud equivalent of sudo access on a server and it's far less monitored.

4. Resilience means redundant hardware

Data center model: Disaster recovery means a second set of hardware in a bunker in another city. The warm site. Physical redundancy for physical failures.

Cloud reality: Hardware is an abstraction. Disaster recovery means the Terraform state file can provision the entire stack in another region. Resilience is a configuration property, not a physical one.

The gap: Organizations perform DR drills where they fail over to another region manually. But because the infrastructure-as-code isn't unified, the DR region has different IAM roles, secrets and DNS settings. The Terraform in the DR region references ARNs from the primary region where ARNs that don't exist when the primary is down.

What it produces: The DR site fails to spin up during a real outage because a hardcoded ARN points to a resource in the dead region. The DR plan was a physical plan for a logic problem. The hardware failed over. The configuration didn't.

What security reasoning requires: Detect ghost references. The resources that point to dependencies that no longer exist or exist in a different region. An ARN referencing a KMS key in us-east-1 from a role in eu-west-1 is a ghost reference. It works until us-east-1 is unavailable. Then the role can't decrypt anything. Ghost reference detection catches these before the outage reveals them.

5. Documentation is the source of truth

Data center model: If a technician changed a cable, they updated a spreadsheet. The spreadsheet was the record of the infrastructure. Audits checked the spreadsheet.

Cloud reality: The Terraform repository is the intended state. The actual cloud environment is the observed state. They are not the same. Someone clicked Modify in the AWS Console at 2 AM to fix a production bug. The Terraform was never updated. The desired state says the bucket is private. The observed state says the public access block was manually disabled three weeks ago.

The gap: Security audits evaluate the Terraform repository which is the desired state. The breach happens against the cloud environment which is the observed state. The audit passes the code. The cloud is leaking.

What it produces: Configuration drift. The security team certifies the infrastructure as compliant based on the IaC source. The infrastructure has drifted from the source because a human made a manual change. The compliance certificate describes a system that doesn't exist. The system that exists wasn't audited.

What security reasoning requires: Evaluate the OBSERVED state, not the desired state. Snapshot the live cloud environment via API calls. Evaluate the snapshot. The snapshot captures observed state, including manual changes, console modifications, and API calls that bypassed the deployment pipeline. If the Terraform says public_access_block = true but the snapshot shows it's disabled, the snapshot is right. The Terraform is a historical document.

This is a fundamental architectural decision for security tooling. Tools that evaluate Terraform source (Checkov, tfsec, Sentinel) answer: "does the intended configuration meet policy?" Tools that evaluate live snapshots answer: "does the ACTUAL configuration meet policy?" Both are useful questions. Only the second catches drift. Breaches hide in the drift because the drifted configuration was never reviewed, approved and tested.

The common pattern

All five models share one structural property: they assume a physical constraint that the cloud removed.

Data center constraint:     Cloud equivalent:
───────────────────────     ─────────────────────────────
Physical network perimeter  → Software-defined identity (IAM)
Physical server             → Immutable image in a registry
Physical OS on bare metal   → API-accessible control plane
Physical hardware in bunker → Configuration state in code
Physical cable + spreadsheet→ API call + observed state

Each physical constraint provided a security property for free. The network perimeter provided trust without IAM. The physical server provided persistence without image management. The OS provided the attack surface without API monitoring. The hardware provided redundancy without configuration unification. The spreadsheet provided truth without drift detection.

The cloud removed the physical constraint. The security property it provided didn't get replaced, it just disappeared. The mental model persists ("we have a firewall, we're segmented") while the reality changed ("identity ignores the firewall entirely").

The Su-Field structure

The pattern is the same across all five:

Before (data center):
    F (physical constraint) produces TWO outputs:
    Output 1: Operational function (networking, patching, hardening, DR, auditing)
    Output 2: Security property (isolation, integrity, surface reduction, resilience, truth)

After (cloud):
    F' (software-defined) replaces F
    Output 1: Operational function preserved (faster, more flexible)
    Output 2: Security property LOST (no physical constraint to provide it)

Resolution:
    Keep F' (the operational benefits are real)
    Add F2 (explicit security mechanism that provides the lost property)

    F2 for identity:     IAM path analysis, privilege escalation detection
    F2 for patching:     Image scanning at the registry, not the instance
    F2 for hardening:    CloudTrail monitoring, API permission evaluation
    F2 for DR:           Ghost reference detection, cross-region config validation
    F2 for truth:        Live snapshot evaluation, drift detection

The physical constraint was one field that produced both operational and security outputs. The cloud split them. The operational output survived (faster deployment, elastic scaling, multi-region). The security output didn't. Each F2 above is a specific mechanism that restores the specific security property that the corresponding physical constraint used to provide for free.

What this means for practitioners

The next time you evaluate your cloud security posture, ask five questions:

1. Am I auditing identity or location? If your security review focuses on security groups and NACLs but doesn't trace IAM role assumption chains, you're auditing the screen door, not the control room.

2. Am I scanning the source or the derivative? If your vulnerability scans target running instances but not the container images in the registry, you're patching cattle that will be replaced tomorrow.

3. Am I hardening the OS or the API? If you have EDR on every EC2 instance but no alerts on CloudTrail for rds:CreateDBSnapshot or iam:CreateAccessKey, you're watching the front door while the data walks out through the API.

4. Am I testing hardware failover or configuration failover? If your DR drill tests "can the hardware come up in another region" but not "do the IAM roles, secrets, and ARN references work in another region," your DR plan is physical in a world that's logical.

5. Am I auditing the code or the cloud? If your compliance assessment evaluates the Terraform repository but not a live snapshot of the actual environment, your certificate describes a system that may not exist. The 2 AM console change isn't in Terraform. It's in the cloud. Only the snapshot sees it.

Five questions. Each reveals whether your security model has been updated for the cloud or whether you're still running data center assumptions against software-defined infrastructure.

Three of these five gaps are addressed directly by Stave: identity path analysis (Problem 1 — privilege escalation controls trace IAM chains through four policy layers), API control plane evaluation (Problem 3 — 4,400+ controls covering cloud API permissions with 670+ compound chains composing cross-resource risk), and observed-state snapshot evaluation (Problem 5 — evaluates live cloud configuration, not IaC source). Ghost reference detection (Problem 4) is a core differentiator — detecting resources that reference deleted dependencies. Apache 2.0.

Configuration vs Runtime: Where Determinism Ends

Bala Paranj — Tue, 25 Aug 2026 12:23:51 +0000

A security group rule allowing inbound 0.0.0.0/0 on port 3389 is a fact about the configuration. Nobody connecting via RDP is a fact about the runtime. Both are true simultaneously. They answer different questions.

Configuration analysis asks: what does the infrastructure PERMIT?
Runtime analysis asks: what is the infrastructure DOING?

The first question has a deterministic answer. The same configuration, evaluated against the same specification, produces the same findings on any machine, at any time, by any operator. The answer is a logical consequence of the state. It can't vary because the inputs don't vary.

The second question has a probabilistic answer. "Is this connection anomalous?" depends on a baseline. "Is this API call malicious?" depends on context. "Is this traffic pattern an attack?" depends on what normal looks like, which changes over time, across environments, and between observers.

Most security tools blur this boundary. They mix deterministic configuration checks with probabilistic runtime assessments in the same dashboard, severity scale and the triage queue. This results in configuration findings that are always true being treated with the same skepticism as runtime alerts that are sometimes wrong. The operator can't tell which findings are facts about the state and which are guesses about the behavior.

Understanding where determinism ends is the first step to using the right tool for each question.

What determinism means here

A deterministic evaluation has three properties:

Reproducibility. The same input produces the same output. Run the evaluation on a Monday. Run it again on a Friday. Same snapshot, same controls, same findings. Share the snapshot with a colleague. They run the evaluation on their laptop. Same findings. Share it with an auditor. Same findings. The evaluation is a function of the input, not of the environment, the time, or the observer.

Completeness. The evaluation answers every question it claims to answer. If the specification says "every S3 bucket must have encryption enabled," the evaluation checks every S3 bucket and reports every one that doesn't. It doesn't sample. It doesn't estimate. It doesn't skip resources that are probably fine.

Explainability. Every finding carries a proof: which resource, which property, which value, which specification it violates. The operator can verify the finding by inspecting the resource and the specification. The finding isn't a score or a probability, it's a logical derivation from observable facts.

Configuration analysis has all three. Runtime analysis has none by nature, not by weakness.

The configuration side: what determinism can answer

These questions have deterministic answers from a configuration snapshot:

Is this resource misconfigured? A security group allowing 0.0.0.0/0 on port 3389 either exists or doesn't. The answer is binary. The evaluation is a comparison between the observed state and the specification.

Can this principal reach this resource? A role assumption chain from role A through role B to S3 bucket C either exists in the IAM configuration or doesn't. The answer is the transitive closure of the permission graph. A mathematical computation with a definitive result.

Do these misconfigurations compose into a known attack path? The Capital One pattern: IMDSv1 enabled AND public IP AND overpermissioned role is a conjunction of three boolean properties. All three are true simultaneously on this instance, or they're not. The compound evaluation is a logical AND over deterministic inputs.

How close is this configuration to a named breach pattern? If two of three controls in a chain are failing and one is passing, the configuration is distance-one from the pattern. The distance is a count of failing members versus total members. It is an integer derived from the evaluation.

What changed between two configuration states? Given snapshot A and snapshot B, the diff is deterministic: these resources were added, these were removed, these properties changed. The risk direction of each change (IMDSv2 enforced → IMDSv2 optional is risk-increasing) is a comparison between the old value, the new value, and the specification.

Every answer above is a function of the configuration state. The state is observable. The specification is declared. The evaluation is a logical derivation. It can't be wrong in the way a probabilistic estimate can be wrong. It can only be wrong if the specification is wrong or the observation is incomplete. Both of those are verifiable.

The runtime side: what determinism can't answer

These questions cannot be answered from a configuration snapshot:

Is anyone exploiting this misconfiguration? The security group allows 0.0.0.0/0 on port 3389. Is anyone connecting? Configuration analysis can't tell. The security group rule is a gate — it permits traffic. Whether traffic flows through the gate is a runtime fact observed in VPC Flow Logs, not in the security group configuration.

Is this API call malicious? A CloudTrail event shows sts:AssumeRole from an IP address in a country where the company has no employees. Is it an attacker, a VPN, or a traveling employee? The answer requires context that no configuration snapshot contains: the employee's travel schedule, the company's VPN exit points, the historical baseline of where this role is typically assumed from.

Is this behavior anomalous? A Lambda function that usually invokes 100 times per hour is invoking 10,000 times per hour. Is it an attack, a traffic spike, or a new feature that went viral? The answer requires a baseline, what "normal" looks like for this function. Baselines are empirical, not specified. They change over time. They differ between environments. They are inherently probabilistic.

Is this user compromised? An IAM user who typically accesses 3 S3 buckets is suddenly listing all buckets in the account and downloading objects from 15 of them. Is the user compromised, running an audit, or doing a data migration? The answer requires understanding intent, which no technical system can determine from telemetry alone.

Each of these questions requires runtime data (logs, metrics, flow records), empirical baselines (what "normal" looks like), and probabilistic judgment (is this deviation significant enough to alert?). The answers are uncertain. A well-tuned GuardDuty deployment with good baselines produces valuable alerts. But the alerts are estimates, not proofs. They can be false positives (the VPN exit point was unusual but legitimate) or false negatives (the attacker's behavior was within the baseline).

The boundary is precise

The boundary between deterministic and probabilistic follows a clear line:

Configuration analysis answers: "Is this state permitted by the specification?" The answer is always deterministic because both the state and the specification are observable, static, and complete.

Runtime analysis answers: "Is this behavior consistent with normal?" The answer is always probabilistic because "normal" is empirical, dynamic, and incomplete.

The confusion happens when tools mix both in one output. A dashboard that shows "S3 bucket is public (severity: HIGH)" next to "Unusual API activity from IAM user (severity: HIGH)" treats a deterministic fact and a probabilistic estimate as equivalent. They're not. The public bucket is public. That finding is true until someone changes the configuration. The unusual API activity might be malicious. That finding is true until the baseline shifts or more context arrives.

An operator who treats both findings with the same confidence either over-trusts the runtime alert (acts on every anomaly as if it's an attack) or under-trusts the configuration finding (treats a verified misconfiguration as "probably fine, like the last three false positive alerts"). The blurred boundary degrades the operator's judgment in both directions.

What each side does better

Configuration analysis is better for:

Prevention. A control that fires before the misconfiguration reaches production prevents the incident. A runtime alert fires after the behavior occurs. The misconfiguration already exists, the traffic is already flowing, the data may already be accessed. Configuration analysis operates on the left side of the timeline. Runtime analysis operates on the right.

Proof. An auditor asking "can you prove your S3 buckets aren't public?" gets a deterministic answer from configuration analysis: here's the snapshot, here are the controls, here are the findings, run the evaluation yourself. Runtime analysis can't prove a negative — "no unusual access was detected" doesn't mean no unauthorized access occurred. It means no access triggered an alert.

Compound risk. The Capital One pattern requires evaluating three configuration properties simultaneously on the same instance. Configuration analysis composes naturally. The compound is a conjunction of properties in the snapshot. Runtime analysis would need to correlate three separate log streams (network flow, IMDS access, IAM activity) across time windows to detect the same compound, which is orders of magnitude harder.

Reproducibility. Two operators evaluating the same snapshot get the same findings. Two SOC analysts reviewing the same SIEM alerts may reach different conclusions, because the assessment requires judgment about baselines, context, and severity thresholds.

Runtime analysis is better for:

Detection of active exploitation. Configuration analysis says "this instance COULD be exploited." Runtime analysis says "this instance IS being exploited." The first is a risk assessment. The second is an incident. When the incident is happening, runtime wins.

Intent determination. A role with broad permissions is a configuration finding. The same role being used at 3am from a new IP to enumerate secrets is a runtime finding that adds intent signal. Configuration can't distinguish an overpermissioned role used legitimately from one used maliciously because both have the same configuration.

Behavioral patterns over time. A user who gradually escalates their access over weeks, requesting slightly broader permissions each time has no single configuration violation. Each permission request is individually reasonable. The PATTERN is suspicious, but the pattern only appears in the log stream over time, not in any single snapshot.

Zero-day and novel attacks. An attacker using a technique that no specification anticipated produces no configuration finding. The configuration permits the behavior because nobody specified that it shouldn't. Runtime analysis may still detect it. The behavior is anomalous even if no specification forbids it.

The complementary architecture

The right architecture uses both and keeps them separate.

Configuration analysis runs periodically (hourly, daily, on every deployment). It evaluates the current state against the specification and produces deterministic findings. The findings are facts about the configuration. They don't expire, shift, or depend on baselines. They're true until the configuration changes.

Runtime analysis runs continuously. It evaluates the event stream against empirical baselines and produces probabilistic alerts. The alerts are estimates about the behavior. They can be false positives. They require human judgment to triage. They're context-dependent.

The two systems connect at one point: the configuration finding contextualizes the runtime alert. When GuardDuty alerts on unusual API activity from a role, the configuration analysis tells you whether that role is overpermissioned (blast radius if compromised), whether it participates in a compound attack path (distance from a named breach pattern), and whether it has compensating controls (MFA required, source IP restricted). The runtime alert says "something is happening." The configuration analysis says "here's how bad it can get."

This connection is one-directional. The configuration finding informs the runtime triage. The runtime alert does NOT change the configuration finding. A public S3 bucket is a configuration finding whether or not anyone is accessing it. An overpermissioned role is a configuration finding whether or not it's being abused. The finding is a fact about the state, not an inference about the behavior.

Where most tools go wrong

Most cloud security platforms combine configuration and runtime in one product. The marketing says "unified visibility." The reality is a blurred boundary that weakens both:

Configuration findings inherit the skepticism of runtime alerts. The operator who has been burned by false positive anomaly detections treats configuration findings with the same suspicion — "this is probably another false alarm." But configuration findings aren't probabilistic. The bucket IS public. The role DOES have AdministratorAccess. These aren't guesses. Mixing them with guesses teaches the operator to treat facts as guesses.

Runtime alerts inherit the resolution process of configuration findings. A configuration finding is fixed by changing the configuration such as a Terraform apply, a console click, an API call. A runtime alert is investigated where logs are reviewed, context is gathered, a judgment is made. When both appear in the same queue with the same severity labels and the same SLA, the operator applies the wrong resolution process to the wrong finding type.

Compound analysis is impossible across the boundary. A compound configuration finding (three controls failing simultaneously) is a deterministic conjunction. The compound is true because all members are true. A compound that mixes configuration and runtime ("this bucket is public AND someone accessed it yesterday") requires correlating a deterministic state with a probabilistic event. The correlation is useful but it's not deterministic. The access event may be legitimate, and the compound inherits the uncertainty of its weakest member.

The design principle

Separate the deterministic from the probabilistic. Evaluate configurations against specifications and produce findings that are facts. Monitor runtime against baselines and produce alerts that are estimates. Connect them at the triage layer. The configuration finding contextualizes the runtime alert. But don't merge them into one output with one severity scale and one triage queue.

The operator needs to know which findings are always true (configuration) and which findings are sometimes true (runtime). The boundary between them is precise. Blurring it helps neither.

Configuration tells you what your infrastructure permits. Runtime tells you what your infrastructure is doing. Determinism answers the first. Probabilism answers the second. Using the wrong method for the wrong question produces the worst of both: unreliable answers about configuration (undermining trust) and over-certain answers about behavior (undermining judgment).

The specification defines what "correct" means. The evaluation verifies whether the state matches. That's where determinism lives and where it should stay. Everything past the boundary is monitoring, detection, and response. Different tools. Different methods. Different epistemology.

Know the boundary. Use the right tool on each side.

Zero Permissions on Secrets Manager. Full Access to Your Secrets.

Bala Paranj — Mon, 24 Aug 2026 11:55:27 +0000

A 2025 CloudGoat walkthrough by VirajMathpati documents a privilege escalation that no permission-based scanner catches. The principal with a synthetic IAM user named cg-sns-user-cgidxi93qpes3g has zero permissions on Secrets Manager. Run their identity through IAM Access Analyzer; it confirms no access. Run it through PMapper; same answer. Run it through any policy-based review; the IAM user cannot read any secret in the account.

The user retrieves a secret in seven API calls.

The trick: the user's permissions don't connect to Secrets Manager directly. The connection is a credential-value flow through four services. SNS publishes an API key as a message payload. The user subscribes to the topic and receives the key in their inbox. The key authenticates to an API Gateway. The gateway's Lambda integration reads from Secrets Manager. The secret comes back as the HTTP response body.

No policy connects SNS to Secrets Manager. The connection is the credential value itself. Every step is individually authorized. The compound chain is invisible to any tool that checks permissions service-by-service.

The four services and the four configurations

[1] IAM user policy
    Allow:  sns:Subscribe, sns:Receive, sns:ListTopics, …,
            apigateway:GET            (Resource: *)
    Deny:   apigateway:GET on 7 specific paths

[2] SNS topic
    Topic ARN:    public-topic-cgidxi93qpes3g
    Topic policy: NONE (defaults to IAM-only gate)
    Publishes:    a message containing an API Gateway key

[3] API Gateway
    Auth:         API key only (no IAM, no Cognito, no Lambda authorizer)
    Integration:  Lambda

[4] Lambda
    Reads:        the target secret in Secrets Manager

Each one is fine by per-service checklist criteria. The user's IAM policy is broad but doesn't grant Secrets Manager. The SNS topic has no resource policy — the standard "topic owner only" default. The API Gateway uses key-only auth, which is a documented authentication option. The Lambda accesses Secrets Manager via its execution role, the standard pattern.

The bug is the connection.

Four Z3 queries

The Z3 prover in stave/examples/sns-secrets-compound-chain/z3prove/ runs four queries against the writeup configuration and four against a remediated version.

Finding 1: the topic is subscribable + publishes credentials

The user's IAM policy admits sns:Subscribe on Resource: "*". The topic has no resource policy. In AWS, the IAM identity policy is the only gate when no topic policy is configured. The user wins.

--- Finding 1: SNS topic subscribable, publishes credential ---
  subscribable + sensitive topics: 1 / 1
  verdict: SAT — witness: arn:aws:sns:us-east-1:676206926638:public-topic-cgidxi93qpes3g
           (publishes api_key targeting arn:aws:apigateway:us-east-1::/restapis/x93anl9mj7)

The data-flow annotation publishes_credential_type=api_key on the topic asset makes this a security concern rather than a subscription pattern concern. Stave's existing broad-subscribe control checks the topic's resource policy. This issue lives in the IAM identity policy plus the absence of a topic policy plus the data-flow fact about what the topic publishes.

Finding 2: the API Gateway deny coverage gap

The user's policy grants apigateway:GET on Resource: "*" and denies seven specific patterns:

{
  "Effect": "Deny",
  "Action": "apigateway:GET",
  "Resource": [
    "/apikeys",
    "/apikeys/*",
    "/restapis/*/resources/*/methods/GET",
    "/restapis/*/methods/GET",
    "/restapis/*/resources/*/integration",
    "/restapis/*/integration",
    "/restapis/*/resources/*/methods/*/integration"
  ]
}

Seven patterns. The author was thinking carefully. They identified specific high-sensitivity paths (API keys, method bodies, integration internals) and denied them. By any reasonable security review, this deny is well-considered.

Z3 walks 24 known API Gateway management paths and finds 21 of them reachable:

--- API Gateway Deny coverage analysis ---
  OPEN      : /restapis
  OPEN      : /restapis/{id}
  OPEN      : /restapis/{id}/resources
  OPEN      : /restapis/{id}/resources/{resource_id}
  OPEN      : /restapis/{id}/resources/{resource_id}/methods/{method}
  BLOCKED   : /restapis/{id}/resources/{resource_id}/methods/{method}/integration
  OPEN      : /restapis/{id}/stages
  OPEN      : /restapis/{id}/stages/{stage}
  OPEN      : /restapis/{id}/deployments
  OPEN      : /restapis/{id}/deployments/{deployment_id}
  OPEN      : /restapis/{id}/models
  OPEN      : /restapis/{id}/authorizers
  ... (more)

The author blocked the API key endpoint. Reasonable. But the principal can enumerate the rest: /restapis lists every API ID. /restapis/{id}/stages lists each API's stages. /restapis/{id}/resources lists the resource paths. With those three calls, the attacker has the full URL. They don't need /methods/GET (denied) or /integration (denied). They have the API key from Finding 1. They invoke the URL directly.

The deny was correct for the things the author thought to protect. AWS API Gateway has more management endpoints than any individual reviewer holds in their head. The deny-list approach loses the race to the service surface area, just like the autoscaling iteration.

Finding 3: the 5-hop data-flow chain

The most novel modeling in this article. The Z3 program computes five booleans:

hop 1: principal can subscribe to topic                       true
hop 2: topic publishes credential                             true
hop 3: API Gateway accepts credential, no IAM auth required   true
hop 4: API Gateway integrates with downstream function        true
hop 5: function reads Secrets Manager                         true

Each hop is a fact read from a different service's observation. The conjunction is the data-flow chain. Z3's And of five booleans tells us the chain is satisfiable.

This is the iteration's distinguishing feature. The five facts come from four different observation assets: the IAM user, the SNS topic, the API Gateway, the Lambda + one cross-asset claim ("function reads Secrets Manager"). No single CEL predicate can express this. The composition is necessarily multi-asset.

--- Finding 3: data-flow credential chain (5 hops) ---
  hop 1 (principal can subscribe to topic):                 true
  hop 2 (topic publishes credential):                       true
  hop 3 (API Gateway accepts credential, no IAM auth):      true
  hop 4 (API Gateway integrates with downstream function):  true
  hop 5 (function reads Secrets Manager):                   true
  verdict:  SAT — full credential-flow chain is reachable
            (zero direct Secrets Manager permissions; access via
             credential value flowing through SNS → API GW → Lambda)

Finding 4: the compound chain

F1 ∧ F2 ∧ F3. The complete escalation:

sns:ListTopics                  → discover the topic
sns:Subscribe                   → subscribe with attacker email
(receive SNS message)           → extract the API key
apigateway:GET /restapis        → enumerate API IDs
apigateway:GET /restapis/.../stages    → enumerate stages
apigateway:GET /restapis/.../resources → enumerate paths
curl https://<api>.execute-api.../user-data
    -H "x-api-key: <leaked-key>"  → invoke API
                                  → API GW invokes Lambda
                                  → Lambda reads Secrets Manager
                                  → secret returned in HTTP body

Seven calls. Three services touched directly. A fourth (Secrets Manager) reached transitively. Zero direct permissions on the target.

Why permission analysis misses this

A permission-based reasoner such as IAM Access Analyzer, Zelkova, PMapper or any policy resolver works by walking the policy graph: "principal P has policy that grants action A on resource R." The graph is complete for permission edges. But:

There is no permission edge from SNS to Secrets Manager. SNS doesn't have permission to read secrets. SNS publishes messages. The "edge" is the credential value that travels in a message payload.
There is no permission edge from the IAM user to the API key. The user has sns:Subscribe. The key is a string in a message body. Receiving the key isn't a permission grant; it's receipt of data.

A permission graph cannot represent "principal P receives credential C via mechanism M, and C authenticates to service S." That isn't a permission relationship. It's a data-flow relationship. The information is in the configuration of what flows where, not in who is allowed to do what.

This iteration extends the model to include data-flow facts as observation annotations. The Z3 program reasons over both permission facts and data-flow facts uniformly. The chain reaches a service the user has no permission on, but the user reaches the service's output.

The remediation

Three changes, each closing one finding:

 // 1. Scope sns:Subscribe to specific topic prefixes.
 {
-  "Action": "sns:Subscribe",
-  "Resource": "*"
+  "Action": "sns:Subscribe",
+  "Resource": "arn:aws:sns:us-east-1:676206926638:ops-alerts-*"
 }

 // 2. Remove apigateway:GET. The user's data-science
 //    role doesn't need it. If a future use case
 //    arises, scope to the specific API ARN.

 // 3. Stop publishing credentials through SNS. The
 //    correct architecture is to put the API key in
 //    Secrets Manager, grant the consumer's IAM
 //    role secretsmanager:GetSecretValue on the
 //    specific secret ARN, and let the consumer
 //    retrieve it via the SDK at runtime. Messaging
 //    services are not credential transport.

Plus changing the API Gateway authorization to AWS_IAM so the resource policy and IAM identity policy both gate access, adding a defense in depth even if a future credential leak occurs.

After remediation, all four Z3 queries return UNSAT. The chain is broken at hop 1 (the user can no longer subscribe to the credential-publishing topic).

The architectural lesson

The published configuration is a teaching exercise. CloudGoat is intentionally vulnerable. But the shape of the bug is real. Production AWS environments do this: a CI/CD pipeline publishes a deployment notification to SNS. The notification includes a deployment URL. The URL is a presigned S3 URL that grants read access to a build artifact. Anyone subscribed to the topic can fetch the artifact. Credential value flowing through messaging that subscribers harvest.

The architectural rule: messaging services are not credential transport. If a credential needs to flow between services, put it in Secrets Manager (or SSM Parameter Store) and grant the consuming service secretsmanager:GetSecretValue on the specific secret ARN. The credential never traverses a notification path; the consumer retrieves it via SDK calls authenticated by IAM.

Two reasons:

SNS subscription is a public surface. Adding a subscriber doesn't require any permission on the thing being notified about. It only requires sns:Subscribe. Anyone with that permission joins the recipient list.
Subscriber list management is loose. Operators add subscribers more freely than they add IAM permissions. The "everyone gets dev-team alerts" pattern is comfortable; "everyone gets production secrets" should not be.

When the credential is in Secrets Manager and the consumer's IAM grants secretsmanager:GetSecretValue on a specific ARN, the access pattern is:

Authentication: SigV4 with the consumer's identity.
Authorization: explicit IAM policy.
Audit: every retrieval is a CloudTrail event with the principal recorded.
Rotation: Secrets Manager handles it; consumers fetch the latest value.

None of these properties hold for a credential delivered via SNS message. The audit trail shows a generic "publish" event; the principal that received it is a subscriber identifier, not an IAM principal; rotation requires re-publishing.

What pattern-matching tools see

Run an IAM policy review on cg-sns-user. The output is "this user has SNS read access and API Gateway read access. No access to Secrets Manager." Correct.

Run a posture scan on the SNS topic. The output is "topic has no resource policy; access governed by IAM." Correct.

Run an API Gateway scan. The output is "API uses key-only auth; consider IAM authorization for defense in depth." Useful but not foundational.

Each per-service tool is correct for what it checks. None of them composes the four answers into "this principal can read your secrets."

The Z3 prover does. The five hops conjunct into one boolean, and the boolean returns SAT.

Checklist

No SNS topic publishes credential values, API keys, or any secret material in message bodies
If a credential needs to flow between services, it lives in Secrets Manager (or SSM) with IAM-scoped retrieval, not in a notification stream
sns:Subscribe in IAM identity policies is scoped to specific topic ARN patterns, not Resource: "*"
API Gateway methods that read Secrets Manager use AWS_IAM authorization, not API_KEY only
CI runs cross-service data-flow analysis on pre-merge observation snapshots; the example shipped with this article is the template

The IAM user has no Secrets Manager permission. The secret comes back anyway. That's not a bug in any single service. It's a bug in the composition. The permission graph is silent. The credential graph talks. Z3 reads both.

How this relates to existing compliance mods

A framework benchmark in turbot/steampipe-mod-aws-compliance will correctly report the IAM user as having zero Secrets Manager permissions. That's what the IAM policy says, and the framework controls are accurately reading the IAM API. The verdict isn't wrong; it's just on the wrong question. The compositional finding ("the secret comes back because a Lambda subscribed to an SNS topic the user can publish to") isn't representable in per-resource framework controls because no framework section names "subscribe to a topic whose subscriber holds a permission you don't." That's the class of finding Stave's compound + chain catalog covers. Frameworks catch what the IAM policy permits; Stave catches what the cross-service composition delivers. Both run against the same snapshot; both render in Powerpipe. Comparison: aws-compliance-mod.

The example is at stave/examples/sns-secrets-compound-chain/. Two binaries side by side: a CEL evaluation via pkg/stave.Apply (the existing CTL.SNS.POLICY.SUBSCRIBE.BROAD.001 per-topic control reports clean on both fixtures because the risk lives in the IAM identity policy and absence of topic policy, not in a topic-side broad subscribe) and a Z3 SAT prover that runs the four queries from this article and prints the API Gateway management deny coverage table. The Z3 binary lives in a sibling Go module so its libz3 link stays out of Stave's main vendored tree. Stave detects this pattern and 31 other H1-grounded scenarios from local AWS configuration snapshots, without cloud credentials.

Why the Capital One Breach Wasn't About One Misconfiguration

Bala Paranj — Sun, 23 Aug 2026 11:11:51 +0000

In 2019, a former AWS employee exploited a server-side request forgery vulnerability in Capital One's web application firewall. The breach exposed over 100 million customer records.

No single misconfiguration caused the breach.

It was the correlation of three weak signals that produced a critical finding:

An SSRF vulnerability in the WAF configuration
An overly permissive IAM role attached to the compromised instance
Unencrypted S3 buckets containing sensitive customer data

Each finding alone? Low to medium severity. Together? A $190 million breach.

The Problem with Flat Scanners

Most cloud security tools evaluate configurations in isolation. They produce a flat list of findings:

WARN  S3 bucket has public access enabled
WARN  IAM role has overly broad permissions
INFO  Server-side encryption not enabled

Three warnings. No critical alert. The tool did its job. It found each misconfiguration. But it missed the compound risk because it never correlated the signals.

This is the gap: a scanner that evaluates one configuration at a time cannot reason about the interaction between configurations.

Introducing Stave

Stave is an offline configuration safety evaluator. It takes two inputs: security controls written in YAML and observation snapshots of your cloud infrastructure and produces a deterministic security assessment. No agents, API keys or network access required. It runs on local data, which makes it suitable for air-gapped environments and audit-sensitive workflows.

The examples below use Stave's control language to illustrate how compound predicate evaluation closes the signal correlation gap.

Signal Correlation: The Missing Primitive

What security teams need is a way to express compound conditions, the predicates that fire only when multiple weak signals appear together on the same asset:

id: CTL.S3.EXPOSURE.COMPOUND.001
name: Unencrypted Public Bucket with Broad IAM Access
description: >
  S3 bucket is publicly accessible, lacks encryption, and is
  accessible by an overly permissive IAM role. This combination
  creates a high-severity data exfiltration risk.
severity: critical

unsafe_predicate:
  all:
    - field: public_access
      op: eq
      value: true
    - field: server_side_encryption
      op: eq
      value: false
    - field: iam_role_scope
      op: eq
      value: "overly_permissive"

The all combinator is the key. This control only fires when every condition is true simultaneously. Three medium-severity signals become one critical finding because the predicate explicitly models the compound risk.

How Compound Predicates Work

The evaluation model supports two combinators:

all — Every condition must be true (logical AND). Use this for compound risk scenarios where the danger comes from the intersection of weak signals.

any — At least one condition must be true (logical OR). Use this for detection breadth where multiple indicators point to the same root cause.

These nest arbitrarily:

unsafe_predicate:
  all:
    - field: public_access
      op: eq
      value: true
    - any:
        - field: server_side_encryption
          op: eq
          value: false
        - field: kms_key_id
          op: missing

This reads: "The bucket is publicly accessible AND (encryption is disabled OR no KMS key is configured)." Two different paths to the same compound risk.

Adding Time as a Correlation Axis

A misconfiguration that appeared 2 hours ago is different from one that's been open for 90 days.

If your evaluation engine tracks when a configuration became unsafe, you gain a temporal correlation axis:

Finding: CTL.S3.PUBLIC.001
  Asset: patient-records-bucket
  First unsafe: 2026-01-01T00:00:00Z
  Duration: 2160 hours (90 days)
  SLA threshold: 168 hours (7 days)
  Status: NON_COMPLIANT

The same public_access: true finding has different severity depending on duration. A bucket that went public 2 hours ago during a deploy is an incident. A bucket that's been public for 90 days is a governance failure.

This requires observation snapshots, point-in-time captures of configuration state. With at least two snapshots, you can compute:

When the unsafe state first appeared
How long it has persisted
Whether it's a new regression or a chronic drift
Whether it exceeds the team's SLA threshold

The Three Correlation Dimensions

Putting it together, configuration risk is a function of three dimensions:

Dimension	Question	Signal
Predicate	What combination of fields is unsafe?	Compound `all`/`any` conditions
Temporal	How long has it been unsafe?	Duration vs SLA threshold
Contextual	What else is true about this asset?	Cross-field correlation

A flat scanner only covers the first dimension and only one field at a time. Adding predicate composition catches Capital One-style compound risks. Adding temporal tracking catches chronic drift that compliance teams care about.

Result of evaluation

Given two observation snapshots of an S3 bucket captured a week apart, and a set of controls with compound predicates, the evaluation produces:

{
  "kind": "ASSESSMENT",
  "summary": {
    "total_assets": 1,
    "violations": 3
  },
  "status": "NON_COMPLIANT",
  "findings": [
    {
      "control_id": "CTL.S3.PUBLIC.001",
      "asset_id": "patient-records-bucket",
      "evidence": {
        "first_unsafe_at": "2026-01-01T00:00:00Z",
        "unsafe_duration_hours": 336,
        "threshold_hours": 168,
        "temporal_risk": "SLA_EXCEEDED"
      }
    }
  ]
}

The output is machine-readable JSON. Teams pipe it into their SIEM, Splunk, or ticketing system. The evidence block provides the temporal proof that auditors need. This has been misconfigured for 336 hours, exceeding your 168-hour SLA.

The Remediation Attestation

After fixing the findings, teams need proof that the fix worked. A before/after comparison produces an attestation:

{
  "kind": "ATTESTATION",
  "summary": {
    "previous_violations": 3,
    "current_violations": 0,
    "remediated": 3,
    "open": 0,
    "regressions": 0
  }
}

Zero regressions. Three remediated. This is the artifact that goes into the compliance record which is formal proof that the compound risk was identified and resolved.

Building This Into Your Security Pipeline

The design decisions:

Controls are data, not code. Express compound predicates in YAML/JSON, not in Go/Python functions. This lets security teams author and review controls without touching the evaluation engine.
Observations are snapshots, not streams. Capture configuration state at discrete points in time. Two snapshots a week is enough for SLA tracking. This works offline and doesn't require agent installation.
The evaluation engine is deterministic. Given the same controls, observations, and timestamp, the output is byte-identical. This matters for audit reproducibility.
Separate the assessment from the remediation. The engine produces findings. A separate step verifies the fix. These are different artifacts with different schemas because they answer different questions.

Conclusion

The Capital One breach wasn't caused by one misconfiguration. It was caused by the correlation of three. Most security tools would have flagged each one individually. None would have flagged the compound risk.

If your configuration evaluation can express all/any predicate composition, track temporal duration against SLA thresholds, and produce machine-readable evidence, you're doing security assessment.

The difference matters when the auditor asks: "How long was this configuration unsafe, and what else was true at the same time?"

This article describes the evaluation model behind Stave, an offline configuration safety evaluator that uses compound predicate composition and temporal tracking to produce security assessments and remediation attestations.

Stop Adding Scanners, Start Deleting Paths

Bala Paranj — Sat, 22 Aug 2026 11:39:54 +0000

OWASP recently published the Subtractive Security project, and it names something the industry has been doing backwards for twenty years.

The default approach to security is additive. Add a scanner. Add a SIEM. Add an EDR. Add a CSPM. Each tool watches more things. Each tool generates more alerts. The budget goes up, the dashboard count goes up, and the question nobody answers is: did any attack paths disappear?

Matt Frenz, the project lead, borrows a hierarchy from reliability engineering and applies it to security:

Erase the attack path. Remove it architecturally so it can't exist.

Constrain the path if erasure isn't possible. Restrict it so exploitation requires more conditions.

Monitor only what can't be erased or constrained. Detect what you couldn't prevent.

The hierarchy is a priority order. Erasure is better than constraint. Constraint is better than monitoring. The industry inverts this: most budget goes to monitoring, some to constraining, almost none to erasure.

Monitoring an attack path that could be deleted is paying to watch a fire you could have prevented.

The problem with the additive model

Every monitoring tool you add to an environment that still has erasable paths creates the same dynamic: more alerts about problems that shouldn't exist.

An IAM role with AmazonEC2FullAccess attached to a Lambda function generates GuardDuty findings when that Lambda makes API calls it shouldn't. The monitoring works. The alert fires. An analyst investigates. They determine it's a misconfiguration, not an attack. The role gets noted for remediation. It sits in the backlog for weeks.

The subtractive approach: scope the role to the specific actions on the specific resources the function needs. The overprivileged path is erased. The GuardDuty finding stops firing. The analyst never sees it. The backlog doesn't grow. The signal-to-noise ratio of every other monitoring tool improves because there's one less thing to watch.

This is Frenz's point. Every path you erase from the configuration improves the effectiveness of every monitoring tool downstream. Erasure doesn't compete with monitoring. It makes monitoring work better.

Erasure in practice

An erasable path is a configuration property that creates attacker optionality and can be changed. In AWS, these are specific and concrete:

A security group with 0.0.0.0/0 egress. An attacker who compromises a workload behind this group can reach any destination on the internet. Scope the egress to what the workload needs. Path erased.

An IAM role with a wildcard action (s3:*) on a wildcard resource (*). Any credential that assumes this role has full S3 access. Scope to the specific buckets and actions required. Path erased.

An unused access key on an IAM user with admin permissions. A credential that works but nobody uses. Delete the key. Path erased.

An EKS node group allowing IMDSv1 with hop limit > 1. Pods on the node can reach the instance metadata service and steal the node's IAM credentials. Enforce IMDSv2 and set hop limit to 1. Path erased.

Each of these is a binary property. The path exists or it doesn't. It's verifiable from a configuration snapshot. No runtime data, behavioral analysis or LLM inference needed. A predicate evaluates to true or false.

Compound paths are higher-value erasures

The subtractive framing becomes most powerful when applied to compound paths. Attack chains that exist only in the composition of individually survivable configurations.

Consider: a Bedrock agent role grants broad lambda:InvokeFunction on Resource: *. Separately, that agent references a Lambda function that no longer exists (a ghost ARN). Separately, the agent role has overbroad S3 write access.

Each configuration is individually common. Together they create a data exfiltration path: prompt control gives the mechanism, the ghost ARN gives the exploitation vector, S3 gives the destination.

The subtractive insight: erasing any one leg erases the entire compound path. Scope the Lambda invoke to specific function ARNs. This erases the compound path. Delete the ghost reference. This erases the compound path. Scope the S3 write. This erases the compound path.

One configuration change, one erasure, multiple TTPs removed. That's the prioritization logic Frenz describes: "path erasure A eliminates 5 TTPs across 1000 assets... readily apparent that action A will result in a greater reduction in attacker optionality."

A monitoring tool that detects the agent making an unusual Lambda call is operating at level 3 of the hierarchy. It's watching a fire. The compound path erasure prevents the fire. Both are useful. One is structurally better.

Path Erasure Rate is the metric executives need

Frenz introduces Path Erasure Rate: paths erased divided by paths eligible for erasure. It's binary, clear, and trends in one direction.

This answers the executive question that the industry has been struggling with for decades: "how do I measure security progress?" Not compliance percentage, alert closure rate or MTTR. Paths erased.

A tool that finds 34 erasable paths in an IAM configuration gives you PER = 0/34. When the team scopes the first overprivileged role, PER = 1/34. When all 34 are resolved, PER = 34/34 = 1.0.

The number goes up. It never comes back down for the same path. Every increment structurally reduces the attack surface, deterministically.

For compound paths, the math is even better. Erasing one leg of a five-leg chain removes one finding but eliminates the entire chain. The PER increment is 1, but the attacker optionality reduction is the full compound path. This is the only metric where a smaller action produces a larger effect.

The hierarchy explains the market's blind spot

Most of the cybersecurity industry's revenue comes from monitoring. EDR, SIEM, NDR, XDR, SOAR. These are all level-3 tools. They detect, alert, investigate and respond. They operate after the path has been traversed.

Level-1 and level-2 tools that erase and constrain paths represent a tiny fraction of the market. Infrastructure-as-code scanners, CSPM tools, and configuration verifiers operate here, but they're treated as compliance tools rather than security tools. "Check the box" rather than "erase the path."

Frenz's hierarchy explains why the industry has 11,000 alerts per day and 40% of them go uninvestigated: the paths that produce those alerts were never erased. The monitoring is correct. The architecture is wrong.

The subtractive approach doesn't eliminate monitoring. It reduces the load on monitoring to the paths that genuinely can't be erased. The ones where detection is the right answer because prevention isn't possible. The remaining alerts are the ones that matter, because every erasable path has already been removed.

What this changes

The OWASP project number makes this a reference, not an opinion. When a practitioner argues that scoping an IAM role is more valuable than adding another SIEM rule, they can now cite a framework with a hierarchy, a metric, and an OWASP project backing it.

The industry's default question has been "what should we monitor?" The subtractive question is "what should we delete?"

The tools that answer the second question, identify erasable paths, prioritize compound erasures, and measure PER trending toward zero are the ones operating at the top of the hierarchy. Everything else is watching.

Stave is an open-source cloud configuration verifier. Each finding names an erasable path. Each chain finding names a compound path where erasing one leg eliminates the entire chain. No credentials required.

The Sandbox Had One Allowed Egress Path. The Model Used it to Escape.

Bala Paranj — Fri, 21 Aug 2026 11:46:30 +0000

Two sandbox escapes in one week. The Hugging Face breach happened because the sandbox had unrestricted egress. The OpenAI ExploitGym escape is more instructive. The sandbox was almost isolated.

The evaluation environment had one sanctioned egress path: an internal JFrog Artifactory proxy that pulled packages from the internet. The model found a vulnerability in that proxy, pivoted through it to a node with internet access, stole credentials from the pivot node, and chained them into production infrastructure. The isolation was real except for one allowed path. That one path was the entire attack.

This does not mean "don't allow any egress." Some sandboxes need package registries, model repositories, or data feeds. The lesson: if your sandbox has a single allowed egress path, that path is your entire security boundary, and every property of that path such as its network rules, IAM role, credential scope, monitoring coverage must be verified.

The anatomy of a sole-egress escape

A properly isolated enclave has no direct internet access. No internet gateway, NAT gateway, VPC peering to an internet-connected VPC, Transit Gateway attachment, VPN or Direct Connect. No public IPs on any instance.

But it needs packages. So you add one thing: a proxy that can reach a package registry. The proxy runs in the same VPC. It has a security group that allows outbound HTTPS to the registry endpoint. Its IAM role has just enough permission to authenticate with the registry. It pulls packages and serves them to the enclave's compute.

The security model says: the proxy is the sole egress path, and the proxy is tightly scoped. The proxy's security group allows only HTTPS to the registry. The proxy's role has only registry read access. The enclave instances can reach the proxy but nothing else.

Here's what goes wrong:

The proxy has a vulnerability. The model exploits it. Now the attacker has the proxy's network position which includes outbound HTTPS to the registry, but also whatever the proxy's security group allows beyond that single rule. If the security group has a broad egress rule (even one that "should only reach the registry"), the attacker has internet access.

The proxy has an IAM role. The attacker can reach the IMDS endpoint from the proxy's network position (unless IMDSv2 is enforced AND the hop limit prevents container escape). Now the attacker has temporary credentials with whatever permissions the proxy's role carries. If the role has more than registry-read even one extra permission, the blast radius expands.

The enclave instances can reach the proxy. If the security group allows broad traffic from enclave to proxy, the attacker can pivot in the other direction. From a compromised enclave instance to the proxy, gaining the proxy's network and credential context.

Three configuration properties. Each seems reasonable in isolation. Together, they create a complete escape path from an isolated sandbox to production.

26 invariants for sandbox isolation

I decomposed sandbox isolation into 26 configuration properties across five areas: the egress inventory (what paths out of the VPC exist), the pivot surface (can enclave instances reach egress-capable nodes), SSRF absorption (can credential theft be prevented at the instance level), proxy egress scope (is the sanctioned egress scope), and credential blast radius (do enclave credentials reach production).

Egress inventory

Ten properties verify that a VPC has no unintended egress paths. Every way traffic can leave a VPC is a checklist item: IGW attachment, NAT gateway routes, IPv6 egress-only gateway routes, VPC peering, Transit Gateway attachments, VPN and Direct Connect, public IPs and Elastic IPs, VPC endpoint policies, and gateway endpoint route entries.

Most of these have existing controls. The main route table is checked for IGW routes. NAT egress is checked. VPC peering has five controls covering bidirectional routing, cross-account trust, and DNS resolution. Transit Gateway has seven controls covering route isolation, auto-accept, and propagation scope.

Two gaps needed contract fields: IPv6 routes to egress-only gateways, and gateway endpoint route table associations. A gateway endpoint without route entries is a ghost endpoint. It exists but traffic doesn't flow through it.

The most important gap was the obvious one: every egress control fires unconditionally on every VPC. There was no way to say "this specific VPC should be isolated, check it more strictly." A VPC with an internet gateway is normal for a web application. The same VPC with an internet gateway is a critical finding if it's supposed to be an evaluation sandbox.

The missing concept: isolation intent

The audit's central finding: there was no representation for "this VPC should be isolated from the internet."

The fix is a tag. The same mechanism that marks an S3 bucket as containing PHI (classification: PHI), or marks an ECS service as read-only (service-type: read-only), marks a VPC as intended to be isolated:

Tag: stave:isolation-intent
Values: airgapped | egress-restricted | internet-facing

An airgapped VPC should have no egress of any kind. An egress-restricted VPC should have egress only through VPC endpoints with resource-scoped policies. An internet-facing VPC is the standard case where egress controls fire at their normal thresholds.

The tag expresses intent. The controls verify reality matches intent. The finding fires when the VPC is tagged as airgapped but has an internet gateway attached. That's the compound check: declared isolation intent plus observed egress path equals a critical finding. Without the intent tag, the same internet gateway is informational. With it, it's a containment breach.

The pivot surface

The hardest gap to detect and the one the ExploitGym attack exploited is the lateral path from an enclave instance to an egress-capable node. The model didn't break through the sandbox's external boundary. It moved laterally to a node that had a legitimate egress path and used that node's network position to escape.

Three properties check this: security group rules permitting enclave instances to reach egress-capable instances, NACL rules permitting enclave subnets to reach subnets with default routes out, and security groups shared between enclave and non-enclave instances. A shared security group is an implicit trust bridge.

Each requires classifying which instances are enclave (isolated, no egress) versus "egress-capable" (has a path to the internet, even if scoped). The isolation intent tag on the VPC provides this classification. Instances in an airgapped VPC are enclave. Instances in the same VPC with egress rules reaching outside the VPC CIDR are egress-capable. The security group check asks: can the first group reach the second?

SSRF absorption

If an attacker reaches an instance through a vulnerability (SSRF, RCE, dependency exploit), the first thing they try is the Instance Metadata Service. IMDSv2 enforcement, hop limit restriction, and IMDS disablement on instances that don't need credentials. These are the controls that prevent SSRF from becoming credential theft.

This area is fully covered. Eighteen IMDS controls across EC2, ECS, and EKS. The Capital One chain (IMDSv1 + public instance + broad role) has existed with incident citation since the control catalog was first built. Five chain variants cover EC2, ECS, EKS, Kubernetes, and container-specific IMDS paths. This is the most thoroughly covered area in the entire audit because the attack pattern has been documented since 2019.

Proxy egress scope

The proxy itself is a security boundary. Its security group determines what it can reach. Its VPC endpoint policies determine what services it can access. Its IAM role determines what it can do with those services.

The control that's been missing: verifying that egress is exclusively through VPC endpoints or an allowlist. Existing controls detect unrestricted egress (security group allows 0.0.0.0/0) and detect broad NAT egress. No control verified the positive case: "this VPC's ONLY egress is through endpoints." Having both an internet gateway and VPC endpoints means the endpoints are bypassable. Traffic can go around them through the IGW. The control checks: if the VPC is egress-restricted, does it have any egress path other than VPC endpoints?

Credential blast radius

The final area checks whether sandbox credentials can reach production. Can an enclave role assume a production role? Does a production role trust an enclave principal? Are resource policies in production granting access to enclave accounts? Does the enclave's Organization OU have SCPs restricting services and regions?

The existing cross-environment controls are strong: transitive role assumption paths, org boundary trust checks, SCP region and service restrictions. The gap: no control checked resource policies (S3, KMS, SQS, SNS, Lambda) for grants to enclave-account principals specifically. A production S3 bucket policy that grants s3:GetObject to an enclave account's IAM role is a data access path that bypasses every AssumeRole control. The resource policy grant doesn't require assuming a role. It's a direct grant from the resource to the foreign principal.

Compound chains

The individual controls check individual properties. Three compound chains compose them into the attack paths the ExploitGym model took:

The enclave egress violation chain fires when a VPC tagged as airgapped or egress-restricted has any internet egress path. The compound check: declared intent says "no egress" AND observed configuration says "IGW attached." This is CHAIN-G from the audit. The chain that couldn't be authored until isolation intent had a representation.

The enclave lateral to egress chain fires when an isolated VPC has both enclave compute and egress-capable compute with security group rules permitting traffic between them. This is the ExploitGym path: the model moved laterally from enclave to proxy because the security group allowed it.

The enclave to production via resource policy chain fires when a production resource policy grants access to an enclave-account principal without requiring AssumeRole. This bypasses every identity-based control (MFA, ExternalId, session naming) because it's a resource-based grant.

The proxy is the perimeter

The operational takeaway is counterintuitive: in a sandbox with one allowed egress path, that path isn't part of the boundary. It is the boundary. Every property of the proxy such as network rules, IAM role, patch level, monitoring has the same security significance as a firewall rule on a perimeter device. But nobody reviews a JFrog Artifactory instance with the same rigor they review a firewall.

The 26 invariants formalize them. They check the entire egress inventory (every way traffic can leave), the pivot surface (can enclave instances reach the proxy), the proxy's scope (is its egress restricted), and the credential blast radius (can proxy credentials reach production). Verified before the model runs. Verified continuously. Verified against declared intent, not assumed isolation.

The sandbox had one allowed egress path. The model used it. The configuration that allowed it is verifiable. Check it.

The 26 egress isolation invariants, the isolation-intent mechanism, and the three compound chains are implemented in Stave. The egress isolation checklist is derived from the OpenAI ExploitGym and Hugging Face incidents (July 2026).

Cloud Security Has No TodoMVC. So We Built One.

Bala Paranj — Thu, 20 Aug 2026 12:02:37 +0000

In JavaScript, when you want to compare frameworks, you go to TodoMVC. Same app, every framework. You compare React and Vue and Svelte against identical requirements. The differences such as bundle size, rendering approach, state management, verbosity reveal themselves on uniform ground.

Cloud security has never had this.

Every vendor demos against their own scenario. Prowler shows you their best findings. Wiz shows you their graph. AWS Config shows you its rules. Nobody runs them all against the same deliberately misconfigured environment and publishes what each one found and what each one missed.

If you're evaluating cloud security tools such as during a POC, a vendor comparison, or an internal assessment, you're comparing demos, not data.

This should change.

The problem with vendor-controlled comparisons

When a vendor runs a demo, they choose the environment. They choose which misconfigurations to show. They choose the narrative. Every tool looks good when the author picks the example.

A POC is better. You run the tool in your own environment. But your environment has whatever it has. You can't control what misconfigurations exist, so you can't tell the difference between "the tool didn't find it" and "it doesn't exist to find." If Tool A reports 200 findings and Tool B reports 150, is Tool A better or just noisier?

What you need is a controlled environment. Known misconfigurations. Documented ground truth. A scorecard. Run every candidate tool against the same environment, fill in FOUND or MISSED for each ground truth item, and compare.

TodoMVC gave that for JavaScript ecosystem. Cloud security doesn't have an equivalent.

What we built

A Terraform-deployed AWS environment containing 30 documented misconfigurations across 8 services, plus 5 multi-resource attack paths. Each misconfiguration has a unique ID, a description, a severity rating, and manual verification steps so you can confirm it's real by checking the AWS console yourself.

The environment deploys in about 10 minutes and costs roughly $2/day to run. You tear it down when you're done evaluating.

The misconfigurations cannot be argued. Not "this IAM role might be overprivileged depending on your risk threshold." Rather: "this S3 bucket allows public read access." Binary. Objectively wrong. Any tool claiming to cover S3 should find it.

The 5 compound paths are the interesting part. These connect misconfigurations across services into attack chains. A publicly reachable instance with an overbroad IAM role that can read from an unencrypted S3 bucket isn't three independent findings, it's one attack path. Some tools find the individual resources. Fewer tools connect them into the path. The scorecard distinguishes between the two.

How it works

Deploy the environment. Run your candidate tools against it. Fill in the scorecard.

The scorecard is a CSV with every ground truth ID pre-filled. For each tool, you mark FOUND, MISSED, PARTIAL, or N/A. At the bottom: an atomic score (out of 30) and a compound score (out of 5).

No software to install beyond the tools you're evaluating. No vendor to contact or account to create.

We include one completed scorecard as a reference. Stave's results against the environment so you can see what a filled-in scorecard looks like and verify the environment is working before running your own tools. It's there as an example, not a claim.

What's in the ground truth

Eight AWS services: S3, IAM, CloudTrail, KMS, EC2, ELBv2, OpenSearch, and Config. The misconfigurations span:

Public exposure such as resources accessible from the internet that shouldn't be. Encryption gaps such as data at rest or in transit without encryption. Logging deficiencies such as audit trails that are disabled or incomplete. Identity issues such as overprivileged roles, unused credentials, missing MFA conditions. Network configuration such as security groups with overly permissive rules.

The compound paths combine these into realistic attack chains. Each path documents the entry point, the pivot, and the target. Each step references a specific ground truth atomic finding, so you can see exactly which individual misconfigurations compose into the chain.

Why compound paths matter

The CSA Top Threats to Cloud Computing 2026 report says: "attackers exploit the seams between products." They use a compromised third-party API to pivot to an overprivileged identity to exfiltrate data from a misconfigured bucket. No single product sees the whole chain.

Most security tools check one resource at a time. Security group: ✓ checked. IAM role: ✓ checked. S3 bucket: ✓ checked. Three passing checks. One exploitable path.

The compound paths in the eval kit test whether a tool can connect the checks. Finding the individual misconfigurations gets you partial credit. Connecting them into the attack path gets you the full score. The scorecard makes this visible.

Standarized environment

This is a standardized environment for running your own evaluations. You deploy it, run your tools, see what they find. The ground truth is public. Your results are yours.

This is not a public ranking, a vendor-shaming exercise, or a definitive verdict on which tool is best. Different tools have different strengths. Some focus on compliance mapping. Some focus on runtime detection. Some focus on code scanning. The eval kit tests configuration posture such as one dimension of a multi-dimensional problem. A tool that scores 25/30 on atomic findings might be exactly right for your use case, and a tool that scores 30/30 might miss what matters to you.

This is not a replacement for evaluating tools in your own environment. The eval kit gives you a controlled comparison. Your production environment gives you the real one. Use both.

Contributing results

If you run the eval kit during a real evaluation and want to share your results, you can submit a completed scorecard to the repo. This is optional. The kit works just as well if you never share anything.

Over time, if enough people share results, a picture emerges. Not a ranking but a coverage map. Which tools find which categories of misconfigurations. Where the compound path detection drops off. What the industry is good at and where the gaps are located.

That picture is missing in security industry now. Every organization runs POCs but because nobody runs them on the same ground.

Try it

The repo is open. The environment is Terraform. The ground truth is documented. The scorecard is a CSV.

If you're evaluating cloud security tools this quarter, run them against the same environment and see what you see.

The eval kit is open source and vendor-neutral. Stave's results are included as a reference implementation. The first completed scorecard, not a claim of superiority. The environment, ground truth, and scoring template stand on their own regardless of which tools you choose to evaluate.

Addressing the CSA Top Threats to Cloud Computing 2026

Bala Paranj — Wed, 19 Aug 2026 10:40:33 +0000

CSA's Top Threats to Cloud Computing 2026 published in August 2026. 507 experts. 11 issues. The scores range from 7.45 to 7.95. A spread of half a point, meaning the industry considers all eleven roughly equally severe. Identity holds the top spot. Two AI-specific threats enter for the first time: AI-Enhanced Attacks at rank 2 and AI System Compromise at rank 6.

The report tells you what to worry about. It doesn't tell you which worries are mechanically checkable.

We took the report and asked a different question: for each issue, what machine-verifiable properties can we extract from a cloud configuration snapshot, and do we already check them?

The results: 112 properties across all 11 issues. 93 are snapshot-verifiable. 91 are at least partially covered. Two are pending on observation data. The controls are designed but the data isn't collected yet. And 19 properties are out of scope because they require runtime behavior, organizational process, or data that doesn't exist in a configuration snapshot.

The methodology

A risk framework and a verification tool operate at different levels of abstraction. The CSA report says things like "implement least-privilege access controls." That's good advice. It's also not checkable as stated.

So the first step was decomposition. For each of the 11 issues, we extracted every concrete property that could be expressed as a predicate over cloud configuration state. "Implement least-privilege access controls" becomes a family of specific checks: IAM roles don't have wildcard actions, execution roles don't carry PowerUserAccess, trust policies require MFA conditions, cross-account roles scope to specific principals.

Then we classified each property into three categories:

Snapshot-verifiable. The property can be checked by examining current AWS configuration such as IAM policies, security group rules, encryption settings, logging configuration, resource tags. This is Stave's domain.

Runtime-behavioral. The property requires observing system behavior over time such as API call patterns, anomalous logins, data access frequency, runtime attestation. The domain of SIEM, EDR, and runtime monitoring. We can't check these from a snapshot.

Procedural. Organizational process such as access reviews, incident response plans, shared responsibility understanding. They are important, but not machine-checkable.

Out of 112 properties: 93 snapshot-verifiable, 19 out of scope. The 19 break down as: 8 runtime/behavioral, 5 non-AWS-Config data, 4 organizational/process, and 2 that we deliberately chose not to implement. One because the property requires application-level authorization logic rather than configuration (OWASP BOLA/BFLA). Another one because including the data in snapshots would create the very exposure it's supposed to detect (secret values in environment variables). The 19 are boundaries, not gaps.

What 97.8% means

93 snapshot-verifiable properties. Of those: 77 fully covered, 14 partially covered, 2 pending on observation data. 91 out of 93 at least partially covered. That's 97.8%.

The 14 partial items are narrow scope boundaries within well-covered areas such as "container base image currency requires layer analysis" when we already check ECR scanning, image signing, digest pinning, and untrusted registry pulls. The configuration side is covered; a thin slice at the edge isn't.

The 2 pending items are both AI/ML-specific: prompt/dataset versioning and model artifact signing. These need observation fields that AWS services don't yet surface as configuration data such as model provenance metadata, dataset lineage, prompt version history. The controls are designed and the predicates are written. The day the observation data is available, 14 controls activate across those two properties. Zero new logic required.

Zero actionable snapshot-verifiable gaps. Every configuration property that can be verified from an AWS snapshot has at least partial coverage.

Eleven issues, one catalog

We checked against a catalog of 3,434 controls and 745 chains across 131 AWS services. Of those, 744 unique controls, 21.6% of the catalog mapped directly to CSA properties. The rest cover areas the CSA report doesn't touch (service-specific best practices, compound attack paths, ghost reference detection) or provide depth beyond what the framework requires.

Here's how each issue broke down.

Issue 01: Inadequate Identity, Credential, Access, and Key Management

The deepest coverage in the catalog. 11 properties, 9 fully covered, 1 partial, 1 out of scope (token replay, which is runtime). 138 controls referenced, spanning trust policies, confused deputy prevention, session management, credential lifecycle, and agent identity.

The compound chain third_party_exposure_path fires when a vendor role is both dormant and overprivileged and lacks an external ID condition. Three facts that are individually common but together create an unmonitored ingress path through a third-party trust relationship. This is the shape that atomic scanners miss: each control might pass a risk threshold alone, but the combination doesn't.

Issue 02: AI-Enhanced Attacks and Weaponization

New in 2026. 9 properties, 6 covered, 1 partial, 2 out of scope (vulnerability remediation velocity and runtime integrity verification). 92 controls referenced.

This issue functions as a threat amplifier. The snapshot-verifiable properties are about reducing the surface that AI-enhanced attacks target. Credential lifetime, logging depth, detection coverage. The mitigations overlap heavily with Issues 01 and 05.

Issue 03: Insecure Third-Party Resources

9 properties, 5 covered, 2 partial, 2 out of scope (SaaS dependency inventory and SBOM presence, both require data sources outside AWS configuration). 82 controls.

EKS image provenance is the deepest area: 10 controls covering public registry pulls, missing digests, unsigned images, and the critical-severity eks_image_provenance_unsafe chain. The partial items are container base image currency (we check signing and scanning, not individual layer freshness) and API key rotation lifecycle (no native API Gateway mechanism exists).

Issue 04: Insecure Interfaces and APIs

10 properties, 7 covered, 1 partial, 2 out of scope. 77 controls. API Gateway carries 105 controls in the catalog including auth, throttling, WAF, logging, mTLS, and orphan detection. One of the out-of-scope items is OWASP BOLA/BFLA which is whether your application checks that User A is allowed to access Object B. That's authorization logic in code, not AWS configuration.

Issue 05: Misconfiguration and Inadequate Change Control

The original cloud security problem. 12 properties, 7 covered, 3 partial, 1 pending (prompt/dataset versioning which is AI/ML-specific, controls designed, waiting on observation data), 1 out of scope (IaC validation is pre-deploy; Stave checks post-deploy state). 93 controls.

The three partials are exposed credentials (we check rotation and storage patterns, not secret content), configuration drift (we compare snapshots, not continuous real-time monitoring), and RAG pipeline misconfiguration.

Issue 06: AI System Compromise and Supply Chain Exploitation

New in 2026. 11 properties, 7 covered, 0 partial, 1 pending (model artifact signing has controls designed, waiting on observation data), 3 out of scope (prompt injection defense, output monitoring, and runtime behavioral analysis are all runtime concerns). 72 controls.

The compound chain bedrock_agent_data_exfiltration is the standout. It fires when three conditions stack: a Bedrock agent role grants broad lambda:InvokeFunction on Resource: *, the agent references a Lambda that no longer exists (ghost ARN), and the role has overbroad S3 write access. Prompt control gives the mechanism, the ghost reference gives the exploitation vector, S3 gives the exfiltration destination. Drop any one leg and the chain breaks.

Three of this issue's out-of-scope properties do have boundary controls. Prompt injection defense is runtime, but Stave verifies that the guardrail configuration exists. The prompt attack filter is enabled at high strength and the agent has a guardrail associated. The control doesn't detect injection at inference time. It verifies the defensive configuration is in place.

Issue 07: Advanced Persistent Threats

10 properties, 7 covered, 1 partial (CodeArtifact supply chain persistence has controls queued pending observation data for package repository policies), 2 out of scope. 112 controls.

Issue 08: Insufficient Cloud Security Architecture and Strategy

10 properties, 6 covered, 2 partial, 2 out of scope (SaaS integration visibility and shadow AI detection). 126 controls. One partial is the human-to-machine identity ratio which is a governance indicator tracking whether non-human identities are proliferating faster than oversight can keep up.

Issue 09: Insecure Software Development Lifecycle

10 properties, 6 covered, 2 partial, 2 out of scope. 118 controls. One out-of-scope item, secret values in environment variables. We deliberately chose not to implement. Including the secret content in configuration snapshots would create the exposure risk the control is supposed to detect. The correct tool for that check is a secrets scanner operating on the live environment, not a configuration verifier storing the values.

The codebuild_build_escape chain maps here: when a CodeBuild project runs with Docker privileged mode and has an overprivileged IAM role, a container escape grants the build role's full permissions on the host. Both conditions must stack.

Issue 10: Accidental Data/AI Data Exposure

10 properties, 7 covered, 1 partial (SageMaker model package group policy closes when the AI/ML governance observation data ships), 2 out of scope. 77 controls. S3 alone carries 149 controls covering public access block, ACLs, bucket policies, website hosting, access points, encryption, and versioning.

Issue 11: System Vulnerabilities

10 properties, 10 covered. 0 partial. 0 gaps. 0 out of scope. 125 controls. 100% coverage.

This is the counterintuitive result. "System vulnerabilities" sounds like CVE patching, which is runtime. But the CSA properties decompose into configuration checks: is Inspector enabled, is SSM Patch Manager configured, is ECR image scanning active, are ECS/EKS clusters running supported versions, are security groups restricting unnecessary exposure. Every one of those is a snapshot-verifiable configuration predicate.

The exclusions

Nineteen properties across all 11 issues are out of scope. None are missing categories. They're individual properties where the required data doesn't exist in an AWS configuration snapshot, or where implementing the check would be counterproductive.

Six of the 19 have boundary controls. These don't cover the runtime property itself, but they verify the infrastructure prerequisite. The distinction matters: when Stave reports that a Bedrock guardrail is configured with prompt attack filtering at high strength, that's not the same as detecting prompt injection at inference time. It's verifying the defensive configuration exists. The gap between configured and effective is real, and we don't paper over it.

The remaining 13 have no boundary coverage at all. Token replay is runtime. SBOM presence is a development artifact. Edge device security isn't AWS configuration data. These aren't gaps in the tool. They're boundaries of the domain.

Where AI/ML observability falls short

The two pending properties: prompt/dataset versioning and model artifact signing are both AI/ML-specific. They need 3 new asset types and 16 observation fields. Currently in development.

This tells you something about the state of the industry. Traditional cloud services (IAM, S3, VPC, EC2, EKS) have mature APIs that produce comprehensive configuration data. AI/ML services (Bedrock, SageMaker) expose their core functionality through APIs but don't yet surface the governance metadata such as model provenance, dataset lineage, prompt version history that security verification needs. The gap isn't in verification logic. It's in what the platform makes observable.

The number that matters

744 controls out of 3,454 which is 21.6% of the catalog map to the 11 CSA threats. That means 78.4% of the catalog covers properties the CSA report never mentions: service-specific best practices, compound attack paths between services, ghost reference detection, denial-of-wallet controls, and controls at depths the framework doesn't reach.

The CSA report is a useful floor. It is not a ceiling. The properties it recommends are the ones 507 experts agreed are important. The properties it doesn't mention are the ones that show up in incident reports.

Stave is an open-source cloud configuration verifier. It evaluates AWS configuration snapshots against system invariants and finds compound attack paths that single-resource scanners miss without requiring any credentials.

Your CloudTrail is Enabled, Multi-Region, and Validated. It Stopped Delivering Events 120 Days Ago.

Bala Paranj — Tue, 18 Aug 2026 12:05:26 +0000

There's a class of failure where every check passes and the system is broken.

CloudTrail is enabled. Multi-region. Log file validation is on. CloudWatch Logs integration is active. The retention policy is configured. The S3 lifecycle rule exists. The compliance dashboard shows green across every line item.

The S3 delivery failed 120 days ago. The bucket policy changed or the bucket was deleted. Or the KMS key was disabled. Or the trail's IAM role lost permission to write. CloudTrail logged the failure in GetTrailStatus under LatestDeliveryError. Nobody called GetTrailStatus. Every tool checked the trail configuration. No tool checked whether the trail was actually delivering events.

Every control downstream of delivery evaluates a stale or empty bucket. The retention policy governs nothing. The lifecycle rule expires nothing. The log file validation validates nothing. The compliance report certifies a system that isn't recording.

The ghost reference in your audit trail

This is the ghost reference pattern where a configuration that appears active but references something that no longer functions. Secrets Manager rotation enabled but the rotation Lambda was deleted. An SNS topic configured as an alarm action but the topic was deleted. A CloudWatch subscription filter pointing to a destination that rejects the data.

The audit trail version is the most dangerous because it's invisible to every other security check. An S3 bucket that's public will fire a control. A trail that's not delivering won't fire anything. Because the non-delivery isn't a configuration property the trail exposes through its standard describe API. It's in a separate API call (GetTrailStatus) that most collectors don't make.

The fix is a derived boolean: s3_delivery_healthy. True when LatestDeliveryError is empty AND LatestDeliveryTime is within 24 hours. The collector makes the GetTrailStatus call, derives the boolean, stamps it on the observation. The evaluator checks the boolean. The complexity of "is delivery working?" is resolved once by the collector. The evaluator sees a field and compares it against an expected value.

What I checked

I decomposed detection blindness. The state where an attacker operates without any detection service noticing into 27 configuration properties across six AWS detection services.

CloudTrail covers the audit trail: enabled, multi-region, management events include reads (not just writes), S3 and Lambda data events, CloudWatch Logs integration, delivery health, log file validation, and retention beyond the 90-day Events History window. Seven properties.

AWS Config covers configuration drift visibility: recorder enabled, all resource types recorded, global resources included, delivery channel with S3 and SNS, and delivery health (snapshot freshness). Five properties.

GuardDuty covers threat detection: detector enabled, all protection features active (S3, EKS, Lambda, RDS, Runtime, Malware), finding export to S3, notification routing to the response team, and detection method count for cross-signal correlation. Five properties.

Security Hub covers finding aggregation: enabled, at least one standard active, auto-enable for new accounts, and cross-region aggregation. Four properties.

VPC Flow Logs cover network visibility: enabled per VPC, status ACTIVE, and custom format with security-critical fields (vpc-id, subnet-id, tcp-flags, pkt-srcaddr). Three properties.

CloudWatch and EventBridge cover the detection pipeline: log group retention policy, subscription filter destination monitoring, and CloudTrail change monitoring (CIS 4.5 — detection self-protection). Three properties.

26 of 27 covered

The existing catalog handles nearly everything. CloudTrail alone has 60 controls. Config has 51. GuardDuty has 19. The per-service coverage is thorough for the properties these services expose through their standard describe APIs.

The one property that isn't covered is CloudTrail S3 delivery health. This requires a different API call than the standard trail description. DescribeTrails returns the configuration. GetTrailStatus returns the operational state. Most collectors call the first. Few call the second. The gap exists because the delivery status lives in a different API endpoint than the trail configuration.

The two compound gaps

Individual service controls fire independently. An attacker who disables GuardDuty triggers one finding. An attacker who also breaks Config delivery triggers another. An attacker who simultaneously breaks CloudTrail CloudWatch integration triggers a third. Each finding arrives separately. No compound finding says "multiple detection services are compromised simultaneously."

Detection response broken. GuardDuty findings are generated but not delivered (notification not configured). Config changes are recorded but not notified (SNS topic missing from delivery channel). CloudTrail events flow but don't reach CloudWatch for real-time detection (integration broken). Each service detects. None responds. A compound chain with threshold 2 fires when any two notification paths are broken. It catches the pattern that individual service checks miss. An attacker who breaks two notification paths simultaneously faces no alert escalation.

Detection quality decay. CloudTrail captures only writes (reads are invisible, an attacker's reconnaissance leaves no trail). Flow logs use default format (TCP flags and packet source addresses are missing. Forensic analysis can't determine connection direction or spoofing). GuardDuty has low detection method diversity. Cross-signal correlation fails because single-signal findings are less reliable. Retention is too short (evidence is gone before the investigation starts). Each is medium severity individually. Three of four simultaneously means the detection stack is structurally insufficient for post-incident forensics. Detection exists. Evidence doesn't.

The quality decay problem

Detection quality decay is subtler than detection absence. An absent detection service triggers an immediate finding. "GuardDuty is disabled" is a clear signal. A degraded detection service looks normal on every dashboard. The trail is enabled. The flow logs are active. GuardDuty is running. The dashboard is green.

But the trail only captures writes. The flow logs use the default format. GuardDuty has one detection method instead of three. Retention is seven days. Each setting is a choice someone made or a default someone didn't change. None triggers a critical finding individually. Together, they mean the organization's detection stack produces evidence that's incomplete in four dimensions simultaneously.

After an incident, the forensic team discovers: they can see what was modified but not what was read (write-only trail). They can see that traffic flowed but not the TCP flags or real source address (default flow log format). GuardDuty produced low-confidence single-signal findings that were dismissed (low method count). The detailed logs from the first three days of the intrusion aged out of the seven-day retention window before anyone started looking (short retention).

The detection stack was never disabled. It was never misconfigured in any way that triggers a critical finding. It was degraded across four dimensions to the point where it couldn't support the investigation it existed to enable.

The compound chain that detects this with threshold 3 out of 4 quality dimensions degraded fires when the detection stack crosses from "partially degraded" to "structurally insufficient." The threshold is deliberate: any 2 of 4 degraded is common and low-risk. Three of 4 means the detection stack is decorative.

The pattern

Every audit I run surfaces the same structural gap: individual controls check individual properties. Compound risks span properties across services. The individual checks pass. The compound path is unsafe.

Detection blindness follows the same pattern. Each detection service is independently configured. No service knows whether the other services are configured. An attacker who impairs defenses across multiple services faces multiple independent medium-severity findings. There is no one critical compound finding that says "your detection stack is structurally compromised."

The fix isn't more individual controls. It's compound chains that compose across services. The detection response chain asks: "are multiple notification paths broken simultaneously?" The quality decay chain asks: "are multiple evidence dimensions degraded simultaneously?" Neither question can be answered by checking one service at a time.

The 27-item detection blindness checklist, the CloudTrail delivery health control, and the two compound chains are implemented in Stave. The ghost reference pattern (configured but not functioning) appears across CloudTrail delivery, Secrets Manager rotation, SNS alarm actions, and CloudWatch subscription filters. The same archetype, different services, same structural shape.

AI Generates Faster Than You Review. Properties Replace the Reviewer.

Bala Paranj — Mon, 17 Aug 2026 13:21:28 +0000

The pull request model worked for twenty years because both sides of the review had the same thing: domain knowledge. The author and the reviewer understood the codebase. The review was a conversation between two experts.

AI broke that assumption. The author is now a machine that doesn't understand the codebase. It lacks company specific business logic spread across different applications, spreadsheets, docs, confluence, wikis, emails and slack threads. The reviewer must now supply ALL the domain knowledge. Reviewers must verify that the generated code fits the architecture, follows conventions, aligns with the domain knowledge, respects invariants nobody wrote down, and doesn't introduce subtle design violations that pass every test.

Review became harder than writing. This doesn't scale. AI agents generate large number of commits in a day. The reviewer is the same person, with the same finite attention, reading diffs at human speed. The options are:

Slow down the agents (defeats the purpose), rubber-stamp the PRs (accepts the risk), or find something that replaces review.

Properties

The instinct is to write better documents. GitHub's Spec Kit crossed 120k stars on this idea: define what to build before building it. The spec becomes the input to the AI agent that generates the code.

This is a better prompt, not a solution. The spec tells the AI what to generate. Nothing tells anyone whether the generated code satisfies the spec. The verification step is missing. A spec written in natural language can't be mechanically verified. It can only be read by another human, which puts you back at the review bottleneck.

Specifications have failed before. Rational Rose generated code from UML diagrams. The diagrams were wrong. The generated code faithfully implemented the wrong design. The problem wasn't the tool. The problem was that a natural-language document that is structured and detailed requires human interpretation. Two humans interpret it differently. Neither can verify it mechanically.

Properties are different. A property is a statement that is true or false, checkable by a machine, with no room for interpretation.

"Every stateful resource must be encrypted at rest" is a property. For any resource, the answer is yes or no. There is no "it depends." There is no ambiguity about what "encrypted at rest" means. The resource either has encryption enabled or it doesn't. A machine checks it. Same answer every time.

"The system shall protect data using industry-standard encryption" is a specification. It requires a human to decide what industry-standard means, which data is covered, and whether the implementation qualifies. That human is the reviewer. The reviewer is the bottleneck.

The difference matters because properties can replace the reviewer. Specifications cannot.

The property is the review

The replacement comes from a discipline that predates software by decades. Systems Engineering is the field that builds nuclear plants, spacecraft, and medical devices. It separates properties from implementations and verifies mechanically that the implementation satisfies the properties.

The human writes the properties. The implementation by a human, AI or bot is verified against the properties mechanically. The properties don't generate the code. No reviewer in the loop.

The model has two sides. The left side is human work, done once. The right side is machine work, done continuously:

    HUMAN (once)                    MACHINE (continuous)

    Write 30 properties             Verify implementations
         │                               ▲
         ▼                               │
    Define domain                   Find gaps in coverage
    (which services apply)          (properties reveal missing controls)
         │                               ▲
         ▼                               │
    Create templates                Systematic fault-finding
    (how a property instantiates)   (structured guidewords on every component)
         │                               ▲
         ▼                               │
    AI generates 4,500 controls  →  Verify each control fires correctly

The human's effort is on the left side only. Each step on the right side corresponds to a step on the left, verifying at the same level. The machine generates, verifies, finds gaps, and catches bugs. This is continuous, at any scale, without a reviewer.

In cloud security, the universal statements look like:

"Every stateful resource must be encrypted at rest." Boolean. Checkable. No interpretation.

"No principal should have unrestricted Permissions Management actions." Boolean. Checkable. No interpretation.

"Credentials must rotate within their scheduled lifetime." Boolean. Checkable. No interpretation.

Each universal statement generates specific properties for every service in its domain. The encryption statement generates a property for S3, a property for RDS, a property for DynamoDB and so on. One for every service that stores data. 30 universal statements. 4,500 properties derived from them.

There are about 30 universal statements. Each applies across every service in its domain, generating hundreds of specific properties. A domain expert can audit all 30 in a day and by auditing 30, they've covered the 4,500 properties that derive from them." They change slowly only when understanding of the domain deepens, not when AWS adds a new service.

The implementation is 4,500+ controls that instantiate these 30 properties across 121 AWS services. Each control checks one property on one resource type. These change constantly when every new service needs new controls. AI generates them from the properties.

The verification runs in both directions. Downward: does every control follow from a property? If a control fires and no property covers it, the control is wrong. Upward: does every property cover every service in its domain? If the encryption property applies to 35 stateful services and the catalog has controls for 27, the other 8 are gaps found mechanically, not through incidents.

No human reviews 4,500 controls. A human reviews 30 properties. The machine does the rest.

Why this scales and PR review doesn't

The PR model puts domain knowledge in the review step. The reviewer must understand each implementation well enough to judge its correctness. When one agent generates code, one reviewer keeps pace. When ten agents generate code, the reviewer falls behind. When a thousand agents generate code, the model collapses.

Properties put domain knowledge in a mechanically verifiable artifact. The expert writes 30 properties. The verification is mechanical. Does the implementation satisfy the property? A machine answers this question at any scale.

One agent generating code: 30 properties, one verification pass, human reviews 30 properties. Ten agents: 30 properties (unchanged), ten verification passes (automated), human still reviews 30 properties. A thousand agents: 30 properties (still unchanged), a thousand verification passes (automated), human still reviews 30 properties.

The properties are constant with respect to the number of agents. The verification scales linearly but is automated. The human effort doesn't grow.

The proof: systematic fault-finding on AI-generated code

Does this work?

We ran the cycle on cloud security. 30 universal properties generated 4,500+ controls across 121 AWS services. After the AI generated a batch of 37 controls from one template, we applied a systematic fault-finding methodology from chemical process safety, adapted for software. Six structured guidewords applied to every component, looking for deviations from the intended behavior.

The analysis found 5 silent false negatives. Five controls had wrong identifiers. They existed in the catalog, appeared correct in every dashboard, but never matched real data. They would have sat there for months, making 5 services appear secure when they weren't. A human reviewer would have needed to read each control, understand the data schema for that service, and verify the identifier matched. This is a multi-day review across 37 services.

The systematic fault-finding found all 5 in one pass. Automated. Systematic. Fixed in one commit.

The cycle validates itself. The properties found 51 gaps in the catalog. The services where the property should hold but no control existed. The machine authored 51 controls to close them. The analysis found 5 were silently broken. The fix went in. Without the systematic fault-finding step, 5 controls sit in the catalog, 5 services appear clean, 5 silent false negatives surface months later when an incident reveals the control didn't fire.

Why cybersecurity proved it first

This approach works well in cybersecurity because of two characteristics.

First, the properties are clear. "Encrypted at rest" is boolean. "Authentication required" is boolean. There's no subjective judgment. Other domains have fuzzier properties such as "the UI should be responsive" or "the code should be readable" that resist formalization.

Second, the verification is decidable. Given a configuration and a security property, the answer is yes, no, or can't tell (with a classification of why). There's no "maybe it's secure depending on context." The verification terminates with a definitive answer.

These two characteristics made cybersecurity the proving ground. But the structure doesn't require them. It requires only that properties can be written by domain experts, implementations can be generated by machines, and verification can compare the two.

For API development, the properties are "every endpoint requires authentication" and "every response includes rate limit headers." The AI generates endpoint handlers. The verification checks handlers against properties.

For data engineering, the properties are "no null in required fields" and "referential integrity across tables." The AI generates pipeline transformations. The verification checks outputs against properties.

For infrastructure as code, the properties are "all subnets have route tables" and "no hardcoded credentials in any Terraform file." The AI generates modules. The verification checks plans against properties.

The pattern is the same. The property constrains. The machine generates. The verification confirms. The human writes properties, not reviews.

The Beck parallel

Kent Beck captured the same insight in 1997 with JUnit: "write the test once, execute it mechanically forever." The human writes the test. The machine runs it. The test is the domain knowledge, extracted from the developer's head into an artifact that the machine can check without the developer being present.

Properties are Beck's insight scaled to the agentic era. Write the property once. Verify implementations against it mechanically forever. The property is the domain knowledge, extracted from the expert's head into an artifact that the machine can check at any scale.

Beck made the developer's judgment reusable across test runs. Properties make the expert's judgment reusable across agents, implementations and the entire development lifecycle. The judgment is in the artifact. The verification is in the machine. The PR is obsolete.

The cycle described in this article properties, generation, verification, systematic fault-finding and gap sweep is implemented in Stave, an open-source AWS configuration verifier. 30 universal properties. 4,500+ controls. 121 services. The property is the review.

Solve It Once: Kelsey Hightower's Talk Applied to Security Verification

Bala Paranj — Sun, 16 Aug 2026 12:42:17 +0000

Kelsey Hightower gave a talk at PlatformCon 2026 that was about the arc of a career, from running commands in SharePoint to writing Go tools that play music on your terminal.

The stories has an architecture principle that applies to how security verification should work. Solve the problem once, encode the solution as a reusable artifact, and never solve that problem again.

The Jira loop

He joined a company where deployments were driven by Jira tickets. Someone opens a ticket with deployment parameters. An engineer would read the ticket, copy the parameters, run the commands, paste the output back into the ticket, close it, and wait for the next one. Every hour, another ticket. Same process, commands and manual steps. The engineer became the loop.

He wrote a Puppet manifest that watches the tickets, extracts the parameters, runs the deployment, posts the output, and closes the ticket. The loop ran once as automation and then it was over. No engineer in the loop or ticket waiting for a human. The problem was solved, permanently, by encoding the solution into a reusable artifact.

Doing a repetitive manual process faster is not the right thing to do. Eliminate the loop by recognizing the abstraction hiding in the repetition and encoding it into an artifact that makes the manual steps unnecessary.

The substrate

This is the pattern that runs through every transition he describes. It's missed by most people when they talk about automation.

System administrators ran deploy.sh manually. Docker didn't automate typing apt-get install. Docker recognized that "application + dependencies + environment" was a repeatable unit. The container image became the substrate. Deployment stopped being a sequence of commands and became a declaration. The commands didn't get faster. They became unnecessary.

Operators placed workloads on servers manually. Kubernetes didn't automate SSH-ing into machines to check available memory. Kubernetes recognized that "place this workload on infrastructure meeting these constraints" was a schedulable unit of work. The pod spec became the substrate. Placement stopped being a human decision and became a computation. The human decisions became unnecessary.

Each time, an outsider saw the abstraction that insiders were too close to see. The outsider didn't build a faster version of the manual process. They built the substrate that eliminated it.

If you don't see the substrate, you'll build faster versions of the old process and call it innovation. You'll automate the Jira loop with a bot instead of writing the manifest that makes the loop unnecessary. You'll put an AI on the SSH session instead of building the scheduler that makes SSH irrelevant.

The security version of the same trap

A security engineer audits IAM policies across an AWS organization. They open each account, read each role's policy, check for wildcards, check for overpermissive managed policies, check for cross-account trust without conditions, file findings, track remediation, and do it again next week. Every week, same process, same checks, same categories of findings on different resources. The engineer is the loop.

A Reddit thread captured this:

Fifty comments. Same pain from every angle. One engineer admitted to wildcarding permissions under deadline pressure. Another described spending 30 iterations at 10 minutes each trying to scope down a single role. Another asked for "an easy, out of the box, fast CI/CD PR check that tells you what permission errors you will receive before you get to prod."

Every commenter was stuck in the loop.

The current market response: "use AI to do the audit faster." Let Claude read the IAM policies. Let an AI agent review the CloudFormation templates. Let ChatGPT analyze the security group rules. Each one automates the manual review step without changing the abstraction. The AI reads the policy, infers the risks, generates prose, costs tokens. Next scan, different policy, same question, same inference, same tokens. A thousand policies, a thousand LLM calls. The same category, wildcard action with wildcard resource
is discovered a thousand times. Each discovery costs the same as the first. The LLM never remembers. The organization never accumulates. The token meter never stops running.

This is Kelsey's Jira loop at machine speed. The AI agent opens the policy, reads it, files a finding, closes the ticket, opens the next policy, reads it, files the same category of finding, closes the ticket. Faster. Same loop, cost structure and lack of accumulation.

The substrate hiding in the security process

The substrate is a formal specification evaluated against a configuration snapshot.

That's the abstraction hiding in the manual security review process. The same way the container image was hiding in the manual deployment process, the same way the pod spec was hiding in the manual placement process.

A Stave control is a solved problem compiled into a reusable artifact. CTL.IAM.ROLE.FULLACCESS.MANAGED.001 checks whether any IAM role has a full-access managed policy attached. It's a CEL predicate that is deterministic, zero tokens per evaluation, zero LLM calls, forever. The discovery that AdministratorAccess on a Lambda role is dangerous happened once. Maybe a human recognized it from a breach. Maybe an LLM helped surface it during authoring. The discovery was encoded into the control, and from that point forward, the loop is eliminated for that category.

The catalog with over 3000 controls is 3000+ loops that each ran once and became reusable artifacts. No LLM call will ever be made again to detect AdministratorAccess on a Lambda role. No human will ever manually check whether an S3 bucket policy grants cross- account access. No security engineer will lose their mind looking at the same wildcards every week. Each control makes the manual step structurally unnecessary.

The token economics are stark:

An AI agent approach: 1,000 accounts, 100 roles each, one LLM call per role per scan, weekly scans — 5.2 million LLM calls per year. Each call re-discovers what a CEL predicate already knows. The same finding, regenerated from scratch, every time, at full token cost.

Stave's approach: 3,000+ predicates evaluated against observations. Cost: milliseconds of CPU. Zero tokens or API calls. The same findings, deterministic, every run. The marginal cost of detecting the same category on the thousandth role is identical to the first: zero.

Train your own model

His closing line: "make sure you train your own model."

He means the mental model. The pattern recognition that practitioners build through decades of incidents, audits, misconfigurations, and fixes. The accumulated judgment that lets a senior engineer look at a configuration and see the attack path that a junior engineer misses. That model is the most valuable thing in the profession, and it lives in one person's head.

The control catalog is that model externalized. Each control encodes a pattern that a practitioner learned from a breach, an audit, a public incident report or a Reddit thread where someone described the pain. The sprint loop that produced the controls this session is the training loop: a gap audit discovers a new pattern, the Red-Green test proves the control catches it, a HAZOP validates the control itself. The model learns permanently. It never forgets, retires or fails to check because it's Friday afternoon.

The difference between Kelsey's trained model and the control catalog: his lives in one person's head and goes away when they retire at 42. The catalog is externalized, formalized, executable, and sharable. Every practitioner who runs stave eval inherits every pattern every contributor ever learned. The decades of experience, encoded once, applied by everyone, forever.

That's the multiplier. Not "AI reviews your policies faster." Instead: every insight, from every practitioner, compiled into a reusable artifact that runs at zero marginal cost, permanently. The model grows. The cost doesn't.

The SharePoint problem

Kelsey's other war story: deploy.sh was the wrong command. It was documented in SharePoint. He followed it, the deployment failed. The right command was this_one_works.sh. Nobody had updated the documentation.

This is the current state of cloud security knowledge. CIS benchmarks in PDFs. Best practices in Confluence. Security checklists written three years ago for services that have changed twice since. A practitioner follows the checklist, checks every box, and still has a misconfigured environment because the checklist doesn't cover the compound path between IAM and network that was discovered last month.

The substrate replaces SharePoint-as-security-knowledge with executable specifications. The control isn't a document someone might read and might follow. It's a predicate that evaluates against every configuration, every time, without anyone remembering to check. There's no this_one_works.sh because there's only one command: stave eval. The knowledge is the execution. If the knowledge is wrong, the execution is wrong, and the Red-Green test catches it before it ships. SharePoint couldn't tell you deploy.sh was wrong. The test suite tells you the control is wrong before anyone depends on it.

The Unix pipeline

Kelsey's progression: learn one command at a time → compose commands with pipes → create your own commands. He didn't jump from SharePoint to writing Go tools. Each step built on the previous one.

The same progression applies to the Stave adoption path:

# Learn one command
iam-explain role.json

# Compose with pipes
iam-explain role.json --output obs | stave apply

# Create your own
# Write a custom control for your organization's specific pattern

Each step builds trust through demonstrated value. The practitioner who starts with iam-explain checking one policy. The same way Kelsey started with find listing files in a directory, graduates to composing it with stave apply when one policy at a time stops being enough. The same way Kelsey graduated from single commands to find | xargs rm.

The pipe character is the transition from "automate the manual step" to "operate on the substrate." iam-explain alone automates the manual policy review. iam-explain --output obs | stave apply changes the abstraction. The output is structural findings from over 3,000 specifications evaluated simultaneously across every role in the observation set, including compound chains that exist only in the conjunction across roles.

Each tool follows the Unix principle that survived 50+ years: do one thing well, compose through a universal interface, stay closed for modification. iam-explain parses IAM policies and emits observations. stave apply evaluates observations against specifications. Neither modifies the other. Neither needs to know the other's internals. The obs.v0.1 JSON format is the text stream between them. It is inspectable at the pipe junction, the same way grep | sort | uniq -c is inspectable at every stage.

The convergence

I was staring at practitioners doing manual security audits. The same wildcards every week, the same compounds nobody could see, the same remediation loop that never converged and I built what they needed.

The architecture that emerged follows his principles anyway:

Solve it once, reuse forever. Each control encodes a discovery that never needs to be re-derived. The catalog is a library of solved problems. The marginal cost of applying the library is zero.

The substrate eliminates the loop. The specification evaluated against the snapshot is the abstraction that makes manual security review structurally unnecessary.

Train your own model. The catalog externalizes practitioner judgment into an executable, sharable, permanent artifact. The model grows monotonically. It never forgets.

The Unix pipeline. Small tools, composed through a universal interface, each doing one thing well, each independently verifiable. The pipe character is the adoption path.

Kelsey arrived at these principles from system administration to Docker to Kubernetes. I arrived from the security verification side, from manual audits to formal specifications to compound chain detection. Different starting points, domains and problems. Same destination.

The destination is the substrate: a formal, reusable, composable artifact that makes the manual process structurally unnecessary. For infrastructure, it's the container image and the pod spec. For security verification, it's the control catalog and the observation schema. For both, the principle is the same: solve it once, compile it into the library, and never solve that problem again.

Stave is an open-source AWS configuration verifier. iam-explain is a single-policy IAM analyzer with Z3 formal verification. The pipe between them is the adoption path.