✓ Human-authored analysis; AI used for formatting and proofreading.
A security group rule allowing inbound 0.0.0.0/0 on port 3389 is a fact about the configuration. Nobody connecting via RDP is a fact about the runtime. Both are true simultaneously. They answer different questions.
Configuration analysis asks: what does the infrastructure PERMIT?
Runtime analysis asks: what is the infrastructure DOING?
The first question has a deterministic answer. The same configuration, evaluated against the same specification, produces the same findings on any machine, at any time, by any operator. The answer is a logical consequence of the state. It can't vary because the inputs don't vary.
The second question has a probabilistic answer. "Is this connection anomalous?" depends on a baseline. "Is this API call malicious?" depends on context. "Is this traffic pattern an attack?" depends on what normal looks like, which changes over time, across environments, and between observers.
Most security tools blur this boundary. They mix deterministic configuration checks with probabilistic runtime assessments in the same dashboard, severity scale and the triage queue. This results in configuration findings that are always true being treated with the same skepticism as runtime alerts that are sometimes wrong. The operator can't tell which findings are facts about the state and which are guesses about the behavior.
Understanding where determinism ends is the first step to using the right tool for each question.
What determinism means here
A deterministic evaluation has three properties:
Reproducibility. The same input produces the same output. Run the evaluation on a Monday. Run it again on a Friday. Same snapshot, same controls, same findings. Share the snapshot with a colleague. They run the evaluation on their laptop. Same findings. Share it with an auditor. Same findings. The evaluation is a function of the input, not of the environment, the time, or the observer.
Completeness. The evaluation answers every question it claims to answer. If the specification says "every S3 bucket must have encryption enabled," the evaluation checks every S3 bucket and reports every one that doesn't. It doesn't sample. It doesn't estimate. It doesn't skip resources that are probably fine.
Explainability. Every finding carries a proof: which resource, which property, which value, which specification it violates. The operator can verify the finding by inspecting the resource and the specification. The finding isn't a score or a probability, it's a logical derivation from observable facts.
Configuration analysis has all three. Runtime analysis has none by nature, not by weakness.
The configuration side: what determinism can answer
These questions have deterministic answers from a configuration snapshot:
Is this resource misconfigured? A security group allowing 0.0.0.0/0 on port 3389 either exists or doesn't. The answer is binary. The evaluation is a comparison between the observed state and the specification.
Can this principal reach this resource? A role assumption chain from role A through role B to S3 bucket C either exists in the IAM configuration or doesn't. The answer is the transitive closure of the permission graph. A mathematical computation with a definitive result.
Do these misconfigurations compose into a known attack path? The Capital One pattern: IMDSv1 enabled AND public IP AND overpermissioned role is a conjunction of three boolean properties. All three are true simultaneously on this instance, or they're not. The compound evaluation is a logical AND over deterministic inputs.
How close is this configuration to a named breach pattern? If two of three controls in a chain are failing and one is passing, the configuration is distance-one from the pattern. The distance is a count of failing members versus total members. It is an integer derived from the evaluation.
What changed between two configuration states? Given snapshot A and snapshot B, the diff is deterministic: these resources were added, these were removed, these properties changed. The risk direction of each change (IMDSv2 enforced → IMDSv2 optional is risk-increasing) is a comparison between the old value, the new value, and the specification.
Every answer above is a function of the configuration state. The state is observable. The specification is declared. The evaluation is a logical derivation. It can't be wrong in the way a probabilistic estimate can be wrong. It can only be wrong if the specification is wrong or the observation is incomplete. Both of those are verifiable.
The runtime side: what determinism can't answer
These questions cannot be answered from a configuration snapshot:
Is anyone exploiting this misconfiguration? The security group allows 0.0.0.0/0 on port 3389. Is anyone connecting? Configuration analysis can't tell. The security group rule is a gate — it permits traffic. Whether traffic flows through the gate is a runtime fact observed in VPC Flow Logs, not in the security group configuration.
Is this API call malicious? A CloudTrail event shows sts:AssumeRole from an IP address in a country where the company has no employees. Is it an attacker, a VPN, or a traveling employee? The answer requires context that no configuration snapshot contains: the employee's travel schedule, the company's VPN exit points, the historical baseline of where this role is typically assumed from.
Is this behavior anomalous? A Lambda function that usually invokes 100 times per hour is invoking 10,000 times per hour. Is it an attack, a traffic spike, or a new feature that went viral? The answer requires a baseline, what "normal" looks like for this function. Baselines are empirical, not specified. They change over time. They differ between environments. They are inherently probabilistic.
Is this user compromised? An IAM user who typically accesses 3 S3 buckets is suddenly listing all buckets in the account and downloading objects from 15 of them. Is the user compromised, running an audit, or doing a data migration? The answer requires understanding intent, which no technical system can determine from telemetry alone.
Each of these questions requires runtime data (logs, metrics, flow records), empirical baselines (what "normal" looks like), and probabilistic judgment (is this deviation significant enough to alert?). The answers are uncertain. A well-tuned GuardDuty deployment with good baselines produces valuable alerts. But the alerts are estimates, not proofs. They can be false positives (the VPN exit point was unusual but legitimate) or false negatives (the attacker's behavior was within the baseline).
The boundary is precise
The boundary between deterministic and probabilistic follows a clear line:
Configuration analysis answers: "Is this state permitted by the specification?" The answer is always deterministic because both the state and the specification are observable, static, and complete.
Runtime analysis answers: "Is this behavior consistent with normal?" The answer is always probabilistic because "normal" is empirical, dynamic, and incomplete.
The confusion happens when tools mix both in one output. A dashboard that shows "S3 bucket is public (severity: HIGH)" next to "Unusual API activity from IAM user (severity: HIGH)" treats a deterministic fact and a probabilistic estimate as equivalent. They're not. The public bucket is public. That finding is true until someone changes the configuration. The unusual API activity might be malicious. That finding is true until the baseline shifts or more context arrives.
An operator who treats both findings with the same confidence either over-trusts the runtime alert (acts on every anomaly as if it's an attack) or under-trusts the configuration finding (treats a verified misconfiguration as "probably fine, like the last three false positive alerts"). The blurred boundary degrades the operator's judgment in both directions.
What each side does better
Configuration analysis is better for:
Prevention. A control that fires before the misconfiguration reaches production prevents the incident. A runtime alert fires after the behavior occurs. The misconfiguration already exists, the traffic is already flowing, the data may already be accessed. Configuration analysis operates on the left side of the timeline. Runtime analysis operates on the right.
Proof. An auditor asking "can you prove your S3 buckets aren't public?" gets a deterministic answer from configuration analysis: here's the snapshot, here are the controls, here are the findings, run the evaluation yourself. Runtime analysis can't prove a negative — "no unusual access was detected" doesn't mean no unauthorized access occurred. It means no access triggered an alert.
Compound risk. The Capital One pattern requires evaluating three configuration properties simultaneously on the same instance. Configuration analysis composes naturally. The compound is a conjunction of properties in the snapshot. Runtime analysis would need to correlate three separate log streams (network flow, IMDS access, IAM activity) across time windows to detect the same compound, which is orders of magnitude harder.
Reproducibility. Two operators evaluating the same snapshot get the same findings. Two SOC analysts reviewing the same SIEM alerts may reach different conclusions, because the assessment requires judgment about baselines, context, and severity thresholds.
Runtime analysis is better for:
Detection of active exploitation. Configuration analysis says "this instance COULD be exploited." Runtime analysis says "this instance IS being exploited." The first is a risk assessment. The second is an incident. When the incident is happening, runtime wins.
Intent determination. A role with broad permissions is a configuration finding. The same role being used at 3am from a new IP to enumerate secrets is a runtime finding that adds intent signal. Configuration can't distinguish an overpermissioned role used legitimately from one used maliciously because both have the same configuration.
Behavioral patterns over time. A user who gradually escalates their access over weeks, requesting slightly broader permissions each time has no single configuration violation. Each permission request is individually reasonable. The PATTERN is suspicious, but the pattern only appears in the log stream over time, not in any single snapshot.
Zero-day and novel attacks. An attacker using a technique that no specification anticipated produces no configuration finding. The configuration permits the behavior because nobody specified that it shouldn't. Runtime analysis may still detect it. The behavior is anomalous even if no specification forbids it.
The complementary architecture
The right architecture uses both and keeps them separate.
Configuration analysis runs periodically (hourly, daily, on every deployment). It evaluates the current state against the specification and produces deterministic findings. The findings are facts about the configuration. They don't expire, shift, or depend on baselines. They're true until the configuration changes.
Runtime analysis runs continuously. It evaluates the event stream against empirical baselines and produces probabilistic alerts. The alerts are estimates about the behavior. They can be false positives. They require human judgment to triage. They're context-dependent.
The two systems connect at one point: the configuration finding contextualizes the runtime alert. When GuardDuty alerts on unusual API activity from a role, the configuration analysis tells you whether that role is overpermissioned (blast radius if compromised), whether it participates in a compound attack path (distance from a named breach pattern), and whether it has compensating controls (MFA required, source IP restricted). The runtime alert says "something is happening." The configuration analysis says "here's how bad it can get."
This connection is one-directional. The configuration finding informs the runtime triage. The runtime alert does NOT change the configuration finding. A public S3 bucket is a configuration finding whether or not anyone is accessing it. An overpermissioned role is a configuration finding whether or not it's being abused. The finding is a fact about the state, not an inference about the behavior.
Where most tools go wrong
Most cloud security platforms combine configuration and runtime in one product. The marketing says "unified visibility." The reality is a blurred boundary that weakens both:
Configuration findings inherit the skepticism of runtime alerts. The operator who has been burned by false positive anomaly detections treats configuration findings with the same suspicion — "this is probably another false alarm." But configuration findings aren't probabilistic. The bucket IS public. The role DOES have AdministratorAccess. These aren't guesses. Mixing them with guesses teaches the operator to treat facts as guesses.
Runtime alerts inherit the resolution process of configuration findings. A configuration finding is fixed by changing the configuration such as a Terraform apply, a console click, an API call. A runtime alert is investigated where logs are reviewed, context is gathered, a judgment is made. When both appear in the same queue with the same severity labels and the same SLA, the operator applies the wrong resolution process to the wrong finding type.
Compound analysis is impossible across the boundary. A compound configuration finding (three controls failing simultaneously) is a deterministic conjunction. The compound is true because all members are true. A compound that mixes configuration and runtime ("this bucket is public AND someone accessed it yesterday") requires correlating a deterministic state with a probabilistic event. The correlation is useful but it's not deterministic. The access event may be legitimate, and the compound inherits the uncertainty of its weakest member.
The design principle
Separate the deterministic from the probabilistic. Evaluate configurations against specifications and produce findings that are facts. Monitor runtime against baselines and produce alerts that are estimates. Connect them at the triage layer. The configuration finding contextualizes the runtime alert. But don't merge them into one output with one severity scale and one triage queue.
The operator needs to know which findings are always true (configuration) and which findings are sometimes true (runtime). The boundary between them is precise. Blurring it helps neither.
Configuration tells you what your infrastructure permits. Runtime tells you what your infrastructure is doing. Determinism answers the first. Probabilism answers the second. Using the wrong method for the wrong question produces the worst of both: unreliable answers about configuration (undermining trust) and over-certain answers about behavior (undermining judgment).
The specification defines what "correct" means. The evaluation verifies whether the state matches. That's where determinism lives and where it should stay. Everything past the boundary is monitoring, detection, and response. Different tools. Different methods. Different epistemology.
Know the boundary. Use the right tool on each side.
Top comments (0)