DEV Community: Bala Paranj

The Agentic AI Landscape: Workarounds All the Way Down

Bala Paranj — Tue, 27 Oct 2026 13:38:20 +0000

The landscape

In the span of eighteen months, AI-assisted development went from "autocomplete for code" to "agents that build entire applications while you sleep." The architecture underneath is not complete.

The industry converged on a dominant frame: Agent = Model + Harness. The model generates. Everything else such as filesystem, sandbox, memory, context management, tools, skills, prompt engineering, loop detection is the harness. The harness serves the model. The model is the system.

Around this frame, a landscape of approaches emerged. Each one addresses a real problem. Each one works, partially, temporarily. And each one is a workaround for an architectural element nobody knows is missing because the frame has no vocabulary for elements that don't serve the model.

Here's the map.

Agent Skills

Encoded domain expertise that tells the model which tools to use, how to interpret results, and what reasoning patterns to apply. Meta's Capacity Efficiency team built the most sophisticated version: standardized tool interfaces composed with skills that capture senior engineers' investigation patterns. Same tools power offense (finding optimizations) and defense (catching regressions).

The problem they solve. Models are general-purpose. They don't know that serialization regressions should be investigated by checking recent schema changes, or that endpoint latency regressions should start with GraphQL endpoint profiling. Skills encode "how to look" so the model doesn't waste tokens discovering investigation patterns from scratch every time.

Why they fall short. Skills encode procedural knowledge of how to investigate after something went wrong. They don't encode declarative knowledge of what must be true regardless of whether anything went wrong. "Consult the top GraphQL endpoints for latency regressions" is a recipe. "Serialization functions must not exceed N milliseconds" is a rule the machine enforces before the regression exists. Skills make the model a better investigator. They don't prevent the problems the model investigates.

They're a workaround for. the missing Engine. It is declared intent in the form of specifications the system checks mechanically. Without specifications, you encode expertise as procedures. With specifications, the expertise becomes invariants the machine enforces.

Superpowers and Context Engineering

Techniques for managing the model's context window to maximize output quality. Includes: static per-repo onboarding docs (AGENTS.md, CLAUDE.md), on-demand compressed context, sub-agents that take vertical slices and build research documents, compaction strategies that summarize and restart in fresh windows. Dex's RPI protocol (Research → Plan → Implement) is the most disciplined version — staying in the "smart zone" of the context window by intentionally compacting between phases.

The problem they solve. Models degrade past roughly 40% of the context window. Too many tools, too much injected context, too much conversation history and the quality drops. Context engineering is the discipline of putting the right tokens in the window so the model produces better output.

Why they fall short. Context engineering optimizes the input to the model. It doesn't verify the output. A model with perfect context can still generate code that violates architectural boundaries, breaks implicit invariants, or produces output that's locally correct but globally inconsistent. Better input reduces the probability of wrong output. It doesn't verify the output is right. The distinction between "probably right" and "verified correct" is the distinction the approach can't cross.

They're a workaround for. the missing Casing. It is enforced module boundaries and information hiding. If the architecture had mechanical boundaries (Parnas-style modules, hexagonal architecture enforced at build time), agents wouldn't need context engineering to avoid crossing boundaries they can't see. The boundaries would be enforced by the architecture, not by carefully curating the context window.

Self-Verification and Reviewer Agents

The model reviews its own output, or a second model reviews the first. LangChain calls models "exceptional self-improvement machines" and lists self-verification as their top performance improvement. Anthropic's three-agent harness uses a separate evaluator agent with Playwright-driven testing. OpenAI runs reviewer agents in CI checking patches against documentation.

The problem they solve. Unreviewed code ships with defects. When models generate at machine speed, human review can't keep pace with review duration up 441%, zero-review merges up 31%. Some automated quality signal is better than none.

Why they fall short. Huang et al. (ICLR 2024) established that models cannot self-correct reasoning without external feedback where performance can degrade after self-correction. The same blind spots that produced the error evaluate the error as correct. Anthropic acknowledged, their evaluator is "still an LLM that is inclined to be generous towards LLM-generated outputs" and spent "several rounds" tuning the evaluator prompt to be sufficiently skeptical. A type checker doesn't need tuning to be skeptical. It doesn't form opinions. It passes or fails.

They're a workaround for. the missing Control Unit. It is an independent oracle that measures output against declared properties and issues a deterministic verdict. Self-verification substitutes a model's opinion for a mechanical check. The workaround exists because the element (independent, deterministic verification against declared intent) is absent from the architecture.

Loop Engineering and the Ralph Loop

Patterns that keep the model generating over long horizons. The Ralph Loop intercepts the model's attempt to stop, reinjects the prompt in a fresh context, and forces continued generation until a completion goal is met. Loop engineering automates this: agent writes, judge agent reviews, loop continues. The aspiration is autonomous execution "every time you must type 'continue' is a failure of the harness."

The problem they solve. Complex tasks exceed a single context window. The model stops before the work is done. Without a continuation mechanism, the human manually restarts, re-explains context, and loses progress.

Why they fall short. The loop optimizes duration of generation. The problem being solved is "the agent stopped too early." The solution is to force it to keep going. What's not being checked: whether what was generated in the last iteration was correct before starting the next one. If iteration three introduced a subtle architectural mistake, iterations four through twenty build on that mistake. A long-horizon execution loop without an independent verification gate between iterations is a mechanism for generating more wrong output, faster, with more confidence.

They're a workaround for. the missing Engine (no specification to converge toward) and the missing Control Unit (no gate between iterations). With a specification, generation converges toward a declared target instead of running indefinitely. With a verification gate, each iteration is checked before the next one builds on it. Without both, the loop is "generate more and hope it improves."

Prompt Optimization and Agent Memory

Improving model output by improving the prompt: system prompts, few-shot examples, chain-of-thought templates, memory files that persist across sessions, AGENTS.md files that encode project-specific conventions. The goal is to give the model enough context to produce better output without human intervention.

The problem they solve. Models are stateless. Without memory and carefully constructed prompts, each interaction starts from zero. Good prompts produce better output. Project-specific memory reduces the learning curve per session.

Why they fall short. A better prompt produces a better guess. It doesn't produce a verified output. "Generate code that follows our coding conventions" is a prompt. "All public functions must have error handling and all errors must be wrapped with context" is a linter rule. The prompt requests. The rule enforces. The prompt might be followed. The rule is always checked.

They're a workaround for. the missing Engine again. Prompts and memory files are proto-specifications. They have intent in them, but they're ephemeral, ambiguous, and model-interpreted. A specification is durable, precise, versioned, and mechanically checkable. Every prompt-engineering improvement is a better version of a thing that shouldn't be a prompt. It should be a declaration machines enforce.

Friday Slop Cleanup

OpenAI's Ryan Lopopolo described running a billion tokens a day through a fully agent-generated codebase and spending every Friday with 20% of engineering time is spent in manually cleaning up accumulated "AI slop." The team later tried to automate this cleanup with more agents.

The problem it solves. Agent-generated code accumulates. Not all of it belongs. Duplication, unnecessary abstractions, dead code, stylistic inconsistency. The codebase grows without any force pushing the other direction. Someone has to clean it up.

Why it falls short. Manual subtraction one day a week is a human performing a function the architecture should perform continuously. The slop was generated on Monday through Thursday. It accumulated all week. On Friday, a human manually reverses the accumulation. Next week, it accumulates again. The cleanup doesn't prevent the accumulation. It reacts to it.

It's a workaround for. the missing Casing's subtraction discipline. If the architecture included a force that asks "does this belong?" on every generation. Measured against a declared essence of the product, the slop wouldn't accumulate. The subtraction would happen continuously, at generation time, not manually, once a week, after the damage is done.

The pattern underneath

Every approach in the current landscape has the same structure:

Approach	Real problem	Function it performs manually	Missing element
Agent Skills	Models lack domain expertise	Procedural investigation	Engine (declared invariants)
Context Engineering	Model degrades with bad context	Manual boundary management	Casing (enforced boundaries)
Self-Verification	Unreviewed code ships defects	Quality opinion from a model	Control Unit (deterministic oracle)
Loop Engineering	Tasks exceed one context window	Duration-based continuation	Engine + Control (spec + gate)
Prompt Optimization	Stateless models need context	Intent injection per session	Engine (durable specification)
Friday Slop Cleanup	Code accumulates without curation	Manual weekly subtraction	Casing (subtraction discipline)

The pattern: a human or a model manually performs a function that a missing architectural element would perform mechanically. Each workaround works partially, temporarily, expensively. Each one will be replaced by the element it's substituting for.

Why the workarounds don't survive

Three forces are converging on the workarounds simultaneously:

They don't scale with generation speed. Workarounds that depend on human intervention (Friday cleanup, prompt tuning, manual context management, human review of agent output) hit the same wall the Osmani data documents: 4x code for 10% more value, reviews up 441%, 31% merged unread. Generation runs at machine speed. Workarounds run at human speed. The gap widens with every model improvement.

They cost more as models improve. A more powerful model generates more output per token. More output means more to review, more to clean up, more to verify, more context to manage. Every workaround's cost scales with the model's capability. Agentic bloat of ten thousand lines to solve a ten-line problem is a direct consequence of a powerful tool with no subtraction discipline. The inference bill is the monthly invoice for missing elements.

They have a proven mathematical ceiling. Vassilev's NIST proof (June 2026) extended Gödel's incompleteness theorems to AI systems: no finite set of guardrails is universally robust. Self-verification, prompt engineering, reviewer agents. Every approach that wraps the model in a finite set of rules has a formal ceiling. The ceiling can be raised. It cannot be removed. The workarounds are not converging toward a complete solution. They are asymptotically approaching a limit.

What aerospace figured out

NASA's Space Shuttle software flew for 30 years with a near-zero defect rate. Because the architecture was complete. Every function had the right element at the right level:

Assertions at code level. Mechanical checks that trigger System Exceptions on failure.
Safety Properties at design level. "Shall not" and "must always" rules enforced as invariants.
Interface Specifications between modules. ICDs with self-checking logic that rejects violations.
Invariants at system level. Boundary conditions that trigger immediate response on breach.

No workarounds. No Friday cleanup. No prompt tuning. No "several rounds" of calibrating the evaluator to stop praising mediocre work. Each element was mechanical, deterministic, and enforced. The workarounds were unnecessary because the elements were present.

The current agentic AI landscape is a collection of brilliant engineering teams building elaborate workarounds for elements that aerospace identified and supplied decades ago. The workarounds will be replaced. The subject of this book is what replaces them, why it works, and how to build it before the workarounds' costs exceed the value they protect.

This chapter surveys the landscape as of June 2026. The approaches described are drawn from published sources: LangChain, OpenAI, Meta, Anthropic. The five-element framework referenced throughout: Altshuller's Law of System Completeness (TRIZ), independently confirmed by Beer's Viable System Model and Nalebuff/Brandenburger's Co-opetition.

Generation is Not Solved: What Google's New SDLC Paper Gets Wrong About AI-Assisted Development

Bala Paranj — Mon, 26 Oct 2026 12:47:25 +0000

Google published The New SDLC With Vibe Coding in May 2026, a 51-page paper positioning agentic engineering as the mature successor to ad-hoc AI-assisted development.

It is full of errors. Claims that sound reasonable until you hold them against the formal traditions that govern whether software works.

This article examines twelve specific misconceptions in the paper and offers the grounded alternative for each. The places where it's wrong are the places where teams will make expensive decisions.

Generation is solved.

The paper's closing line: "Generation is solved. Verification, judgment, and direction are the new craft."

Generating code that compiles is solved. Generating code that passes a test suite is mostly solved. Generating code that is correct meaning it satisfies a complete specification of all required behaviors, including behaviors the test suite doesn't cover is not solved. Conflating these three things is a category error.

Dijkstra spent a career distinguishing between "testing shows the presence of bugs, never the absence of bugs." That distinction is the central problem of software verification. It applies with greater force to AI-generated code because AI models produce plausible-looking output that passes shallow checks while harboring semantic errors that only manifest at system boundaries.

The correction: Generation is automated. It is not solved. Production outages, security vulnerabilities, and data corruption live in the gap between "compiles and passes tests" and "correct under all specified conditions". Any engineering process that treats generation as solved will systematically undercount that gap.

Context engineering is a new discipline.

The paper presents context engineering as deciding what information an AI agent needs to do its job as an emergent practice driven by AI tooling. It identifies six types of context: instructions, knowledge, memory, examples, tools, and guardrails.

This is Parnas (1972) in a new hat.

Parnas's "On the Criteria To Be Used in Decomposing Systems into Modules" is about one thing: what information does a module need, and what information should be hidden from it? His information hiding principle is the design decision is choosing what each module doesn't know. It maps directly onto the paper's static/dynamic context split. The six context types the paper lists are preconditions, postconditions, invariants, and interface contracts, reorganized into a taxonomy that obscures their origin.

The paper even rediscovers the core trade-off Parnas identified: "Too much static context wastes tokens and dilutes signals. Too little means the agent forgets critical rules." Substitute "too much coupling increases complexity" and "too little information causes incorrect behavior" and you have the original formulation.

The correction: Context engineering is specification engineering applied to AI agents. The medium changed (natural language prompts instead of formal interface definitions), but the principles are identical. Teams that treat context engineering as a novel skill without studying its theoretical foundations will rediscover every mistake the specification community already catalogued.

3. Tests + evals is a sufficient verification model.

The paper's verification rests on two mechanisms: tests verify deterministic properties, evals (scored by datasets, rubrics, and LM judges) verify non-deterministic properties.

Absent from the paper's framework:

Formal verification — proving that a property holds for all possible inputs, not just sampled ones
Model checking — exhaustive exploration of state spaces to find violations of temporal properties
Static analysis — detecting classes of bugs (null dereference, integer overflow, use-after-free) without executing code
Type systems as proof systems — using the compiler to enforce invariants at the language level
Property-based testing — generating random inputs guided by algebraic properties, mentioned once in passing and never integrated into the framework
Abstract interpretation — computing sound approximations of program behavior

The paper claims the difference between vibe coding and agentic engineering is "how outputs get verified." But its own verification toolkit can only sample behaviors. It cannot prove properties. A system verified only by tests and evals has no guarantees about behaviors outside the test distribution. This is where the catastrophic failures live.

The correction: A production verification framework needs at least three layers: deterministic tests for known behaviors, property-based tests for algebraic invariants, and static analysis for structural properties. For security-critical paths, formal verification of specific properties (e.g., "this function never reads outside the bounds of its input") is not optional. Evals scored by LM judges add a fourth layer for subjective quality, but they are the weakest layer, not a replacement for the first three.

The Factory Model is the right mental model.

The paper proposes the "factory model" as its central organizing metaphor: the developer's primary output is not code but "the system that produces code" such as specifications, agents, tests, feedback loops, and guardrails.

This metaphor has been critiqued and rejected for 40 years.

Brooks's No Silver Bullet (1986) argues that software development is different from manufacturing because each unit of software is unique. Manufacturing produces identical copies of a designed artifact. Software development is the design process. There is no separate production step. The factory metaphor misleads because it implies that once the assembly line is configured correctly, output is reliable and repeatable. Software does not work this way. Each prompt produces different output. Each context window yields different behavior. The factory has no quality floor.

What the paper describes, specifying properties, constraining agent behavior, verifying outputs against specifications is closer to Lamport's framework for specifying concurrent systems. In Lamport's model, you specify what the system should do (the safety and liveness properties), not how to build it. The specification is the primary artifact. Implementation is a refinement of the specification. Verification proves the refinement preserves the properties.

The correction: Replace the factory metaphor with the specification-refinement model. The developer's primary output is the specification. The precise description of what the system must do and must not do. AI agents perform refinement (translating specification into implementation). Verification proves the refinement is correct. This model is rigorous, has formal foundations, and doesn't mislead teams into thinking they're running a production line.

The model is not the engine.

The paper's equation is "Agent = Model + Harness." The harness is "the car, the road, and the traffic laws." The model is "the engine." Figure 7 labels the model at ~10% and the harness at ~90%.

The metaphor is wrong, and it leads to the wrong investments.

An engine has a defining property: it converts energy into reliable, predictable work. A combustion engine produces torque proportional to fuel input. You can characterize its output curve, its failure modes, its efficiency envelope. It degrades gracefully under load. It does the same thing every time you turn the key.

An LLM does none of this. Given identical inputs, it produces different outputs (temperature > 0). Its failure modes are not characterizable in advance. You cannot predict which prompts will produce hallucinations. It does not degrade gracefully; it fails silently by producing plausible-looking but incorrect output. Calling it an engine smuggles in connotations of reliability and predictability that the component does not possess.

In TRIZ's Law of System Completeness, a framework for analyzing whether a technical system has all necessary components. A complete system requires an Engine, Transmission, Tool, Control Unit, and Energy Source. Applied to an AI agent system, the components map differently than the paper assumes. The Tool is the model. It's the thing that acts on the workpiece (the codebase). The Engine is the element that drives the system forward is the specification-verification-learning loop. The Transmission is the harness, which translates the engine's output into the tool's motion. The Control Unit is the human exercising judgment over whether the system's output satisfies the specification.

This distinction matters. If the model is the engine, you improve the system by upgrading the model. If the model is the tool, you improve the system by improving the loop with tighter specifications, faster verification, better learning from failures. The paper's own Terminal Bench data (harness-only changes moved an agent from outside Top 30 to Top 5) proves the tool framing. They cited the evidence for the right conclusion and drew the wrong one.

The correction: The model is a stochastic tool. The engine of an AI-assisted development system is the specification-verification-learning loop: specify what correct means, generate a candidate, verify it against the specification, learn from the failures, revise the specification. Every investment decision should optimize the loop, not the tool. A better model inside a broken loop produces faster garbage. A weaker model inside a tight loop converges on correct output.

6. The new SDLC is waterfall in disguise. Learning is absent.

The paper's AI-driven SDLC (Figure 5) prescribes: Requirements → Design → Implementation (agents) → Testing/Eval → Review & Deploy → Maintenance. The recommendations reinforce the sequence: write specs and architecture docs upfront, configure the harness, define eval criteria, then let agents run.

This is waterfall with a faster implementation phase.

The point of the Agile movement, whatever you think of its current institutional form was one empirical insight: you cannot fully specify a system before building it. Requirements emerge from building. The user doesn't know what they want until they see what they don't want. The developer doesn't understand the problem until they've tried to solve it and failed.

The paper eliminates the primary mechanism by which this learning happens.

In traditional development, the developer writes code and learns from the act of writing. The resistance of the material teaches them about the problem's structure. They discover edge cases by hitting them. They revise their mental model of the requirements in real time. The feedback loop between "what I think the system should do" and "what the system does" is tight and continuous.

In the paper's orchestrator model explicitly positioned as the more advanced mode, the developer writes a specification, hands it to agents, and reviews output asynchronously. The human is removed from the implementation loop. But most learning about requirements happens during implementation. When the agent runs autonomously in the background and produces a pull request, the human reviews the output but never observes the intermediate states that would have taught them about the problem.

The paper even identifies "specification quality is the new bottleneck" without recognizing the contradiction: if writing good specs requires understanding the problem deeply, and understanding the problem deeply requires building and failing and learning, then removing humans from the building phase degrades the specs that the paper positions as the foundation of everything.

This shows up in two failure modes:

First, spec drift without revision. The paper recommends writing architecture docs and AGENTS.md files upfront, then running agents against them. But specifications written before implementation reflect pre-implementation understanding. If the human never touches the implementation, they never learn what's wrong with their specification. The spec becomes a stale document that agents faithfully implement, correctly satisfying the wrong requirements.

Second, the METR tax compounds. The paper's own data shows experienced developers take 19% longer with AI assistance. Part of that tax is the cost of understanding code you didn't write. But part of it is also the cost of discovering, at review time, requirements you didn't know you had. Things you would have discovered earlier if you'd been in the implementation loop. Deferring that discovery to review time is more expensive than discovering it during implementation, for the same reason that bugs found in production cost more to fix than bugs found during development.

The correction: The SDLC needs a learning loop, not just a feedback loop. The paper's feedback loop routes test failures back to agents for correction. The agent learns and retries. But the human doesn't learn. A learning loop routes surprises such as unexpected behaviors, discovered requirements, spec violations back to the human to revise the specification. This means: build small, verify early, revise the spec based on what you learned, then build more. It means the specification is a living document that co-evolves with the implementation, not an upfront artifact that agents execute against. It means the human must remain in the loop during implementation, not just before and after.

The conductor mode (real-time, in the IDE) preserves this learning loop. The orchestrator mode (async, background agents) breaks it. The paper positions orchestrator mode as the more advanced practice. It's the more dangerous one. It's only safe for tasks where the specification is already known to be complete which is a small subset of real work.

7. The 80% problem is a workflow problem.

The paper observes that AI agents can generate approximately 80% of a feature's code, but the remaining 20% such as edge cases, error handling, integration points requires deep contextual knowledge. The paper treats this as an empirical observation about current model limitations and prescribes focusing human attention on "what AI struggles with."

The diagnosis is wrong. This is not a workflow problem. It's a specification completeness problem.

The gap between the 80% and the 100% is not between "easy code" and "hard code." It is between what is specified and what is assumed. Parnas called these the "unstated requirements" — the behaviors the system must exhibit that nobody wrote down because they seemed obvious to the humans who understood the domain.

AI models cannot infer unstated requirements. They generate code that satisfies the stated specification (the prompt) and fills in the rest with statistical patterns from training data. The 20% gap is the set of requirements that exist in the developer's head but not in the prompt.

The correction: The 80% problem is a specification problem. The fix is not "focus human attention on the hard parts." The fix is: make the specification complete enough that correctness is mechanically verifiable. This means writing preconditions, postconditions, and invariants for every interface — not as documentation, but as executable checks. Property-based testing (e.g., with Hypothesis, Rapid, or gopter) is the practical tool for discovering unstated requirements before they become production bugs.

8. Intent as the new Interface is pure progress.

The paper's thesis — developers now express what they want rather than how to build it is presented as an unambiguous advance. The conclusion is titled "Intent as the new Interface."

This framing ignores a critical regression: natural language is a lossy and ambiguous specification medium.

TLA+, Alloy, Z, VDM, and even CEL predicates have been "intent as interface" for decades. They let you say what the system should do without saying how. The difference is that these languages are formal — they have precise semantics, they're mechanically checkable, and they eliminate ambiguity by construction.

Natural language does none of this. "Build a secure API" is ambiguous in ways that a TLA+ specification of the API's safety properties is not. The move from formal specifications to natural language prompts trades precision for accessibility. That trade-off may be worth making for some use cases, but presenting it as pure progress without acknowledging the precision cost is misleading.

The correction: Natural language intent is a useful starting point, not a finished specification. A disciplined workflow starts with natural language intent, refines it into testable properties (property-based test generators, CEL predicates, formal invariants), and uses the testable properties as the contract with the AI agent. The natural language prompt is the sketch. The executable specification is the blueprint.

9. The METR finding is a footnote, not the headline.

The paper mentions in one paragraph that a METR study found experienced developers took 19% longer when using AI assistants on certain tasks, primarily because of time spent verifying, debugging, and correcting AI output. The paper then moves on.

This is the most important data point in the entire paper, and it's treated as a caveat.

For experienced practitioners working on complex, unfamiliar, or correctness-critical tasks, current AI tooling may impose a net tax rather than providing a net benefit. The verification burden — reading AI-generated code you didn't write, understanding its assumptions, checking its correctness can exceed the time saved by not writing the code yourself.

This finding is consistent with what the theoretical literature predicts: the cost of understanding a system scales with the system's complexity, not with who wrote it. AI doesn't reduce the complexity of the software; it shifts the work from writing to reading. For systems where understanding is the bottleneck (legacy codebases, security-critical paths, concurrent systems), this shift may not be favorable.

The correction: Teams should measure total task completion time including verification, not just code generation time. For correctness-critical work, the right metric is "time to verified-correct implementation," not "time to first output." The 25-39% productivity gains cited in the paper likely measure the wrong thing — they measure generation speed, not delivery speed. Until the industry converges on measuring delivery of verified-correct code, productivity claims from AI tooling are suspect.

10. The harness model (Agent = Model + Harness) lacks a correctness owner.

The paper defines "Agent = Model + Harness" and describes harness components: instructions, tools, sandboxes, orchestration, guardrails, observability. It claims that ~90% of agent behavior is determined by the harness, not the model.

The claim may be directionally correct, but the model is missing a critical component: who owns correctness?

The conductor/orchestrator framework defines developer roles around workflow mode — real-time direction versus async delegation. Neither role, as defined, owns the system's properties. In Lamport's framework, someone must own:

The safety properties — "the system never does X" (never exposes credentials, never processes a malformed input, never violates an invariant)
The liveness properties — "the system eventually does Y" (every request gets a response, every failure triggers recovery, every finding gets reported)

Neither conductor (focused on keystroke-level direction) nor orchestrator (focused on task delegation) is responsible for specifying, verifying, and maintaining these properties. The result is that system invariants become emergent properties of the agent's behavior rather than designed properties that the harness enforces.

The correction: Add a third role: the specifier. The specifier owns the system's safety and liveness properties. They write the invariants that the harness must enforce and the properties that verification must prove. The specifier reviews not individual code changes but the system's property set: "After this change, do all invariants still hold?" This role maps to what Stave calls the Judge. The component that evaluates whether the system's output satisfies the specification, independent of how the output was produced.

11. Compound failure modes are absent.

The paper discusses each SDLC phase independently and evaluates AI output per-task. Its verification framework (test individual outputs, eval individual trajectories) operates at the single-agent, single-task level.

Production systems fail at boundaries. The most dangerous bugs are in the interactions between components where individually-correct pieces compose into incorrect systems.

This is the graph problem. A function that correctly validates input and a function that correctly processes validated input can still compose into a system that processes unvalidated input, if the edge between them (the call site, the data flow, the trust boundary) is missing or incorrect. AI agents generating code module-by-module have no mechanism to verify that the edges compose correctly.

The paper never addresses compositional correctness. There is no discussion of interface contracts between AI-generated modules, no discussion of how to verify that independently-generated components compose into a system that satisfies global properties, and no discussion of the well-known theoretical results (Hoare logic, rely-guarantee reasoning, separation logic) that govern compositional verification.

The correction: Verification must operate at three levels: unit (does this function satisfy its contract?), compositional (do these modules compose correctly across their interfaces?), and system (does the assembled system satisfy its global properties?). AI-generated code requires more attention to compositional verification, because the agent generating module A has no awareness of the assumptions module B makes about A's behavior. Architectural enforcement tools (dependency guards, interface lint, contract-based design) are not optional when AI is generating the components.

12. The economics model ignores escape cost.

The paper's CapEx/OpEx analysis compares vibe coding (low CapEx, high OpEx from token burn and maintenance) with agentic engineering (high CapEx from upfront specification, low OpEx from structured output). The crossover point where agentic engineering becomes cheaper is presented as the key decision point.

The model is incomplete because it doesn't account for escape cost. The cost of bugs that pass both tests and evals and reach production.

For security-critical systems (cloud infrastructure, financial services, healthcare, compliance), a single undetected vulnerability in AI-generated code can exceed the entire cost savings from AI-assisted development. The TCO model needs a third term: risk-adjusted cost of escapes, weighted by the probability that AI-generated code contains defects that evade the verification framework, multiplied by the cost of those defects in production.

The paper's verification framework (tests + evals) has a known escape rate. No test suite has 100% fault detection. AI-generated code may have a higher escape rate than human-written code because AI errors are more likely to be semantically plausible (they look right) and less likely to trigger the pattern-matching heuristics that experienced reviewers use to catch bugs.

The correction: Add escape cost to the TCO model: TCO = CapEx + OpEx + (P(escape) × Cost(escape)). For systems where Cost(escape) is high (security, compliance, financial), the investment required to drive P(escape) acceptably low may dominate both CapEx and OpEx. This is where formal verification, static analysis, and defense-in-depth verification strategies pay for themselves by reducing the expected cost of undetected defects.

The Underlying Problem

These twelve issues share a root cause: the paper treats workflow advice as engineering principles without grounding either in the formal traditions that govern whether software works.

The paper never cites Dijkstra, Parnas, Lamport, Hoare, Liskov, or Brooks. It never references TLA+, Alloy, separation logic, or rely-guarantee reasoning. It never distinguishes between testing (sampling behaviors) and verification (proving properties). It never acknowledges that natural language specifications are lossy.

This matters because the paper will be read by engineering leaders making investment decisions. Those decisions such as how much to invest in verification infrastructure, whether to adopt formal specification practices, how to measure AI-assisted development productivity will be worse if the conceptual foundations are wrong.

The tools are changing. The principles are not. AI agents are a new implementation medium, not a new theory of correctness. The teams that build reliable systems with AI assistance will be the ones that apply the existing theory such as specification, verification, compositional reasoning to the new medium.

Generation is automated. Correctness is still the hard problem.

Stave and the AWS Compliance Mod: why we ship both

Bala Paranj — Sun, 25 Oct 2026 11:29:53 +0000

The Capital One incident in 2019 exfiltrated 100 million customer records. The path was specific: an EC2 instance with IMDSv1 enabled, accessible via SSRF from the public-facing application, running with an IAM role that had s3:GetObject on a bucket holding the credit card data. Each individual setting was reviewable; each individual setting could have been "fixed" without anyone realizing the architecture composed into an exfiltration path.

If you had run turbot/steampipe-mod-aws-compliance against that account the morning of the breach, every CIS check would have passed. The IAM user had MFA, the bucket had encryption, the security group had documented inbound rules, the trail was logging. The composition failed. Per-resource scanners check the rows; the breach is in the join.

AWS Compliance Mod job is framework coverage of single-resource properties, and it does that job extremely well. Stave's job is the compositional layer above. Different tool, different question.

What the AWS Compliance Mod is good at

540+ controls across 16+ frameworks. CIS AWS Benchmark v1.4/v3.0/v4.0, PCI-DSS v3.2.1/v4.0, HIPAA Security Rule, NIST 800-53 Rev 5, SOC 2, FedRAMP Moderate, GxP, AWS Foundational Security Best Practices, more. Each control is a SQL check against Steampipe tables that wrap live AWS APIs. Mature ecosystem. Wide adoption.

If your question is "am I CIS-compliant right now?", install it. The framework-cited dashboards render exactly the evidence pack auditors expect. The per-resource property checks are fast, debuggable, deterministic. The contributor velocity is real.

For the atomic-controls slice, "is encryption enabled," "is MFA on," "is logging configured", the AWS Compliance Mod's framework breadth is its strength. Stave's catalog has those controls too (~86% of the Stave catalog is atomic), but Stave doesn't compete on framework coverage of the atomic axis. That territory is theirs.

Where Stave's territory starts

The 14% of Stave's catalog that's compound which is currently 4,755 controls and 770 chains, growing is the part that catches what per-resource scanners can't see by construction. Three classes:

Compound risk chains. A set of controls that, when they co-fail, create more risk than the sum of individual findings. A public-read S3 bucket plus access logging disabled is an exfiltration path with no audit trail. The chain engine fires when the composition's escalation threshold is met; the compound finding carries its own severity that may exceed any member.

Ghost references. Dangling cross-resource pointers such as a Cognito user pool referencing a deleted Lambda trigger, a CloudFront distribution whose S3 origin no longer exists, an ECS task naming a removed IAM role. The reference exists in one asset's configuration; the absence is in another asset's inventory. Per-resource scanners that don't model cross-resource invariants miss every one of these. Stave's catalog has ~99 ghost-reference controls today across APIGateway, CloudFront, Cognito, DynamoDB, ECS, EKS, ELB, EventBridge, Secrets Manager, SNS, SQS.

Snapshot-anchored reproducibility. Re-running the same obs.v0.1 snapshot produces byte-identical output. Two weeks later, a different engineer points the same snapshot at Stave and gets the same verdict. This is useful for audit timelines, regression bisects, "what did we know at time T" questions. Live-API tools can't offer this.

Side-by-side: the same intent, two tools

Both detecting "cross-account role assumption without an external ID."

AWS Compliance Mod — per-resource SQL check

The shape (representative, paraphrased from the live mod's IAM section):

-- One row per IAM role; check the role's own trust policy.
select
  role_arn as resource,
  case
    when trust_requires_external_id then 'ok'
    else 'alarm'
  end as status,
  case
    when trust_requires_external_id then role_name || ' requires ExternalId.'
    else role_name || ' allows cross-account assume without ExternalId.'
  end as reason
from aws_iam_role
where assume_role_policy_document_has_cross_account_principal;

Per-role, status, reason, rendered in a CIS-cited benchmark dashboard. The check is per-role and SQL-shaped; the "is this role cross-account?" question is answered from the role's own trust policy.

Stave — compound control reasoning over an IAM-graph computation

CTL.IAM.IDENTITY.BLASTRADIUS.002 — abridged:

id: CTL.IAM.IDENTITY.BLASTRADIUS.002
name: Cross-Account Role Must Require External ID
severity: critical
scope: compound
corpus_reference: MITRE:T1550
applicable_asset_types: [aws_iam_role]
unsafe_predicate:
  all:
    - field: properties.identity.kind
      op: eq
      value: role
    - field: properties.identity.role.blast_radius_scope
      op: eq
      value: cross_account
    - field: properties.identity.role.cross_account_trust_without_external_id
      op: eq
      value: true

The predicate checks three properties on a single role. But two of those properties (blast_radius_scope and cross_account_trust_without_external_id) are pre-computed by Stave's observation extractor by reasoning across the role's trust policy, attached permissions, and the reachable resources in other accounts. The extractor walks the IAM graph; the predicate evaluates the result.

The control fires when the composition produces the unsafe state. AWS Compliance Mod's per-row SQL shape can't express the composition. The "is the trust missing an external ID?" question is per-role and SQL-shaped; the "does the role have cross-account reach AND does the trust permit assumption from any principal in the trusted account?" question is graph-shaped, computed once during observation extraction, and predicated on at query time.

When to use which

Your question	Use
Am I CIS-compliant right now?	AWS Compliance Mod
Am I PCI/HIPAA/SOC 2/FedRAMP compliant?	AWS Compliance Mod
Do my IAM users have MFA? (per-resource check)	AWS Compliance Mod
Would the Capital One pattern get through my CI?	Stave
Where are my dangling cross-resource references?	Stave
Can I prove this verdict was deterministic against this snapshot?	Stave
Can I export detection logic to Z3 / Soufflé for formal analysis?	Stave
I want both auditor reporting and compound-risk detection	Both, side by side

Running both together

Both tools render in Powerpipe and consume Steampipe / DuckDB backends. Install side by side:

# Framework benchmarks against live AWS.
powerpipe mod install github.com/turbot/steampipe-mod-aws-compliance

# Compound risk + ghost references over snapshots.
powerpipe mod install github.com/zepho/powerpipe-mod-stave

# Stave itself produces the snapshot the powerpipe-mod-stave consumes.
go install github.com/sufield/stave/cmd/stave@latest

A typical workflow:

Daily / on-snapshot: stave apply --format json > out.stave.json produces the deterministic evaluation. CI fails on new compound-risk chains or ghost references via stave ci gate.
Pre-audit: powerpipe benchmark run cis_v3 --mod aws_compliance renders the framework-cited evidence pack auditors expect.
Architecture review: powerpipe server opens both dashboards side by side. The AWS Compliance Mod's severity rollup answers the framework question; Stave's stave_posture dashboard answers the composition question.

The two surfaces are intentionally orthogonal. The Stave Powerpipe mod deliberately does not render severity-count cards (that surface duplicates AWS Compliance Mod territory, and Stave's first dashboard widget is the chain graph instead). The AWS Compliance Mod does not render compound-risk chains (by design its job is per-resource framework coverage). Neither tries to be the other.

TL;DR

Run both. Auditor reports come from the AWS Compliance Mod. Composition risk comes from Stave. The full canonical comparison with every section above expanded is at aws-compliance-mod. The Capital One framing is the canonical wedge. The rest is just what falls out of using the right tool for each question.

We use VPC endpoints for S3, are we compliant? VPC endpoint without a policy is a wormhole to every S3 bucket

Bala Paranj — Sat, 24 Oct 2026 13:56:24 +0000

One of the confident statements on r/aws: "We route all S3 traffic through a VPC endpoint, so our data stays on the private network." VPC endpoints do keep traffic off the public internet. But the confidence creates a blind spot.

The Default Policy Problem

When you create a VPC gateway endpoint for S3, AWS attaches a default policy:

{
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "*",
      "Resource": "*"
    }
  ]
}

Any principal on the VPC can reach ANY S3 bucket in ANY AWS account. Not just your buckets or buckets in your account. Any bucket, anywhere, that allows access from the calling principal.

The endpoint restricts the network path but not the destination. A compromised EC2 instance on your VPC can exfiltrate data to an attacker-controlled S3 bucket in a different account, and the traffic never touches the internet. Your VPC flow logs show "private network." Your data is in someone else's bucket.

The HIPAA Citation

HIPAA section 164.312(e)(1) requires transmission security. The technical measures to guard against unauthorized access to electronic PHI during transmission. A VPC endpoint that allows unrestricted destination access doesn't satisfy this. It secures the transport but not the authorization of where data can flow.

How Stave Detects It

Stave evaluates this as a compound risk using two controls:

Control	What it checks	Result
CTL.S3.NETWORK.VPC.001	VPC endpoint or IP condition exists	PASS
CTL.S3.NETWORK.POLICY.001	Endpoint policy restricts access	FAIL

The compound finding is the dangerous one. CTL.S3.NETWORK.VPC.001 passes — yes, you have a VPC endpoint. But CTL.S3.NETWORK.POLICY.001 fails. The endpoint policy is missing or uses the default full-access policy. Together, this creates a false sense of security: the VPC endpoint exists, but it's a wormhole to the entire S3 namespace.

stave evaluate \
  --controls controls/s3/network/ \
  --observations observations/ \
  --eval-time 2026-04-08T00:00:00Z

Security State: NON_COMPLIANT

Findings:
  phi-vpc-endpoint:
    CTL.S3.NETWORK.POLICY.001  UNSAFE  VPC endpoint policy unrestricted
      VPC endpoint policy is missing or uses the default full-access
      policy. Any principal on the VPC can reach any S3 bucket.

      Remediation: Replace the default endpoint policy with one that
      restricts Resource to specific bucket ARNs and Action to required
      S3 operations only.

The fix is to replace the default policy with a restrictive one:

{
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": [
        "arn:aws:s3:::phi-data-bucket/*",
        "arn:aws:s3:::phi-logs-bucket/*"
      ]
    }
  ]
}

A VPC endpoint without an endpoint policy is security theater. It looks compliant where the traffic stays on the private network, the checkbox is checked. But the default Allow * policy turns the endpoint into an unrestricted gateway to every S3 bucket in existence. Stave detects when the endpoint exists but the policy doesn't restrict it, surfacing the compound risk that a single-control scanner would miss.

Three S3 risks AWS Trusted Advisor can't see

Bala Paranj — Fri, 23 Oct 2026 12:51:00 +0000

Trusted Advisor is a useful baseline. It checks whether buckets have public access and whether logging is enabled. But it operates on a narrow threat model: "Is this bucket public right now?" Three categories of S3 risk are invisible to it.

Blind spot 1: Incomplete observation data (CTL.S3.INCOMPLETE.001)

Before Trusted Advisor can check a bucket, it needs to read the bucket policy and ACL. What happens if the IAM role running the check lacks s3:GetBucketPolicy permission? Or if the bucket policy explicitly denies the scanner's principal?

Trusted Advisor reports nothing. The bucket doesn't appear in findings because the check couldn't execute. No error or warning. Silence.

Stave treats missing data as a finding, not a pass. The CTL.S3.INCOMPLETE.001 control fires when policy or ACL data is absent from the observation snapshot:

stave evaluate --snapshot observations/snap.json --profile hipaa

── LOW ──

  [FAIL] INCOMPLETE.001 — low
  Finding: Bucket policy data missing from snapshot — safety cannot be proven
  Remediation: Re-run the observation collector with s3:GetBucketPolicy,
  s3:GetBucketAcl, and s3:GetBucketPolicyStatus permissions

Absence of evidence is not evidence of absence. If you cannot read the policy, you cannot prove the bucket is safe.

Blind spot 2: Latent public exposure behind PAB (CTL.S3.PUBLIC.005)

A bucket can have a policy that grants s3:GetObject to Principal: "*" and still appear private if Block Public Access (PAB) is enabled. PAB overrides the public policy at the API layer. Trusted Advisor sees "not public" and reports green.

But PAB is a single toggle. One API call s3api put-public-access-block with all flags set to false and the bucket is instantly public. The underlying public policy is latent, waiting to activate.

Stave's CTL.S3.PUBLIC.005 detects this latent exposure:

── MEDIUM ──

  [FAIL] PUBLIC.005 — medium
  Finding: Public read policy masked only by Public Access Block —
  removing PAB would immediately expose the bucket
  Remediation: Remove the underlying public-granting policy statement
  so the bucket does not depend solely on PAB for protection

AWS Config rule changes, Terraform drift, and well-intentioned "fix the CORS issue" tickets have all disabled PAB on production buckets. If the underlying policy is already public, the exposure is instant.

Blind spot 3: ACL escalation (CTL.S3.ACL.ESCALATION.001)

S3 ACLs have a permission that most engineers have never heard of: WRITE_ACP. This permission lets the grantee modify the ACL itself. If AllUsers has WRITE_ACP, any anonymous user can grant themselves FULL_CONTROL with read, write, delete, and further ACL modification.

Trusted Advisor checks for public READ and WRITE. It does not check for WRITE_ACP or READ_ACP, because these are not direct data access permissions. They are escalation paths.

Stave's CTL.S3.ACL.ESCALATION.001 catches it:

── HIGH ──

  [FAIL] ACL.ESCALATION.001 — high
  Finding: Bucket ACL grants WRITE_ACP to AllUsers — any anonymous user
  can modify the ACL to grant themselves FULL_CONTROL
  Remediation: Remove WRITE_ACP grants and enable S3 Public Access Block
  with BlockPublicAcls set to true

Trusted Advisor answers: "Is this bucket public right now?" That is a useful but incomplete question. For HIPAA compliance, you also need to answer: "Can I prove this bucket's state?" (incomplete data), "Would this bucket become public with one config change?" (latent exposure), and "Can anyone escalate their own permissions?" (ACL escalation). These three blind spots require a tool that models threat paths, not just current state.

HIPAA compliance: AWS Config Conformance Pack vs Stave

Bala Paranj — Thu, 22 Oct 2026 14:00:14 +0000

Two approaches to the same regulation

AWS provides a HIPAA Security Conformance Pack which is a CloudFormation template that deploys ~100 AWS Config managed rules across S3, IAM, EC2, RDS, ELB, Lambda, VPC, CloudTrail, and more.

Stave provides a HIPAA profile consisting of 14 controls + 3 compound risk detectors focused on S3 and IAM, evaluated offline against JSON snapshots with duration tracking and regulatory citations.

They solve the same regulation at different depths.

Conformance Pack

The pack deploys AWS Config rules as CloudFormation resources:

S3BucketPublicReadProhibited:
  Properties:
    ConfigRuleName: s3-bucket-public-read-prohibited
    Scope:
      ComplianceResourceTypes:
        - AWS::S3::Bucket
    Source:
      Owner: AWS
      SourceIdentifier: S3_BUCKET_PUBLIC_READ_PROHIBITED
  Type: AWS::Config::ConfigRule

Each rule is an opaque AWS-managed check. You deploy the template, AWS Config evaluates your resources continuously, and results appear in the Config dashboard.

Strengths:

Covers 20+ AWS services in one template
~100 rules including S3, IAM, EC2, RDS, ELB, Lambda, VPC, CloudTrail, KMS, Secrets Manager, SageMaker, DynamoDB, Redshift, OpenSearch
Continuous evaluation — no manual snapshots
Native AWS integration — no external tools

Limitations:

Requires AWS Config enabled ($0.003 per rule evaluation per resource)
Requires live AWS API access and IAM permissions
Rules are opaque — you can't read the evaluation logic
Results are in the AWS Config dashboard — vendor lock-in
No HIPAA section citations on findings
No duration tracking
No compound risk detection
No offline capability
No proof of pass

What Stave does differently

1. Every finding cites the regulation

[FAIL] ACCESS.001 — critical
Compliance: §164.312(a)(1) — Access control
Finding: Bucket phi-patient-records: Block Public Access is not fully enabled

The AWS Config finding says "S3_BUCKET_PUBLIC_READ_PROHIBITED — NON_COMPLIANT." It doesn't tell the auditor which HIPAA section applies. Stave maps every control to its specific Security Rule citation.

2. Compound risk detection

AWS Config rules evaluate independently. Two rules can both pass while their combination is dangerous.

Stave detects three compound patterns:

[critical] COMPOUND.001 (triggers: ACCESS.001, ACCESS.002)
Public access with overly broad IAM permissions — the S3 + IAM lateral
movement pattern present in the majority of documented AWS breaches.

[high] COMPOUND.002 (triggers: ACCESS.001, CONTROLS.001)
Encryption at rest is configured but the bucket is publicly accessible.
Encryption provides no confidentiality benefit while public access is enabled.

[high] COMPOUND.003 (triggers: ACCESS.003, ACCESS.006)
VPC endpoint restricts this bucket but the endpoint policy does not
restrict which bucket ARNs are reachable — a wormhole.

AWS Config has no cross-rule reasoning. Each rule runs in isolation.

3. Duration tracking

AWS Config tells you a resource is non-compliant right now. It doesn't tell you how long it's been non-compliant.

Finding: Bucket phi-patient-records
  First unsafe: 2026-01-01T00:00:00Z
  Last seen:    2026-01-11T00:00:00Z
  Duration:     240 hours (threshold: 168 hours)

For HIPAA, duration matters. A bucket that was public for 5 minutes during a deploy is different from one that's been public for 6 months. The Conformance Pack can't distinguish them.

4. Acknowledged exceptions with compensating controls

# stave.yaml
exceptions:
  - control_id: ACCESS.001
    bucket: my-cloudfront-origin
    rationale: "CloudFront + OAI pattern — bucket is private to CloudFront"
    acknowledged_by: security-team@example.com
    requires_passing:
      - CONTROLS.001
      - CONTROLS.004
      - AUDIT.001

The Conformance Pack supports suppression via Config rule suppressions. But without mandatory compensating controls. You can suppress a finding without documenting what alternative protection exists.

Stave requires compensating controls. If the compensating control fails, the original finding stands.

5. Offline and deterministic

The Conformance Pack requires AWS Config running in your account. Stave evaluates a JSON snapshot locally:

stave evaluate --snapshot snap.json --profile hipaa --format json

Same input always produces the same output. No AWS credentials needed. Air-gapped environments supported. The logic trace provides a verifiable reasoning chain for every verdict.

6. CMK vs AWS-managed key distinction

The Conformance Pack has S3_DEFAULT_ENCRYPTION_KMS. It checks if KMS is the default encryption. But it doesn't distinguish between the AWS-managed key (alias/aws/s3) and a customer-managed key.

Stave's CONTROLS.001.STRICT fails specifically on the AWS-managed key because it can't be revoked during a breach:

[FAIL] CONTROLS.001.STRICT — critical
Compliance: §164.312(a)(2)(iv) — CMK required for key revocation
Finding: SSE-KMS uses the AWS-managed key (alias/aws/s3).
CMK required for key revocation during breach response.

Head-to-head: S3 HIPAA controls

Check	AWS Config Rule	Stave Control	Stave Extra
Public read	`S3_BUCKET_PUBLIC_READ_PROHIBITED`	`ACCESS.001`	+ HIPAA citation, + duration
Public write	`S3_BUCKET_PUBLIC_WRITE_PROHIBITED`	Built-in (CTL.S3.PUBLIC.003)	+ severity tiering
Encryption at rest	`S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED`	`CONTROLS.001`	—
KMS encryption	`S3_DEFAULT_ENCRYPTION_KMS`	`CONTROLS.001.STRICT`	+ CMK vs AWS-managed distinction
SSL/TLS only	`S3_BUCKET_SSL_REQUESTS_ONLY`	`CONTROLS.004`	+ HIPAA citation
Logging	`S3_BUCKET_LOGGING_ENABLED`	`AUDIT.001`	+ "logs cannot be obtained retroactively"
Versioning	`S3_BUCKET_VERSIONING_ENABLED`	`CONTROLS.002`	+ MFA delete recommendation
ACLs disabled	`S3_BUCKET_ACL_PROHIBITED`	`GOVERNANCE.001`	+ AWS Backup exception note
Object Lock	`S3_BUCKET_DEFAULT_LOCK_ENABLED`	`RETENTION.002`	+ severity tiering (none/governance/compliance)
CloudTrail data events	`CLOUDTRAIL_S3_DATAEVENTS_ENABLED`	`AUDIT.002`	+ object-level audit trail context
Account-level PAB	`S3_ACCOUNT_LEVEL_PUBLIC_ACCESS_BLOCKS_PERIODIC`	Built-in	+ severity downgrade when account-level present
Lifecycle rules	`S3_VERSION_LIFECYCLE_POLICY_CHECK`	`LIFECYCLE.001`	+ PHI minimum retention check
Compound: public + wildcard	Not detected	COMPOUND.001	Cross-rule reasoning
Compound: encrypted + public	Not detected	COMPOUND.002	False security detection
Compound: VPC + no policy	Not detected	COMPOUND.003	Wormhole detection

Where the Conformance Pack wins

Capability	AWS Config	Stave
Services covered	20+ (EC2, RDS, ELB, Lambda, VPC, etc.)	S3 + IAM
Total rules	~100	14 + 3 compound
Continuous	Yes (Config evaluates on change)	No (snapshot-based)
Native AWS	Yes	No (offline tool)
Setup effort	Deploy one CloudFormation template	Build extractor + run CLI

If you need broad coverage across many AWS services with minimal setup, the Conformance Pack wins on breadth.

Where Stave wins

Capability	AWS Config	Stave
HIPAA citations	No	Every finding cites §164.312
Compound risk	No	3 cross-rule detectors
Duration tracking	No	SLA-based with temporal evidence
Exceptions	Suppression only	Compensating controls required
Offline	No	Yes — air-gapped capable
Deterministic	No	Same input = same output
Proof of pass	No	Logic trace reasoning chain
CMK distinction	No	Fails on AWS-managed key
Multi-cloud	No (AWS only)	Yes (S3, IAM, GCS, DNS)
Readable rules	No (opaque)	Yes (YAML + CEL)

If you need deep S3/IAM analysis with auditor-ready evidence, Stave wins on depth.

Use both

Deploy the Conformance Pack for broad continuous monitoring across all AWS services. Run Stave for deep S3/IAM analysis with compound risk detection, duration tracking, and HIPAA-cited evidence that auditors can verify offline.

The Conformance Pack is your dashboard. Stave is your proof.

Your SaaS vendor said 'use S3 for HIPAA recordings'. Did you secure the bucket?

Bala Paranj — Wed, 21 Oct 2026 13:50:56 +0000

The handoff problem

SaaS vendors that handle PHI such as telehealth platforms, EHR integrations, diagnostic tools tell customers: "Store your data in your own S3 bucket for HIPAA compliance."

This is architecturally sound. The vendor avoids storing PHI. The customer controls their own data. But it creates a responsibility handoff: the vendor provides instructions, the customer must execute them correctly.

Whereby, a video conferencing platform used in telehealth, tells HIPAA customers to store cloud recordings in their own S3 bucket with this policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowListAndPutRecordings",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<aws_account_id>:user/<aws_user>"
      },
      "Action": ["s3:ListBucket", "s3:PutObject"],
      "Resource": [
        "arn:aws:s3:::<recording_bucket>",
        "arn:aws:s3:::<recording_bucket>/*"
      ]
    }
  ]
}

Good template. Scoped principal, minimal actions, specific resources. But the template only covers the bucket policy. It says nothing about encryption, logging, versioning, Public Access Block, Object Lock, or transport encryption.

The customer is responsible for all of it. They can get it wrong.

What goes wrong

1. No encryption at rest

The customer creates the bucket, pastes the policy, starts recording. Nobody enables server-side encryption. The recordings which contain patient video, audio, and potentially screen-shared medical records sit unencrypted in S3.

Stave catches it:

[FAIL] CONTROLS.001 — high
Compliance: §164.312(a)(2)(iv) — Encryption and Decryption
Finding: Bucket telehealth-recordings: encryption at rest not enabled

2. AWS-managed key instead of CMK

The customer enables SSE-S3 (AES256) which is the default. But AES256 uses an AWS-managed key that the customer cannot revoke. During a breach, the customer needs to disable the key immediately to render recordings unreadable. With the AWS-managed key, they can't.

Stave catches it:

[FAIL] CONTROLS.001.STRICT — critical
Compliance: §164.312(a)(2)(iv) — CMK required for key revocation
Finding: SSE uses AES256 (AWS-managed). CMK required for breach response.

Whereby even notes that "key rotation will provide additional security." But they stop short of requiring CMK. Stave's HIPAA profile makes the distinction explicit.

3. No access logging

The bucket stores PHI recordings but has no server access logging enabled. If an unauthorized party downloads recordings, there's no forensic evidence.

Stave catches it:

[FAIL] AUDIT.001 — critical
Compliance: §164.312(b) — Audit Controls
Finding: Bucket telehealth-recordings: logging not enabled.
Logs cannot be obtained retroactively from AWS.

This is the retroactivity problem. You can't turn on logging after a breach and recover what happened before. The evidence window is zero.

4. Public Access Block not enabled

The customer pastes the bucket policy but doesn't enable Public Access Block. A future policy change such as a Terraform apply with a stale config, a console click could make the bucket public. PAB is the safety net.

Stave catches it:

[FAIL] CTL.S3.CONTROLS.001 — high
Finding: Public Access Block not fully enabled. No safety net against
accidental public exposure from policy or ACL changes.

5. Wildcard actions in the policy

Instead of the vendor's scoped template, the customer pastes a broader policy they found on Stack Overflow:

{
  "Effect": "Allow",
  "Principal": "*",
  "Action": "s3:*",
  "Resource": "arn:aws:s3:::telehealth-recordings/*"
}

Now anyone can read, write, delete, or modify ACLs on the recordings bucket.

Stave catches it:

[FAIL] ACCESS.002 — high
Compliance: §164.312(a)(2)(i) — Least privilege
Finding: Policy grants Allow with wildcard action s3:*

And the compound detector fires:

[critical] COMPOUND.001
Public access with overly broad IAM permissions — the S3 + IAM lateral
movement pattern present in the majority of documented AWS breaches.

6. No Object Lock for retention

HIPAA requires 6 years of PHI retention. Without Object Lock in Compliance mode, recordings can be deleted accidentally, maliciously, or by a lifecycle rule that expires objects too early.

Stave catches it:

[FAIL] RETENTION.002 — critical
Compliance: §164.316(b)(2) — 6-year PHI retention
Finding: Object Lock not enabled — recordings can be deleted.

7. No transport encryption enforcement

The bucket policy doesn't deny non-TLS requests. Someone accessing the bucket via HTTP (not HTTPS) transmits recordings in plaintext.

Stave catches it:

[FAIL] CONTROLS.004 — high
Compliance: §164.312(e)(2)(ii) — Encryption in transit
Finding: No Deny statement for aws:SecureTransport=false

The compound picture

Run all 14 HIPAA controls against a typical "I followed the vendor's template" bucket:

stave evaluate --snapshot recordings-bucket.json --profile hipaa

═══ HIPAA Security Rule ═══
Bucket:    telehealth-recordings
Result:    FAIL

  critical: 0/4 passed
  high:     1/7 passed  (only the vendor's bucket policy is correct)
  medium:   0/2 passed

Overall: FAIL

The vendor's bucket policy template is the only thing that passes. Everything else such as encryption, logging, retention, transport security, PAB is the customer's responsibility. And the customer didn't do it.

The duration dimension

The worst version of this: the bucket has been in this state since the customer set it up 18 months ago. Every recording since then is stored unencrypted, unlogged, without retention protection.

Duration: 13,000+ hours (threshold: 168 hours)

No other tool tracks this. AWS Config says "non-compliant now." Stave says "non-compliant for 13,000 hours." That temporal evidence changes the regulatory conversation.

The lesson for SaaS vendors and customers

For vendors: providing a bucket policy template is necessary but insufficient. Document the full bucket security posture: encryption (CMK, not AWS-managed), logging, PAB, versioning, Object Lock, transport encryption. Or better: provide a Stave observation contract so customers can evaluate their bucket against your HIPAA requirements.

For customers: the vendor's HIPAA compliance ends at their boundary. Your S3 bucket is your responsibility. Run Stave's HIPAA profile against every bucket that stores vendor-delegated PHI:

stave evaluate --snapshot my-recordings-bucket.json --profile hipaa

If it doesn't say PASS across all 14 controls, the bucket isn't HIPAA compliant regardless of what the vendor's documentation says.

HIPAA on Kubernetes: what kops issue #1776 asked for and what Stave covers now

Bala Paranj — Tue, 20 Oct 2026 14:41:24 +0000

The question that's still open

In February 2017, kops issue #1776 asked: "What is kops missing to support a HIPAA compliant cluster?"

The issue is still open. The discussion produced one of the most practical HIPAA checklists for AWS infrastructure, contributed by engineers running PHI workloads:

The checklist covers 6 domains: data encryption, access control, audit logging, network security, compute security, and operational procedures. Most of these are still valid requirements in 2026.

The checklist mapped to Stave

Here's every technical requirement from the kops discussion, mapped against what Stave evaluates today:

Data Encryption

Requirement	HIPAA Section	Stave Status	Control
ePHI encrypted at rest (S3)	§164.312(a)(2)(iv)	Covered	`CTL.S3.ENCRYPT.001`
CMK instead of AWS-managed key	§164.312(a)(2)(iv)	Covered	`CONTROLS.001.STRICT`
Encrypted EBS volumes	§164.312(a)(2)(iv)	Covered	`CTL.EC2.EBS.DEFAULT.001`
Encrypted EBS root AMI	§164.312(a)(2)(iv)	Covered	`CTL.EC2.AMI.ENCRYPTION.001`
etcd encryption at rest	§164.312(a)(2)(iv)	Covered	`CTL.K8S.ETCD.CERT.001`
S3 transport encryption (TLS)	§164.312(e)(2)(ii)	Covered	`CTL.S3.ENCRYPT.002`
Network encryption (pod-to-pod)	§164.312(e)(1)	Covered	`CTL.K8S.NETPOL.001`

Access Control

Requirement	HIPAA Section	Stave Status	Control
S3 Public Access Block	§164.312(a)(1)	Covered	`CTL.S3.CONTROLS.001`
No wildcard IAM actions	§164.312(a)(2)(i)	Covered	`CTL.S3.ACCESS.002`
Root account MFA	§164.312(d)	Covered	`CTL.IAM.ROOT.MFA.001`
Console user MFA	§164.312(d)	Covered	`CTL.IAM.CONSOLE.MFA.001`
No root access keys	§164.312(a)(1)	Covered	`CTL.IAM.ROOT.ACCESSKEY.001`
Unused credentials disabled	§164.312(a)(2)(i)	Covered	`CTL.IAM.CRED.UNUSED.001`
Access key rotation (<90 days)	§164.312(a)(2)(i)	Covered	`CTL.IAM.CRED.ROTATION.001`
No inline IAM policies	§164.312(a)(2)(i)	Covered	`CTL.IAM.POLICY.INLINE.001`
Password policy (length, complexity)	§164.312(a)(2)(i)	Covered	`CTL.IAM.PASSWORD.*`
Default-deny security groups	§164.312(a)(1)	Covered	`CTL.VPC.SG.DEFAULT.001`
Kubernetes RBAC enforcement	§164.312(a)(1)	Covered	`CTL.K8S.RBAC.WILDCARD.001`
Pod security policies	§164.312(a)(1)	Covered	`CTL.K8S.POD.PRIVILEGED.001`

Audit & Logging

Requirement	HIPAA Section	Stave Status	Control
S3 access logging	§164.312(b)	Covered	`CTL.S3.LOG.001`
CloudTrail S3 data events	§164.312(b)	Covered	`CTL.S3.AUDIT.OBJECTLEVEL.001`
VPC flow logging	§164.312(b)	Covered	`CTL.VPC.FLOWLOG.001`
Kubernetes audit logging	§164.312(b)	Covered	`CTL.K8S.AUDIT.001`
kops/kubectl action logging	§164.312(b)	Not covered	Operational — tool-specific
File system auditing (OSSEC)	§164.312(b)	Not covered	Runtime agent — not config

Network Security

Requirement	HIPAA Section	Stave Status	Control
VPC endpoint for S3	§164.312(e)(1)	Covered	`CTL.S3.NETWORK.VPC.001`
VPC endpoint policy	§164.312(e)(1)	Covered	`CTL.S3.NETWORK.POLICY.001`
Private topology (no public IPs)	§164.312(e)(1)	Covered	`CTL.EC2.PUBLIC.001`
Restricted egress	§164.312(e)(1)	Covered	`CTL.K8S.NETPOL.EGRESS.001`
Network policy enforcement	§164.312(e)(1)	Covered	`CTL.K8S.NETPOL.DENY.001`

Data Integrity & Retention

Requirement	HIPAA Section	Stave Status	Control
S3 versioning	§164.312(c)(1)	Covered	`CTL.S3.VERSION.001`
S3 Object Lock (compliance mode)	§164.316(b)(2)	Covered	`CTL.S3.LOCK.001/002/003`
S3 lifecycle rules	§164.316(b)(2)	Covered	`CTL.S3.LIFECYCLE.001/002`
Backup encryption	§164.312(a)(2)(iv)	Covered	`CTL.RDS.ENCRYPT.BACKUP.001`
Backup testing	§164.308(a)(7)	Not covered	Operational — cadence policy

Operational (Policy-based, not technical)

Requirement	HIPAA Section	Stave Status	Notes
Unique user identification	§164.312(a)(2)(i)	Partially	IAM controls, not K8s users
Emergency access procedure	§164.312(a)(2)(ii)	Not covered	Operational policy
Auto-logoff	§164.312(a)(2)(iii)	Not covered	Application-level
Security incident reporting	§164.308(a)(6)	Not covered	Operational policy
User off-boarding	§164.308(a)(3)	Not covered	Operational policy
Malware protection	§164.308(a)(5)	Not covered	Runtime security
BAA with AWS	§164.308(b)	Acknowledged	Disclaimer in every report

Scorecard

Domain	Total Checks	Stave Covers	Gap
Data Encryption	7	7	0
Access Control	12	12	0
Audit & Logging	6	4	2 (kops action logging, filesystem auditing)
Network Security	5	5	0
Data Integrity	5	4	1 (backup testing cadence)
Operational	7	0	7 (policy-based, not technical)
Total	42	32	10

Stave covers 32 of 42 requirements across S3, IAM, EBS, EC2, VPC, RDS, ELB, and Kubernetes. The 10 remaining gaps are operational procedures (7), runtime agents (2 — filesystem auditing, kops action logging), and backup testing cadence (1). None of these are infrastructure configuration that Stave's evaluation model addresses.

Out of scope (operational)

These requirements are policy-based, not infrastructure-based. Stave evaluates technical controls only:

Emergency access procedures
User off-boarding processes
Security incident reporting
Malware protection
Backup testing cadence
Auto-logoff configuration
kops/kubectl action logging (tool-specific operational logging)
File system auditing / OSSEC (runtime agent, not config)

Stave's HIPAA report includes the BAA disclaimer: "Stave evaluates technical controls only. A BAA with AWS is a contractual prerequisite for HIPAA compliance that Stave cannot verify."

What Stave already does beyond the kops checklist

The kops discussion focused on infrastructure configuration. Stave adds dimensions the checklist doesn't cover:

Compound risk detection — public access + wildcard actions = lateral movement
Duration tracking — how long has each misconfiguration persisted?
CMK vs AWS-managed key — the breach response distinction
Object Lock compliance mode — not just "enabled" but which mode
Presigned URL restrictions — controls the kops checklist never mentions
Acknowledged exceptions — documented risk acceptance with compensating controls
Proof of pass — logic trace proving why something is compliant
Offline evaluation — air-gapped, deterministic, no credentials needed

The coverage

Stave covers 32 of 42 requirements from the kops HIPAA checklist across S3, IAM, EBS, EC2, VPC, RDS, ELB, and Kubernetes. The catalog now includes 3,558 controls with compound risk detection, duration tracking, and HIPAA citations.

The 10 remaining gaps are operational procedures and runtime agents that fall outside infrastructure configuration evaluation. Every technical requirement from the original 2017 discussion is covered.

The kops HIPAA discussion referenced in this article is github.com/kubernetes/kops/issues/1776.

Is my S3 bucket HIPAA compliant? The question AWS can't answer

Bala Paranj — Mon, 19 Oct 2026 12:19:26 +0000

Recurring theme on r/aws:

The top reply is always some version of: "AWS won't tell you that. You need to evaluate your configuration against the Security Rule yourself."

That reply is correct and does not help.

What HIPAA requires for S3

The HIPAA Security Rule doesn't mention S3. It defines four categories of safeguards that apply to any system storing electronic protected health information (ePHI):

Access Control (SS 164.312(a)(1)) -- Only authorized users can reach PHI
Audit Controls (SS 164.312(b)) -- All access to PHI is logged
Encryption (SS 164.312(a)(2)(iv), SS 164.312(e)(2)(ii)) -- Data encrypted at rest and in transit
Retention (SS 164.316(b)(2)) -- PHI retained for minimum 6 years

Teams get stuck mapping these to S3 settings. Encryption at rest is one checkbox. But which KMS key type? Is server access logging enough, or do you need CloudTrail object-level events? Does Block Public Access count if it's only set at the account level?

How Stave evaluates HIPAA compliance

Stave is an open-source CLI that evaluates S3 configurations against compliance profiles. The built-in HIPAA profile maps 28 controls to specific Security Rule sections and evaluates them against an observation snapshot of your bucket configuration.

stave evaluate --snapshot observations/snap.json --profile hipaa

The output is a structured report with pass/fail verdicts, regulatory citations, and remediation steps:

═══ HIPAA Security Rule ═══
Bucket:    phi-patient-records
Account:   ***456
Snapshot:  2026-04-08T14:30:00Z
Result:    FAIL

── CRITICAL ──

  [FAIL] CONTROLS.001.STRICT — critical
  Compliance: §164.312(a)(2)(iv) — CMK required for key revocation during breach response
  Finding: Encryption uses AWS-managed key — customer cannot revoke
  Remediation: Change default encryption to SSE-KMS with a customer-managed key

  [PASS] CONTROLS.004 — critical
  Compliance: §164.312(e)(2)(ii) — Transmission Security

  [PASS] AUDIT.001 — critical
  Compliance: §164.312(b) — Audit Controls

  [PASS] ACCESS.001 — critical
  Compliance: §164.312(a)(1) — Access Control

── HIGH ──

  [FAIL] AUDIT.002 — high
  Compliance: §164.312(b) — Object-level logging for PHI access audit trail
  Finding: CloudTrail object-level data events not configured
  Remediation: Configure a CloudTrail trail with S3 data event selector

Each finding maps directly to a HIPAA citation. Your auditor can trace the control back to the regulation.

AWS signs the Business Associate Agreement. AWS provides the encryption toggles, the logging infrastructure, and the access controls. But AWS will never evaluate whether your specific configuration satisfies the Security Rule. That is the customer's responsibility under the shared responsibility model.

Stave bridges that gap. Instead of a 47-page checklist, you get a single command that produces a pass/fail verdict with the regulatory receipts to back it up.

Prowler has HIPAA checks. What checklist-based compliance still misses.

Bala Paranj — Sun, 18 Oct 2026 11:30:06 +0000

The 2018 conversation

In Prowler issue #227, a user asked: "any plan to add [HIPAA checks]?"

A contributor laid out a checklist consisting of MFA, VPC flow logs, encrypted EBS volumes, S3 bucket policies, encrypted RDS. The maintainer's response:

They shipped the checks and have continued expanding them. Prowler's release v5.22.0, March 2026 includes HIPAA compliance frameworks for AWS and Azure, listed as a supported framework in the README. The issue was closed in 2018, and HIPAA has been part of Prowler for years.

MFA enabled? Check. Encrypted volumes? Check. Access logs? Check. These are useful checks. They're the same class of checks AWS Config provides. They're necessary but not sufficient.

What the checklist misses

The Prowler HIPAA checks (and AWS Config Conformance Pack, and Security Hub) all share the same architecture: individual rules evaluated independently against individual resources. This creates three blind spots that no checklist can fill.

1. No compound risk

Prowler checks if a bucket is public. Prowler checks if a policy has wildcard actions. But it can't detect that the combination is the S3+IAM lateral movement pattern behind most documented AWS breaches.

Stave detects compound patterns across services:

[high] ec2_to_s3_lateral
Overbroad EC2 instance profile + unrestricted egress = S3 data exfiltration.
(CTL.EC2.PROFILE.OVERBROAD.001 + CTL.VPC.SG.EGRESS.001)

[critical] public_phi_exposure
Public S3 bucket + no encryption enforcement = PHI exposure.
(CTL.S3.PUBLIC.005 + CTL.S3.ENCRYPT.KMS.OWNERSHIP.001)

[high] vpc_endpoint_bypass
VPC endpoint with default policy = any principal on the VPC reaches any S3 bucket.
(CTL.VPC.ENDPOINT.GATEWAY.DEFAULTPOLICY.001)

No individual check catches these. They require cross-rule reasoning.

2. No duration tracking

Prowler tells you a resource is non-compliant right now. It doesn't tell you it's been non-compliant for 6 months.

Finding: phi-patient-records
  First unsafe: 2025-06-01T00:00:00Z
  Last seen:    2026-01-11T00:00:00Z
  Duration:     5,400+ hours (threshold: 168 hours)

For HIPAA, this distinction matters. A bucket that was briefly public during a deploy and caught within the SLA is different from one that's been exposed for 7 months. Duration is the missing dimension in every cloud security checklist.

3. No PHI-specific depth

The Prowler contributor's checklist is service-level: encrypted volumes, access logs, VPC flow logs. These are correct for general AWS hardening. But HIPAA S3 requires deeper analysis:

PHI-Specific Check	Prowler	Stave
CMK vs AWS-managed key	No — checks "is KMS enabled?"	Yes — `CTL.S3.ENCRYPT.KMS.OWNERSHIP.001` fails on `alias/aws/s3`
Object Lock compliance mode	No	Yes — `CTL.S3.LOCK.001`–`003` tiers: none=critical, governance=high, compliance=pass
Presigned URL restrictions	No	Yes — `CTL.S3.PRESIGNED.001` checks `s3:signatureAge`/`s3:authType` conditions
VPC endpoint without policy	No	Yes — `CTL.VPC.ENDPOINT.GATEWAY.DEFAULTPOLICY.001` detects the wormhole
ACL escalation (WRITE_ACP)	No	Yes — `CTL.S3.ACL.ESCALATION.001` detects privilege escalation path
Latent exposure behind PAB	No	Yes — `CTL.S3.PUBLIC.005` detects public policy masked by PAB toggle
HIPAA section citations	No	Yes — every finding cites §164.312

The Prowler maintainer was right: full HIPAA compliance depends on how each user handles PHI. But that's an argument for a tool that evaluates the actual PHI storage configuration, not just generic service settings.

What the user in #227 needed

The user asking for HIPAA checks in 2018 needed:

A HIPAA profile — a curated set of controls mapped to Security Rule sections, not a generic checklist
Compound risk detection — dangerous combinations, not just individual failures
Duration tracking — how long has this been non-compliant?
PHI-specific controls — CMK enforcement, Object Lock compliance mode, presigned URL restrictions
Acknowledged exceptions — documented risk acceptance with compensating controls
Offline evaluation — auditor-ready evidence, not dashboard access

Prowler shipped item 1 (a checklist with HIPAA framework tags). Items 2-6 remain unaddressed.

How Stave fills the gap

stave apply --observations phi-observations/ --format text

═══ HIPAA Security Rule ═══
Bucket:    phi-patient-records
Result:    FAIL

── COMPOUND RISKS ──

  [critical] public_phi_exposure (triggers: CTL.S3.PUBLIC.005, CTL.S3.ENCRYPT.KMS.OWNERSHIP.001)
  Public bucket with no CMK encryption = PHI exposure

── critical ──

  [FAIL] CTL.S3.ACCESS.001 — critical
  Compliance: §164.312(a)(1) — Access control

  [FAIL] CTL.S3.ENCRYPT.KMS.OWNERSHIP.001 — critical
  Compliance: §164.312(a)(2)(iv) — CMK required for key revocation

── Summary ──

  critical: 0/4 passed
  high:     2/8 passed
  medium:   0/2 passed

3,500+ controls + 770+ chain detectors. Every finding cites the HIPAA section. Duration tracked. Exceptions require compensating controls. Runs offline.

The answer to #227

The Prowler maintainer said full HIPAA compliance checks aren't possible because it depends on how each user handles PHI. It's true that no tool can verify organizational policies, BAAs, or workforce training.

But technical controls on PHI storage infrastructure are verifiable. A bucket either has CMK encryption or it doesn't. Object Lock is either in compliance mode or it isn't. The VPC endpoint either has a restrictive policy or it doesn't.

Stave evaluates what's verifiable and says so explicitly:

That disclaimer present in every HIPAA report is the honest boundary. Everything inside that boundary, Stave proves. Everything outside it, Stave acknowledges it can't.

That's the answer issue #227 needed in 2018.

Coverage: the full Prowler HIPAA checklist

Against the Prowler HIPAA checklist, every item is covered plus depth beyond the checklist:

Prowler Check	Stave Controls	Depth beyond the checklist
Account: MFA Enabled	`CTL.IAM.ROOT.MFA.001`, `CTL.IAM.CONSOLE.MFA.001`	Root vs console distinguished
Account: Root Credentials	`CTL.IAM.ROOT.ACCESSKEY.001`	Duration tracking on exposure
S3: Encryption + IP Filter	`CTL.S3.ENCRYPT.001`–`004`, `CTL.S3.NETWORK.001`, `CTL.S3.NETWORK.VPC.001`	CMK vs AWS-managed key; key policy audit
S3: IAM Permissions	`CTL.S3.ACCESS.001`–`003`, `CTL.IAM.POLICY.*`	ACL escalation, presigned URL restrictions
S3: Access Logs	`CTL.S3.LOG.001`, `CTL.S3.AUDIT.OBJECTLEVEL.001`	Log bucket integrity, retention, prefix
VPC: Flow Logging	`CTL.VPC.FLOWLOG.001`, `CTL.VPC.FLOWLOG.STATUS.001`, `CTL.VPC.FLOWLOG.FORMAT.001`	Bidirectional check, NAT/TGW flow logs
VPC: Flow Logs Encrypted	`CTL.VPC.FLOWLOG.ENCRYPT.001`	Ghost S3 destination detection
VPC: ELB Logging	`CTL.ELB.LOG.001`, `CTL.ELB.NLB.ACCESSLOG.001`, `CTL.ELB.NLB.CONNLOG.001`	Permission audit, ghost bucket, lifecycle
EC2: Encrypted EBS Volumes	`CTL.EC2.EBS.ENCRYPT.001`, `CTL.EC2.EBS.ENCRYPT.CMK.001`	KMS policy audit, SCP enforcement
EC2: Encrypted EBS Snapshots	`CTL.EC2.EBS.SNAPSHOT.ENCRYPT.001`, `CTL.EBS.ENCRYPT.KEYORIGIN.001`	Shared unencrypted snapshot detection
RDS: Encrypted RDS	`CTL.RDS.ENCRYPT.001`, `CTL.RDS.ENCRYPT.CMK.001`, `CTL.RDS.SNAPSHOT.ENCRYPT.001`	Backup encryption, cross-region, SSL enforcement

What Stave does that no checklist can

Beyond checklist parity, Stave provides capabilities the Prowler architecture has no equivalent for:

Compound risk detection across services (770+ chain detectors)
Duration tracking on every finding (SLA-based)
CMK vs AWS-managed key distinction for PHI encryption
Object Lock compliance mode tiering for PHI retention
Presigned URL restrictions for PHI buckets
Latent exposure detection (public policy behind PAB)
ACL escalation detection (WRITE_ACP privilege escalation)
Acknowledged exceptions with mandatory compensating controls
HIPAA §164.312 citations on every finding
Proof of pass via logic trace

The checklist covers breadth. Stave covers depth and cross-service reasoning.

Are presigned URLs a HIPAA risk? The presigned URL gap in HIPAA S3 compliance

Bala Paranj — Sat, 17 Oct 2026 12:54:28 +0000

We use presigned URLs to share reports with providers. Are we HIPAA compliant? The discussion is split between "they're temporary, it's fine" and "they're a ticking time bomb." The nuance is in what your bucket policy enforces.

The Problem

A presigned URL embeds IAM credentials into the URL itself. Anyone with the URL can download the object without AWS credentials or MFA check. Default expiration is up to 7 days for IAM user credentials. If a presigned URL to a PHI object leaks via email, Slack, or a log file, you have an uncontrolled access window to protected health information.

The bucket policy doesn't restrict presigned URLs by default. Presigned URLs bypass IAM policies, VPC endpoint conditions, and IP restrictions. Unless the bucket policy explicitly includes conditions for s3:signatureAge or s3:authType.

The HIPAA Citation

HIPAA section 164.312(a)(1) requires access controls that restrict access to electronic PHI to authorized persons. A presigned URL that can be forwarded to any person on the internet is not an access control. It's an access delegation with no audit trail of who used it.

How Stave Detects It

Stave control CTL.S3.PRESIGNED.001 checks whether the bucket policy includes presigned URL restrictions:

id: CTL.S3.PRESIGNED.001
name: Presigned URL Access Must Be Restricted
severity: medium
compliance:
  hipaa: "164.312(a)(1)"

unsafe_predicate:
  all:
    - field: properties.storage.kind
      op: eq
      value: bucket
    - field: properties.storage.access.presigned_url_restricted
      op: eq
      value: false

The control checks for two specific guardrails in the bucket policy:

s3:signatureAge Limits how long a presigned URL remains valid. A Deny statement with NumericGreaterThan on s3:signatureAge of 600000 (10 minutes) forces short-lived URLs.
s3:authType A Deny statement with StringNotEquals on s3:authType set to REST-HEADER blocks presigned URL access, requiring direct IAM authentication.

stave evaluate \
  --controls controls/s3/access/ \
  --observations observations/ \
  --eval-time 2026-04-08T00:00:00Z

Security State: NON_COMPLIANT

Findings:
  phi-reports-bucket:
    CTL.S3.PRESIGNED.001  UNSAFE  Presigned URL access unrestricted
      Bucket policy does not restrict presigned URL access. Presigned
      URLs can provide long-lived unauthenticated access to objects.

      Remediation: Add a Deny statement with Condition NumericGreaterThan
      s3:signatureAge (e.g., 600000 for 10 minutes) or StringNotEquals
      s3:authType REST-HEADER to block presigned URL access.

Presigned URLs are inherently uncontrolled. The security comes from the bucket policy conditions that constrain them, not from the URLs themselves. If your PHI bucket policy doesn't include s3:signatureAge or s3:authType conditions, every presigned URL is a potential breach vector with no expiration guardrail beyond the IAM credential lifetime. Stave flags this gap before an auditor or an attacker finds it.

Do we need Object Lock for PHI? HIPAA S3 retention: Object Lock compliance mode is not optional

Bala Paranj — Fri, 16 Oct 2026 12:53:47 +0000

The thread repeats on r/aws: a team stores Protected Health Information (PHI) in S3, has versioning enabled, and assumes that's sufficient for retention. Someone suggests Object Lock. The team pushes back. It's complex and they already have lifecycle policies.

The answer: Object Lock in Compliance mode is not optional for HIPAA PHI buckets.

What HIPAA Requires

HIPAA section 164.316(b)(2) mandates a 6-year retention period for documentation related to PHI. This is a regulatory requirement that auditors verify.

Versioning alone doesn't satisfy this. A versioned object can still be permanently deleted by removing all versions and delete markers. Lifecycle policies can expire versions. An IAM principal with s3:DeleteObject can remove data before the retention period expires.

Object Lock provides WORM (Write Once Read Many) protection. But the mode matters:

Governance mode: Users with s3:BypassGovernanceRetention permission can override the lock. This is a privileged bypass.
Compliance mode: No user, including the root account, can delete or modify protected objects during the retention period. This is the only mode that provides the tamper-proof guarantee HIPAA requires.

How Stave Detects It

Stave evaluates this through three controls that tier the severity:

Control	What it checks	Severity
CTL.S3.LOCK.001	Object Lock not enabled on compliance-tagged bucket	MEDIUM
CTL.S3.LOCK.002	Object Lock uses GOVERNANCE mode instead of COMPLIANCE	MEDIUM
CTL.S3.LOCK.003	Retention period shorter than 2190 days (6 years)	MEDIUM

This creates a severity cascade. A PHI bucket without Object Lock fails CTL.S3.LOCK.001. A PHI bucket with Object Lock in Governance mode passes LOCK.001 but fails LOCK.002. A PHI bucket with Compliance mode but a 365-day retention period passes both but fails LOCK.003.

stave evaluate \
  --controls controls/s3/lock/ \
  --observations observations/ \
  --eval-time 2026-04-08T00:00:00Z

Output showing the tiered findings:

Security State: NON_COMPLIANT

Findings:
  phi-data-bucket:
    CTL.S3.LOCK.001  UNSAFE  Object Lock not enabled
      Compliance-tagged bucket does not have S3 Object Lock enabled.
      Objects can be deleted or overwritten without WORM protection.

  phi-archive-bucket:
    CTL.S3.LOCK.002  UNSAFE  GOVERNANCE mode instead of COMPLIANCE
      PHI bucket with Object Lock is using GOVERNANCE mode instead of
      COMPLIANCE mode. Users with special permissions can override retention.

  phi-logs-bucket:
    CTL.S3.LOCK.003  UNSAFE  Retention period 365 days < 2190 days required
      PHI bucket Object Lock retention period is shorter than the HIPAA
      minimum of 6 years (2190 days).

HIPAA PHI retention is not "enable versioning and hope for the best." It's a specific requirement: 6 years, tamper-proof, no privileged override. Object Lock in Compliance mode with a 2190-day minimum retention period is the only S3 configuration that satisfies all three conditions. Stave detects each gap independently, so you know where your buckets fall on the compliance spectrum.