DEV Community

Architecting Deterministic Safety Gates for Agentic Workflows

Debashish Ghosal on November 02, 2026

Architecting Deterministic Safety Gates for Agentic Workflows TLDR: The moment you let agents open PRs, mutate infra, or run migrations...
Collapse
 
max_quimby profile image
Max Quimby •

The "machines don't grade their own homework" framing is the right one, and I think the part people underestimate is that separating the planner and the critic isn't automatically enough. If the critic is another LLM fed the same context the planner used, it inherits the same blind spots — it'll happily approve a confidently-wrong plan because it was reasoned into the same corner. The separation that actually buys you safety is the one you name last: policy-as-code running before anything can act, because that's the only link in the chain that's genuinely deterministic and doesn't share the planner's framing.

One thing I'd add from running agents against infra: the hard artifact boundary works best when the plan is emitted as data the policy engine can parse (a structured diff, a Terraform plan JSON), not prose the critic has to re-interpret. The moment the gate has to "read" intent, you've reintroduced the probabilistic link you were trying to remove. How are you representing the plan artifact at the boundary?

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

In agreement with what you said. Emphasizing the 2nd paragraph, this is absolutely important that we treat the agent artifact outputs closer to how logs look that does not require one to re-interpret using another LLM, they should be as close to clear text to read and understand as possible. This is a very important criteria to follow in agentic design. This sort of detailed artifact format is useful for both agents, and humans to read and understand instead of attempting to re-interpret

Collapse
 
panthpatel profile image
Panth Patel •

Our line is drawn by stage, not by model. The orchestrator is a plain script; an agent starts only when there is code to write or a conflict to resolve. It can't push. Its endWork call is refused until its plan checklist is done, the stack is healthy and the merge is clean, and the script pushes only after a human moves the card. Files the script manages are read-only to the agent. Who reads your plan artifacts before execution, a person or the critic?

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

Thank you, your endWork gate design is the shape I would want: the script owns the push, the agent cannot, and the files it manages stay read-only. To your question, it is both, at different thresholds. The critic ranks findings, the policy gates decide pass or fail, and a human approves intent at the plan level by default only when policy says the change is protected, meaning money, prod destructive ops, or a shared blast radius. Routine changes do not wait on a human reading intent, and the protected ones wait on the plan, not on a 400-line diff. The bit I would keep from your setup is that the refusal is mechanical. A clean merge is a git fact, and you can stack a third condition on it, whether this plan touches a protected resource, without asking a human to eyeball risk.

Collapse
 
panthpatel profile image
Panth Patel •

Routine changes skipping the human is where we differ. Every ticket stops twice for a person: the executioner opens the running app at Review, and the lead checks the production preview at Confirm. How does your policy decide shared blast radius, a list of paths or something derived from the code?

Thread Thread
 
debashish_ghosal profile image
Debashish Ghosal •

Derived, not listed — hand-curated path lists rot silently, and the blast radius they guard drifts with every refactor. What's been working: compute the touched modules from the diff, expand one hop through the ownership/import graph, and let category rules set thresholds — anything touching migrations, credentials, or public interfaces escalates regardless of size; everything else scales with dependent count. It's the same argument as config-vs-code in the gate: the moment the map of "what shares this fate" lives in a file somebody maintains, it's already behind the code it describes. (Your twice-per-ticket human stop is stricter than mine, and I suspect you've earned it.)

Thread Thread
 
panthpatel profile image
Panth Patel •

Dependency order is the one part we derive too. A change at Envizom usually touches 6 repos, so the merge step checks that every PR is mergeable first, then merges libraries before the repos that use them, and a conflict never leaves half the repos merged. Does your one-hop expansion cross repo boundaries, or stop at the repo edge?

Thread Thread
 
debashish_ghosal profile image
Debashish Ghosal •

It stops at the repo edge today, and your merge step is the honest answer for the space beyond it: check every PR mergeable first, merge libraries before the repos that use them, and never let a conflict leave half the estate merged. Dependency order across repos is a different problem from blast radius within one, because the graph lives in no single place the tooling can read, and a one-hop expansion that pretended otherwise would be a hand-curated list with extra confidence. What I would add on top of your sequence is the plan artifact evaluated before execution across all six repos at once, because ordering saves you from half-merged, but only the plan-level review catches the rule nobody wrote down. At six repos per change, that review is cheap compared to the alternative.

Collapse
 
paullovelace profile image
Paul Lovelace •

"Machines don't grade their own homework" is the line, but the critic inheriting the planner's blind spot is the one that worries me more — separating them on paper doesn't separate the context. The artifact-boundary idea feels like the actual fix: something a human can read without another model translating it. Do you enforce that boundary in CI, or is it still a convention people can skip when they're in a hurry?

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

Thank you, and you are right that separating them on paper does not separate the context. I think that is the honest limit of the planner and critic fix: different model, different system prompt, different role, but both can share the same blind spot if you feed them the same summary. That is why the artifact boundary, not the topology, is the load-bearing part for me. To your question, it is enforced rather than convention. The gates run pre-merge and pre-execution as policy-as-code, and the property that matters is that a policy rule does not read the agent's explanation of why the change is good. It evaluates resulting state against declared policy, so you cannot talk your way past it, which is exactly why it survives a deadline. The plan artifact is the object those gates evaluate, not a document a human is supposed to remember to open.