A loop that refactors code until a check passes is a few lines of shell. Most guardrails beside such a loop only watch what an agent already wrote; the one worth building refuses a bad edit before it lands, and that is what this piece constructs.
The rule it enforces: production code under src/ must not import a mock library. That rule can live in two places. Put it in the prompt, do not import mock, and it is clear and still only a request: text loaded into a model is a lever on probability, not a switch, so it leaves the model free to write the import anyway. Put it in a gate that fires before a write lands, and it refuses the edit outright. Both state the same rule; only one can make it hold every time.
The ask channel has a ceiling: a prompt pushes the odds up, never to one. How close a prompt gets to that ceiling is measurable work. Reporails reads the asking channel and shows you, with measured evidence, which instructions pull the lever and which are text the model is free to ignore, so you can write the ask against evidence instead of on faith. Even a well-measured prompt tops out below certainty, because the model underneath stays probabilistic. Closing that last gap is what the gate is for.
The loop, and where it can only ask
This is the second component of the loop taken on its own: the gate. The series opener named the loop (generate, check, steer, retry, stop) and posed three questions about the components it turns on. The first is the check, the arm that decides good enough, stop. This piece takes the second: the rules that can refuse a diff, and the difference between a channel that can ask and a channel that can say no.
Here is the loop the series is dissecting. It refactors src/ until a guard holds. The agent never decides it is done; the guard does.
#!/usr/bin/env bash
# work-until-checked: refactor src/ until the guard holds.
# The steering rule lives in the agent's own instructions (CLAUDE.md),
# stated once; the loop feeds back only what changed, the guard's output.
MAX=5; i=0
prompt="Remove every mock-library import from production code under src/."
while [ "$i" -lt "$MAX" ]; do
run_agent --task "$prompt" # GENERATE (rule already in context)
if bash no-mocks.sh; then # CHECK
echo "stop: guard holds after $i retries"; exit 0
fi
prompt="The last attempt still tripped the guard; fix it:
$(bash no-mocks.sh 2>&1)" # STEER: only the new signal
i=$((i + 1))
done
echo "stop: budget exhausted, guard still red"; exit 1
Read the arms one at a time. The steering rule lives once in the agent's own instructions, its CLAUDE.md, so run_agent --task "$prompt" never re-ships it. That first $prompt is the ask, and the rule already in context is text the model reads as a lever on probability, not a switch: Remove every mock-library import moves the odds that the next diff is clean without setting them to one. The STEER arm rewrites $prompt to carry only what changed, the guard's latest output, and asks again against that. A tighter, more specific ask, but an ask.
no-mocks.sh is the only arm that is not asking. It scans src/ after run_agent returns, so it is a detector: it reads the diff the agent already wrote and returns a verdict on it. When the verdict is red, the loop retries. Nothing here stops the bad write from landing in the first place. The guard catches after the fact, and the loop re-rolls. Between the write and that catch, a forbidden import can sit in production code for a whole iteration before the detector sees it.
For a refactor loop that converges, the window is usually harmless, because the next iteration overwrites it. But "usually harmless" is doing a lot of work, and the cases where it is not are exactly the constraints you cared enough to write down twice.
Two channels: one asks, one refuses
A prompt is the channel that can only ask. Everything on the context surface, the task, the agent's instructions, the system prompt, is an input to a probabilistic process. It shifts the distribution of what the model produces, which is why prompt and context engineering are real disciplines. It is also, by construction, unable to force any single outcome. The prompt asks, and any given iteration is free to ignore it.
A gate is the channel that can refuse. It is a deterministic function that fires at a fixed point and returns a hard verdict the model does not get to route around. It does not shift odds; it either lets the diff through or it does not.
The base loop has an asker (the task), a steerer (the retry), and a detector (the guard). It has no refuser. Nothing in it can stop a write before the write happens, so a loop that checks for exactly the thing you forbade can still let it land and catch it a beat too late. Adding the refuser is the rest of this piece.
Build the missing arm: a hook that refuses
Claude Code fires hooks at named transitions in the harness: SessionStart, PreToolUse, PostToolUse, Stop. A PreToolUse hook runs before a tool call executes and can block it. That is the transition to key on, because it is the one point where you can refuse a write before it happens rather than detect it after. Here is where the gate sits in the lifecycle of a single write:
The gate is the diamond, and everything downstream of the allow edge is the write reaching disk. Refuse there and the write never reaches the Tool executes node.
Here is a hook that blocks any Write or Edit whose content adds a mock import under src/. Save it as .claude/hooks/block-mock-imports.sh:
#!/usr/bin/env bash
# block-mock-imports.sh: refuse any Write/Edit that adds a mock import to src/.
# Runs as a Claude Code PreToolUse hook. Exit 2 blocks the tool call.
payload=$(cat) # the tool call arrives as JSON on stdin
path=$(jq -r '.tool_input.file_path // ""' <<<"$payload")
content=$(jq -r '.tool_input.content // .tool_input.new_string // ""' <<<"$payload")
case "$path" in src/*) ;; *) exit 0 ;; esac # only guard production code
if grep -qE '^(import mock|from mock|from unittest import mock)\b' <<<"$content"; then
echo "blocked: '$path' would add a mock import to production code. \
Use a real dependency or a constructor-injected fake." >&2
exit 2 # exit 2 == hard block; the message on stderr goes back to the model
fi
exit 0
Wire it up as a PreToolUse hook matched to the write tools, in .claude/settings.json:
{
"hooks": {
"PreToolUse": [
{
"matcher": "Write|Edit",
"hooks": [
{ "type": "command", "command": ".claude/hooks/block-mock-imports.sh" }
]
}
]
}
}
Now the agent tries to write import mock into src/badcache.py. The transition fires before the write:
The write never happened. src/badcache.py is untouched. The hook read the pending tool call, matched ^import mock in the content the agent was about to write, and returned exit code 2, which Claude Code treats as a block with the stderr message handed back to the model. The model sees the refusal in the same turn and has to try something else. There was no window, because there was no write to catch.
Notice the pattern is anchored: ^import mock, not a bare substring. A docstring that merely names the rule does not start a line with an import, so it does not trip the hook. Anchoring matters more here than in a detector that only reports: a match keyed too broadly refuses clean work, and that carries higher stakes at a gate, which is the next section.
Compare the hook to no-mocks.sh. They match nearly the same rule. What differs is where they sit. The guard sits after the write and reports; the hook sits before the write and refuses. Same constraint, two channels, and only one of them closes the window between the write and the catch.
Which rule goes in which channel
Not every rule belongs in the refusing channel, and filing one in the wrong channel fails in a specific, quiet way in each direction.
A must-hold constraint filed as mere steering gets ignored the one time it matters. The prompt asks, the model is free to decline, and on the iteration where it declines, the diff lands. A must-hold rule in an ask-only channel holds on every iteration except the one where the model declines, and that iteration is the case you filed it for.
A rule that only needed to steer, but is hard-gated, causes friction and false blocks. This is the over-matching failure pointed at a gate instead of a detector, and it costs more here. A gate keyed too broadly refuses clean work: the developer who names the rule in a docstring, the test file that legitimately imports a mock, the production helper named mock_response that a bare substring match trips on. A false alarm from a detector is noise you can ignore for one iteration. A false block from a gate is work that cannot proceed until someone fixes the gate. Refusal is load-bearing in the moment, so an over-broad refuser has a larger blast radius than an over-broad detector.
The heuristic: gate what must hold every time and is cheaply, deterministically checkable; steer what needs judgment or flexibility. no mock import in production code is a fixed, must-hold, one-line-of-grep constraint, so gate it. prefer constructor injection over a service locator is a judgment call with real exceptions, so steer it and let a check observe rather than refuse. A criterion you cannot express as a deterministic match at a transition is a criterion you cannot gate; if it needs a model to judge it, it belongs in the asking channel, with all the probability that implies.
The generator's "done" is a claim to re-derive
Underneath both channels is the posture the series keeps returning to. An agent's done is a claim to re-derive, not a fact to relay. The loop already refuses to trust the generator's report that the work is finished; that is what the check is for. The gate extends the same distrust one step earlier, to the moment of the write. The agent believes its edit is fine. The gate does not accept the belief; it re-derives the verdict from the diff, deterministically, before the edit is allowed to exist. Steering is where you tell the generator what you want. Gating is where you stop taking its word for it.
What holds, and what it still costs to keep loaded
With the hook wired in, the rule you needed to hold every time does. do not import mock holds on every iteration now, on the ones you read and the ones you never open, because a PreToolUse gate does not depend on your attention or the model's cooperation. That is what "hold every time" costs: a channel that can refuse. No amount of rewording the ask gets you there. The prompt still does its job, moving the odds toward a good diff, and the gate makes the one diff you cannot afford impossible. You want both, filed correctly.
One component the opener flagged sits underneath both channels: the context surface. Every rule you steer with is text loaded into the model, and the whole surface is paid for on every turn, whether the model needs a given rule that turn or not. The loop here states its one rule once, but most agents carry far more. What does an instruction cost to keep loaded, and what changes when you load it only where it applies instead of keeping the whole surface in front of the model every turn? That is the question the next piece takes up.
I work on Reporails, deterministic diagnostics and governance for the instruction files, rules, and prompts that steer coding agents. It reads the steering channel, the text you use to ask, and tells you, with measured evidence, where it drifts. The hooks here are the enforcing channel that sits beside it: what a rule refuses at a transition, not what it asks for on the surface.
Top comments (8)
The gate/steer split matches what we converged on after ~8 weeks of running deterministic turn-end hooks on autonomous agent sessions. Two field notes that might be useful:
1. Gates can false-block their own documentation. We run a deterministic check for Cyrillic lookalike characters slipping into English technical words (a real failure mode we observed 4 times in transcripts before promoting it from guidance to a hard check). Two minutes after the promotion commit landed, the gate flagged the rule document describing it — because writing a literal example of the violation is the violation, as far as a regex is concerned. We had to rewrite the rule to point at examples stored outside the gated path. So "over-broad gates false-block legitimate code" has a self-referential family member that is easy to miss at design time: the gate and its own paper trail.
2. Promotion wants a measured threshold, not a mood. @sarracin0's decision-record question is the right one — our answer is a physical counter: a violation observed N times in real transcripts (we use 3) gets promoted from steering to gate, and the promotion commit links every observed instance. The evidence list is the rationale, so the "why was this made a hard block" record falls out for free.
One addition to the classification heuristic: gates work at the exit, not only before tool calls. Our turn-end hook checks whether progress claims in the final message match physical artifacts (mtimes, line counts, permalinks) before the turn may close quietly — the "confident done-message with nothing behind it" failure that LLM judges systematically anchor on.
Curious how reporails handles the promotion side: do you measure ignore-rate per steering instruction, and is there a threshold where you'd recommend graduating one to a gate?
The docstring false-block has a direct analog on the code side: a grep 'import mock' gate fires on # do not import mock in a comment, because grep matches the string and not the statement. Two ways out - anchor the pattern to
^import mockso prose stops matching, or keep the rule's own examples outside the gated paths, which is the route you took. Our forbidden-term gate needs an exclude-list for the same reason: without it, it flags the file that lists the terms.Measuring ignore-rate per instruction is the core of what Reporails reads: how far each line moves the model against how often the model skips it. That's the signal an N=3 rule runs on. The threshold recommendation on top of it isn't built yet; the measurement under it is.
The promotion-commit-links-instances habit is the piece @sarracin0 is chasing from the other end of this thread.
Anchoring wasn't available to us, which is why we ended up on the relocation route: a Cyrillic lookalike can occur inside any English word, so there is no syntactic anchor like
^importseparating statement from prose — the violation and the description of the violation are byte-identical. That's the class where exclude-lists start growing, and the exclude-list itself becomes exactly the aging decision @sarracin0 described upthread. Moving the examples outside the gated path froze ours at one entry.On the threshold side, one datum from our promotion pipeline that might be useful when you build the recommendation layer: frequency alone mispriced our gate. The Cyrillic rule was promoted at N=3 observed violations, and 114 seconds after the promotion commit landed, the gate false-blocked its own rule document. The false-block surface only became visible after graduation. So a graduation signal probably wants a denominator: something like ignore-rate × cost-of-violation ÷ false-block surface. The interesting part is that false-block surface looks statically readable — "does this pattern also match prose" correlates strongly with whether a syntactic anchor exists, which seems like something Reporails could score before anyone promotes.
One measurement question, since you have the per-line signal: can ignore-rate be stratified by position in the run? Our turn-end checks exist because proof-type instructions decay late — followed early in a session, silently skipped in the back half of long runs. A whole-run average would undercount exactly the instructions that most want an exit gate.
Yes, and position is the axis the whole thing gates on. A whole-run average buries exactly the instructions you're pointing at: the proof-type ones (don't close the turn until the done-claim matches a real artifact) decay late, so we read context occupancy as a first-class signal, real token count rather than turn index, and the back half of the run is where the ignore-rate worth reading sits.
What runs on top of that today is re-injection, utilizing the recency law. Once a run crosses a context threshold, the exit-gate instructions that decayed get re-primed, so they land back at full strength in the band where they'd otherwise be skipped. Stratified ignore-rate is the read; the re-inject is what we do with it.
... and yes on the static score: "does this pattern also match prose" is readable before you promote, which is the same admission an exclude-list makes after the fact.
ps.: fun fact, I just released a small article to demonstrate context suffocation today: dev.to/cleverhoods/see-how-ai-inst...
The switch is deterministic in execution, not in correctness. Moving from prompt to hook doesn't remove the judgment call, it relocates it from "will the model comply" to "did I write the check right." Your own guardrail-fired-wrong post is the proof: the anchor bug was a wrong check, not a probabilistic slip, and it still needed a human to notice the false alarm and fix the regex.
Smaller place to be wrong, and a much cheaper one to audit, but not a place where judgment disappears.
The gate that refuses before the write lands is the right move, and "text loaded into a model is a lever on probability, not a switch" is the whole thing. One regress to close before enforce actually means enforce: the gate is a script and the rule is a file, and if both live in the tree the agent edits, refuses-the-edit is a convention until a principal outside the agent holds the readonly bit. An agent that can write to src/ can usually write the guard that checks src/, and then the switch is back to being an ask. The boundary needs the same treatment one level up: who owns the gate, and is that owner outside the agent's reach.
Second, no-mock-in-src is a syntactic gate. It catches the import line, not the hand-rolled fake or the mock reached through dependency injection, so it holds the rule exactly to the depth the violation is spellable. Both real, both worth stating: the gate is only a switch if the producer cannot reach the gate, and only complete where the fault is textual.
The gate versus steer split is the cleanest explanation of ask vs refuse I have read, and the heuristic, must-hold and cheaply checkable goes in the gate, judgment stays in the prompt, is the right cut. The part I would add is that the heuristic itself is a decision that ages. A rule starts as a steer because it looked like judgment, then an incident proves it was actually must-hold, and someone promotes it to a hook mid project. Six months later nobody remembers why that specific grep pattern is a hard block instead of a docstring note, so the next person who finds it over matching either loosens it wrong or leaves a false block problem alone because touching it feels risky. The gate enforces the rule perfectly forever, nothing enforces that the classification, gate or steer, stays reviewed. I keep that kind of call as a decision record next to the rule itself: the constraint, why it got promoted, a status, revisited on a cadence instead of trusted forever, which is the same reasoning behind cowork-os as a decisions layer next to the rules a project runs on. Do you track why a given rule ended up in the refusing channel instead of the asking one, or does that reasoning live only in whoever wrote the hook?
No, Reporails reads the ask side, not the decision log. It scores each instruction by how far the model's output moves with the line present versus absent - the number that tells you a rule gets ignored often enough to belong behind a gate.