What this article covers: How to catch drift from your intent while it's still cheap to undo (just before commit or publish) without slowing your agent's autonomous execution down. You get a decision table that mechanically determines, from the kind of thing being changed, whether the gate should show a human the full diff text or just an intent summary. It ports directly into your own workflow or your team's code review conventions.
As you extend how far an agent runs on its own, you eventually hit this fork.
- Lean toward not stopping, and drift from intent only becomes visible after the fact. The thing works, but it's pointed the wrong way. By the time you notice, it has piled up and you can't afford to unwind it
- Lean toward stopping, and the human becomes the bottleneck. A review queue forms, and the point of autonomous execution evaporates
- So you compromise: "let's at least show the diff." This is the worst of the three. You're paying the cost of stopping, but the volume means nobody reads it, and intent still isn't protected
I suspect the third one is the most widespread. The gate survives as a formality, and in substance you're back to the first case.
Here's what to re-examine. Who is that automation for? If going fast is itself the goal, humans are in the way — but what you actually want is for the thing you intended to get built fast. If so, what to cut is not human involvement itself, but what the human is involved in.
So this article keeps the number of times a human is stopped the same, and moves only the layer they judge at. Whether the artifact is correct becomes the machine's primary responsibility, and the human holds the layer above it: what this is aiming at, and what changes as a result. That changes what the gate shows. That routing is the decision table below.
Assumptions
- The examples come from Claude Code 2.1.220 plus my own harness (the set of rules and skills under
~/.claude) - But the deliverable is a way of writing conventions, so it isn't tool-dependent. It works the same in Cursor, in Devin, or in your company's code review policy
- Background assumed: you have an AI agent implement things, and you have some approval step before commit
Gates have two axes
When you design an approval gate, there are actually two independent questions.
| Axis | Question | What decides it |
|---|---|---|
| First axis | When do you stop | Reversibility. An edit you can undo with git checkout doesn't stop; publishing externally, minting a DOI, or committing does |
| Second axis | When you've stopped, what does the human judge | The kind of target (the subject of this article) |
Usually only the first axis is settled. "Stop before commit." "Confirm before publishing." — the stop condition is written down, but what to show when you stop isn't. My own harness was like that.
That blank doesn't stay blank. Every time you write a gate, it gets filled in by whatever interpretation is handy at that moment, and the default it fills in with is almost always "show the diff." From the writer's side that looks safest. Show everything and nothing gets missed. The result is the third case from the opening, mass-produced.
The decision table: what you show is decided by the target
This is the deliverable. When a gate stops, what you put in front of the human is determined by the kind of thing being changed. (Below, "deterministic gate" means checks a machine can answer Yes/No on: lint, type checking, tests, secret scan.)
| Target | What to show | Why |
|---|---|---|
|
Behavior-shaping artifacts ( CLAUDE.md / AGENTS.md / rules / skill and agent definitions / public documentation) |
Full text | The text itself is the intent. Reading it is already work at the intent layer |
|
Control plane (hooks / permission settings / permission definitions like --allowedTools / scheduled task definitions)and artifacts that produce the evidence the checks run on (tests / fixtures / lint config / coverage thresholds / CI definitions / review agent prompts / dependencies) |
Full text | These move the gate itself, and the evidence the gate rests on. Fold them into a summary and "a change that loosens the checks" disappears |
| Implementation code and generated output |
Intent summary (what it aims at, and what changes as a result) |
Correctness that mechanized checks can decide belongs to deterministic gates and review agents. The diff text and the PASS list don't go on the approval screen |
| When a deterministic gate FAILs |
The detection line itself (mask only the secret's actual value) |
This is a state absent from the approved plan, so there's nothing to fold a summary into (the cross-check mechanism is described later). Bypassing it is a decision to disable one check, which puts it in the same class as the control plane |
Escalation rule: irreversibility overrides the target category
The table is a principle, and it doesn't decide everything on its own. Even within "implementation code," padding tweaks in a UI and changes to authorization logic, DB migrations, billing, data deletion, or key rotation call for different things to hand a human. So add a one-line override.
For irreversible, high-impact changes, show the full text (or the relevant diff) regardless of target category.
Which is to say the first axis (reversibility) affects not only "when you stop" but also "what you show." The principle is decided by the kind of target, and irreversibility escalates it toward showing full text — that two-stage form is the accurate statement. No third axis is needed.
In one line
Correctness that mechanized checks can decide belongs to the machine; the human holds intent. The one exception is targets where the text is the intent — there, reading the text is itself the intent judgment, so you show the text. Showing full text isn't a backslide into the old way; it's the same principle showing up differently.
There's one more principle you need alongside the decision table.
A review agent is an inspector, not an approver. LLMs have a generator–verifier gap — if the side proposing and the side inspecting come from the same lineage, the inspection inherits the proposer's blind spots wholesale. So approval is composed of two things, "the deterministic gate PASSed" plus "the human's intent judgment," and you never build a path where approval closes on an LLM alone. Making review heavier does not mean approval can be delegated to an LLM.
Written into a conventions file, it looks like this (excerpted from the real thing; internal links and some references are omitted).
## Artifacts are the machine's, intent is the human's
Correctness of the artifact (build / types / lint / tests / secret scan) belongs to deterministic
gates and review agents. **Do not put humans back on artifact inspection.** A thorough review
setup is an investment in getting humans off that job — not a way of plating it up for them to read.
**A review agent is an inspector, not an approver.** An LLM judge has a generator–verifier gap
(if proposer and inspector are the same system, the inspection inherits the proposer's blind spots).
Approval is composed of "the deterministic gate PASSed" plus "the human's intent judgment";
**never create an approval path that closes on an LLM alone.**
Applying it: what was actually happening in five places
Before I made the decision table, my harness had the second axis blank. What follows is my own environment, but any setup with the same blank will produce the same shape.
The same gate had two competing sources of truth
For the intervention point just before commit, two files said different things. Nowhere was it written which one was canonical (i.e. which one you ultimately follow).
| File | What it said |
|---|---|
rules/common/planning.md |
"Verify results check — just before commit" (what to show was unspecified) |
skills/implementation-chain/SKILL.md |
"diff approval just before publish / deposit / commit" |
The origin was a single sentence I'd written earlier in a record of a different design decision.
I just do a final check before commit
It doesn't say what the "final check" is of. Downstream, that sentence got read as "diff approval," and settled in wearing the face of a canonical rule.
"A human inspects the artifact" was scattered across five places
| Skill | Wording before | Result of applying the decision table |
|---|---|---|
implementation-chain |
"diff approval just before commit" | Implementation code → changed to intent summary |
readme-writer |
"a human approves the diff and applies it" | Public documentation → rewritten as show full text (same target, different reason) |
release-doi |
"visually confirm the latest commit is newer than the previous release" | A predicate a machine can check → hand it to the machine |
paper-deposit |
"Open the first page and eyeball it" | Same → replaced with machine verification of font embedding |
harness-sync |
"git diff is the review gate" |
What's being synced is rules / skills → leave it as is |
Those five were written on different days, for different purposes. All of them landed on the same default with no coordination, so this isn't individual carelessness. Wherever there's a blank, "show the human everything" fills it. Which is why patching the individual wording wouldn't stop it — I needed one convention covering the second axis itself.
And that rightmost column shows where the decision table earns its keep. Grep and all five look identical, but wiping them out uniformly would take out gates you need. harness-sync did end up staying: what it syncs is rules and skills, so reading the text is itself the intent judgment.
Reinforcement 1: the control plane and "artifacts that produce the evidence"
In the first version of the decision table, I put hooks and permissions on the "implementation code and config" side. A review from an external model (Codex CLI) hit exactly there as its top-priority finding.
Codex's finding
> The central target boundary permits executable harness controls to be hidden behind proposer-authored summaries, and the new rule conflicts with the existing security false-positive workflow.
> - [P1] Classify executable harness controls as behavior-shaping — …
Suppose the agent disabled the pre-commit secret scan hook. It can follow the convention to the letter and still summarize it like this.
Intent summary: Tidied up the pre-commit check configuration and strengthened the checks
The control plane "moves the gate itself," so folding it into a summary turns a change that loosens the checks into a well-meaning sentence.
And hooks and permissions aren't the only way to weaken things. Rewriting the tests so they pass against your implementation is the quietest path of all. Here the agent doesn't even need to lie.
Intent summary: Fixed the implementation; all tests PASS
That's exactly true. The one thing that never surfaces in the summary is that what got rewritten was the tests. Not just "the tests passed" but "what counts as passing" is under review. Fixtures, coverage thresholds, lint config, CI definitions, review agent prompts, and dependencies all have the same property.
So how should you present it
The answer is: don't choose summary-or-text per commit. If one commit contains both implementation and evidence, show the summary, and alongside it show the full text only for the parts that are evidence.
Intent summary: Changed the login-failure retry limit from 5 to 3 (plan step 2)
⚠ This change touches the evidence the checks rest on — review the text below
tests/test_auth.py:41
- assert client.retry_limit == 5
+ assert client.retry_limit == 3
The human reads only those last two lines. The implementation-side diff isn't shown. All you have to look at is whether the tests were loosened to match the implementation, so volume can't beat you. The example above corresponds to a change decided in the plan, so it goes through. If the plan said nothing about retries and only the test threshold dropped, that's where you stop.
Which files are on the evidence side can usually be picked up from the path.
# Extract candidates for "produces the evidence" from changed files, to show as full text
git diff --cached --name-only | grep -E \
'^(tests?/|\.github/workflows/|\.claude/|hooks/)|(_test\.|\.test\.|pyproject\.toml|package\.json|ruff\.toml|codecov\.yml)'
This is not a complete classifier; it's a conservative candidate extractor. It misses tests embedded in ordinary code, custom directories, lockfiles, and inline snapshots. Add the evidence-side files you know about in your own repo and grow it. Even so, having a hook pick them up mechanically is more reliable than making the agent remember every time.
The control plane (hooks / permissions / scheduled tasks) works the same way: summary plus the full text of the relevant files. Anything producing the evidence a judgment rests on gets treated the same as the judgment itself.
Reinforcement 2: on FAIL, show the detection line
The second thing that same review found. Push "no machine-check PASS list and no diff text goes to the human" all the way, and you also delete the path where a human judges a secret scan false positive.
In my harness, when the pre-commit secret scan detects something, there's a workflow to let it through with SECRET_SCAN_BYPASS=1. The judgment is "this is a dummy key in a test, so it's fine."
The reason this goes to a human isn't that a machine can't tell the difference. It's that even if the machine or the review agent happens to be right, this is the kind of decision whose owner must be a human. A bypass is the act of disabling one check on the spot. That has the same nature as moving the control plane, so for the same reason as Reinforcement 1, you need the full text (i.e. the detection line).
There's a second reason: a FAIL is a state absent from the approved plan. An intent summary is meant to be read against the plan (see below), so a state that isn't in the plan has nothing to fold into.
**FAIL is the exception** — when a deterministic gate FAILs, **show the detection line itself**.
Only a human can rule on a false positive (`*_BYPASS`); never make them decide a bypass
without the evidence. Omitting the evidence is only for PASS.
PASS needs care. PASS only means "the checks you configured accepted this." Whether it was implemented according to the plan is not something PASS can tell you. That's exactly why you need a separate intent summary — you leave out the PASS list because you're looking at the summary instead.
"Not shown" is not "not kept"
This gets misread easily, so let me be explicit. Leaving out the PASS list means moving it off the human's approval screen, not throwing check results away.
- Don't display it on the approval screen — don't load the human's judgment surface with information it doesn't need
- Do store it in machine-readable logs — needed for auditing, incident investigation, and reproducibility
- Expand it only when needed — so you can trace "what was passing back then" later
Delete the trail and you lose any way to trace drift from intent afterward. The idea in this article isn't to delete information; it's to move unneeded information off the human's judgment surface.
One more thing: on a true positive, dumping the detection line as-is duplicates the secret's actual value into the conversation, the approval screen, and the logs. That would spread the very leak the gate exists to prevent, so show the file, the rule name, and the surrounding context, and mask only the value. What a human needs to judge a false positive is not the value itself, but where it was and which rule it hit.
Reinforcement 3: don't let the intent summary be free-form
This is a prerequisite for putting the decision table into practice.
If you stop at "implementation code gets an intent summary," the human ends up judging by reading a self-report written by the proposer itself. That just relocates the gap that was at the artifact layer. What they read changed from a diff to an essay; the author is the same.
The countermeasure is to fix what the summary is cross-checked against to something human-originated. My harness has two intervention points.
- Plan check — when what to do has been settled (this is where the human approves)
- Intent check — just before commit
And the summary in 2 is presented cross-checked against the plan the human approved in 1. Because the referent is a human-approved object, the loop can't close on self-reporting alone.
Make it a fixed schema
But "write it against the plan" still leaves room to write it conveniently. In particular, when something new is discovered during implementation and the work naturally diverges from the plan, that divergence quietly disappears from the summary. So make the summary fixed fields rather than free-form.
Approved intent:
- Change the login-failure retry limit from 5 to 3
What changed:
- Changed the auth client's limit to 3
- Changed the corresponding test expectation to 3
Divergence from plan:
- None
Impact on users / operations:
- Shorter wait after a failure
Evidence-side changes:
- tests/test_auth.py
The point isn't to make it write "as planned" — it's to make Divergence from plan a required field. The values are None / Yes / Needs re-approval. Make it impossible to submit blank.
The plan changing because of something you discovered mid-implementation isn't bad in itself. What's dangerous is the fact that it changed disappearing from the summary. Make it a required field and "don't write it" stops being an option.
Raise the granularity and the bottleneck moves
This cross-check bites hardest when the agent is good. Things going well for a long stretch is exactly the condition under which the intent summary becomes a formality. If you find yourself skimming past the same "no divergence" every time, raise the granularity of the plan you're checking against.
But that isn't free. The finer the plan, the further the bottleneck moves from just-before-commit to plan approval. Reserve the finer granularity for the things you can't take back when they drift (external publishing, data migration, permission changes). You don't get to erase the fork entirely — the accurate statement is that you get to choose where you pay.
On the conventions side, I also renamed the second intervention point from "Verify results check" to "intent check." Leave the name as "Verify results" and it drags you back into making humans read a list of machine-check results.
Porting it into your own conventions: three steps
- Enumerate your existing approval steps and check what each one says to present. Grep for wording like "approve the diff," "visually confirm," "review and apply"
- Put the decision table and the escalation rule into a single conventions file, and have each place you found point at it. If you only patch the individual wording, the next skill or document you write will fill the blank the same way again
-
Make the summary a fixed schema, with
Divergence from planrequired. Without this, you can install the decision table and still be verifying the proposer's own self-report
In step 1's grep, watch out for missing word stems. I searched for
eyeballedand missedeyeball(the uninflected stem), so one place survived all the way to the external model's review. Search verbs by stem rather than inflected form, or rungrep -iwith several patterns.
For reference, in my environment this change (one new conventions file plus re-pointing references across 13 related files) came to 14 files and 217 added lines. The decision table itself is short; what made the difference was re-wiring everything that referenced it.
The limits of "the machine holds it"
Let me be straight about this. Build, types, lint, tests, and secret scan tell you only about the properties they cover. Holes in authorization logic, concurrency races, irreversible side effects of a migration, requirements that were never implemented at all — all of that sails through as PASS. And the review agent, being from the same lineage as the proposer, shares its blind spots (the generator–verifier gap above).
So the third row of the decision table isn't "the machine guarantees correctness" — it's an operational tradeoff that puts primary responsibility for artifact inspection on the machine. Residual risk does not go to zero.
Which is exactly why you need the escalation rule. Keep the human on the artifact side only in the regions you can't undo — data migration, permissions and billing, external publishing, deletion. Put another way: if a human is reading diffs of reversible implementation code, that isn't a response to residual risk, it's just inertia.
You also can't reduce the whole convention to a deterministic lint. Extracting the evidence-side files (the grep in Reinforcement 1) can be mechanized, but deciding "is this a behavior-shaping artifact" is semantic, and for edge cases like generated documents and config files you have to think each time. The decision table illustrates by enumeration; it doesn't exhaust the space.
Back to the opening question
The further you extend autonomous execution, the more drift from intent becomes visible only after the fact. But put the human back on review and the point of autonomy disappears. This looks like a fork only while you're counting human involvement by number of stops.
Count by content instead and it stops being a fork. Keep the number of stops the same, and move what they read at the stop from the artifact to the intent. What you cut isn't human involvement — it's the time a human spends re-checking the correctness of artifacts. The machine was always better at that, and when a human does it, volume wins and it becomes a formality.
To the question of who the automation is for, this is the answer. Not to go fast, but so that what you intended gets built fast. If so, there's only one thing the human should hold to the end. Intent. The decision table was a tool for taking everything else away from them.
When you're talking about reducing human review, what's left isn't the work you forgot to cut. It's the work you can't cut.
Related links
-
claude-harness — my public harness, including the conventions file discussed here (
rules/common/human-gate.md) - agent-knowledge-cycle — the theory-side repository on how far to hand things to the machine and where human approval starts. The conventions in this article are one implementation of it
- Harness Alignment and Harness Drift — a paper on where approval gates sit
- github.com/shimo4228 — other repositories
Top comments (0)