Last month I watched a teammate demo an agent workflow: connect the model, point it at an open issue, walk away. Ten minutes later the demo ended early, because the agent had helpfully reformatted two hundred files it was never asked to touch. Nothing was broken — the tests even passed — but the diff was unreviewable, and unreviewable is a failure mode of its own.
The lesson I took from that wasn't "write better prompts." It was that an agent's output should land in a quarantine lane first, and only earn its way onto a real branch. This post describes the lane I use: a small Bash gate, a routing rule for which tasks are allowed near it, and a zero-cost setup for the propose-and-retry loop that makes the whole thing affordable. You can reproduce every piece from what's written below.
Why "just review the diff" stops working
Manual diff review assumes two things: that diffs arrive at a pace a human can absorb, and that each diff is small enough to actually read. Agents break both assumptions. A single afternoon of agent-assisted work can produce dozens of candidate patches, and the temptation to skim — or to accept anything with green tests — grows with every one.
So the review has to be mechanical first, human second. Mechanical checks are cheap, consistent, and never get tired at 6pm. The human then only looks at patches that already passed a filter, which means the human's attention goes where it matters.
Routing rule: three lanes, decided before the agent runs
Before any task reaches the agent, it gets assigned a lane. I use three, and the assignment happens in my head in about ten seconds:
- Read-only lane. Summarize this log, explain this traceback, sketch an approach. The agent produces text, nothing touches the filesystem, and the only review is whether the answer is right.
- Quarantine lane. Write or modify code, but the result is a patch file that goes through the gate below before it exists anywhere near a working branch.
- Hands-off lane. Schema changes, CI configuration, anything that deletes or renames across the tree, anything involving credentials files. These stay with a human, full stop. The agent may draft a plan, but it does not execute.
The hands-off lane is the one people resist, so let me argue for it once: the value of an agent on a migration script is bounded (it saves maybe an hour), while the cost of a bad migration script is unbounded (broken deploys, broken rollbacks, a very long evening). Tasks with that payoff shape don't get delegated, regardless of how good the model is.
The gate: a Bash harness with three failure exits
Here's the quarantine lane as a shell script. It clones the repo into a scratch directory, applies the candidate patch there, runs a few deliberately crude checks, then runs your test command under a timeout with the network stubbed out. The source repo is only ever read.
#!/usr/bin/env bash
# quarantine.sh <repo_dir> <patch_file> <test_command>
# Exit 0: patch is a candidate for human review.
# Exit 1/2/3: rejected at the apply, tripwire, or test stage.
set -u
REPO="$1"; PATCH="$2"; TEST_CMD="$3"
SCRATCH="$(mktemp -d)"
trap 'rm -rf "$SCRATCH"' EXIT
# Patch size limit: big diffs get split, not reviewed.
LINES=$(wc -l < "$PATCH")
if [ "$LINES" -gt 400 ]; then
echo "REJECT: ${LINES}-line patch. Split the task into smaller units." >&2
exit 1
fi
cp -r "$REPO" "$SCRATCH/work"
cd "$SCRATCH/work" || exit 1
if ! git apply --check "$PATCH" 2>/dev/null; then
echo "REJECT: patch does not apply to a clean tree." >&2
exit 1
fi
git apply "$PATCH"
# Tripwires: strings that route a patch straight to a human.
# Dumb on purpose — false positives are acceptable, silence is not.
if git diff HEAD | grep -nE '(chmod |chown |base64|eval |/etc/|~/.ssh|rm -rf)' ; then
echo "REJECT: tripwire pattern matched. Human review required." >&2
exit 2
fi
# No new files outside src/ and tests/ without a human looking first.
NEW_FILES=$(git status --porcelain | awk '/^\?\?/ {print $2}')
for f in $NEW_FILES; do
case "$f" in
src/*|tests/*) ;;
*) echo "REJECT: new file outside allowed paths: $f" >&2; exit 2 ;;
esac
done
# Run tests with no network and a hard time limit.
HTTP_PROXY="http://127.0.0.1:9" HTTPS_PROXY="http://127.0.0.1:9" \
timeout 240 bash -c "$TEST_CMD" >/tmp/gate_test.log 2>&1
STATUS=$?
if [ $STATUS -eq 124 ]; then
echo "REJECT: test run hit the 240s limit." >&2
exit 3
elif [ $STATUS -ne 0 ]; then
echo "REJECT: tests failed. Tail of output:" >&2
tail -n 40 /tmp/gate_test.log >&2
exit 3
fi
echo "PASS: $(git diff --stat HEAD | tail -n1). Ready for human review."
exit 0
A few notes on choices that look arbitrary but aren't:
The exit codes are the API. Distinguishing "doesn't apply" (exit 1) from "tripwire hit" (exit 2) from "tests failed" (exit 3) lets the retry loop respond differently: re-generate the patch, escalate to a human, or feed the test output back into the prompt.
Rejecting oversized patches is the highest-value check in the script. Every reliability problem I've had with agents traces back to tasks that were too big. A 400-line cap sounds strict until you notice that patches under it fail in ways you can diagnose in one glance.
The proxy trick is a tripwire, not a wall. Pointing HTTP_PROXY at a dead port makes accidental network calls fail loudly. A process that deliberately unsets its environment can walk right past it — for genuine isolation you want a container with networking disabled. For catching the common case (an agent that decided to pip install something mid-task), it's enough.
Paying for the retry loop with nothing
Here's the part where this usually gets impractical: a gate is only useful if you run candidates through it repeatedly — draft, reject, re-prompt, reject, re-prompt, pass. On a metered API and rented CI runners, that loop has a real price tag, and the natural response is to run it fewer times, which defeats the purpose.
My current answer is to put the entire loop on free infrastructure. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Concretely, I've been using MonkeyCode's free model access for the drafting side of the loop and its free server option to host quarantine.sh as a small always-on job, so neither the model calls nor the compute show up on a bill. I'm intentionally not stating quotas, model names, or availability guarantees — free-tier terms move, and anything I printed here could be stale by the time you read it, so check the current terms yourself before wiring this into something you depend on.
The routing between cheap and expensive resources ends up looking like this:
| Stage of the loop | What runs it | Why |
|---|---|---|
| Read-only lane tasks | Free model | Wrong answers are caught by reading them |
| First drafts in the quarantine lane | Free model | A bad draft costs one rejected gate run |
| Third attempt at the same task | Your best paid model | Two distinct rejections mean the task is hard, not the prompt |
| Hands-off lane + final review of every PASS | A human | Non-negotiable |
That third row matters. Cheap models are a filter, not an oracle — when the free tier fails twice with different reasons, the right move is usually to spend money on one good attempt rather than burn ten more cheap ones.
Where this setup breaks
I want to be plain about the edges, because this is where gate-based workflows get oversold:
-
mktemp -dis not a security boundary. It protects your working tree from a careless patch. It does not protect your machine from hostile code execution. If the threat model includes malicious input, use containers or VMs with actual isolation. - Passing tests certify your test suite, not the patch. Weak tests plus a gate equals confidently accepted wrong code. The gate inherits every blind spot your suite already has.
- Free infrastructure is a dependency with no SLA. Quotas change, services get sunset. The script above takes a patch file as input and doesn't care who generated it — keep that property. The moment your workflow only works with one provider's free tier, you've built on sand.
- One patch at a time. Two agents proposing overlapping patches into the same quarantine lane need merge handling and ordering that this script deliberately doesn't do. Serialize, or build something bigger.
If you're in a regulated environment, or you need audit trails and guaranteed isolation, this isn't the tool — start with purpose-built sandboxed infrastructure instead.
Try it on one task this week
Take a single quarantine-lane task from your backlog, run the loop, and write down one number: how many gate rejections it took to reach a PASS. That count is a more honest measure of your agent setup than any benchmark leaderboard. High rejection counts point at prompt or task-decomposition problems; low counts with suspicious diffs point at tripwires that aren't tuned yet.
If you want somewhere to run the experiment at zero cost, MonkeyCode's free model access plus the free server covers both halves of the loop — the script above is the rest.
One thing I'm still tuning: the tripwire regex. Mine catches the obvious cases and misses an embarrassing one roughly once a month. What's in yours? I'd like to steal some patterns in the comments.
Top comments (0)