Coding agents are fast. That's the problem. An agent that edits twelve files in forty seconds can also break twelve files in forty seconds, and you find out when the tests go red or, worse, when they don't.
I don't think the answer is "don't use agents". The answer is a loop that keeps you in charge. Here's the one I use.
Step 1. Make it plan before it touches anything
Don't write code yet. List the files you'd change, what changes in each, and what could go wrong. Wait for my OK.
Read the plan like a pull request description. Wrong file list? Wrong assumption about the framework? You just saved yourself a 300-line diff. Most agent disasters are visible at this stage.
Step 2. Keep each step small enough to review
One plan item, one change. If the agent proposes "refactor auth and update the API and fix the tests", split it:
Do only item 1 from the plan. Stop after that.
A diff you can read in two minutes gets read. A diff you can't gets approved blind.
Step 3. Make it explain the diff
Explain this diff file by file. Mark anything you're not sure about, and anything you assumed without checking.
This is where invented function names and made-up config options show up. If the explanation is vague, the code usually is too.
Step 4. Verify with something that isn't the agent
The agent saying "this should work now" is not evidence. Evidence is:
- the test suite, run by you or by CI;
- the exact reproduction steps of the bug, repeated;
- for anything touching auth, payments or data deletion, a second review with a fresh prompt:
You didn't write this code. Find the three most likely ways it fails in production.
What goes wrong when you skip a step
- Skip the plan: the agent solves a different problem than the one you had.
- Skip small steps: you can't tell which change broke things.
- Skip the explanation: hallucinated APIs reach your main branch.
- Skip verification: you ship "should work".
A rule for commits
Commit after every verified step. It costs ten seconds and turns "the agent broke something an hour ago" into a one-line revert.
How do you keep agents on a leash? I'd like to hear the rules other people use.
Disclosure: I work on MadGPT, which includes a Coder model and a VS Code agent. The loop above works with any coding agent. Drafted with AI assistance, edited by a human.
Top comments (0)