More code is being written by AI now.
Oddly, reading it is not the tiring part.
The tiring part is this:
“Maybe I should check one more thing.”
And then another thing.
A review can continue forever if nobody gives it permission to end.
At some point, I realized I was asking the wrong question.
I kept asking the AI to find more problems.
But the real problem wasn’t finding things.
It was deciding which findings still deserved attention.
So I started sorting them into three boxes.
Three boxes
- A human must decide
- Safe to skip
- Not verified yet
The second box is the interesting one.
“Looks fine” is not enough.
If the AI says something is safe to skip, it has to explain what it actually checked: ran the code, counted all references, found a rule in documentation, or proved something from the structure.
And it has to point to path:line.
That changes the job.
Instead of solving the problem again, I can look at the evidence and ask a much cheaper question:
“Yes, is that actually there?”
Sometimes.
Because there’s a catch.
An AI can also confidently write “structurally impossible” next to a very convincing wrong line number.
A citation is not truth.
It is only a place to look.
And “safe to skip” probably won’t feel safe at first.
I’ll still read that pile.
Probably for a while.
So this won’t save time on Tuesday.
Maybe after three weeks.
But I think that’s the useful part of the idea.
AI code review doesn’t necessarily need to find fewer things. It needs a rule for stopping.
Tomorrow, I’d try only this:
Sort findings into human decision / safe to skip / unverified.
Anything “safe” needs evidence andpath:line. Otherwise, it isn’t safe yet.
AI is very good at continuing.
Humans are too, unfortunately.
Someone has to say, “Enough.”
Top comments (2)
"A citation is not truth. It is only a place to look." That line earns the whole post. In our review setup, what eventually made the "safe to skip" pile trustworthy was a second adversarial pass of our own: an independent check whose only job is to falsify each safe-to-skip claim. Because the finding already names path:line and what was checked, that job is bounded - follow the citation, try to break the claim, stamp it CONFIRMED or leave it PLAUSIBLE. Humans then read only the PLAUSIBLE pile, which turns out to be the minority.
The other cheap signal we feed your box 1: run two reviewers independently and diff their verdicts. Wherever they disagree, that finding was never "safe to skip" no matter what either one wrote next to it. Agreement is not proof, but disagreement is a reliable flag for "a human must decide", and it costs one extra model call instead of a policy document.
This is really useful. I like the idea of turning “safe to skip” into something that has to survive an independent falsification pass, rather than treating the first citation as enough.
And the disagreement signal is especially clean: agreement isn’t proof, but disagreement is a very cheap reason to escalate to a human.
That feels like a much more concrete stopping rule than what I had in mind. Thanks for pushing the idea further.