Don't Review Agent Code. Interrogate It.

Lightman Miao — Sun, 25 Oct 2026 04:08:11 +0000

I don't really review agent-generated code anymore, and I doubt most of you do either. We can keep this between us.

An agent can write 100K lines or open 30 PRs in a day. Nobody has time to read that. The obvious bugs are mostly gone by the time I see the PR anyway, because the agent already checked its own work a few times while building it. It's very proud of that.

So the usual move is to start a fresh agent, tell it "review this PR", skim what it says, and merge.

I did that for a while. For one whole feature I even had Claude and Codex cross-review each other's PRs, which felt very responsible at the time. After 30 merged PRs, the agent handed the feature back to me for manual testing, and the whole thing was completely over-engineered. It was full of verification steps and strict validation I never asked for, and every one of those PRs had passed a cross-review.

Two agents politely approving each other's work for 30 PRs. I had built a very efficient LGTM machine.

The problem with a fresh reviewer agent

The reviewer starts with nothing. It doesn't know why the PR exists or what the feature was supposed to cover, so it can't tell when the code wanders past the scope. It has no idea what you're building next month either, so it can't judge whether an abstraction will ever pay off.

What bugs me most is how it handles over-engineering. Hand it a PR with a generic retry framework or a brand new caching layer and it assumes somebody had a good reason. It might suggest a rename or one more test. It almost never asks whether the thing should exist.

You end up with a review that looks thorough, full of small comments, and the extra code goes through untouched.

What I do now

I've been doing this for about a year, and it's how I catch most of the over-engineered code now. I still don't read most of the diff. I treat the reviewer like a witness: it reads the code, and I ask the questions.

1. Ask what the PR does

Fresh agent, and a prompt like this:

Read this PR and give me a report on exactly what it changes. List:

- Database schema changes (new tables, columns, indexes, migrations)
- Scheduled job / cron / background worker changes
- A summary of every feature added or modified
- New external services, APIs, or third-party libraries
- New environment variables or config flags
- New public endpoints or interfaces
- Anything deleted or deprecated

Don't judge the code yet. Just tell me what it does.

The last line is there on purpose. I want the facts before any opinions. Left alone, it will happily jump straight to "Overall this is a clean, well-structured PR."

2. Find the vague parts

The report is usually long and looks detailed. Read it slowly and you'll hit sentences like these (made up, but you'll recognize the type):

"Adds a sync_state table to support improved synchronization reliability."

"Introduces a background job for periodic maintenance."

"Adds FEATURE_X_MODE to allow flexible configuration."

When the agent can't say plainly why something exists, there's a decent chance nobody asked for it.

One time I asked the implementer agent to "send a notice to the admin if a third-party service is down and returning error codes." It added two new tables to track Slack notification and email status. I never asked for either. I wanted a message. I got a data model.

Agents love tables the way some people love storage bins. There is always room for one more.

3. Interrogate

Now push on anything that doesn't make sense to you. Pull up a chair, point the desk lamp at it, and start asking:

"Talk to me about this new table. Who asked for it? Because it wasn't me. Why can't the existing table handle this?"
"This cron job. What does it do every night at 2 a.m.? We already have a service doing exactly that. Explain."
"FEATURE_X_MODE. Name one person who will ever set it to anything other than the default. I'll wait."
"This new dependency showed up out of nowhere. What's it doing here? Something we already use can do this job. Convince me otherwise."
"Walk me through it. An invalid input comes in. Where exactly does it get rejected? Which test proves it? Show me the line."
"The feature was supposed to do X. That's it. So which parts of this PR were not part of X? Take your time."

Be ready for "You're absolutely right!" on the first question. Keep going anyway. Agreeing with you is easy for it. Explaining why the table exists is the part that takes work.

Then slide the evidence across the table, the stuff it couldn't have known:

"We will only ever have one region. So tell me again why we need this abstraction."

"We're going to plug in more external tools like this in the next phase. Does the current design make that easy?"

That's the point where the reviewer starts being useful. Once it knows the scope and the plan, it points at real problems, and the comments about variable names mostly go away.

Why it works

The agent can read 3,000 lines in a few seconds, but it has no idea what's supposed to be there. I can't read 3,000 lines for every PR, but I know what the feature is for and where the product is headed. The questions are where those two meet.

Me, a few questions in, finally understanding why there's a cron job.

By the end I have a pretty clear picture of what's going into the codebase, and I still haven't read most of the diff.

Short version

Ask the agent what the PR changes, not whether it's good. Look for the vague answers. Push on every table, job, env var, and dependency you can't explain, and tell the agent what it doesn't know. Whatever nobody can justify gets cut. The agent will say you're absolutely right about that too.

Developers "Before" vs "Today"

Lightman Miao — Thu, 22 Oct 2026 06:28:42 +0000

Before: one dev, one keyboard, and a hallway error logs full of NullPointerExceptions. You took them down one at a time, and every fix was earned.

Today: we're strapped into a mech with unlimited firepower, and the swarm is still coming: hallucinations, context compaction, and an agent that apologizes and then makes the same mistake again.

The bugs got smarter and so did the weapons, but we're still fighting.

What's the enemy you miss least, and which one do you fight most now? 👇

DEV Community: Lightman Miao