DEV Community

Cover image for How to Stop a Coding Agent and an AI Reviewer From Looping on One Pull Request
Mayur Agnani
Mayur Agnani

Posted on Edited on Originally published at theaicommit.com

How to Stop a Coding Agent and an AI Reviewer From Looping on One Pull Request

A coding agent fixes every comment an AI reviewer leaves, so the loop ends only when a review comes back empty. Let only real problems block, and ask for two clean reviews in a row.

Engineer's view. This is "retry until green" on a flaky test suite. The job stops at the first green run, so noise decides when it ends, and one green run proves little. An agent and an AI reviewer stop the same way: at the first review with nothing to fix. It costs rounds and missed bugs.

TL;DR. An agent and an AI reviewer stop looping only when a review finds nothing to fix. Let only new, important problems on changed lines block, and stop after two clean reviews in a row.

Explain Like I'm 5

You ask a friend to check your story for mistakes. Every time they find something, you fix it and hand it back. Some of their notes are real mistakes. Some are just "I'd use a different word." Each fix gives them a fresh page to find new little things on. So you go back and forth all afternoon. Then one time they glance and say "looks fine," and you stop. But a glance can miss a real mistake. Better rules: only fix real mistakes, and stop when they've read it carefully twice and found nothing.

The Problem

You have shipped this loop before, in CI.

A test suite has a few flaky tests. Someone adds a retry: run the suite until it passes, up to five times. Two things go wrong. Builds take longer, because the flaky tests keep failing for no reason. And a real failure that only shows up half the time now gets through. One lucky green run is enough. The stop rule was "the first green run," and noise decided when that came.

A coding agent working with an AI reviewer runs the same loop. The reviewer reads the pull request and leaves comments. The agent fixes every one, because to an agent a comment is an instruction. Each fix is a new commit, so the reviewer runs again. The loop ends at the first review that finds nothing to fix.

Anthropic's Claude Code team, where Claude writes about 90% of the code, measured this. Before they changed their rules, 8.9% of internal pull requests needed five or more review rounds. One pull request ran more than fifty rounds overnight. About 59% of what the reviewer kept flagging in later rounds was code that was already there before the change.

The long tail costs CI time and attention. The quieter cost is the one the flaky suite taught you: a single clean review is weak evidence when the reviewer misses half the bugs on any one pass.

Decide what is allowed to block, and decide when the loop may stop. Block only new, important findings on lines the change touched. Stop after two clean reviews in a row. After three fix rounds, hand the pull request to a person.

Read the full post on The AI Commit

How to Stop a Coding Agent and an AI Reviewer From Looping on One Pull Request carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.


Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

Top comments (0)