The volume of AI-generated code is the question every eng manager is asking now, and the common answer is "have an AI review it." But teams keep wiring it up as: the same model that wrote the diff reviews the diff. That is the one setup an AI code review cannot be trusted in, and it's an eval-design issue, not a quality-of-tool issue.
A model reviewing its own output is one blind spot measured twice. It doesn't know what it got wrong in generation because it has no independent read of the code, it has the same priors, the same truncation, the same confidence in the same places. If the generator thought the happy path covered the edge case, the reviewer probably thinks so too, because it's the same weights. Studies on LLM-as-judge (herding, correlated judges) show exactly this: when the judge shares lineage with the thing being judged, N votes is really one opinion counted N times. The self-review loop is the same failure, applied to code.
Three things that actually break the correlated blind spot:
Cross-model review. Have a different model read the diff than the one that wrote it. Different training lineage, different priors, and a genuine chance it catches what the generator assumed. This is cheap to do with agent APIs and it's the single biggest correction you can make.
Independent verification. Tests are the review tool that has no opinion. If the code and its tests come from the same file, they share a blind spot, so look at what the tests never assert and whether anything outside the diff gets touched. A build, a type check, a static-analysis pass that aren't part of the model's output count as signal.
Pin your harness. Review quality is model plus scaffold, so write down what carried context into the review, what tool loop the reviewer ran, and what judge decided "pass." If the score moves, you want to know whether the model changed or the harness did.
The fix for AI code volume isn't a better self-reviewer. It's making sure the thing that reviews the code is not the thing that wrote it.
Top comments (1)
The correlation is located in the weights here, so the top fix is different weights. But in a working review pipeline most of it lives in the inputs, and swapping lineage leaves that part alone. Look at what the reviewer actually receives: the diff, the commit message, the PR description, the docstrings, the test names. Almost all of it is the generator's own output. A second model of entirely different lineage still reads the generator's account of what the change was supposed to do, mistaken premise included. And the most common generation defect is a misread requirement rather than a coding slip. Hand that framing to reviewer B and the misread transfers cleanly. Shared priors never enter into it.
Point 2 quietly undermines the ranking in point 1. What makes a build or a type check strong is that its verdict is producible without consulting the artifact's account of itself. A compiler never sees the PR description. Being non-model is incidental; the evidence path is what carries the independence. Score the list on that property and cross-model review does poorly, because both models are working from the same story about intent, while a type check does well despite looking like the trivial item. So the one change described as the single biggest correction is also the only one that leaves the dominant correlation in place.
There is a cheap control that would settle this for any given setup. Collect diffs carrying a known injected defect that contradicts the stated requirement, then run reviewer B against each one twice. First pass keeps the generator-authored framing intact, PR description and commit message and generated test names. Second pass restates the requirement from the original ticket, drops the generator's prose, and renames the tests to neutral identifiers. If detection rate moves between the two passes, reviewer B was leaning on framing and the lineage swap is buying less than the wiring suggests. If it holds steady, cross-model review earned the top slot.
Does your cross-model setup put the generator's PR description and commit message in front of the reviewer at all? That is the doorway the shared premise walks through.