Our team grew this year, and the PR volume has grown with it. Certainly faster than the ticket generation. New people means more code moving through the pipeline. It doesn't mean more context. A lot of mass PRs needed real thought just to read through: what problem this is solving, why this approach, what actually matters here versus what's incidental. That gap, more code, same amount of shared understanding, is where our troubles started.
When reviews got harder and slower, people reached for the tool that could help them attempt to keep pace. We have an AI code review bot that runs as an automated reviewer on every PR now, and separately, plenty of us run different LLM and coding agents of choice locally to draft review comments. Put those two together and you get exactly what you'd expect: a bot commenting on a PR, another bot replying to it. Some of the LLM created reported bugs, were created in the first place with total confidence by a model that had no way to know it wasn't real. That is a real drawback about the AI coding assistance, confident wrongness.
What it felt like
Comments stopped sounding like the colleagues who supposedly wrote them. Verbose. Every possible reference and citation included, whether the moment called for it or not. We're a team that talks about code plainly, in our own words, and PRs started filling up with AI slop.
One complaint came up more than once: having to leave the PR to get real context, pinging a person or a model outside the thread because the comment itself didn't actually contain enough, or just way, way too much filler. The entire point of a pull request comment is that you shouldn't have to go anywhere else for it.
Another: duplication, and the cost of re-reading. A paragraph explaining code you could read directly in a minute doesn't save you time. It costs you time, on top of whatever the comment was supposed to save you from working out yourself.
Someone on the team put it plainly: not sure what the right balance is now, but the whole paradigm has clearly shifted. Fair. What's less fair is assuming the way we're doing it right now is the only way it could be done.
The scarce resource
A few weeks into this, our CEO said something in Slack that really got to the heart of the whole problem for me. The PR-review complaints were real, but they were a symptom, not the disease. His point: the actual bottleneck isn't PR review specifically, it's that the finite attention of skilled teammates can be easily overwhelmed by LLM-generated content. Human attention is the scarce resource worth guarding most zealously, and inflicting verbose, cheap-to-produce-but-expensive-to-consume writing on each other should be treated as a real anti-pattern. His term for it was Return-on-Attention, ROA: every word you ask someone else to read has to be worth what it costs them to read it.
He backed it with two examples from that same week. Some ADRs had gone out with a lot of repetition in them, and a colleague had to read through all of it. Terrible ROA. A PR that gitignored a single directory, one line of code, ten characters, came with a description running 1,430 characters. Terrible ROA for whoever had to review it.
Neither example is about code quality. Both are about what you're allowed to cost another person's attention to save yourself thirty seconds of editing.
The same idea has a public version. noslopgrenade.com calls it a "slop grenade": pasting a massive AI-generated response into a chat or email where a human would have written one sentence. It's aimed at chat, not code review, but it's the identical failure. An LLM can produce more words than a task needs, for free, and the cost of that surplus doesn't disappear. It just moves to whoever has to read it.
The part I don't love admitting
I noticed the same pull in myself, not as a reviewer but as a thinker. There were stretches where I'd hand a problem to the model before I'd actually sat with it, and I could feel my own workflow discipline getting looser. Not because the answers were wrong. Because I'd stopped doing the part where I figure things out first and ask questions second.
I'm also mid-fight with a specific habit of Claude's: narrating design decisions into comments and code. "We decided this, not that." It shows up unprompted, and it's never once been useful to me. A decision like that belongs in a PR description or a commit message, where it has context and a date and an author. Sitting in a code comment, it's just noise that will be wrong the next time someone changes their mind and nobody remembers to delete it. I'm building a skill specifically to stop Claude from writing that pattern into a codebase.
Where the line actually is
Bot review isn't the problem. Misusing it is.
It's useful when it gives a reviewer an angle they wouldn't have found on their own, or catches a bug, a missed edge case, a real issue sitting in the diff. That's a second pair of eyes, really helping.
It stops being useful the moment it costs the reader more attention than it saves them. Review exists to ship value and to pass knowledge between people. Code quality matters because it serves that, not the other way around. If a comment leaves the code a little better but leaves the reviewer more drained, that's not a trade worth making.
Attributed to you
Here's a rule I use. When posting a review comment, it's attributed to you. Sure, a coding agent might be a co-author…but that isn't who's going to have to follow up. It isn't the one the PR author is going to come to with questions, or pushback. It's you. The PR author has no way to know how it got written. Sure they probably know, especially if you're slapping em dashes all over the place. They're going to read it as your judgment, in your voice, and hold you to it exactly the way they would if you'd typed every word yourself.
So that's the standard before you hit submit: would you have said this, at this length, in this tone? If the answer is no, the fix isn't a better prompt. It's an edit. Leave off what isn't you.
Top comments (44)
I'm seeing organizations doing random "PR reviews" during weekly standups. The team lead will randomly select a PR from the last week and walk through the code changes with the team with the person whose name is on the PR being the one answering the questions around coding style, edge cases, etc. If the code and PR comments are all AI generated, that's fine, you just need to be able to speak to it and defend it.
Different teams handle the "well, that's just what Claude Code wrote" answers and reasoning differently.
Nice post.
Hmmmm interesting, that probably is a good check on PR Reviews...if a bit nerve wrecking for some devs that might feel called out. I would hope they frame it as a learning opportunity not a pile on anyone 😅
I suppose that it depends on the team's dynamic, but the intent as I most often see it is a learning opportunity, yes.
"Return-on-Attention" is a great way to think about it. One thing I've noticed is that AI doesn't just generate more code, it generates more things to verify. If every suggestion requires careful validation, you've only moved the work around. The best AI tools reduce the number of decisions you have to make, not increase them.
Oh boy does it generate more things to verify!
Return on Attention - That's a good way to see it, better than describing the actual activity, but describing the value of attention as a scarcity.
You guys remember when XMLHttpRequest came out? There we thousands of libraries abstracting it, many were bad, some lived on - axios comes to mind. It's such a vital part of software and it took so long to get it down. Using it was tricky, there were many nuances you needed to know.
Now the abstraction is just await fetch(). We had to get await, and a standard to get a simple abstraction that simplifies understanding it.
With AI we seem to be going the other direction. Everyone is in a frenzy to build stuff, so we're building "long tail" software that aren't good under the hood. We are actually turning attention away from the more important nuances.
I think we need to formalize AI development - like we do programming standards. It would be more fuzzy, but it needs better structure. MCP is a great example, but we need something similar for programming structures.
Before we had design patterns to guide the fuzzy stuff, formal standards that guided abstractiosn and implementations. We need these built in, real AI frameworks that choose these and keep the AI in line.
Like how you'd use package.json to describe your dependencies, we need a schema to describe AI development that is predictable.
I believe that would allow us to automate most of the review - the implementation. We just need to focus on the - did it work (because the implementation is standardized it can be tested).
I'm just rambling but right now it seems we're driving AI blindly most the time - or stuck in review. AI will produce more and more faster - we need to change how we build the structure that produces what's reviewed.
I appreciate your thoughts here! It is always nice to see another dev who has been through multiple generations of software. I still remember J2EE and such and oof full hand generated WSDLs, man those sucked.
Return on attention is the right metric for AI review. More comments are not automatically more review. If the bot creates five weak notes for every one real issue, it turns human attention into the bottleneck. The best AI reviewer may be the one that stays quiet until it has evidence worth interrupting for.
Thank you for sharing such an excellent post. I really enjoyed reading it.
I’m a Python Full-Stack Engineer with over 10 years of experience designing and building scalable software solutions for clients across a variety of industries. Along the way, I’ve learned that successful projects depend not only on strong technical execution but also on creating real business value.
With my recent contract completed, I’m exploring new opportunities to collaborate with professionals who value innovation, practical problem-solving, and long-term partnerships. I enjoy discussing ideas that combine technical excellence with sound business strategy, creating outcomes that benefit everyone involved.
I believe every connection has the potential to become something meaningful. If you're interested in exchanging ideas, exploring opportunities, or simply connecting with someone who enjoys building impactful technology, I'd be happy to hear from you.
Wishing you success in your future endeavors, and I look forward to connecting.
Michael, thanks for reading the post. And best of luck out there with your next gig! If I hear anything Python-wise, I will send the details your way.
Thanks. Christine
If you allow, I wanna discuss about Opportunities and collaboration with you as soon as possible. Of course, this is an excellent opportunity for you.
How about you?
If you have free time, I wanna discuss each other to get income.
Best
The ROI collapse in code review is real, but I'd push back slightly on where the problem originates.
You describe bots reviewing bots â but the underlying issue isn't the AI per se, it's that we're running AI review as a parallel human-reviewer simulation. We give it the same artifact (the PR diff), the same output format (comments), and the same authority level (official review status), and then act surprised when it generates the same noise we were already drowning in.
What if AI review worked as a hypothesis generator rather than a verdict machine? Not "this is wrong" but "consider: what if this error propagates to the billing layer?" â a question only a reviewer with full system context would think to ask.
The trickier failure mode is when the PR is fine but the original ticket was flawed. Should the review bot surface that the problem being solved is the wrong problem? That's where human attention genuinely earns its keep.
Have you seen teams successfully reframe AI review as an async analysis layer rather than a parallel reviewer?
Yea, our teams do consider a good amount of the AI review items as nice to haves or nitpicks, so they really aren't considered blocking. Some of the time, then the dev will just make a follow up pr if they need.
The fatigue makes sense once you count what a reviewer actually has to check per PR now: not just "is this correct" but "is this still the pattern we agreed on," and nothing in the diff tells you which of those two questions you're supposed to be answering. That ambiguity is exhausting even before you get to the line-by-line read.
“Return on attention” is a useful way to name the problem.
A lot of AI review tooling optimizes for finding more things to say, not for preserving reviewer attention. That creates the strange failure mode where the team gets more comments, more apparent rigor, and less shared understanding.
The best pattern I’ve seen is to force AI review output into a smaller contract: severity, evidence, reproduction path, and why a human should care. If it cannot provide those, it should stay silent or ask a question. Otherwise the bot is not reducing review load; it is spending the team’s attention budget on its behalf.
The idea of "Return on Attention" really stands out. As AI-generated reviews become more common, success shouldn't be measured by the number of comments produced but by how quickly a reviewer can make a confident decision. High-quality feedback reduces cognitive load rather than adding to it.
Return on attention is the right metric. AI review can add value, but only if it reduces the human's uncertainty instead of creating another inbox. A good review bot should surface evidence, risk, and suggested next action, not just more fluent commentary to process.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.