Our team grew this year, and the PR volume has grown with it. Certainly faster than the ticket generation. New people means more code moving throug...
For further actions, you may consider blocking this person and/or reporting abuse
I'm seeing organizations doing random "PR reviews" during weekly standups. The team lead will randomly select a PR from the last week and walk through the code changes with the team with the person whose name is on the PR being the one answering the questions around coding style, edge cases, etc. If the code and PR comments are all AI generated, that's fine, you just need to be able to speak to it and defend it.
Different teams handle the "well, that's just what Claude Code wrote" answers and reasoning differently.
Nice post.
Hmmmm interesting, that probably is a good check on PR Reviews...if a bit nerve wrecking for some devs that might feel called out. I would hope they frame it as a learning opportunity not a pile on anyone ๐
I suppose that it depends on the team's dynamic, but the intent as I most often see it is a learning opportunity, yes.
"Return-on-Attention" is a great way to think about it. One thing I've noticed is that AI doesn't just generate more code, it generates more things to verify. If every suggestion requires careful validation, you've only moved the work around. The best AI tools reduce the number of decisions you have to make, not increase them.
Oh boy does it generate more things to verify!
Return on Attention - That's a good way to see it, better than describing the actual activity, but describing the value of attention as a scarcity.
You guys remember when XMLHttpRequest came out? There we thousands of libraries abstracting it, many were bad, some lived on - axios comes to mind. It's such a vital part of software and it took so long to get it down. Using it was tricky, there were many nuances you needed to know.
Now the abstraction is just await fetch(). We had to get await, and a standard to get a simple abstraction that simplifies understanding it.
With AI we seem to be going the other direction. Everyone is in a frenzy to build stuff, so we're building "long tail" software that aren't good under the hood. We are actually turning attention away from the more important nuances.
I think we need to formalize AI development - like we do programming standards. It would be more fuzzy, but it needs better structure. MCP is a great example, but we need something similar for programming structures.
Before we had design patterns to guide the fuzzy stuff, formal standards that guided abstractiosn and implementations. We need these built in, real AI frameworks that choose these and keep the AI in line.
Like how you'd use package.json to describe your dependencies, we need a schema to describe AI development that is predictable.
I believe that would allow us to automate most of the review - the implementation. We just need to focus on the - did it work (because the implementation is standardized it can be tested).
I'm just rambling but right now it seems we're driving AI blindly most the time - or stuck in review. AI will produce more and more faster - we need to change how we build the structure that produces what's reviewed.
I appreciate your thoughts here! It is always nice to see another dev who has been through multiple generations of software. I still remember J2EE and such and oof full hand generated WSDLs, man those sucked.
Return on attention is the right metric for AI review. More comments are not automatically more review. If the bot creates five weak notes for every one real issue, it turns human attention into the bottleneck. The best AI reviewer may be the one that stays quiet until it has evidence worth interrupting for.
Thank you for sharing such an excellent post. I really enjoyed reading it.
Iโm a Python Full-Stack Engineer with over 10 years of experience designing and building scalable software solutions for clients across a variety of industries. Along the way, Iโve learned that successful projects depend not only on strong technical execution but also on creating real business value.
With my recent contract completed, Iโm exploring new opportunities to collaborate with professionals who value innovation, practical problem-solving, and long-term partnerships. I enjoy discussing ideas that combine technical excellence with sound business strategy, creating outcomes that benefit everyone involved.
I believe every connection has the potential to become something meaningful. If you're interested in exchanging ideas, exploring opportunities, or simply connecting with someone who enjoys building impactful technology, I'd be happy to hear from you.
Wishing you success in your future endeavors, and I look forward to connecting.
Michael, thanks for reading the post. And best of luck out there with your next gig! If I hear anything Python-wise, I will send the details your way.
Thanks. Christine
If you allow, I wanna discuss about Opportunities and collaboration with you as soon as possible. Of course, this is an excellent opportunity for you.
How about you?
If you have free time, I wanna discuss each other to get income.
Best
The ROI collapse in code review is real, but I'd push back slightly on where the problem originates.
You describe bots reviewing bots รขยย but the underlying issue isn't the AI per se, it's that we're running AI review as a parallel human-reviewer simulation. We give it the same artifact (the PR diff), the same output format (comments), and the same authority level (official review status), and then act surprised when it generates the same noise we were already drowning in.
What if AI review worked as a hypothesis generator rather than a verdict machine? Not "this is wrong" but "consider: what if this error propagates to the billing layer?" รขยย a question only a reviewer with full system context would think to ask.
The trickier failure mode is when the PR is fine but the original ticket was flawed. Should the review bot surface that the problem being solved is the wrong problem? That's where human attention genuinely earns its keep.
Have you seen teams successfully reframe AI review as an async analysis layer rather than a parallel reviewer?
Yea, our teams do consider a good amount of the AI review items as nice to haves or nitpicks, so they really aren't considered blocking. Some of the time, then the dev will just make a follow up pr if they need.
The fatigue makes sense once you count what a reviewer actually has to check per PR now: not just "is this correct" but "is this still the pattern we agreed on," and nothing in the diff tells you which of those two questions you're supposed to be answering. That ambiguity is exhausting even before you get to the line-by-line read.
โReturn on attentionโ is a useful way to name the problem.
A lot of AI review tooling optimizes for finding more things to say, not for preserving reviewer attention. That creates the strange failure mode where the team gets more comments, more apparent rigor, and less shared understanding.
The best pattern Iโve seen is to force AI review output into a smaller contract: severity, evidence, reproduction path, and why a human should care. If it cannot provide those, it should stay silent or ask a question. Otherwise the bot is not reducing review load; it is spending the teamโs attention budget on its behalf.
The idea of "Return on Attention" really stands out. As AI-generated reviews become more common, success shouldn't be measured by the number of comments produced but by how quickly a reviewer can make a confident decision. High-quality feedback reduces cognitive load rather than adding to it.
Return on attention is the right metric. AI review can add value, but only if it reduces the human's uncertainty instead of creating another inbox. A good review bot should surface evidence, risk, and suggested next action, not just more fluent commentary to process.
As someone working on GPU infrastructure for secure ML workloads, Iโve seen how LLMs are increasingly used to automate code reviews, but the real challenge remains contextโespecially when models run on different hardware stacks like VoltageGPU. The fatigue mentioned in the article hits close to home; without proper alignment between tooling and actual deployment environments, even the best AI suggestions can miss the mark.
The 'slop grenade' concept is so real. Just because an LLM can generate 500 words in two seconds, we forget that someone has to spend their limited cognitive energy reading it. Love the 'Return on Attention' frameworkโdefinitely a great boundry to set!
Thanks for reading!
This is a really good one. I often catch Claude adding lengthy explanations to my comments that I race to remove because a human would neither understand them, nor believe me ๐๐
๐ I have definitely done this too.
I started treating AI review comments like draft commits. If I wouldn't say it myself, I delete it. Most PRs get better after cutting half the words.
Great way to frame it.
well soooooooooo true but some AI generate oversimplified code (ChatGPT) some produce duplicated code (Cluade) lol
The "1430 chars for 10 chars of change" hit home. I've started treating AI comments like spam filters โ if it's longer than my attention span, it's getting collapsed.
Yea....that was definitely not a popular PR ๐
The "Return on Attention" idea really stood out. Definitely something worth keeping in mind while using AI.
Thanks for reading Muhammad! I appreciate the comment ๐
every AI comment added is another triage call - trust it or flag for juniors. started treating AI review feedback like junior contributions: signal, not authority. that second-order cost never shows in the faster-reviews pitch
The bot-replying-to-bot loop is the failure I find hardest to contain, because confident wrongness from one model becomes input the next one treats as settled fact. The line about having to leave the PR to get context points at the actual metric worth watching: a review comment that sends you elsewhere has negative value, no matter how thorough it reads. I have started scoring review comments on whether they are self-contained and whether the flagged bug reproduces, and most of the noise fails both checks.
Then do you still respond on those comments? I have a co-worker who refuses to review a PR if it has too long of commit messages and just way too much in the PR description. Which is fair. Her time, and energy can be better focused with shorter, more concise artifacts....and it actually helps everyone on the team.
As someone working with GPU infrastructure for confidential computing, Iโve seen how LLMs are increasingly used to manage code review noiseโbut the real bottleneck isnโt the code, itโs the signal-to-noise ratio in team communication. Tools like VoltageGPU help isolate sensitive data during training, but they donโt fix burnout from endless bot-reviewed PRs.
Hey Christine, your "Return on Attention" piece was great (and clearly did numbers on dev.to). I noticed it lives on christine-seeman.com first: how do you handle getting it onto dev.to after, by hand or a script? I'm digging into that exact problem before building anything, so I'd love to hear your actual workflow.
I do this pretty much by hand. So I store my draft posts locally as .md, then have a python script to post on my site (WordPress). Then to x-post to dev.to I have it pretty much cat the markdown to my clipboard, and then paste into a new post on here. There is a small .py script I use for that cat-ing of the markdown and such ...it's here: github.com/cseeman/blog-scripts#to... . I think there is an API here...but this was what has worked for me without too much effort to post both on WordPress and here. I had RSS setup for a while, but I changed themes and that stopped working. I think the RSS feed into dev.to is a good approach, if that works for your site.
the 'bot commenting on a bot's suggestions' loop is the one that finally made us gate AI review behind a delay. we had the bot run 5 minutes after the first human review, not before. completely changed the dynamic: humans anchored the thread, the AI added instead of defining it.
the attention cost is the part your post names correctly and most tools don't acknowledge. you can't review 40 AI comments at the same depth as 5 human ones, and the signal to noise ratio isn't there yet.
are you seeing different patterns across PR sizes โ does the bot noise get proportionally worse on larger PRs, or is it a flat overhead regardless?
If you're tired of code reviews, you might want to change your programming language. 20% of the tokens compared to Python. 10% relative to C# and Java. And 10 times as high success ratio as Fable on Python, C#, JS, etc ...
Return on attention is the right unit, and the version that bites is that a review comment has to be cheap to verify or cheap to dismiss, and slop is neither. Confident wrongness is expensive for one reason: at a glance it is identical to confident rightness, so the reader has to spend attention to tell them apart, and that cost lands on every comment whether or not it earned reading.
A colleague's terse note is cheap to act on because it carries the why in the same breath. A bot comment that makes you leave the thread to reconstruct context has already cost more attention than the bug it found was worth. The move that would actually shift your metric is not better-sounding bots, it is making each automated comment carry the evidence that lets you resolve it in one look, cite the line it checked and what it expected, or do not post. Volume was never the problem. Unverifiable-per-glance was.
Return on attention is the metric most AI review tooling ignores. A bot can produce technically relevant comments and still make the team worse if it increases the number of decisions humans must triage. The better product question is not โdid it find issues,โ but โdid it spend reviewer attention well?โ
The 10 character diff getting a 1,430 character description is the exact failure mode I hit running a couple of coding agents on the same repo. The AI reviewer treats verbosity as thoroughness, so I end up scanning three paragraphs to find the one line that actually changed behavior. Your accountability question lands hard because I have caught myself pasting a bot's explanation into a PR description without reading whether it was even true. The real fix might not be a better AI reviewer at all. It might be a rule that nothing gets attached to a PR unless a human read it first and would defend it under questioning.