DEV Community: Arnaud Debray

A reviewer's attention is a fixed budget

Arnaud Debray — Tue, 08 Sep 2026 14:38:12 +0000

The PR size grew significantly this year. But a reviewer can still only pay attention to a finite number of changes. For the past months we have been interviewing the tech leads who review AI-generated code for a living, and tinkering with our own review surface around what they told us. These are our conclusions so far.

PRs are now larger. Human attention hasn't grown

Every quote below is verbatim from the people we interviewed for our last post: four months, more than 100 tech leads, architects, and senior engineers, on how their teams actually use AI.

"Now that you remove the bottleneck of writing the code, we've just moved the bottleneck to reviews."
— an engineering manager

The surge of coding agents has created strong control concerns for tech leads. Many try to limit this loss of control by reviewing as many PRs as possible. But that is where the bottleneck appears. What makes this a losing battle is that it isn't only a question of time. It's a question of cognitive capacity. No matter the skill level, no matter how much of a superhero you are, nobody is built to build a global understanding of a 2000-line AI-generated PR touching 100 files. The human attention window is limited.

Three problems with the GitHub PR review interface for AI-generated PRs

Every file has the same weight and is presented in simple alphabetical order.
It shows what changed and not why. As reviewer, you must reconstruct the reasoning from the code, one file at a time.
Not everything is in GitHub. You must keep GitHub open to comment, then have a local Claude Code session in another window, then your IDE again in another window. You constantly switch from one to the other.

In the rest of this post, we'll describe what we built to address these 3 problems that came up again and again in our interviews. We're being fully transparent about what worked, what didn't, and what we still want to test. And of course, we're very open to feedback.

1. Triage by criticality

In the last post we said that industry-standard code, a boilerplate integration or reading a CSV in Python, is easy for a model, and that as models commoditize, that kind of code commoditizes with them. This is the first point towards deciding what we should review and what we should not. The seniors we sat with already split their reviews in two, although they do not do it consciously.

Risky code gets read first, every time. Depending on the profile of the reviewer, their definition of risky code changes. For a backend, the data model, the APIs, the contracts between services, auth, payments, anything that touches money or identity. This risky code is read first, every time. One CTO wrote their order down for us in a Slack message.

"1. Check we haven't dumped [junk] into the data models. 2. Check we haven't modified contracts between micro-services 3. Then, once those parts are validated, I look at the coherence and readability of the code."

Boilerplate is skimmed over. This type of code gets the last part of the review, when the reviewer is already not really paying attention anymore, e.g., the helpers, the tests, the connectors to a well-documented API. If it is not business critical, it is not your expertise, and it does not need to be non-standard; the model has seen the pattern ten thousand times and it will probably produce code that is good enough.

What we built

The first thing we built is a change map. Every changed file gets a criticality tier, and the tiers are set per viewer rather than per repo: a backend reviewer marks the data model and the APIs critical and the frontend tests low; a frontend reviewer flips the same PR the other way. This is done by matching file paths with rules. Before reading a line, you see the shape of what you are about to spend attention on, and you discard what you will not open.

the change map on a 93-file PR — 6 critical / 22 high / 5 medium / 50 low / 10 needs triage]

"Where I'm losing time is I need to skim through the changes to see what is touchy in the PR, now I can just see in a minute. This PR has critical files touched, or it's just low impact."
— a lead architect

Result: we shipped the criticality ordering and triage first: you can discard files you don't want to read, then the diff is displayed with critical files first, noise last. Users do like it, although they feel a bit uncomfortable explicitly saying "I'm not reading this" at first. Our users asked for the alphabetical file tree to navigate as well, because that is what they are used to, so we provide both and they choose. Either you navigate via alphabetical order, or you navigate by criticality.

This is not perfect yet. A one-line change in a utils file can be the whole blast radius, and a path rule will not flag it. Our belief is that those changes are being treated as the last part deserving attention and thus not read carefully anyway. The tiers do not make it worse.

A way to improve would be to make the rules smarter and flag tiers based on the semantic meaning of the changes. Another addition we're looking at is a clean way to mark "this we did not really check, and that is fine according to our policy." We have not shipped that yet.

2. Show the why behind the changes

The reasoning that used to live in a dev's head, or on a whiteboard when the feature needed some more design, the edge cases, the alternatives that were rejected, the constraints, now gets typed into a prompt before the agent writes a line. Recovering what the human thought about and determining if the logic is correct has always been the slow part of the review. Now the issue is that there are many more decisions in a single PR, and you do not know whether a change was backed by actual thinking from the developer or was an arbitrary choice by the agent.

"I spend my time on PRs writing why, why, why [...] I'm forcing the devs to own their decisions."
— a tech lead

What we built

Hervé captures the developer's thinking (the prompts) and puts it next to the diff in each PR. That way, we can expose important decisions that were made during implementation, e.g., we cache X but not Y. Our first version surfaced all of it at the top of the page next to the PR description. That was too much, and a user told us so within a week:

"The info at the top is just too much and I just skip it."
— a CTO

It felt like reading the PR twice. The second version keeps only what deserves the top of the page: the intent shifts, where the agent added or dropped scope against the spec it was given, and the undiscussed decisions, where the agent chose something on its own without asking the developer. Everything else is available on demand. Hervé can read the AI sessions behind the PR, so you can ask it whether the developer prompted for a given edge case, and it finds the answer in the session, or tells you it is not there.

an undiscussed decision card — the agent added a cache on its own, and no turn in the session asked for it

One thing that surprised us: the PR author finds it useful to read the undiscussed list, to see what the agent decided on their behalf.

3. The IDE feeling in the review surface

Reviewing one PR today takes three tools. GitHub or GitLab, to read the diff and leave comments. An IDE on a local checkout, to navigate the code: go to definition, find usages, search for a symbol. And a local agent, to ask the higher-level questions about the change. Three applications for a single review.

"I checkout locally on the PR to get the code in the IDE, asks questions via Claude-code, and make comments on Github. The worst is when I review multiple PRs in parallel, I start mixing the windows and do not know which is which anymore."
— a lead dev at a startup

What we built

We built the IDE features the reviewer needs into the review surface: go to definition, find usages, symbol search, on the review page. That folds two of the three tools into Hervé. The third, the local agent/harness, we have not folded in yet. It comes with a subscription most developers already pay for, so one more question about a PR costs them nothing, and we have not found a way to do better or cheaper than that yet. For now, we have decided not to compete with it. Developers love their harness anyway. In fact, they're asking us to expose Hervé's features to their harness rather than trying to replace it for review. This is probably where we will be going.

Conclusion

We are asking people to digest an amount of information per PR that is far more than a head was built to hold. The tooling has not caught up to the AI revolution. We have been coming at it from a few angles: criticality and ordering, so the attention goes where it counts; the captured intent, so less of it goes to reconstructing the why; the IDE features, so it stops requiring three windows for a single review. Some of it worked, some of it did not.

What caught your attention in our approach? Would you rather have a new review interface, or more information available through the CLI so your local Claude sessions can access the insights and the captured intent? You can try Hervé!

We talked to 100+ dev teams about how they use AI. Here are some things we learned.

Arnaud Debray — Tue, 01 Sep 2026 12:03:55 +0000

We spent four months talking to more than 100 tech leads, architects, and senior engineers about how their teams actually use AI. Review time is a real constraint. But the one they kept circling back to was harder to name: control.

The dominant story about AI and code review is a throughput story: agents write faster than any human can read, the PR queue backs up, and the fix is to review faster with better bots, more QA, more automation. Everyone we spoke to feels that pressure. But ask them whether humans are still needed and the answer is a firm yes: people still own the moments that decide quality, setting up how the agents work, writing the spec, and reviewing what ships. It is that last one, review, where the deepest worry sits.

Humans didn't leave the loop. They moved to its edges.

That yes has gotten more specific over the last six months. The fever that peaked after the late-2025 model releases (also known as the Saaspocalypse era) has cooled into a hangover: a few months of uncoordinated AI use that produced years of tech debt. Three moments came back again and again as irreducibly human.

The agentic software factory. Before a line is generated, someone sets up the system the agents work inside: CLAUDE.md and AGENTS.md, the conventions, the rules, the specs format, the skills that standardize the workflows, the CI checks. Someone also keeps auditing the architecture and the codebase as the agents reshape it. It is the same job the tech lead always had, now aimed at agents instead of people. One finding was consistent: teams that let the agents run that structuring themselves backed off. Code files ballooned to thousands of lines, the layout scattered, and past a certain size the agents lost the thread of their own code. Keeping the system legible stayed a human job.

Writing the specification. It is easy for an LLM to produce industry-standard code, such as a boilerplate integration or reading a CSV in Python. As LLMs commoditize, that kind of software commoditizes with them. That pushes the value into the business-specific details, where the prompter's domain expertise is what makes the software non-standard.

Reviewing and validating. Early on we met a few believers, convinced they could get away without ever reading their code again. Almost no one we spoke to still believes they can hand off ownership of the codebase: roughly 80% still keep a human reviewing what ships. AI is a tool, and a fast one. Speed without a gate has one destination: a Ferrari is a very fast way to get into a wall. Someone has to hold a model of what the codebase is becoming, because that is the person you will need the day it breaks.

"You get systems that look right but aren't, and when one thing goes wrong, you have no idea what happened."

— a founding engineer

The clean exception is the early-stage startup, where nothing matters except finding the market before the runway ends. Fair. But we heard the same bill come due later, once they scaled: the codebase turned unmaintainable, new hires could not ramp, and more than one team told us they paused growth to clean up or rebuild. The debt is fine to take. It just belongs in the plan.

Time is the real constraint. Control is the deeper worry.

The review bottleneck is almost always framed as a time problem, and it genuinely is one. Addy Osmani's Code Review in the Age of AI frames it that way, and so did our own manifesto. Agents outproduce reviewers, the queue grows, reviewers go underwater. About 2/3 of the teams we spoke with named review as their bottleneck.

But that is not what surprised us. Sitting next to tech leads while they reviewed, the concern underneath was not time. It was control: being able to say, with confidence, what a change does to the system they answer for. Around 40% named that loss of grip outright; more circled it without quite having a word for it.

A senior dev carries a mental model of the codebase that lets them review a PR extremely fast: check the data-model changes, check the APIs, check the business logic, skim the rest. Done. That single act signs off the change and keeps the mental model current. That model is what makes them the right person to fix the thing at 2am when it breaks.

That is the pass agents break. For as long as people wrote the code, review had a safety net: the author was a teammate you knew, and you could ask what they were thinking, which edge cases they weighed, why they went this way instead of the obvious one. Now the author is effectively a stranger who has already left the building. The reasoning that produced the change lives in the agent's session, not in the diff, so all the reviewer has is the code. Instead of skimming against a model they already hold, they read every change and try to infer, from the code alone, why it was made.

"Now I have a 14,000-line PR. How the hell am I going to understand what happened? Can I trust it?"

Review bots do not close the gap either. They flag individual lines, one more stream to read, while the question that actually matters goes unanswered: what is this change doing to the system? Push it far enough and the code is written by one model and reviewed by another, until people quietly learn to stop looking at that signal.

"In the end, it's just AI talking to AI."

— a team lead

The binding constraint moved from writing code to understanding it.

New tools for writing code do not change what a software process is for. Thirty years of practice still holds. What changed is where the constraint binds: writing is cheap now, and reading a change, understanding it, and answering for it is the step that limits everything. That step still runs on an interface built for a diff a human could hold in their head. Watching reviewers work, we saw five things they used to do cheaply get expensive.

The reviewer's job	Before	Under agentic development
Hold the spec in your head	PR description and a linked ticket; a whiteboard for the hard ones	More spec than ever, in prompts, tickets, PRDs, ADRs, and repo docs, but none of it is easily accessible when you review the diff
Find the poorly-built parts	Scan a small diff; lean on conventions, tests, and CI	Everything looks fine and well tested at a glance, across too many files. It takes GitHub, the IDE, and an agent to review a single PR: three interfaces for one review
Confirm nothing exceeded scope	Obvious on a scroll, if you know the codebase	Buried in the hundreds of files changed, and "out of scope" barely means anything now: agents scope-creep by default, so it mostly slips through
See how the team actually works	PR reviews, peer coding, lints, CI, conventions, DORA metrics	You just see an LLM's output, with lints and CI enforced on it. No way to know which model was used, how the dev prompted, or which skills they used, and no obvious way to level the team or break the silos that leave it behaving like a pack of solo devs
See every change	Read every PR, top to bottom	Impossible at volume; you have to focus on what matters, not the boilerplate

Humans are necessary. They just don't have the right tools.

None of this makes the reviewer's job automatable, and that is the point. Not because an LLM cannot summarize a change; increasingly it can. Because someone human still has to own the result and answer for it: to the on-call rotation, to the customer, to whoever asks why it broke. And owning a decision means understanding it. A better reading interface does not decide whether a change is right; it gives back the context to decide faster. It will not rescue a team that never wrote the spec down, and it will not turn a 300-file PR into a good idea.

Humans are nowhere close to leaving the software development process. Their role is moving: from typing the code to owning the judgment about it, and to protecting the one asset agents erode fastest, a live model of what the system is becoming. We think the binding constraints of the SDLC have changed, and the tooling has to be rebuilt to adapt to them.

That is the interface we are building Hervé to be: a review surface that hands the reviewer the intent alongside the diff, so they can rebuild their mental model instead of reverse-engineering it from the diff.

Do these observations resonate with you? What about you, how do you keep control over your codebase?