DEV Community

Remdore
Remdore

Posted on AI-assisted

I don't read most of my AI's pull requests. I look at the ones I can see.

I don't read most of the pull requests my AI writes anymore. I read the title, the description, whether CI is green and what the agent says it checked, and then I merge. The exception is anything I can look at: a page, a screenshot, a diagram. Those I still open, every time.

That sounds reckless written down, so I went back through my own repos to see what it actually looks like in numbers. These are the two main repos behind ExtraTime, my football quiz site: a Django API and a Next.js frontend. I'm the main author of both, with a couple of friends sending PRs now and then.

How it got here

I started letting an agent open pull requests for me in April 2026. Since then I've merged 1,396 PRs into those two repos, and 1,241 of them say in the description that they were written with Claude Code. In October alone I merged 563, with about 370,000 lines changed between them.

There's no way I read 370,000 lines in a month. I don't think anyone would, and pretending I did would be worse than saying so.

But the part that surprised me is what came before. In 2025, before I let an agent open PRs, I merged 321 into the same repos. The median description was empty. The median time from opening a PR to merging it was 2 minutes on the API and 4 on the frontend. I wasn't reviewing my own code then either. I was the reviewer and the author, so the PR was just a button I pressed on the way to deploying.

So what I gave up wasn't careful review. It was the feeling of knowing every line, because I'd typed every line.

What I read instead

The title and description. Since April the median PR description in these repos is 2,400 to 2,800 characters, against zero before. The agent writes what it changed, why, and how it tested it. That's the part I read properly. If the description doesn't make sense, or it describes a different change than the one I asked for, the PR doesn't get merged, whatever the diff says.

CI. Every API PR runs the Django test suite in two shards plus a Pyflakes check for new findings. The frontend builds, lints, type-checks, runs its unit tests and then runs Playwright end-to-end tests in Chromium and Firefox. What's changed is how much there is to run. On 1 January the API had 442 test functions. On 1 April it had 1,496, and today it has 6,383. The frontend went from 236 on 1 April to 2,483. Most of the tests added since April came in the same PR as the change they test.

The agent's own review. The agent leaves a comment on the PR saying what it checked, along the lines of "Reviewed: the PENDING to CORRECT transitions are now a compare-and-set, a loser drops only the row it inserted". GitHub's Copilot reviewer also reviews some of them. I read those comments the way I'd read a colleague's summary, not as proof.

And mostly, I trust it. That's the honest version. If the description is right and CI is green, I merge without opening the diff.

What I still look at

Anything that ends up on a screen. For frontend changes I open the Vercel preview and click through it. If the PR has screenshots, I look at them. If it has a diagram, I read the diagram.

The reason is simple. A test can tell me a function returns the right thing. It can't tell me the page looks wrong. Going back through the frontend fixes since April, about two dozen have titles like these:

  • fix(lobby): no sideways scrolling on phones
  • fix: five clubs — wrap long footballer names on the card
  • fix(lineup): stop the mobile hydration swap that shifted the footer (CLS 0.79 → ~0.06)
  • fix: prevent burger menu items from being clipped when Notifications is expanded

Each of those is about how a page looks or moves, not what it returns. A layout shift of 0.79 means the page visibly jumps under your thumb. The end-to-end tests click through those same pages in two browsers, and they pass, because a button that moved is still a button. Someone had to look.

So the rule I've ended up with is: I review what I can see, and I trust what I can test.

What I got wrong

Only 81 of the 1,396 PRs since April have an image in the description. 4 have a diagram. So "I look at anything visual" mostly means I have to remember to open the preview. When I don't, nothing reminds me. I should have made the agent attach screenshots to every frontend PR from the start, rather than when I happened to ask.

The other thing is where the bugs go when nobody reads the diff. In 2025, 10 to 13% of my merged PRs were fixes. In October it was 46 to 47%, and 121 of October's PRs mention a Sentry issue. Some of those are noise rather than bugs (one is literally about dropping planned Postgres stops from Sentry), and I can't separate fixes for code nobody read from fixes for code from years ago. But the shape is clear. Not reading the code doesn't make the bugs disappear. It moves them to production, where Sentry finds them instead of me. For a quiz site that's a trade I'm fine with. For a bank it wouldn't be.

Not one PR since April has been reverted. I first read that as good news. It more likely means I fix forward: when something breaks, the answer is another PR, not undoing the first one, and that shows up in the fix count instead.

Caveats

This is one person's two repos, with a small team and a hobby project's stakes. "Written with Claude Code" is what the PR description says, and some of those may have my edits in them too. The fix and Sentry counts are from PR titles and the first part of each description, so they're approximate. And a test count isn't a measure of test quality. 6,383 tests can still all test the wrong thing.

Where I ended up

I don't think the answer is to go back to reading every line. I didn't do that before AI either, and the volume now makes it impossible. What I'd say instead:

  • Read the description like it's the contract. If it's wrong, nothing else matters.
  • Make the tests the agent's job, in the same PR. They're what I actually trust.
  • Look at anything a user will see. That's where tests are blind and I'm not.
  • Ask for screenshots by default, so looking isn't something you have to remember.
  • Watch production. If you don't read the diff, Sentry is your reviewer now, and you should treat its alerts that way.

Top comments (0)