DEV Community

Altair
Altair

Posted on

Ask, Don't Just Read: Keeping Up With Code an AI Wrote

#claudecode #ai #codereview #productivity

I'm the person in charge of digital transformation at a Japanese company. I build internal tools and workflow automations, and these days an AI coding agent (Claude Code) writes almost all of the code.

I thought my problem was "code review can't keep up." When I looked closer, it was something else.

The bottleneck wasn't review. It was understanding.

To be honest, for most of this work I no longer read diffs line by line. What I was actually stuck on was:

  • I couldn't tell whether the change did what I intended.
  • I couldn't explain it to anyone else.

Tools that make diff review faster don't help if you've already stopped reading diffs. I needed a way to say "I understand this" without reading everything.

It turns out everyone is hitting the same wall

  • Anthropic's own Claude Code team has said the bottleneck has moved from writing code to reviewing it.
  • Margaret-Anne Storey (2026) splits the problem into three kinds of debt: technical debt in the code, cognitive debt (the team's understanding erodes), and intent debt (the reasons and constraints never get written down). It's a conceptual paper, not an experiment.
  • In a 2026 survey in Japan, 86.3% of engineers who review AI-generated code said the burden had increased, and the top problem (49.5%) was "the submitter can't explain the code."

The study that changed how I approached it was Anthropic's randomized trial from January 2026. 52 mostly junior engineers learned a new library. The group using AI scored 50% on a comprehension quiz versus 67% for the hand-coding group, and the biggest gap was in debugging.

But the result depended on how people used the AI. People who fully delegated to it scored under 40%. People who asked conceptual questions and fixed errors themselves scored 65% or higher. (The authors are clear that this is correlation, not causation.)

So maybe reading an AI's explanation isn't enough. Maybe what matters is the human asking the question.

What I do now: an intent check

Spec-driven development plus automated review is great for a big team, but it's too heavy for one person. I cut it down to four things.

1. Write the intent before building

One line of purpose, then 3–5 success criteria in the form "when X, Y happens." The AI drafts them, I confirm. One round-trip.

This is where mismatches between the goal and the approach show up — much cheaper than finding them afterward.

2. Judge the result with a table, not the diff

# Success criterion Result Evidence
1 … ✓ / ✗ / unverified test run, actual output, screen, counted numbers

Evidence must come from actually running it. "The code says so" doesn't count as ✓. The last row is always: "Do the comments and design notes still match what it does now?"

3. Leave one or two questions — without answers

Under the table, the AI writes one or two "questions worth asking to understand this better." It doesn't write the answers. I decide whether to ask. If I don't, that's fine.

If the study above is right, asking beats reading. This just leaves the door open.

4. Keep four lines for explaining it to others 

- What changed: (from the user's point of view)
- Why:
- How to check it:
- What it doesn't do:
Enter fullscreen mode Exit fullscreen mode

These go into the design notes. The table is for me to judge "did it do what I meant." These four lines are for explaining it to someone else.

A one-line change, two surprises

The first change I tried this on was a single line: keep 3 daily backups instead of 7. The goal was to free up disk space.

The table showed:

  • ✓ It keeps the latest 3 plus one per week for 4 weeks.
  • ✗ It barely frees any space. Old backups go to the Recycle Bin, so the disk space doesn't come back until the bin is emptied.
  • ✗ The comment at the top of the file still said "keep 7."

Neither problem was visible from the diff. They showed up because I wrote down the purpose first.

What it is not

  • Not a replacement for automated code review. That finds defects; this finds gaps against intent.
  • Not a replacement for having the actual users check the output against real data.
  • When something is ✗, the AI doesn't quietly fix it. A human decides whether the intent or the implementation was wrong.

What I don't know yet

I've only just started. To avoid judging it by how it feels, I log four things each time:

  1. Number of ✗ results
  2. Time spent on the up-front back-and-forth
  3. Whether I asked the questions
  4. Later, when someone asked me about it, whether I could answer without asking the AI again

In one study, developers felt 20% faster with AI while actually being 19% slower (METR, 2025). I'll write a follow-up once I have a few months of data.

As AI writes more of our code, the human job seems to be shifting from "read everything" to "decide what to trust." If so, the key to keeping our understanding might be not how much we read, but how often we ask.

References

Top comments (0)