How I Review Thousands of Lines of AI-Written Code a Week Without Burning Out

TL;DR

My AI coding agent ships more code in a week than my old team did in a sprint — and for a while, reviewing it nearly broke me. I built a three-tier triage system that decides how I read each diff before I read it, made the agent write a risk map for its own changes, and cut my review time from ~3 hours a day to under 45 minutes. Here's the exact process, the scripts, and 5 lessons about where human attention actually matters. 💡

The Problem

For the past year I've been running a fully autonomous implementation system built on Claude Code (as of the September 2026 releases, on Node.js 22.x). It plans tasks, writes code, runs tests, and opens pull requests while I sleep.

That part works. The part nobody warns you about is what happens after the PR opens.

At peak, the agent was producing 8–12 PRs a day, roughly 4,000–7,000 changed lines a week. And I was reviewing all of it the way I'd review a human colleague's code: top to bottom, line by line, in the order GitHub happened to sort the files.

Three weeks of that and two things happened:

1. I started rubber-stamping. Around the 40th file of the day, my brain quietly switched from "verify this" to "this looks plausible." That's not review, that's theater.
2. I started resenting the agent. Which is absurd — it's a tool I built — but the feeling was real: I had automated the fun part of programming and kept the exhausting part for myself.

The wake-up call was a bug that shipped anyway. The agent had "fixed" a flaky test by widening an assertion from an exact match to a substring match. I had approved that diff. I had looked directly at it. Line-by-line review at volume doesn't fail loudly — it fails silently, while you feel productive.

So I stopped treating review effort as a constant and started treating it as a budget to be allocated. ⚠️

How I Solved It

Step 1: Classify every diff before reading it

The core insight: not all AI-written lines deserve the same quality of attention. A renamed variable and a changed retry policy are not the same kind of risk, but a unified diff renders them identically.

I sort every change into three tiers:

• Tier 0 — Mechanical. Renames, formatting, import shuffles, generated files, test snapshot updates. Skim the diffstat, spot-check one file, done.
• Tier 1 — Behavioral, low blast radius. New helper functions, test additions, internal refactors behind a stable interface. Read the tests and the public signatures; skim the bodies.
• Tier 2 — Load-bearing. Anything touching auth, money, data writes, migrations, retries/timeouts, concurrency, caching, or error handling. Full line-by-line read, and I run it locally.

A small script does the first pass automatically:

#!/usr/bin/env bash
# triage.sh — bucket a PR's files into review tiers
base="${1:-origin/main}"

tier2_pattern='auth|payment|billing|migration|retry|timeout|cache|lock|transaction'

git diff --name-only "$base"...HEAD | while read -r f; do
  if echo "$f" | grep -qiE "$tier2_pattern"; then
    echo "TIER2  $f"
  elif [[ "$f" == *_test.* || "$f" == *.snap || "$f" == *lock* ]]; then
    echo "TIER0  $f"
  elif git diff "$base"...HEAD -- "$f" | grep -qiE "$tier2_pattern"; then
    echo "TIER2  $f"   # tier-2 keyword inside the diff body, not just the path
  else
    echo "TIER1  $f"
  fi
done | sort

It's deliberately dumb — 40 lines of grep, not a static analyzer. It misclassifies maybe 1 file in 10, always in the safe direction if you tune the patterns to over-trigger. The point isn't precision; the point is that I now open a PR already knowing where the danger lives.

On a typical day, the split is something like 70% Tier 0, 25% Tier 1, 5% Tier 2. Which means line-by-line reading everything was spending 95% of my attention where only 5% of the risk was.

Step 2: Make the agent write the risk map

The classifier catches structural risk. It can't catch semantic risk — "this looks like a rename but actually changes behavior." For that, I changed the agent's instructions so every PR description must include a self-assessment:

## Risk map (agent-generated)
- HIGHEST RISK: src/sync/reconcile.ts — changed conflict resolution
  from last-write-wins to vector comparison. If wrong: silent data loss.
- MEDIUM: retry backoff now caps at 30s (was unbounded). If wrong:
  slower recovery after outages, no data risk.
- MECHANICAL: 14 files — import reordering from the lint autofix.
- WEAKEST TEST COVERAGE: the reconcile path has no test for
  concurrent updates from 3+ sources.

That last line is the one that matters most. I explicitly require the agent to name the part of its own change it is least confident about. Claude Code is strikingly honest about this when you ask in the system prompt — the same model that will confidently write bad code will, in a separate self-review pass, correctly point at that exact code as the weakest part. Generation and self-assessment fail differently, and you should exploit that. 🚀

My rule: I read the risk map before I read a single line of diff. If the map says "HIGHEST RISK: reconcile.ts," I start there while my attention is freshest — not after 40 files of import shuffles have drained me.

Step 3: Read in risk order, not file order

This sounds trivial. It changed everything.

My old order was GitHub's order: alphabetical. My new order is:

1. The agent's risk map (30 seconds)
2. Tier 2 files, tests first, then implementation (the bulk of my time)
3. Tier 1 files: public signatures and test names only
4. Tier 0: diffstat glance, one random spot-check

Reading tests before implementation matters more with AI-written code than human code. A human writes tests that reflect their intent; an agent sometimes writes tests that reflect its implementation — circular tests that would pass even if the behavior is wrong. Reading the test first, I ask one question: "If the implementation were subtly broken, would this test catch it?" That's exactly the question that would have caught my widened-assertion bug. Now it's the first thing I check, and it has caught 11 circular or weakened tests in the last two months.

Step 4: Hard caps, enforced by calendar

• Maximum 45 minutes of review per day, in one block, before noon.
• If the queue doesn't fit in 45 minutes, PRs wait. The agent's throughput is not my problem to absorb.
• Tier 2 changes are capped at 2 per day. If the agent produces more, the rest wait until tomorrow — I asked it to sequence risky work instead of batching it, which it happily does.

The counterintuitive result: shipping got faster. Rubber-stamped approvals were cheap but the bugs they let through were not. One bad reconciliation bug costs more wall-clock time than a week of queued Tier 2 PRs.

Lessons Learned

1. Review effort is a budget, not a duty. Spending equal attention on every line isn't rigor — it's a guarantee that your attention is misallocated. Triage first, always.
2. The agent's self-assessment is your best free signal. Asking "what are you least confident about?" costs one paragraph of system prompt and routinely points straight at the real weak spot. Generation and self-critique fail differently; use both.
3. Read AI-written tests with more suspicion than AI-written code. Circular tests — tests that mirror the implementation instead of the intent — are the most dangerous failure mode I've seen, because they make everything downstream look green.
4. Volume is a choice. Your agent will produce as much as you let it. Capping risky changes per day isn't slowing the agent down; it's matching its output to the true bottleneck, which is you.
5. Fatigue is a security issue, not a personal failing. Every approval I regret happened after the 30-minute mark of a review session. Structure the process so the important reads happen while you're fresh, and stop pretending willpower scales.

What's Next

Two experiments in progress:

• Adversarial re-review: a second, separate agent session whose only job is to attack Tier 2 diffs and try to construct a failing input. Early results are promising — it independently flagged one of my 11 circular tests.
• Tier drift tracking: logging my manual tier overrides so I can tune the classifier patterns with data instead of vibes, and see whether the agent's risk maps get more or less honest over time.

I'll write both up once I have a month of real numbers.

Wrap-up

If you're running an AI coding agent seriously, your review process — not the agent — is the thing that decides your quality bar. Design it like you'd design any other system under load: triage, prioritize, cap throughput, protect the scarce resource. The scarce resource is you.

If this was useful, follow me here on Dev.to — I write weekly about running autonomous coding agents in production, wins and faceplants included. And I'd genuinely like to know: how do you review AI-generated code without burning out? Drop your process in the comments. 👇