DEV Community

Emery Chen
Emery Chen

Posted on

AI Fatigue Is Review Fatigue: Budget the Diff, Not the Model

AI Fatigue Is Review Fatigue: Budget the Diff, Not the Model

The model is rarely your bottleneck. Your review queue is.

You switched models twice this quarter. Your review queue did not shrink.
That is a measurement gap, not a model gap.

So stop shopping for models. Start budgeting the diff.

Your fatigue is review fatigue

Faster generation raises output volume. Higher volume raises review cost.

Nothing in that loop improves when you upgrade a model. The queue just fills faster.

Most AI workflow debates argue about the wrong number. They compare tokens out per second, not review hours per merge.

That is why the fatigue persists. You optimized the input. You never measured the output.

Define the unit you actually pay for

Tokens are an input. Merged changes are the result.

The honest unit is tokens per merged change. It answers one question leaderboards cannot. Did this run pay for itself?

Two log files are enough:

  • calls.jsonl — one record per model call.
  • merges.jsonl — one record per merged pull request.
{"run_id":"r-1042","tier":"free","in":3120,"out":840}
{"run_id":"r-1042","merged_at":"2026-10-14T09:12:00Z","files":3,"added":48,"removed":11}
Enter fullscreen mode Exit fullscreen mode

Join them with a small script:

# cost_per_merge.py — reference implementation, not benchmarked here
import json
from collections import defaultdict

def load(path):
    with open(path) as f:
        return [json.loads(line) for line in f if line.strip()]

spend = defaultdict(int)
for c in load("calls.jsonl"):
    spend[c["run_id"]] += c["in"] + c["out"]

merges = defaultdict(int)
for m in load("merges.jsonl"):
    merges[m["run_id"]] += 1

rows = []
for run_id, tokens in spend.items():
    if merges[run_id]:
        rows.append((tokens / merges[run_id], run_id, tokens))

rows.sort(reverse=True)
for per_merge, run_id, tokens in rows[:10]:
    print(f"{run_id}\t{tokens:>9} tok\t{per_merge:>9.0f} tok/merge")

total = sum(spend.values())
merged = sum(merges.values())
print(f"\nportfolio: {total} tok, {merged} merges, {total/max(merged,1):.0f} tok/merge")
Enter fullscreen mode Exit fullscreen mode

Run it every Friday. Then read the top of the list.

A run that burns 400k tokens for one merge is your problem. The model version is not.

If the top row is a free-tier run, that is fine. Cheap tokens spent on abandoned work still cost you review time.

Gate the diff before you review it

The second half is a hard budget on what an agent may hand you.

# diff_budget.py — fail the build when the agent overreaches
import fnmatch, subprocess, sys

MAX_FILES = 6
MAX_LINES = 250
ALLOW = ["src/**", "tests/**", "docs/**"]
DENY = ["**/migrations/**", "**/*.lock", "infra/**"]

def changed(base="origin/main"):
    out = subprocess.run(
        ["git", "diff", "--numstat", f"{base}...HEAD"],
        capture_output=True, text=True, check=True).stdout
    for line in out.splitlines():
        added, removed, path = line.split("\t")
        yield path, int(added) + int(removed)

files = list(changed())
lines = sum(n for _, n in files)
bad = []

if len(files) > MAX_FILES:
    bad.append(f"files={len(files)} > {MAX_FILES}")
if lines > MAX_LINES:
    bad.append(f"lines={lines} > {MAX_LINES}")

for path, _ in files:
    if any(fnmatch.fnmatch(path, p) for p in DENY):
        bad.append(f"denied path: {path}")
    elif not any(fnmatch.fnmatch(path, p) for p in ALLOW):
        bad.append(f"outside allowlist: {path}")

if bad:
    print("\n".join(bad))
    sys.exit(1)

print(f"ok: {len(files)} files, {lines} lines")
Enter fullscreen mode Exit fullscreen mode

Wire it in as one CI step:

python diff_budget.py || { echo "over budget; split the change"; exit 1; }
Enter fullscreen mode Exit fullscreen mode

Why a budget beats a better model

A diff you can hold in your head gets reviewed honestly. A 900-line diff gets approved by fatigue.

Budgets are negotiable. Silent overruns are not. Raise MAX_LINES when the work justifies it, in the open.

Where a free allowance belongs in this loop

Here is the part that needs a caution label.

MonkeyCode offers free model access and a free server option. The operator also describes a free allowance of 10 million tokens; confirm the current figure in the product before you plan around it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use that capacity for work that is high volume and low stakes:

  1. Classify incoming issues into two or three buckets.
  2. Draft commit messages from the gate-approved diff.
  3. Propose test names that a human then rejects most of.
  4. Re-run a draft pass on a branch that will never merge.

This keeps paid spend on the merge path. It also keeps free spend measured, because calls.jsonl does not care which tier answered.

The limits are real. Shared capacity is shared. Do not put a latency-sensitive pipeline there. Do not run a forty-minute agent loop and call it production.

Who should not do this

  • Teams with no CI that can run one extra script.
  • Teams that cannot log model calls at all.
  • One-shot migrations where a review budget has no meaning.
  • Anyone who needs a vendor SLA on day one.
  • Anyone hoping the budget replaces a reviewer instead of protecting one.

The position

Free tokens are a subsidy, not an architecture. Review capacity is the constraint you cannot subsidize away.

So make the constraint visible. Log calls. Join them to merges. Gate the diff at six files and 250 lines.

Then argue about models, if you still have the energy. That argument is cheap. Review time is not.

If you want somewhere to run the high-volume, low-stakes half of this loop, the free tier and free server are worth a trial run. Bring your own logs and see what the week says.

Top comments (0)