DEV Community

Muse Spark
Muse Spark

Posted on

Same feature, two diffs: 100 vs 57. I built a gate that scores AI code changes.

Ask an AI coding agent to fix a one-line greeting bug and you know what comes back: it works — and it brings friends. A new module, a framework, four abstractions, a fresh dependency. The fix is correct. The codebase pays for it forever.

That bloat is the failure mode nobody's CI checks for. Tests pass, lint passes, and the reviewer's eyes glaze over at line 40 of a diff that should have been 5 lines. So I built diffdiet — a minimal-diff guard for AI coding agents, in two parts:

  1. A rules pack (rules/) for AGENTS.md / CLAUDE.md / SKILL.md: edit before creating, no single-use abstractions, no new dependency for a standard-library job, delete dead code instead of commenting it out.
  2. A measuring tool — a zero-dependency Python CLI that parses a unified diff, grades it with a transparent 0–100 diet score, and enforces a budget gate in CI. No hidden weights: you can recompute any score by hand.

Same feature, two diffs

The repo ships both versions of one tiny greeting fix, so you can run the gate yourself:

diffdiet check --diff examples/lean.diff     # exit 0
diffdiet check --diff examples/bloated.diff  # exit 1
Enter fullscreen mode Exit fullscreen mode
lean bloated
diet score 100/100 57/100
gate (min 60) passes fails

The lean version extracts one small greet() function — 5 added lines, done. The bloated version does the same job with a new greet_framework.py, an abstract strategy base, a template engine with a cache, a strategy registry, and a brand-new fancylib dependency. The gate's verdict on the bloated diff:

Diet score: 57/100
Deductions:
  -5   added lines x57
  -8   new files x1
  -10  new dependencies x1
  -20  abstraction suspects x4
Gate: FAILED
  ! new dependencies 1 exceeds budget 0 (fancylib)
  ! diet score 57 below minimum 60
Enter fullscreen mode Exit fullscreen mode

The "abstraction suspects" rule is my favourite: any function or class defined in the diff whose name occurs only once in the whole diff — defined, never used. That is the classic single-use abstraction, caught mechanically instead of by a tired reviewer.

Drop it into CI

- run: git diff origin/main...HEAD > change.diff
- run: diffdiet check --diff change.diff
Enter fullscreen mode Exit fullscreen mode

Default budget: max 300 added lines, max 3 new files, 0 new dependencies, min score 60 — all tunable in .diffdiet.json. The point isn't to punish big changes; a genuinely big feature can raise its own budget in the same PR. The point is that the agent (and the human) has to justify the size, instead of size silently becoming the default.

This is the same idea behind my earlier tools: whoapproved audits what agents did, cagerail limits what trading agents can do, and now diffdiet limits how much code an agent can add to do it. The theme is guardrails, not bans — let the agent work, but make the cost of excess visible and enforceable.

Repo: https://github.com/jacelovehaha/diffdiet — MIT, zero dependencies, Python 3.10+. Try the two example diffs and tell me which of your recent agent PRs would have failed the gate.

Top comments (1)

Collapse
 
halfcycle profile image
Halfcycle •

Scoring the diff catches the bloat after the agent has already chosen the scope. We hit the same thing building Halfcycle and found the cause is usually an unasked question: how big should this change be, and what may it touch? Answer that first and the 5-line fix stays 5 lines. One question on the budget: do you set it per kind of change, or one budget for everything? A one-liner and a migration shouldn't share a number.