The Diff Budget: Prove the Smallest Safe Refactor Before You Merge It

The smallest safe refactor is a number, not an instinct. Freeze the behavior first. Then cap the diff.

Most messy-repo refactors fail on scope, not on skill. One extraction turns into a rename. The rename pulls in a signature change. Two days later the characterization tests are red and nobody knows which of the four moves broke them.

This post adds the missing gate: a machine-checked budget for how much a "behavior-preserving" commit may change. It is the last step in a series about freezing behavior. It is also the step that keeps the freeze honest.

The rule: one rung, one budget, one green run

A refactor commit gets exactly one move and one measured budget. The characterization suite must be green before and after. Any patch that exceeds the budget is not a smaller refactor — it is a redesign wearing a refactor's clothes.

The gate is a script, not a review habit. Reviews drift; a script returns exit code 1.

Step 0 — Freeze one behavior path, not all of them

You do not need full coverage to start. You need one entry point that you will touch, pinned at its output bytes.

python -m pytest -q tests/characterization/parser_entry_test.py | tee baseline.txt
sha=$(git rev-parse --short HEAD)
echo "frozen at $sha" >> baseline.txt

If that test file does not exist yet, stop here and write it. The budget gate without a frozen suite is just a diff counter.

Step 1 — Make the gate fail on an empty diff first

A gate you have never seen fail is a gate you cannot trust. Start from a clean worktree. Run it. It must report zero changed lines and pass.

Then touch one line. Run it again. It must count that line.

Step 2 — Pick the rung before you type the patch

Decide the size of the move up front. The ladder below is the part reviewers should argue about, not the code.

Rung	Move	Line budget	Extra evidence
1	Rename a local or private helper	≤ 15	Suite green, no public symbol touched
2	Extract one branch into a private function	≤ 60, 1–2 files	Suite green, call count on the entry point unchanged
3	Move one function to a new module	≤ 80, ≤ 3 files	Suite green, import graph diff read by a human
4	Change a public signature	not a refactor	Split: rung 2 plus a separate adapter commit

Rung 4 is the trap. Adding a parameter with a default looks tiny. It changes the contract, so it belongs in its own commit with its own tests.

Step 3 — Write the budget file

The budget lives in the repo. It is versioned, reviewable, and boring on purpose.

{
  "base": "9f3c1ad",
  "glob": "src/**/*.py",
  "suffixes": [".py"],
  "max_changed_lines": 60,
  "max_files": 3,
  "forbid_public_signature_changes": true
}

base is the frozen commit from Step 0. Never move it inside the same branch. Moving the base is how budgets quietly reset.

Step 4 — Run the gate

The script reads git diff --numstat for size and git diff -U0 for symbol churn. Both are plumbing commands, so they behave the same in CI and locally.

#!/usr/bin/env python3
"""refactor_gate.py — fail a refactor that exceeds its scope budget."""

import json
import re
import subprocess
import sys

SYMBOL = re.compile(r"^\s*(?:async\s+)?(?:def|class)\s+(\w+)")


def git(*args: str) -> str:
    return subprocess.run(
        ["git", *args], check=True, capture_output=True, text=True
    ).stdout


def numstat(base: str, glob: str):
    rows = []
    for line in git("diff", "--numstat", base, "--", glob).splitlines():
        added, removed, path = line.split("\t")
        if added == "-":
            continue  # binary file, not counted
        rows.append((path, int(added), int(removed)))
    return rows


def symbol_churn(base: str, suffixes: tuple[str, ...]):
    gone, added = set(), set()
    path = ""
    for line in git("diff", "-U0", base).splitlines():
        if line.startswith("+++ b/"):
            path = line[6:]
        elif line[:1] in ("+", "-") and path.endswith(suffixes):
            match = SYMBOL.match(line[1:])
            if match:
                name = f"{path}::{match.group(1)}"
                (added if line[0] == "+" else gone).add(name)
    return gone, added


def main() -> int:
    with open("refactor_budget.json", encoding="utf-8") as fh:
        budget = json.load(fh)

    rows = numstat(budget["base"], budget["glob"])
    changed = sum(a + r for _, a, r in rows)
    failures = []

    if changed > budget["max_changed_lines"]:
        failures.append(f"lines {changed} > {budget['max_changed_lines']}")
    if len(rows) > budget["max_files"]:
        failures.append(f"files {len(rows)} > {budget['max_files']}")

    gone, added = symbol_churn(budget["base"], tuple(budget["suffixes"]))
    if budget["forbid_public_signature_changes"] and (gone - added):
        failures.append(
            "symbols removed or renamed: " + ", ".join(sorted(gone - added))
        )

    for path, a, r in rows:
        print(f"{a:>5} +  {r:>5} -  {path}")
    print(f"total: {changed} lines across {len(rows)} files")

    if failures:
        print("\nGATE FAILED")
        for item in failures:
            print(" -", item)
        return 1

    print("GATE PASSED")
    return 0


if __name__ == "__main__":
    sys.exit(main())

The rename check is deliberate. A renamed symbol shows up as a removal without a matching addition, so renames land in their own commit. That keeps git log -S useful six months later.

$ python refactor_gate.py
$ python -m pytest -q tests/characterization
$ git add -p && git commit -m "rung 2: extract tag-validation branch"

Step 5 — Set the rollback rule before you need it

Two failed gate runs on the same rung means the move is wrong, not the budget. Revert to base. Re-read the branch you are extracting. Shrink the move.

One line of policy beats an hour of negotiation:

git reset --hard "$(python -c 'import json;print(json.load(open("refactor_budget.json"))["base"])')"

Mechanical churn needs its own lane. Run the formatter as a whitespace-only commit, add it to .git-blame-ignore-revs, then start the budget clock afterward.

Where a free model and a free server genuinely help

Two parts of this loop are cheap to parallelize and easy to verify.

First, candidate patches. Ask for the smallest diff that keeps the frozen test green, then reject anything the gate flags. The gate is the judge; the model is a suggestion box.

Second, the gate run itself. If your laptop is busy with a long suite, run the suite plus refactor_gate.py on a remote box and read the exit code.

MonkeyCode's free model access and free server option are useful for exactly those two slots — drafting candidate diffs and running the frozen suite off your machine. I am not claiming benchmark numbers, quota details, or hardware here; those are operator-supplied availability claims, not measured results in this post.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A model that drafts a 400-line "cleanup" is not failing at code. It is failing at scope. The budget catches that in one second.

What this gate does not prove

The budget is a proxy, not evidence. It cannot tell you the refactor is correct; it only tells you the diff stayed small enough to reason about.

Three gaps are real:

1. A green suite over golden files hides behavior the golden files never captured.
2. Line counts reward dense edits over honest ones. A 20-line change can still alter control flow.
3. Renaming a private helper inside one file still trips the symbol check, by design. Expect to split those commits.

If your repo has no runnable entry point at all, do not start with the gate. Start with one call, one input, one recorded output. The gate becomes meaningful at line two.

Who should not use this

Skip it if the change is genuinely a redesign. Skip it if your team squashes every branch into one commit, because the base commit loses meaning. Skip it if the code under change is generated and the generator is the real source.

Use it when the code is messy, load-bearing, and poorly understood. That is exactly the repo where the smallest safe change is the only change you can defend in review.