Cap the Diff, Not the Ambition: A Line-Budget Gate for Messy-Repo Refactors

Refactor size is a testable property. Green tests do not prove the diff stayed small.

Two gates close that gap. A snapshot proves the output did not change. A diff budget proves the edit did not sprawl. Run both before review.

Everything below is illustrative. The scripts were not executed for this article, and the numbers are budgets you choose, not measurements.

Prerequisites

• Python 3.10+ and git.
• One messy module you can call from a CLI or a thin adapter.
• Inputs you can replay offline, without network or database state.

Gate 1: pin behavior with one snapshot file

A snapshot answers a narrow question. Does this module still return the same bytes for this input set?

Step 1 — collect replayable inputs. Keep them tiny and hand-written.

{"id": "empty", "input": "fixtures/in/empty.csv"}
{"id": "header_only", "input": "fixtures/in/header.csv"}
{"id": "negative_price", "input": "fixtures/in/neg.csv"}
{"id": "unicode_sku", "input": "fixtures/in/unicode.csv"}

Save as fixtures/cases.jsonl. One JSON object per line.

Step 2 — record the baseline from the untouched code.

# tools/snapshot.py -- illustrative, not executed here
import json, subprocess, sys

CASES = 'fixtures/cases.jsonl'
BASELINE = 'fixtures/baseline.json'

def record(case):
    proc = subprocess.run(
        [sys.executable, '-m', 'legacy.report', case['input']],
        capture_output=True, text=True, timeout=30,
    )
    return {
        'returncode': proc.returncode,
        'stdout': proc.stdout,
        'stderr_tail': proc.stderr.strip().splitlines()[-3:],
    }

def main(mode):
    cases = [json.loads(l) for l in open(CASES) if l.strip()]
    observed = {c['id']: record(c) for c in cases}
    if mode == '--write':
        json.dump(observed, open(BASELINE, 'w'), indent=2, sort_keys=True)
        print(f'wrote {len(observed)} cases')
        return 0
    expected = json.load(open(BASELINE))
    drift = [k for k in expected if expected[k] != observed.get(k)]
    for key in drift:
        print(f'DRIFT {key}')
    return 1 if drift else 0

if __name__ == '__main__':
    sys.exit(main(sys.argv[1] if len(sys.argv) > 1 else '--compare'))

Step 3 — commit the baseline with the tests. Run --compare in CI forever after.

One caveat. If the module prints timestamps or ordering noise, strip those fields before comparing. Do not widen the assertion to "contains substring". That deletes the test's value.

Gate 2: cap the diff

Snapshots cannot see footprint. Gate 2 measures it.

Step 1 — declare a budget as data, next to the tests.

{
  "base": "origin/main",
  "max_changed_lines": 60,
  "max_files": 3,
  "allowed_paths": ["src/report/**", "tests/test_report_*.py"],
  "forbidden_paths": ["src/db/**", "migrations/**"]
}

Step 2 — enforce it with a short script.

#!/usr/bin/env python3
# tools/diff_budget.py -- illustrative, not executed here
import fnmatch, json, subprocess, sys

def git(*args):
    return subprocess.run(args, capture_output=True, text=True, check=True).stdout

def main():
    cfg = json.load(open('tools/budget.json'))
    numstat = git('git', 'diff', '--numstat', cfg['base'] + '...HEAD')
    changed = files = 0
    problems = []
    for line in numstat.splitlines():
        added, deleted, path = line.split('\t', 2)
        if path.startswith('fixtures/baseline'):
            continue
        files += 1
        changed += int(added or 0) + int(deleted or 0)
        if any(fnmatch.fnmatch(path, p) for p in cfg['forbidden_paths']):
            problems.append(f'forbidden path touched: {path}')
        elif not any(fnmatch.fnmatch(path, p) for p in cfg['allowed_paths']):
            problems.append(f'outside declared scope: {path}')
    if changed > cfg['max_changed_lines']:
        problems.append(f'{changed} lines > budget {cfg["max_changed_lines"]}')
    if files > cfg['max_files']:
        problems.append(f'{files} files > budget {cfg["max_files"]}')
    print(f'files={files} changed_lines={changed}')
    for p in problems:
        print('FAIL', p)
    return 1 if problems else 0

if __name__ == '__main__':
    sys.exit(main())

Step 3 — read the printed counters, not just the exit code. A refactor that passes at 58 of 60 lines is one hunk away from failing. That is a signal to split early.

Known false positive: renames. git diff --numstat can report {old => new} style paths, which break glob matching. Check the output on a rename commit before trusting the gate in CI.

What to do when a gate fails

Result	Likely cause	Action
Snapshot drifts before any edit	Nondeterministic output or stale baseline	Fix fixtures or strip volatile fields. Do not start editing.
Snapshot passes, budget fails on line count	Edit is too wide	Ship the smallest hunk. Open a second PR for the rest.
Budget fails on forbidden path	Scope creep into another module	Revert that hunk. File a follow-up issue.
Both pass on the first try	Test may not assert anything	Put a wrong expected value in the baseline, confirm failure, restore.

Where a free model and a free server actually help

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Two operator-supplied facts matter here. MonkeyCode offers free model access, and a free server option. I treat both as access, not as a quality claim, and I have not measured model accuracy for this workflow.

Useful, bounded roles:

1. Draft candidate inputs. Feed the model a recorded log or stack trace and ask for 10 to 15 edge-case inputs. You still verify each one runs.
2. Draft the adapter. A CLI wrapper around a legacy function is low-risk scaffolding. Test it before you trust it.
3. Host the harness. If your laptop setup is the blocker, run the snapshot comparison on a scratch server. Confirm current terms and limits on the vendor's own pages before you rely on them.

Keep one rule absolute. The model never edits tools/budget.json. The budget is the human's declaration of acceptable scope. Handing that file to a generator deletes the gate.

Verify the harness in five minutes

1. On an untouched branch, run the compare mode. Expect exit code 0.
2. Add 60 comment lines to one file. Expect the snapshot to pass and the budget to fail.
3. Delete one case from the baseline. Expect the snapshot to fail.
4. Rename a file inside src/report/. Inspect the counter output manually.
5. Clone fresh and repeat step 1. Confirm no hidden local state.

If step 2 passes, your gate is decorative.

Boundaries and who should skip this

• Output you cannot make deterministic. Snapshot noise becomes false failures, and reviewers start ignoring the gate.
• Genuinely cross-cutting changes. Renaming a shared type touches many files. Raise the cap once, record the reason in the PR, and expect a slower review.
• Concurrency, performance, or security behavior. A snapshot compares bytes. It does not model races or timing.
• Large monorepos. allowed_paths globs go stale fast. Assign one owner per budget file.
• Throwaway scripts under a few hundred lines. The setup costs more than the risk it removes.

A budget gate does not make refactors safe. It makes unsafe scope visible before merge, which is the part tests cannot do alone.

Pick your messiest module. Write four inputs, record a baseline, set a 60-line cap, and let the gate reject your first attempt. Then ship the version it accepts.