The Refactor Receipt: Freeze Behavior, Cap the Diff, Then Ask a Model for Edge Cases

Example scenario (hypothetical, not a real repository): you inherit a 4,000-line pricing module with three entry points, one global cache, and tests that assert only that the import succeeds. A coding agent proposes a 300-line cleanup that claims to preserve behavior and adds twelve new tests. The diff looks plausible, but plausible is not evidence.

The useful question is not whether the refactor looks cleaner. It is whether you can produce a small, reviewable receipt showing which behavior was frozen, which lines changed, and which checks failed. This article describes one workflow for that receipt, plus a gate you can adapt to your repository.

Why the smallest safe change needs a receipt

AI coding tools are now good at producing plausible diffs and plausible tests. That makes generation cheap and acceptance expensive. A characterization suite helps, but a suite alone can be defeated by editing the golden expectations until the tests pass again.

The receipt adds two constraints that a test run alone does not provide. First, it hashes the frozen behavior file, so changing the expectations invalidates the run. Second, it budgets the product diff, so a cleanup cannot quietly grow into a rewrite.

The goal is not to prove the refactor is correct in every possible environment. The goal is to make the accepted change small, attributable, and easy to reject when the evidence is weak.

Step 1: Declare the behavior surface before touching code

Write down the behavior you intend to freeze before you edit the module. Keep this list short enough to review, and make the exclusions explicit. A useful first pass covers four categories.

• Entry points: public functions, CLI commands, HTTP handlers, and jobs that external code calls.
• Input shapes: normal cases, boundary values, empty inputs, and malformed inputs observed in logs.
• Normalization rules: rounding, time zones, locale, float tolerance, and ordering.
• Out of scope: internal helper behavior, log text, and performance characteristics you are not changing.

If the module mutates shared state, add a side-effect ledger to the characterization suite. This article focuses on the receipt and the diff budget; it does not assume that a return-value check is sufficient for a stateful module.

Step 2: Write characterization tests against a frozen case file

The test should read cases from one file and compare the current implementation to the recorded expectations. The case file is the behavior fingerprint, so it must be treated as a reviewed artifact rather than a scratch file. The following example is illustrative and has not been executed for this article.

import json
from pathlib import Path

import pytest

from legacy.pricing import quote

CASES = json.loads(Path(__file__).with_name('quote_cases.json').read_text())

@pytest.mark.parametrize('case', CASES, ids=[c['id'] for c in CASES])
def test_quote_contract(case):
    actual = quote(case['items'], case['customer'])
    assert actual == pytest.approx(case['expected'], abs=0.001)

@pytest.mark.parametrize('case', [c for c in CASES if 'expected_error' in c])
def test_quote_errors(case):
    with pytest.raises(Exception) as exc:
        quote(case['items'], case['customer'])
    assert type(exc.value).__name__ == case['expected_error']

Narrow the exception assertion to the module's real exception types before you rely on this pattern. A broad Exception catch will record the wrong failure as expected behavior, which makes the fingerprint less useful.

Step 3: Build a call-shape inventory from real traffic

Do not start by asking a model for test cases in the abstract. Start with the argument shapes your system actually receives, because those shapes determine which branches are worth freezing. A log scan, a queue sample, or a sample of production requests can provide the inventory.

Shape	Example signal	Why it matters
Empty collection	zero items	Often bypasses totals and discount logic
Single item	quantity of one	Exposes rounding and minimum-charge branches
Mixed tiers	gold plus standard customers	Catches tier checks that depend on order
Boundary total	exactly at a threshold	Reveals inclusive versus exclusive comparisons
Malformed field	missing price or quantity	Freezes the current error contract

For each shape, record only the input fields and the source of the observation. Do not record expected outputs from memory, because memory is exactly what the characterization suite is meant to replace.

Step 4: Use free models for candidate cases, not for verdicts

The operator describes MonkeyCode as an open-source project with free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator-supplied offer also describes a 10-million-token allowance; verify current quotas, model names, and server limits in the project's own documentation before relying on them.

The productive use of a free model here is candidate generation. Give the model the call-shape inventory and ask for additional input cases that fit the same schema. Do not ask it for expected values, and do not accept its explanation as evidence.

You are given a list of call shapes for a legacy pricing function.
Propose up to 20 additional input cases that exercise branches not yet listed.
Return one JSON object per line with fields: id, items, customer, why.
Do not include expected outputs.
Do not include prose outside the JSON lines.

Run every candidate through the current implementation, record the actual result, and review whether the case is stable and meaningful. Keep only cases that execute the current code and produce deterministic output, then add them to the frozen case file. The model proposes inputs; the runtime supplies truth.

This is also where a free model's cost advantage matters least. A cheap wrong expected value is still a wrong expected value, so the review step is not optional. Use the model to widen the search, not to sign off on the change.

Step 5: Budget the diff before you review it

A diff budget turns the phrase smallest safe change into a reviewable number. The values below are an example, not a universal standard; choose them from your repository's risk profile and review capacity.

Dimension	Example budget	Why it is gated
Product lines changed	60	Stops a cleanup from becoming a rewrite
New product files	2	Limits structural churn during behavior freeze
Dependency files	0	Prevents a refactor from changing the runtime surface
Forbidden paths	migrations/, generated/	Protects areas that need a separate review process
Characterization fingerprint	SHA-256	Catches golden-file edits that make tests pass

Store the budget in a file that reviewers can see in the pull request. The following one-line example is deliberately small; add fields only when the gate actually uses them.

{"max_product_lines_changed": 60, "max_new_files": 2, "forbidden_paths": ["migrations/", "generated/"], "fingerprint_file": "tests/characterization/quote_cases.json", "fingerprint_sha256": "REPLACE_AFTER_FREEZE"}

Step 6: Emit a refactor receipt and make it a gate

The script below collects the commit, test exit code, product diff size, changed dependency files, and fingerprint hash. It fails when any budget dimension is exceeded or when the frozen case file changes. The script is illustrative and was not executed for this article; adapt the path filters and test command to your repository.

#!/usr/bin/env python3
# Illustrative gate; not executed for this article.
from __future__ import annotations

import argparse
import hashlib
import json
import shlex
import subprocess
from pathlib import Path

DEPENDENCY_FILES = {'pyproject.toml', 'requirements.txt', 'poetry.lock', 'uv.lock'}

def run(cmd: list[str], cwd: Path) -> subprocess.CompletedProcess[str]:
    return subprocess.run(cmd, cwd=cwd, text=True, capture_output=True)

def sha256_file(path: Path) -> str:
    digest = hashlib.sha256()
    digest.update(path.read_bytes())
    return digest.hexdigest()

def is_product(path: str) -> bool:
    excluded = ('tests/', 'docs/', '.github/')
    return not path.startswith(excluded) and path not in {
        '.refactor-budget.json', 'refactor-receipt.json'
    }

def numstat(base: str, cwd: Path) -> list[tuple[int, int, str]]:
    proc = run(['git', 'diff', '--numstat', f'{base}...HEAD'], cwd)
    if proc.returncode != 0:
        raise SystemExit(proc.stderr.strip())
    rows = []
    for line in proc.stdout.splitlines():
        added, deleted, path = line.split(maxsplit=2)
        rows.append((0 if added == '-' else int(added),
                     0 if deleted == '-' else int(deleted), path))
    return rows

def name_status(base: str, cwd: Path) -> list[tuple[str, str]]:
    proc = run(['git', 'diff', '--name-status', f'{base}...HEAD'], cwd)
    if proc.returncode != 0:
        raise SystemExit(proc.stderr.strip())
    rows = []
    for line in proc.stdout.splitlines():
        status, path = line.split(maxsplit=1)
        rows.append((status, path))
    return rows

def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument('--base', default='origin/main')
    parser.add_argument('--budget', default='.refactor-budget.json')
    parser.add_argument('--test-cmd', default='python -m pytest -q tests/characterization')
    parser.add_argument('--out', default='refactor-receipt.json')
    args = parser.parse_args()

    repo = Path.cwd()
    budget = json.loads(Path(args.budget).read_text())
    changes = numstat(args.base, repo)
    statuses = name_status(args.base, repo)

    product = [(a, d, p) for a, d, p in changes if is_product(p)]
    product_lines = sum(a + d for a, d, _ in product)
    new_files = [p for s, p in statuses if s.startswith('A') and is_product(p)]
    forbidden = [p for _, _, p in product if any(p.startswith(x) for x in budget['forbidden_paths'])]
    dependencies = [p for s, p in statuses if p in DEPENDENCY_FILES]

    fingerprint = Path(budget['fingerprint_file'])
    actual_hash = sha256_file(fingerprint) if fingerprint.exists() else None
    expected_hash = budget['fingerprint_sha256']
    test_proc = run(shlex.split(args.test_cmd), repo)

    violations = []
    if test_proc.returncode != 0:
        violations.append('characterization tests failed')
    if product_lines > budget['max_product_lines_changed']:
        violations.append(f'product lines changed: {product_lines}')
    if len(new_files) > budget['max_new_files']:
        violations.append(f'new product files: {len(new_files)}')
    if forbidden:
        violations.append(f'forbidden paths touched: {forbidden}')
    if dependencies:
        violations.append(f'dependency files changed: {dependencies}')
    if actual_hash != expected_hash:
        violations.append('fingerprint hash changed')

    head = run(['git', 'rev-parse', 'HEAD'], repo).stdout.strip()
    receipt = {
        'base': args.base,
        'head': head,
        'test_command': args.test_cmd,
        'test_exit': test_proc.returncode,
        'product_lines_changed': product_lines,
        'new_product_files': new_files,
        'fingerprint_sha256': actual_hash,
        'expected_fingerprint_sha256': expected_hash,
        'violations': violations,
        'passed': not violations,
        'test_output_tail': test_proc.stdout[-2000:],
    }
    Path(args.out).write_text(json.dumps(receipt, indent=2) + chr(10))
    print(json.dumps(receipt, indent=2))
    return 0 if not violations else 1

if __name__ == '__main__':
    raise SystemExit(main())

Run the gate locally before opening the pull request. The first run should fail if the fingerprint hash is still a placeholder, which is the intended safe default.

python -m pytest -q tests/characterization
shasum -a 256 tests/characterization/quote_cases.json
python tools/refactor_receipt.py --base origin/main

On Linux, use sha256sum tests/characterization/quote_cases.json instead of shasum. Paste the resulting hash into the budget file, then rerun the gate to confirm that an unchanged branch passes.

Step 7: Make the smallest safe change and read the receipt

Use a short loop rather than a large batch of edits. The receipt is most useful when each change has a narrow hypothesis.

1. Confirm that the characterization suite passes on the base commit.
2. Make one bounded cleanup that does not intend to change behavior.
3. Run the receipt gate and inspect every violation, not just the first one.
4. If the fingerprint hash changed, revert the golden-file edit and explain the behavior change separately.
5. If the product diff exceeds budget, split the refactor before asking for review.

A passing receipt is not proof that the refactor is correct. It is evidence that the frozen cases still pass, the golden file was not edited, and the diff stayed inside the declared budget. That is a much narrower claim, and it is one a reviewer can actually check.

Where the free server fits

An optional execution path is MonkeyCode's free server option. Treat it as a remote shell with a known commit, not as a source of truth. Record the commit hash, Python version, and lockfile hash in the receipt; otherwise a green run is not attributable to the code you reviewed.

git rev-parse HEAD
python --version
python tools/refactor_receipt.py --base <base-sha> --budget .refactor-budget.json

Verify whether the environment is ephemeral and which artifacts persist before you rely on it for release evidence. If the free server cannot pin the same runtime as your CI, use it for exploratory runs and keep the merge gate on a controlled runner.

Limitations and who should not use this

This workflow is a poor fit when the refactor intentionally changes behavior. Bug fixes, migrations, and pricing changes need a separate behavior-change plan, not a frozen contract. The gate will correctly block them, and weakening the gate to make them pass defeats its purpose.

It is also weak when the module is nondeterministic. Times, random values, floating-point noise, and concurrency need normalization or a different evidence strategy. A fingerprint that changes on every run will create false violations and train reviewers to ignore the gate.

Teams without a deterministic test environment should fix that first. The receipt does not replace a reproducible runner, a lockfile, or a test command that fails for the right reason.

Finally, do not use model-generated cases as a substitute for domain review. A model can propose an input shape that looks plausible but encodes a business rule nobody agreed to. The runtime records what the current code does; a human still decides whether that behavior is the contract you want to preserve.

Pre-merge checklist

• The characterization case file is reviewed and its hash is recorded in the budget.
• The product diff stays inside the declared line and file budget.
• No dependency file changed during a behavior-preserving refactor.
• The receipt includes the commit hash, test exit code, and fingerprint hash.
• Any intentional behavior change is split into a separate branch with a new baseline.

If this loop saves a review, keep the receipt in the pull request so the next reviewer can see exactly what was frozen. If you need a runner for the gate, the operator's free server option is one place to try the workflow, but confirm its current terms first.