A Refactor Without a Baseline Is a Trust Fall

A refactor without a baseline is a trust fall. You close your eyes and hope. The hope usually lands somewhere else.

AI reviewers are good at reading diffs. They cannot read your intent. A diff review checks shape. It misses behavior changes. You need an external memory. Characterization tests are that memory.

The baseline contract

Fix the contract before you refactor. Pick one messy pure function. Keep it small. Avoid time, random, network, and file I/O. Pure functions give stable baselines.

The contract has three parts. Inputs go in. Outputs come out. Exceptions are also outputs. Snapshot all three.

Step 1: Write the snapshot runner

Put this script outside the function file. It loads a module, runs named cases, and writes JSON.

# snapshot.py
import json
import sys


def run(path, cases):
    ns = {}
    with open(path) as f:
        exec(compile(f.read(), path, "exec"), ns)

    results = []
    for c in cases:
        item = {"name": c["name"]}
        try:
            item["ok"] = True
            item["result"] = repr(
                ns[c["fn"]](*c.get("args", []), **c.get("kwargs", {}))
            )
        except Exception as exc:
            item["ok"] = False
            item["exception"] = type(exc).__name__
            item["detail"] = str(exc)
        results.append(item)
    return results


if __name__ == "__main__":
    cases = json.load(open(sys.argv[2]))
    result = run(sys.argv[1], cases)
    json.dump(result, open(sys.argv[3], "w"), indent=2)

The runner never imports your production package. It compiles one file. That keeps the harness safe for old code.

Step 2: Design probe cases

Name every case. Unnamed cases make diff reading painful. Cover the edges that old code tends to hide.

[
  {"name": "two items", "fn": "calculate_total", "args": [2, 10.0]},
  {"name": "zero quantity", "fn": "calculate_total", "args": [0, 10.0]},
  {"name": "negative price", "fn": "calculate_total", "args": [1, -2.0]},
  {"name": "missing price", "fn": "calculate_total", "args": [1, null]}
]

Start with five to ten cases. You can expand later. More cases means more signal. Most of the value comes from edge cases.

Step 3: Record the baseline

python3 snapshot.py order.py order_cases.json baseline.json

Commit baseline.json first. The commit message can say exactly what you are preserving. Reviewers get a concrete artifact. Nobody has to trust the old function's reputation.

Step 4: Generate the refactor against the contract

Now ask a model to rewrite the function. Give it the baseline file. Give it the case file. Tell it the before/after output must match exactly.

The model can propose a new version. The baseline is not a suggestion. It is a pass/fail gate.

Step 5: Run the new code and diff

python3 snapshot.py order_new.py order_cases.json after.json
diff baseline.json after.json

A clean diff means behavior survived. A noisy diff shows exactly which case broke. This turns a vague review note into a reproducible test.

Which functions deserve a snapshot?

Function trait	Snapshot before AI edits?	Why
Fan-in greater than 3	Yes	Many callers broaden the blast radius.
Many branches or early returns	Yes	Bugs hide in branch paths.
Time, random, or network calls	No	The baseline will flap.
Trivial accessor	No	The snapshot adds noise.

Use the table before every refactor. A cheap snapshot on the wrong function costs maintenance time. No snapshot on the right function costs an incident.

Why the free options matter

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The generate step in this loop needs an AI backend. I used MonkeyCode's free model access for that step. It fits because the refactor iteration is the part that can cost money. A free tier removes the anxiety from retrying. The free server option matters for the other side. Some repos cannot leave a private network. A local server keeps the baseline, the candidate, and the diff in the right place.

The value of the workflow does not depend on MonkeyCode. The script, the cases, and the diff are plain JSON. That is the point.

Limitations

Snapshots preserve bugs. If the original output is wrong, the baseline is wrong. They catch behavior changes, not design problems. They are not a replacement for real tests. They are a wrapper around old behavior.

Use this only on pure functions first. For stateful code, snapshot the input object too. For I/O-heavy functions, build explicit fixtures. Otherwise the baseline lies.

Who should not use this

Teams with full branch coverage should skip the snapshot. A well-tested function already has a baseline. Snapshotting it adds a second version of the same test.

People who want a fully automated refactor should also skip this. The model still makes choices. You still read the diff. The loop only removes the guesswork.

And never write PII into a snapshot file. Characterizing a legacy function does not justify leaking user data.

Next time you ask a model to clean up a mess, record the behavior first. Then the review is a diff. The refactor stops being a trust fall.