Do not refactor a messy module by rewriting it.
Capture current outputs with characterization tests before edits.
Then change one seam and keep the snapshot green.
Hope is not a contract for legacy Python.
Call results, exceptions, and types are the contract.
Freeze those three signals before you rename anything.
1. Name the module and the public surface
Select one importable file, not a package tree.
List public names with a one-line interpreter probe.
Treat dunder names as out of scope for this pass.
python -c "import billing_legacy as m; print(sorted(n for n in dir(m) if not n.startswith('_')))"
Record that list in a text file beside the tests.
Do not add new public names during characterization.
The fixture matrix must describe today's surface only.
Stop if import prints, writes files, or opens sockets.
Quiet import is a precondition, not a later chore.
Come back only after that import stays silent.
2. Classify each name before you call it
Build a four-column table for every public name.
Columns are name, kind, purity, and risk.
Kind must be function, class, constant, or module.
name kind purity risk
quote_total function reads env high
TAX_TABLE constant pure low
Client class network skip
Skip network and clock names on the first freeze.
Those need fakes you do not have yet.
Low-risk constants can wait until after functions.
Purity should answer one question with evidence.
Does the name read env, time, disk, or globals?
If yes, mark it high risk and isolate later.
3. Encode one fixture row per interesting input
Write fixtures as data, not as narrative comments.
Each row needs args, kwargs, and an id.
Keep values JSON-serializable so the diffs stay readable.
# proposal: fixtures.py — example matrix, not a live customer suite
FIXTURES = [
{"id": "empty-cart", "args": [[]], "kwargs": {}},
{"id": "one-item", "args": [[{"sku": "A", "qty": 1, "cents": 199}]], "kwargs": {}},
{"id": "zero-qty", "args": [[{"sku": "A", "qty": 0, "cents": 199}]], "kwargs": {}},
{"id": "neg-cents", "args": [[{"sku": "A", "qty": 1, "cents": -5}]], "kwargs": {}},
]
Four fixture rows beat one happy-path test.
Edge rows catch the behavior you plan to preserve.
Do not add rows that the old code never accepted.
Derive ids from real call sites when you can.
A ripgrep pass is enough for a first inventory.
Copy argument shapes and drop production secrets.
rg "quote_total\(" -n --glob "*.py"
Drop rows that need live credentials or private data.
Replace every customer identifier with synthetic tokens now.
Characterization is about shape, not production data dumps.
4. Capture result, exception type, and repr
Each characterization record should hold three fields only.
Use result on success and error on failure.
Store type as a string, never as a live class.
# proposal: capture.py
import json
from pathlib import Path
def capture_call(fn, args, kwargs):
record = {"result": None, "error": None, "type": None}
try:
value = fn(*args, **kwargs)
record["result"] = value
record["type"] = type(value).__name__
except Exception as exc:
record["error"] = type(exc).__name__
record["type"] = type(exc).__name__
record["result"] = str(exc)
return record
def write_snapshot(path, rows):
text = json.dumps(rows, indent=2, sort_keys=True, default=str)
Path(path).write_text(text + "\n")
The default str hook keeps datetimes and decimals printable.
Sorted keys make git diffs stable across runs.
That stability is the whole point of the freeze.
Do not capture full traceback strings in the snapshot.
Tracebacks include paths and line numbers that churn.
The exception class name is the stable signal.
5. Run the harness twice before any edit
The first run writes the snapshot JSON to disk.
The second run must produce a byte-identical file.
If it does not, you still have hidden inputs.
python -m harness.characterize --write snapshots/billing_legacy.json
python -m harness.characterize --check snapshots/billing_legacy.json
A flaky snapshot is not a characterization test.
Find the env var, clock, or global cache next.
Do not refactor while the snapshot still drifts.
# proposal: billing_legacy.py — stand-in module for local rehearsal
def quote_total(items):
total = 0
for item in items:
total += int(item["qty"]) * int(item["cents"])
return total
# proposal: characterize.py
import json
import sys
from pathlib import Path
from capture import capture_call, write_snapshot
from fixtures import FIXTURES
import billing_legacy
def run():
rows = []
for fix in FIXTURES:
rec = capture_call(
billing_legacy.quote_total,
fix["args"],
fix["kwargs"],
)
rec["id"] = fix["id"]
rows.append(rec)
return rows
if __name__ == "__main__":
rows = run()
path = Path(sys.argv[-1])
if "--write" in sys.argv:
write_snapshot(path, rows)
elif "--check" in sys.argv:
frozen = json.loads(path.read_text())
if frozen != rows:
raise SystemExit("snapshot drift")
print("snapshot ok", len(rows))
Commit the snapshot with the harness in one change.
Reviewers then see behavior, not only new test code.
That pair is the baseline for the later edit.
Pin environment keys that the module already reads.
Print a sorted environment subset before the write.
Store that list next to the snapshot file.
python -c "import os; print('\n'.join(sorted(k for k in os.environ if k.startswith('BILLING_'))))"
6. Make the smallest safe change, then stop
Safe here means the snapshot stays byte-identical.
Small means one function body, not a layer rewrite.
Extract a helper only if the snapshot still matches.
# after: one helper, same quote_total contract
def _line_cents(item):
return int(item["qty"]) * int(item["cents"])
def quote_total(items):
return sum(_line_cents(item) for item in items)
Re-run the check immediately after that extract.
If the snapshot drifts, revert the helper and stop.
The characterization suite is the merge gate here.
Do not rename public functions in the same change.
Do not clean exception types while you extract.
Those are new products, not this refactor step.
Count the diff with git before you open a PR.
If more than one public function moved, split again.
git diff --stat
git diff -U3 -- billing_legacy.py
Review cost grows faster than the helper saves.
One seam per change keeps git bisect usable.
7. Where a free coding model fits this loop
Drafting fixture rows is slow on a wide surface.
A model can propose extra ids from the signature.
You still decide which rows the old code accepts.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode's free model access can draft those fixture rows.
The free server option can run the harness remotely.
Use that split only if this access already fits.
Paste the public signature and the four-column table.
Ask for fixture ids, not for a rewritten implementation.
Reject any row that needs network, time, or extra imports.
This split matters more than the tool brand.
Models invent plausible edge cases you did not run.
Unrun cases remain fiction, not real characterization data.
8. Decision table for the next edit
| change | snapshot required | allowed in same PR |
|---|---|---|
| extract private helper | identical | yes |
| rename public function | identical plus alias | no |
| swap exception class | new snapshot, reviewed | no |
| add logging | identical | yes if no return change |
| cache results | identical across two runs | no until freeze is stable |
If two boxes say no, split the work.
Put characterization first and behavior change second.
Mixing them hides the bug you just shipped.
9. Limitations
This harness does not prove functional correctness yet.
It only proves yesterday's behavior still happens.
Wrong totals stay wrong, only in a stabler form.
JSON cannot hold naive graph objects or sockets.
A default str hook can hide field changes.
Prefer explicit encoders for money and dates.
The method fails on unreproducible process state still.
Threads, caches, and lazy imports still leak state.
Fix those leaks before you trust the check.
Byte-identical JSON is weaker than a typed codec.
Two floats can print the same and still differ.
Prefer integer cents for money during the freeze.
10. Who should not use this approach
Do not use this on greenfield modules with no users.
There is no behavior worth freezing in empty code.
Write ordinary unit tests instead of snapshots here.
Do not use this as a license to skip design.
A frozen mess is still a mess after extract.
Plan a later and separate behavior-change series.
Do not outsource the freeze to a model run.
The snapshot must come from your local interpreter.
A generated JSON file is not evidence.
Skip this loop when the module is a thin HTTP wrapper.
You would freeze vendor errors and transport noise.
Test your mapping layer with fakes instead.
Close
Characterization turns a messy module into a measured one.
The smallest safe change is the only change that follows.
Keep the snapshot in git until the public surface moves.
Top comments (0)