Do not extract a module from a messy script first. Freeze clocks, working directories, and path roots first.
Then record a golden surface for later comparison. Only then make the smallest safe module split.
Messy repos often hide time and path coupling. Unit tests still pass after a bad extract. Golden files then flip for purely non-functional reasons.
The failure this harness targets
God scripts mix reporting, I/O, and business rules. They print timestamps on almost every run. They write files under the current working directory.
Absolute paths leak into JSON and log lines. A later extract changes import order or tempfile use. Observable behavior matches, yet the goldens fail.
That noise blocks the smallest safe change. You cannot trust a red test after the split. You also cannot trust a green test without frozen roots.
File counts can stay stable while paths shuffle. Coverage percent can stay high while order changes. Neither metric is a split gate.
What smallest safe change means here
Smallest means one extract, one module, one import. Safe means the observed CLI surface stays byte-stable. The surface is exit code, streams, and files.
Do not rename, reformat, and extract together. Do not polish log wording during the split. Extra diffs destroy the characterization signal fast.
Keep the original module as the CLI entry. Import path moves belong in a later commit. One behavioral surface belongs in each change set.
Inventory the script surface
Map outputs with a throwaway command first. Treat the block below as an unlabeled example. It is not a measured production run.
python messy_report.py --root ./samples --out ./out
echo $?
wc -c ./out/*
Record four channels after that command finishes. Channel one is the process exit code. Channel two is raw stdout bytes.
Channel three is raw stderr bytes. Channel four is the output directory tree. Skip pretty printers until those four stay still.
Freeze three sources of golden noise
1. Clock and timezone
Patch datetime.now and time.time in tests. Prefer a fixed timezone, not the host zone. Do not freeze time by editing production first.
# example harness fragment — unexecuted here
from datetime import datetime, timezone
from unittest.mock import patch
FIXED = datetime(2026, 9, 23, 12, 0, tzinfo=timezone.utc)
def run_with_frozen_clock(fn):
with patch("messy_report.datetime") as dt:
dt.now.return_value = FIXED
dt.datetime = datetime
dt.timezone = timezone
return fn()
Keep the freeze inside the test process. Do not ship a global clock monkeypatch. Production should take a clock argument later.
If the script shells out to date, patch that too. Host clocks will still leak through subprocesses. Frozen Python objects do not cover those calls.
2. Working directory and temp roots
Run the script under a temporary cwd. Point --out at a temp directory you own. Never characterize against $HOME or the repo root.
# example harness fragment — unexecuted here
import subprocess, tempfile, pathlib
def run_script(sample_root: pathlib.Path) -> dict:
with tempfile.TemporaryDirectory() as tmp:
tmp_path = pathlib.Path(tmp)
out = tmp_path / "out"
out.mkdir()
proc = subprocess.run(
[
"python", "messy_report.py",
"--root", str(sample_root),
"--out", str(out),
],
cwd=tmp_path,
capture_output=True,
text=False,
check=False,
)
files = {
p.relative_to(out): p.read_bytes()
for p in out.rglob("*")
if p.is_file()
}
return {
"code": proc.returncode,
"stdout": proc.stdout,
"stderr": proc.stderr,
"files": files,
"tmp": str(tmp_path),
"out": str(out),
}
The subprocess boundary is the characterization surface. In-process calls miss argv and cwd effects. Keep the first goldens at process level.
Pin PYTHONHASHSEED=0 in the test environment. Hash randomization reorders some debug maps. That reorder looks like a behavior change.
PYTHONHASHSEED=0 python characterize.py
3. Absolute paths inside payloads
Rewrite temp prefixes after the process exits. Replace the sample root with a stable token. Replace the output root with another token.
# example harness fragment — unexecuted here
def normalize(blob: bytes, roots: dict[str, str]) -> bytes:
text = blob.decode("utf-8", errors="surrogateescape")
items = sorted(roots.items(), key=lambda kv: len(kv[0]), reverse=True)
for raw, token in items:
text = text.replace(raw, token)
text = text.replace(raw.replace("\\", "/"), token)
return text.encode("utf-8")
Sort replacements by path length, longest first. Short prefixes otherwise eat longer paths. Cover both Windows and POSIX separators.
Also strip trailing spaces on log lines. Some extract tools change wrapping only. Those edits are not the split you wanted.
Build the golden files
Store normalized bytes, not pretty-printed JSON. Pretty printers reorder keys and change floats. Characterization wants accidental stability, not style.
# example harness fragment — unexecuted here
import hashlib, json, pathlib, os
GOLDEN = pathlib.Path("goldens/messy_report")
def dump_surface(surface: dict, sample_root: pathlib.Path) -> None:
GOLDEN.mkdir(parents=True, exist_ok=True)
roots = {
os.path.abspath(surface["out"]): "__OUT__",
os.path.abspath(surface["tmp"]): "__TMP__",
os.path.abspath(sample_root): "__ROOT__",
}
(GOLDEN / "exit.txt").write_text(str(surface["code"]) + "\n")
(GOLDEN / "stdout.bin").write_bytes(normalize(surface["stdout"], roots))
(GOLDEN / "stderr.bin").write_bytes(normalize(surface["stderr"], roots))
files = {
str(k): hashlib.sha256(normalize(v, roots)).hexdigest()
for k, v in sorted(surface["files"].items())
}
(GOLDEN / "files.json").write_text(
json.dumps(files, indent=2, sort_keys=True) + "\n"
)
Hash file bodies after path normalization. Keep relative paths as map keys. Sort keys so git diffs stay small.
Commit samples next to those goldens. A fixture edit is a behavior change. Review it with the same care as code.
Prove the harness fails for real changes
A green first run is not evidence. Mutate one log line and rerun. The golden must go red on that edit.
# example mutation check — unexecuted here
python -c "from pathlib import Path; p=Path('messy_report.py'); t=p.read_text(); p.write_text(t.replace('done', 'DONE', 1))"
PYTHONHASHSEED=0 python characterize.py; echo exit:$?
git checkout -- messy_report.py
If goldens stay green, the harness is blind. Fix the recorder before any extract. Blind goldens are worse than no tests.
Mutate a timestamp format as a second check. The freeze must catch that class of drift. If it does not, expand the clock patch.
Mutate an absolute path in one JSON field. The token rewrite must catch that class. If it does not, sort replacements again.
Numbered split after the harness is green
- Commit goldens and the harness with no product edits.
- Extract one pure function that performs no I/O.
- Re-export that function from the original module path.
- Rerun the process-level characterization command without edits.
- Stop if exit, streams, or file hashes drift at all.
- Only then extract a second function in a new commit.
Do not move argparse during step two. Do not relocate logging configuration then either. Those extracts need their own goldens later.
Do not retouch comments to satisfy a linter yet. Comment-only diffs still hide a bad split. Keep the diff about the extract only.
Where a draft model fits
A model can propose the extract after goldens exist. It cannot invent the frozen surface for you.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option. Use that pair to draft the one-function extract. Paste the failing golden diff back when the draft drifts.
Keep the harness and samples on your machine. Do not treat hosted drafts as source of truth. Process exit and file hashes remain the gate.
This article does not name models, quotas, or hardware. Those details change and are not required here. The method only needs a draft plus a local check.
Decision table
| Change type | Freeze first | Extract in this commit? |
|---|---|---|
| Clock format in logs | Frozen datetime | No |
| Absolute path in JSON | Path tokens | No |
| Hash order in debug maps | PYTHONHASHSEED=0 |
No |
| Pure helper, bytes unchanged | Goldens green | Yes, one helper |
| Argparse layout | New CLI golden | Not this commit |
| Log wording cleanup | New stream golden | Not this commit |
Read the table before you open an editor. If the cell says no, stop. Characterization only pays when the change is tiny.
Limitations
This method misses in-process races. It misses network order without a fake transport. It misses float noise unless formats are frozen.
Binary files need explicit encodings. surrogateescape is a recorder default only. Do not copy it into production parsers.
Subprocess tests are slower than unit tests. They are not a substitute for type checks. They only lock the observable CLI contract.
Golden files rot when samples change. Pin fixtures beside the golden directory. Review fixture edits as behavior changes.
Hosted drafts can rewrite nearby code. Adjacent cleanup looks helpful and still breaks goldens. Reject any diff outside the named function.
Who should not use this
Do not use this on cryptographic code. Byte-stable logs are the wrong contract there. Do not use this on GUIs with layout noise.
Do not use this to rubber-stamp a large rewrite. A rewrite needs new goldens, not old ones. Do not use a model to replace the harness.
Skip this if outputs are not deterministic. Random IDs without seeds will never stabilize. Seed or inject those IDs first.
Skip this if the script must touch a live network. Recorded bytes then include foreign clocks and hosts. Fake the transport before you freeze goldens.
Close
Freeze time, cwd, and path tokens first. Record exit code, streams, and file hashes. Extract one function only after that surface holds.
Draft the extract on a free hosted model later. Fail closed if the golden files drift.
Top comments (0)