Freeze the Mess With Golden Files, Then Change One Seam

Do not rewrite a messy module on the first pass. Freeze every observable output with golden files first. Then change one seam that those files still cover.

A messy script is not an architecture problem yet. It is a measurement problem with mixed I/O. Until bytes stay still, every cleanup is a guess.

This article is a proposed workflow with labeled examples. It does not report production metrics or customer results. Treat the sample files as teaching fixtures only.

The failure mode in 2026

Coding agents now propose full-file rewrites cheaply. That cheapness does not create a specification. The live script still encodes the only honest contract.

Assumed invariants die first in brownfield report jobs. Hidden clocks, CSV quirks, and print side effects leak. Users keep depending on accidents nobody wrote down.

Characterization does not bless those accidents as product intent. It only blocks silent drift during one structural move. Feature changes still need an explicit owner decision.

What counts as messy here

Use this checklist before you open a rewrite branch.

1. The module writes files, stdout, or both together.
2. Time, locale, or cwd can change the bytes.
3. There are no tests around the current outputs.
4. Helpers cannot be named without moving I/O.
5. A model already offered a cleaner class design.

If four of those five hold, stop the rewrite. Pin outputs. Then extract one pure seam.

This is not a public-API snapshot of a clean library. The oracle is the file the script already emits. The unit under test is the whole messy entrypoint.

Labeled example: a tangled report job

The module below is synthetic on purpose. It mixes parsing, clocks, formatting, and disk writes. Use it as a stand-in for a brownfield report job.

# messy_report.py — labeled example, not production code
from datetime import datetime
from pathlib import Path
import csv
import sys

OUT = Path("out/report.html")

def run(path="in/orders.csv"):
    rows = list(csv.DictReader(open(path, newline="")))
    now = datetime.now().strftime("%Y-%m-%d")
    total = 0.0
    body = [f"<h1>Orders {now}</h1>", "<table>"]
    for row in rows:
        amt = float(row["amount"])
        total += amt
        flag = "late" if row["status"] == "late" else "ok"
        body.append(
            f"<tr><td>{row['id']}</td><td>{amt:.2f}</td><td>{flag}</td></tr>"
        )
    body.append(f"</table><p>total={total:.2f}</p>")
    OUT.parent.mkdir(parents=True, exist_ok=True)
    OUT.write_text("\n".join(body), encoding="utf-8")
    print(f"wrote {OUT} rows={len(rows)}")

if __name__ == "__main__":
    run(sys.argv[1] if len(sys.argv) > 1 else "in/orders.csv")

Four concerns share one function without names. The clock is not injectable today. The output path is a module global. Money format lives inside the write loop.

A rewrite would move all four concerns together. That combined move is the usual outage. The protocol below forbids that combined move.

Step 1 — Record a frozen input set

Create fixtures before any production edit. Keep one boring file and one ugly file.

fixtures/in/basic.csv
fixtures/in/ugly.csv

id,amount,status
1,10.5,ok
2,3.25,late

id,amount,status
10,0,ok
11,999999.999,late
12,-1.5,ok

Capture commands must not include a code edit. Copy outputs into a goldens directory immediately. Commit those copies only after the clock is frozen.

mkdir -p fixtures/goldens out
python messy_report.py fixtures/in/basic.csv
cp out/report.html fixtures/goldens/basic.html
python messy_report.py fixtures/in/ugly.csv
cp out/report.html fixtures/goldens/ugly.html

Live clocks will poison these files within a day. Do not treat the first copies as law yet. Step 2 makes the timestamp repeatable.

Step 2 — Add the smallest repeatability hook

This hook is still characterization, not a redesign. Patch time at the import site used by run. Leave formatting and file writes untouched.

# proposed characterization wrapper — unexecuted example
from datetime import datetime
from unittest.mock import patch
import messy_report

FIXED = datetime(2026, 9, 5, 12, 0, 0)

def capture(input_csv: str) -> str:
    with patch("messy_report.datetime") as dt:
        dt.now.return_value = FIXED
        messy_report.run(input_csv)
    return messy_report.OUT.read_text(encoding="utf-8")

Re-record goldens under that patched clock. Commit the hook and the goldens together. That commit must not change HTML structure or totals.

If datetime was imported as from datetime import datetime, the patch target changes. Patch the name the module actually binds. A wrong target leaves the clock live.

Step 3 — Install a byte-for-byte gate

The gate is ordinary pytest, not a new framework. Fail the build when emitted bytes diverge. Run it on every later structural edit.

# test_characterize_report.py — proposed harness
from pathlib import Path
from datetime import datetime
from unittest.mock import patch
import messy_report

FIX = Path("fixtures")
FIXED = datetime(2026, 9, 5, 12, 0, 0)

def _run(name: str) -> str:
    src = FIX / "in" / f"{name}.csv"
    with patch("messy_report.datetime") as dt:
        dt.now.return_value = FIXED
        messy_report.run(str(src))
    return messy_report.OUT.read_text(encoding="utf-8")

def test_golden_basic():
    actual = _run("basic")
    expected = (FIX / "goldens" / "basic.html").read_text(encoding="utf-8")
    assert actual == expected

def test_golden_ugly():
    actual = _run("ugly")
    expected = (FIX / "goldens" / "ugly.html").read_text(encoding="utf-8")
    assert actual == expected

python -m pytest test_characterize_report.py -q

A red test means the pin is incomplete. Do not extract helpers until both cases stay green. Green only means those two traces are stable.

Step 4 — Freeze-versus-change table

Write this table before touching production lines. If column three has two Yes marks, reject the diff.

Observation	Freeze now	Change now	Reason
HTML tag order and field order	Yes	No	Downstream scrapers may exist
Date text inside <h1>	Yes, via clock patch	No	Nondeterminism hides real diffs
Money text with .2f	Yes	No	Totals are an implicit contract
CSV header names	Yes	No	Upstream drops already exist
Global OUT path	Yes in tests	Not this commit	Extra seam, extra risk
late versus ok flags	Yes via goldens	No	Behavior is still unknown
Pure format_amount helper	N/A	Yes, only this	Smallest measurable seam
New template engine	No	No	Too large a jump
“Fix” negative amounts	No	No	That is a product change

The table is the load-bearing artifact in this protocol. Code without the table invites a second concern into the commit. Two concerns make golden failures unreadable.

Step 5 — Make the smallest safe change

Extract one pure function. Leave I/O in run. Money formatting has no clock and no filesystem.

def format_amount(amt: float) -> str:
    return f"{amt:.2f}"

Call it from the existing loop only. Do not rename variables in the same commit.

body.append(
    f"<tr><td>{row['id']}</td>"
    f"<td>{format_amount(amt)}</td>"
    f"<td>{flag}</td></tr>"
)
body.append(f"</table><p>total={format_amount(total)}</p>")

Re-run the golden gate before any other edit.

python -m pytest test_characterize_report.py -q

Green means the helper preserved recorded bytes. Red means the helper is not equivalent. Stop. Do not “improve” rounding while the gate is red.

Step 6 — When the gate goes red

Read the diff as a failure analysis, not as noise. Classify the delta before you touch goldens or code.

1. Timestamp changed: the clock patch missed an import path.
2. Only whitespace changed: a join or newline shifted.
3. One numeric cell changed: formatting is not equivalent.
4. Row count changed: the parser accepted different CSV rules.
5. File missing: OUT path now depends on cwd.

Update goldens only for an explicit behavior decision. Never update goldens to hide a refactor. If the helper caused the delta, revert the helper.

A useful command for that classification is a plain diff.

python -m pytest test_characterize_report.py -q --tb=short
diff -u fixtures/goldens/basic.html out/report.html

If the HTML is large, diff will still name the line. Humans still have to read that line. Golden tests do not replace judgment.

Step 7 — Draft extra traces without rewriting code

Two fixtures do not describe a messy module. More recorded traces beat imagined edge cases. A coding model may propose extra CSV files from current traces.

Keep the model away from messy_report.py in this step. Ask only for fixture candidates and review questions. You still capture goldens from the current program.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode provides free model access and a free server option, which can host this fixture-drafting and pytest loop. The model still must not edit production code before the gate exists.

A prompt that stays inside that boundary looks like this.

You are drafting characterization fixtures, not a rewrite.
Given messy_report.py and fixtures/in/basic.csv,
propose three additional CSV files that stress:
1) extra columns the parser should ignore
2) blank amount fields
3) mixed-case status values
Do not modify messy_report.py.
Do not invent expected HTML.
Goldens will be captured from the current script.

Review every proposed CSV by hand. Drop rows that cannot occur in your pipeline. Capture goldens only for files you keep.

If blank amounts currently crash, pin the crash. An expected exception is a valid characterization test. Do not fix that crash beside the helper extract.

Stop conditions

Stop after one green seam. Do not extract a second helper now. Do not rename globals. Do not switch path handling everywhere.

A later commit may inject a clock argument. A later commit may pass an output path. Each commit keeps the same golden bytes, or explains a golden update.

If a product owner wants different totals, that is a feature. Update goldens after that decision, in a dedicated commit. Refactor commits and behavior commits must not mix.

Limitations

Golden files lock today’s accidents into the suite. Encoding, trailing newlines, and OS line endings will bite. datetime patches are brittle across import styles.

Large HTML goldens hide the one drifted cell. Byte equality also fails on harmless whitespace. Normalized comparators need their own documented rules.

This protocol does not prove functional correctness. It proves stability under the recorded traces only. Behavior outside those traces remains unknown.

Models can hallucinate fixtures that never happen. They can also emit expected HTML without running code. Discard any expected output the current script did not produce.

A remote server does not freeze time by itself. If tests use the live clock, HTML will drift. Keep the clock patch inside the test process.

Who should not use this approach

Skip this when precise unit tests already guard the module. Skip this when current behavior is known to be harmful. Skip this on safety-critical paths that must change immediately.

Do not use golden HTML as a design spec for new apps. Do not characterize and “fix” money bugs together. Do not let a model rewrite the file before the gate exists.

Teams that never review fixtures will fossilize junk data. That outcome is worse than an untested script you still understand. The protocol needs a human editor for traces.

Checklist

1. Record inputs and outputs with a frozen clock.
2. Commit goldens before any structural edit.
3. Add a byte-for-byte pytest gate.
4. Fill the freeze-versus-change table in writing.
5. Extract one pure helper, then rerun the gate.
6. Optionally draft more CSVs, then recapture goldens.
7. Ship one seam. Open a new branch for the next seam.

The core conclusion does not depend on the editor you use. Freeze the mess with golden files first. Change one seam. Keep the bytes.