Freeze CLI Side Effects Before One God-Module Split

Messy-repo refactors fail at the boundary, not the rename. Green tests still miss files, clocks, and stdout. Freeze those observables before any module extract. Then extract exactly one function and stop.

A later split can keep helpers equal. The written report still changes. CI stays green while customers see drift. That gap is characterization, not naming.

The failure this workflow targets

A god-module often writes reports and prints JSON. It also mutates cwd as a hidden step. Internal unit tests assert helper return values only. After a split, those helpers can stay equal. File order, key order, or timestamps still move. The public artifact changes without a red test.

This workflow freezes the CLI contract first. It then allows one extract. It does not start with a new facade. It does not start with a model-written rewrite.

What to freeze before any extract

Pin the public contract, not private helpers. Record exit code, stdout bytes, and written files. Record relative paths and content hashes together. Record the clock only if reports embed time.

Observable	Freeze method	Split risk if skipped
exit code	assert integer equality	wrapper swallows errors
stdout JSON	parse, then compare canonical form	key order or type drift
files written	sorted relative paths	extra sidecar appears
file bytes	sha256 after canonicalize	whitespace or float drift
cwd	run only under tmp_path	leftover files in repo
env	explicit dict, no ambient bleed	locale and timezone leaks
clock	freeze datetime or drop field	timestamps churn goldens

Skip fields you cannot stabilize on demand. Do not freeze wall-clock strings by default. Drop the field when the report allows it. Canonicalize JSON before hashing any bytes.

Lab fixture, not production telemetry

The module below is a synthetic messy report path. Treat it as a lab fixture only. Do not treat it as measured production traffic.

# messy_report.py — lab fixture for characterization
from __future__ import annotations

import json
import os
from datetime import datetime, timezone
from pathlib import Path


def run(argv: list[str]) -> int:
    out_dir = Path(argv[1] if len(argv) > 1 else "out")
    out_dir.mkdir(parents=True, exist_ok=True)
    payload = {
        "generated_at": datetime.now(timezone.utc).isoformat(),
        "files": sorted(os.listdir(".")),
        "cwd": str(Path.cwd()),
        "count": float(len(list(Path(".").glob("*.txt")))),
    }
    text = json.dumps(payload, indent=2)
    print(text)
    (out_dir / "report.json").write_text(text, encoding="utf-8")
    (out_dir / "report.txt").write_text(
        f"count={payload['count']}\n", encoding="utf-8"
    )
    return 0

The contract is messy on purpose here. It mixes cwd, clock, and float formatting. A naive extract will move at least one field. Characterization must fail before that extract lands.

Numbered workflow

1. Isolate a disposable workspace

Create a temp tree with known inputs. Do not characterize against the real repo root. Ambient files will pollute os.listdir results fast.

python -m venv .venv
. .venv/bin/activate
pip install pytest
mkdir -p lab/data
printf 'a\n' > lab/data/a.txt
printf 'b\n' > lab/data/b.txt

Keep the fixture module beside that tree. Keep production packages off PYTHONPATH during the capture. One mixed import path will poison the golden.

2. Capture one golden run under pins

Freeze cwd, argv, env, and hash seed. Drop time or stub time next. Hash seed still matters for some maps. Locale still matters for some number formats.

cd lab
PYTHONHASHSEED=0 TZ=UTC LC_ALL=C \
  python -c "from messy_report import run; raise SystemExit(run(['prog','out']))"
echo exit:$?
find out -type f | sort

Record the exit code from that process. Save stdout to a sidecar file. List every path under out/ in sorted order. Do not trust directory mtime as a contract.

3. Canonicalize before you store bytes

Raw pretty JSON is a weak golden. Indent, key order, and time all churn. Load JSON, drop unstable fields, then dump with sorted keys. Hash that canonical form, not the pretty printer.

# canonicalize.py — lab helper
from __future__ import annotations

import hashlib
import json
from typing import Any

UNSTABLE = frozenset({"generated_at", "cwd"})


def canonical_bytes(payload: dict[str, Any]) -> bytes:
    stable = {k: v for k, v in payload.items() if k not in UNSTABLE}
    text = json.dumps(stable, sort_keys=True, separators=(",", ":"))
    return text.encode("utf-8")


def sha256_hex(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()

Drop cwd because tests already control tmp_path. Drop generated_at unless a freeze clock exists. Keep count only after you fix float formatting. A later extract must not change that hash.

4. Encode the golden as tests

The tests below are executable characterization, not style checks. They pin exit code, stdout keys, and written paths. They also pin canonical hashes for both outputs.

# test_messy_report_contract.py
from __future__ import annotations

import io
import json
import os
from contextlib import redirect_stdout
from pathlib import Path

import pytest

from canonicalize import canonical_bytes, sha256_hex
from messy_report import run


def _write_inputs(root: Path) -> None:
    data = root / "data"
    data.mkdir()
    (data / "a.txt").write_text("a\n", encoding="utf-8")
    (data / "b.txt").write_text("b\n", encoding="utf-8")


def test_report_contract(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
    _write_inputs(tmp_path)
    monkeypatch.chdir(tmp_path)
    monkeypatch.setenv("TZ", "UTC")
    monkeypatch.setenv("LC_ALL", "C")
    os.environ["PYTHONHASHSEED"] = "0"

    buf = io.StringIO()
    with redirect_stdout(buf):
        code = run(["prog", str(tmp_path / "out")])

    assert code == 0
    stdout_payload = json.loads(buf.getvalue())
    assert set(stdout_payload) >= {"files", "count"}

    out_dir = tmp_path / "out"
    written = sorted(p.relative_to(out_dir).as_posix() for p in out_dir.rglob("*"))
    assert written == ["report.json", "report.txt"]

    report = json.loads((out_dir / "report.json").read_text(encoding="utf-8"))
    assert sha256_hex(canonical_bytes(report)) == sha256_hex(
        canonical_bytes(stdout_payload)
    )
    assert (out_dir / "report.txt").read_text(encoding="utf-8") == "count=0.0\n"

Note the expected count=0.0 line. The fixture globs *.txt in cwd, not under data/. That surprise is the point of characterization. A cleanup extract might "fix" the glob and break the file.

Run the tests before touching structure.

PYTHONHASHSEED=0 pytest -q test_messy_report_contract.py

If this test is red, stop the refactor. Repair the freeze list first. Do not extract on a moving golden.

5. Make the smallest safe change

After the contract test is green, extract one function. Keep run() as the CLI owner. Move payload assembly or file writes, not both.

# proposed extract — review against the contract test
from __future__ import annotations

import json
from pathlib import Path
from typing import Any


def write_reports(out_dir: Path, payload: dict[str, Any]) -> None:
    text = json.dumps(payload, indent=2)
    print(text)
    out_dir.mkdir(parents=True, exist_ok=True)
    (out_dir / "report.json").write_text(text, encoding="utf-8")
    (out_dir / "report.txt").write_text(
        f"count={payload['count']}\n", encoding="utf-8"
    )

That extract is a proposal, not a measured win. Re-run the same contract test after the move. If hashes move, revert the extract immediately. Do not stack a second extract on a red contract.

A safe change set stays tiny. One function. One call site. Same argv. Same files.

6. Use a model only after the freeze

A model can draft the extract diff later. It should not invent the golden. Local tests remain the only merge gate.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can host that draft step. Feed the frozen test file and the god-module only. Ask for one extract that keeps the contract test green. Reject extra cleanup, glob "fixes," and formatter churn.

Do not send production secrets to any hosted model. Do not treat a free server as a CI replacement. Re-run pytest on your machine after the draft returns.

Failure analysis checklist

Use this list when the contract test flips red.

1. Diff written path lists before blaming JSON.
2. Parse both JSON blobs, then compare canonical hashes.
3. Check whether count changed type from int to float.
4. Check whether pretty indent changed hash input by mistake.
5. Check cwd: did the extract start listing data/?
6. Check env: did locale rewrite the text report line?
7. Check clock: did a dropped field return in stdout only?

Most red tests after a split are extra files. The second cluster is canonical JSON drift. Clocks rank third when goldens store strings.

Limitations

This method needs a freezeable public contract. It needs a temp directory and a test runner. It does not prove internal algorithmic equality. It does not prove performance, security, or thread safety.

Canonical JSON drops unknown keys only when you list them. A new sidecar file still needs an updated path list. Float formatting remains a silent killer in text reports. Hash seed pins do not freeze every interpreter detail.

Hosted free models can draft wrong extracts. They may rename keys that tests do not pin. They may "fix" the glob that the golden encoded. That is why the contract test runs first.

Who should not use this approach

Skip this if the module has no observable CLI. Skip this if outputs are encrypted blobs you cannot canonicalize. Skip this if the only contract is a live network round trip. Skip this if policy forbids sending source to a hosted model.

Also skip stacked extracts in one pull request. One frozen contract plus one function move is the unit. Broader architecture work needs a different plan.

Close

Freeze exit code, paths, and canonical hashes first. Extract one function only after that test is green. Keep models downstream of the freeze, never upstream of it.