Lock Report Bytes Before You Extract One Function

Do not extract helpers from a messy report module yet. First lock every byte the job writes today.

Characterization tests record current behavior without blessing it. The smallest safe change is one extract behind those tests.

Why extracts fail in messy repos

Untested god modules hide parsing, aggregation, and file writes. A cleaner helper can still reorder rows or change rounding.

Generated patches make that class of failure cheap. Cheap edits without an oracle multiply silent report drift.

Numeric drift rarely shows up in a visual diff. Hash comparison catches it before anyone ships the report.

What you freeze, and what you ignore

Freeze output paths, file bytes, line counts, and exit codes. Ignore internal names, comment noise, and helper shape.

Do not freeze wall-clock time or random temp paths. Those signals are not behavior you intend to keep.

Also skip virtualenv paths leaked into error strings. Normalize those strings inside the characterization wrapper only.

Labeled example: a mixed report job

The Python below is a labeled proposal, not field history. It mixes CSV parsing, totals, and a JSON write.

# report_job.py — messy on purpose; proposal only
from __future__ import annotations

import csv
import json
import sys
from pathlib import Path


def run(csv_path: str, out_dir: str) -> int:
    rows = []
    with open(csv_path, newline="", encoding="utf-8") as handle:
        for row in csv.DictReader(handle):
            sku = (row.get("sku") or "").strip()
            if not sku:
                continue
            qty = int(row.get("qty") or "0")
            cents = int(row.get("cents") or "0")
            rows.append(
                {"sku": sku, "qty": qty, "cents": cents, "line": qty * cents}
            )

    totals = {}
    for row in rows:
        bucket = totals.setdefault(row["sku"], {"qty": 0, "cents": 0})
        bucket["qty"] += row["qty"]
        bucket["cents"] += row["line"]

    out = Path(out_dir)
    out.mkdir(parents=True, exist_ok=True)
    lines_path = out / "lines.json"
    totals_path = out / "totals.json"
    lines_path.write_text(
        json.dumps(rows, sort_keys=True, indent=2) + "\n", encoding="utf-8"
    )
    totals_path.write_text(
        json.dumps(totals, sort_keys=True, indent=2) + "\n", encoding="utf-8"
    )
    print(f"wrote {lines_path}")
    print(f"wrote {totals_path}")
    return 0


if __name__ == "__main__":
    raise SystemExit(run(sys.argv[1], sys.argv[2]))

The job reads one fixture and writes two artifacts. Totals live beside parsing, which makes extracts tempting.

sort_keys=True is required before you pin hashes. Without it, honest extracts can fail on key order.

A tiny checked-in fixture should look like this. Keep it boring, complete, and secret-free.

sku,qty,cents
A-1,2,199
A-1,1,199
B-9,0,50
,3,100
"C,9",4,50

Artifact: a file-ledger characterization harness

This harness hashes each output after a fixed fixture run. It also records line counts and the process exit code.

# characterize_report.py — proposal harness
from __future__ import annotations

import hashlib
import json
import subprocess
import sys
from pathlib import Path


def sha256_file(path: Path) -> str:
    digest = hashlib.sha256()
    with path.open("rb") as handle:
        for chunk in iter(lambda: handle.read(65536), b""):
            digest.update(chunk)
    return digest.hexdigest()


def run_job(job: Path, fixture: Path, out_dir: Path) -> dict:
    if out_dir.exists():
        for child in out_dir.iterdir():
            if child.is_file():
                child.unlink()
    out_dir.mkdir(parents=True, exist_ok=True)
    proc = subprocess.run(
        [sys.executable, str(job), str(fixture), str(out_dir)],
        check=False,
        capture_output=True,
        text=True,
    )
    files = []
    for path in sorted(p for p in out_dir.iterdir() if p.is_file()):
        files.append(
            {
                "name": path.name,
                "sha256": sha256_file(path),
                "bytes": path.stat().st_size,
                "lines": path.read_text(encoding="utf-8").count("\n"),
            }
        )
    return {
        "exit_code": proc.returncode,
        "stdout_lines": proc.stdout.splitlines(),
        "stderr_sha256": hashlib.sha256(proc.stderr.encode("utf-8")).hexdigest(),
        "files": files,
    }


def main() -> int:
    job, fixture, out_dir, ledger = map(Path, sys.argv[1:5])
    payload = run_job(job, fixture, out_dir)
    ledger.write_text(
        json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8"
    )
    print(ledger)
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Store goldens next to fixtures, never beside generated reports. Treat hash mismatches as failed characterization, not style noise.

python characterize_report.py report_job.py fixtures/normal.csv /tmp/out goldens/normal.json
python characterize_report.py report_job.py fixtures/empty.csv /tmp/out goldens/empty.json
python characterize_report.py report_job.py fixtures/messy.csv /tmp/out goldens/messy.json

Run the capture step once on a clean worktree. Commit goldens only after you inspect the raw files.

# assert_ledger.py — proposal
from __future__ import annotations

import json
import sys
from pathlib import Path

from characterize_report import run_job


def main() -> int:
    job, fixture, out_dir, golden = map(Path, sys.argv[1:5])
    actual = run_job(job, fixture, out_dir)
    expected = json.loads(golden.read_text(encoding="utf-8"))
    if actual != expected:
        print("ledger drift")
        print("expected", json.dumps(expected, sort_keys=True, indent=2))
        print("actual", json.dumps(actual, sort_keys=True, indent=2))
        return 1
    print("ledger ok", golden)
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

python assert_ledger.py report_job.py fixtures/normal.csv /tmp/out goldens/normal.json
python assert_ledger.py report_job.py fixtures/empty.csv /tmp/out goldens/empty.json
python assert_ledger.py report_job.py fixtures/messy.csv /tmp/out goldens/messy.json

Numbered workflow

1. List every file the job creates

Execute the job against a tiny checked-in fixture. Record stdout, stderr, exit code, and created paths.

Delete unknown files from the output directory first. A dirty output dir will poison every later hash.

2. Pin a fixture corpus

Keep three fixtures: empty, normal, and messy rows. Messy rows must include quotes, blanks, and duplicate keys.

Do not generate fixtures from the module under change. Circular fixtures hide the bugs you need to see.

Name fixtures after the behavior, not after ticket numbers. Ticket names go stale and confuse later reviewers.

3. Capture golden hashes

Hash file bytes with sha256, not a pretty printer. Pretty printers hide key order and trailing whitespace.

Write a JSON ledger with path, sha256, and line_count. Keep one ledger file per fixture name.

Stdout lines belong in the ledger when they are stable. Drop timestamps from stdout before hashing that stream.

4. Fail the build on ledger drift

Re-run the job in a temporary directory every edit. Compare the new ledger to the committed golden ledger.

Any hash change is a failed test, full stop. Do not skip stderr in the ledger comparison.

Warnings often encode real branches you still need. Wire the three assert commands into the existing test runner.

Local runs must match CI, including Python version. A hidden interpreter mismatch will look like job drift.

5. Extract one pure function

Choose one function with no filesystem and no clock. Totals or tax rounding are usual first candidates.

Move only that function and leave parse mixed. Writers stay in run until a later change.

If the extract needs extra arguments, stop and split later. Argument lists that grow fast mean the cut was wrong.

A labeled extract target for the sample job is totals. Keep CSV reading inside run for this change.

# labeled extract; apply only after goldens stay green
def accumulate_totals(rows: list[dict]) -> dict:
    totals = {}
    for row in rows:
        bucket = totals.setdefault(row["sku"], {"qty": 0, "cents": 0})
        bucket["qty"] += row["qty"]
        bucket["cents"] += row["line"]
    return totals

Call accumulate_totals from run and write the same files. Do not rename keys during that first cut.

6. Re-run the ledger, then stop

Green ledgers mean the extract preserved observable bytes. Do not chain a second extract in the same change.

Commit the extract with the unchanged golden files. A golden diff in that commit means you went too far.

If goldens change, revert the extract and inspect the writer. Most accidental diffs are key order or extra newlines.

Decision table

Use this table before you accept any extract patch. When two rows conflict, keep the module mixed.

Signal	Extract now	Wait
Filesystem in the function	No	Yes
Clock, network, or RNG	No	Yes
Golden ledger green on three fixtures	Yes	No
Patch also rewrites goldens	No	Yes
New arguments exceed two	No	Yes
stdout or stderr text shifts	No	Yes

The table is a gate, not a scoring rubric. One red cell is enough to reject the patch.

Reading a failed ledger

A hash mismatch on totals.json means aggregation changed. Check integer division, key filters, and skip rules.

A hash mismatch on lines.json means parse changed. Check strip logic, blank rows, and DictReader restkeys.

Exit code change means a new exception path fired. Capture stderr and match it to the fixture row.

Line count drop with a stable hash cannot happen. Treat that pair as harness bug, not job drift.

Byte size growth with a stable line count means padding. Look for extra spaces, BOM marks, or changed indent.

After goldens: a free model can draft the cut

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option.

Use them only after the golden ledger is already green. Paste the pure function and the characterization command, nothing else.

Ask for one extract that keeps the ledger hashes stable. Reject any patch that rewrites fixtures or goldens.

Those files are the oracle, not model scratch space. The harness remains useful if you never call a model.

If outbound code is allowed, try one extract draft there. If policy blocks it, keep the harness and extract by hand.

Limitations

Hash equality does not prove the report is correct. It only proves the bytes match the last pinned run.

Nondeterministic JSON key order will fail honest extracts. Sort keys in the writer before you pin goldens.

Binary assets and timestamps inside JSON will thrash hashes. Strip or freeze those fields in the characterization wrapper.

This workflow does not replace unit tests for new logic. New branches still need explicit cases after the extract.

Large binary reports make sha256 slow on tiny laptops. Hash a canonical text projection instead of raw binaries.

Who should not use this

Skip this if the job has no stable fixture corpus. Skip it if outputs include secrets or customer payloads.

Skip it when the change must alter report bytes on purpose. Update goldens in a dedicated commit with a human diff.

Teams without hash tooling can start with line counts only. Line counts catch drops, not silent numeric drift.

Do not use this method to bless known wrong totals. Pinning a bug is valid only when you schedule a later fix.

What this does not claim

No timing data is offered for models or servers here. No model names, quotas, or hardware claims are made.

The method stands if you never call an external model. The ledger is the only required product of this workflow.

The extract remains optional after the hashes stay green. Stop when the first pure function is isolated.

Merge checklist

1. Confirm fixture files are checked in and free of secrets. Drop any customer payload before the commit.
2. Confirm golden ledgers match a clean run on this branch. Confirm stderr, exit code, and hashes stay recorded.
3. Confirm the diff extracts one function and leaves writers mixed. Confirm no second extract rides along in the same patch.

Messy repos become safer when bytes are pinned first. Extract once, then stop, then schedule the next cut.