DEV Community: Dakota Huang

Pin Walk Order and Byte Totals Before One Relpath Extract

Dakota Huang — Sun, 20 Sep 2026 04:22:53 +0000

Do not extract a path helper from a messy walker first. Freeze sorted relative paths and integer byte totals before any cut. Glob order, symlink follows, and float rounding change reports without a traceback.

Silent drift, not a crash

os.walk follows disk order, not your mental model. Two clones can list the same files in different sequences. JSON dumps then shuffle object keys unless you sort them.

A size total stored as float drifts across platforms. One extra symlink visit doubles a counted directory. Characterization must pin those three channels before a cleanup commit.

Exit-code-only tests miss this class of bug. The process still returns zero. The report JSON just permutes, and reviewers treat the diff as noise.

A tangled reporter worth pinning

The module below is a labeled proposal, not production history. It mixes walking, formatting, and I/O in one function. That mix is the refactor target, not a claimed private codebase.

# proposal: messy inventory reporter (not a live service)
from __future__ import annotations

import json
import os
from pathlib import Path
from typing import Any


def dump_inventory(root: str, out_path: str) -> dict[str, Any]:
    report: dict[str, Any] = {"root": root, "entries": [], "bytes": 0.0}
    for dirpath, dirnames, filenames in os.walk(root, followlinks=True):
        dirnames.sort()
        for name in filenames:
            full = os.path.join(dirpath, name)
            try:
                st = os.stat(full, follow_symlinks=True)
            except OSError as exc:
                report["entries"].append({"path": full, "error": str(exc)})
                continue
            report["entries"].append(
                {
                    "path": full,
                    "size": float(st.st_size),
                    "mtime": st.st_mtime,
                }
            )
            report["bytes"] += float(st.st_size)
    Path(out_path).write_text(json.dumps(report, indent=2), encoding="utf-8")
    return report

The function writes a file and returns a nested dict. Tests that only assert a zero exit miss reorder bugs. Pin the dict shape after canonicalization, not the raw print stream.

Absolute paths embed the temp root into every golden. Float sizes and raw st_mtime values embed host noise. followlinks=True can re-enter the same tree through a symlink.

What to pin, and what to drop

Pin four observables and ignore the rest on purpose. Relative paths must be POSIX-style and lexicographically sorted. The walker must set followlinks to false unless a test says otherwise.

Byte totals must be integers, never floats from os.stat. JSON must use sort_keys and separators that you freeze. Skip inode, device, and owner fields; they are host noise.

Skip mtime unless a downstream consumer prints timestamps. If you must print times, convert to UTC and drop subseconds. Locale-dependent date strings are not a stable golden.

Unicode names need an explicit normalization form. NFC and NFD can look identical in a terminal. They are not equal as golden strings.

Freeze protocol

Isolate the reporter behind a function that accepts a root path. Keep stdout unused during characterization. Return a dict the test can canonicalize.
Build a temp fixture tree with files, a gap directory, and a symlink. Include one broken symlink and one nested empty folder. Keep the fixture under 20 files so diffs stay readable.
Run the reporter twice against that tree in one process. Do not rebuild the tree between the two calls. A pure read path must be stable inside one run.
Canonicalize both outputs with the same dump helper. Compare the encoded text, not Python object identity. Hash the text if the payload grows later.
Fail the test if the two canonical payloads differ. Intra-process drift means the walker is already unsafe. Do not extract helpers until that check is green.
Write the canonical payload to a committed golden file. Store it next to the test, not in /tmp. Reviewers then see inventory changes as test diffs.
Re-run on a second machine or container before you extract. Linux and macOS can disagree on symlink stat results. Windows path separators will fail an unnormalized golden.
Only then extract one relpath helper and keep the walker intact. Do not sort, filter, or change followlinks in the same commit. One behavior change per diff.

# proposal: characterization harness
import hashlib
import json
import os
import tempfile
import unittest
from pathlib import Path

GOLDEN = Path(__file__).with_name("inventory.golden.json")


def posix_rel(root: str, full: str) -> str:
    rel = os.path.relpath(full, root)
    return rel.replace("\\", "/")


def canonicalize(report: dict) -> str:
    entries = []
    for item in report["entries"]:
        path = item["path"]
        if path.startswith(report["root"]):
            path = posix_rel(report["root"], path)
        entries.append(
            {
                "path": path,
                "size": int(item.get("size", 0)),
                "error": item.get("error"),
            }
        )
    entries.sort(key=lambda row: row["path"])
    payload = {
        "entries": entries,
        "bytes": int(report["bytes"]),
        "count": len(entries),
    }
    return json.dumps(payload, sort_keys=True, separators=(",", ":"))


def build_tree(root: Path) -> None:
    (root / "a").mkdir()
    (root / "a" / "one.txt").write_bytes(b"aaaa")
    (root / "b").mkdir()
    (root / "b" / "two.txt").write_bytes(b"bb")
    (root / "b" / "gap").mkdir()
    (root / "link_to_a").symlink_to(root / "a", target_is_directory=True)
    (root / "broken").symlink_to(root / "missing.txt")


class InventoryGoldens(unittest.TestCase):
    def test_intra_process_stability_and_golden(self) -> None:
        with tempfile.TemporaryDirectory() as td:
            root = Path(td)
            build_tree(root)
            from inventory_report import dump_inventory  # labeled proposal import

            first = canonicalize(dump_inventory(str(root), str(root / "out1.json")))
            second = canonicalize(dump_inventory(str(root), str(root / "out2.json")))
            self.assertEqual(first, second)
            if not GOLDEN.exists():
                GOLDEN.write_text(first + "\n", encoding="utf-8")
                self.fail("wrote inventory.golden.json; re-run to pin")
            expected = GOLDEN.read_text(encoding="utf-8").strip()
            self.assertEqual(first, expected)
            digest = hashlib.sha256(first.encode("utf-8")).hexdigest()
            self.assertEqual(len(digest), 64)

The first run may write the golden and fail on purpose. That is a recording step, not a product bug. Commit the file only after you inspect every path and byte total.

Decision table

Signal	Pin in golden	Skip	Reason
Relative POSIX paths, sorted	Yes		Walk order otherwise shuffles JSON arrays
Integer byte totals	Yes		Float addition is not a stable identity
`followlinks=False`	Yes		Symlink loops inflate counts
Broken symlink error string class	Yes		Missing targets must stay visible
Raw `st_mtime`		Skip	Host clocks and FS precision differ
Inode / device ids		Skip	Values are machine-specific
Absolute temp roots		Skip	CI workspaces never match laptops
Pretty-printed JSON whitespace		Skip	Canonical dump already freezes separators

Use the table when a reviewer asks to "just snapshot stdout." Stdout still contains host paths. The table names the exact fields that survive a machine change.

The only allowed change

Extract _relpath(root, path) -> str and nothing else. Leave os.walk, os.stat, and JSON writing in the original function. The helper must not follow symlinks and must not call stat.

# proposal: smallest extract after goldens pass
def _relpath(root: str, full: str) -> str:
    rel = os.path.relpath(full, root)
    return rel.replace("\\", "/")

Wire the helper only at the dict-building site. Do not "improve" error messages in the same patch. Do not switch followlinks to false in that commit if the golden still encodes the old visits.

If the golden still includes symlink-followed duplicates, split the work. First commit: pin current follow behavior. Second commit: set followlinks=False and update the golden in isolation. Third commit: extract _relpath.

Commands that keep the pin honest

python -m unittest test_inventory_goldens -v
python -c "import json,pathlib; p=pathlib.Path('inventory.golden.json'); print(p.read_text()[:200])"
git diff -- testdata/inventory.golden.json

Run the unittest module, not a custom wrapper, on the first pass. Print a prefix of the golden so you see POSIX slashes. Inspect git diff on the golden before you squash.

Add a second command that hashes the golden in CI. A hash mismatch is cheaper to read than a huge JSON fail. Keep the full JSON for humans, the hash for the gate.

python -c "import hashlib,pathlib; b=pathlib.Path('inventory.golden.json').read_bytes(); print(hashlib.sha256(b).hexdigest())"

Failure analysis

Windows backslashes break an unnormalized path golden. NFC versus NFD names break macOS checkouts of the same repo. Trailing slashes on root change relpath output for the root itself.

Concurrent writers in the fixture directory race the walker. Network filesystems can return stale stat sizes. Short-lived temp files from antivirus scanners appear as extra entries.

Broken symlinks must be present in the fixture. If you omit them, the extract can swallow OSError later. Empty directories never appear in filenames; pin that absence explicitly in comments, not in fake rows.

Who should not use this approach

Skip this protocol on a greenfield package with pure functions. You already control order and types. Characterization goldens would only freeze accidental design.

Skip it when the walker is a secret scanner. Inventory snapshots can copy credential filenames into git. Redact or exclude those trees before recording.

Skip it when no test runner exists yet. A committed golden without an automated compare will rot. Do not extract helpers on reviewer memory alone.

Skip it for one-off scripts you will delete the same day. The freeze cost exceeds the edit cost. Use the protocol on modules that will outlive the current ticket.

Limitations

This artifact does not prove semantic correctness of the report. It only proves the next edit did not change pinned bytes. A wrong total that is stable will stay wrong.

It does not replace property tests for path traversal. A helper that accepts .. can still escape root. Add a dedicated unit test for that case after the extract.

It does not pin permission bits or owner names. Those fields need a dedicated fixture user, which most CI images lack. Leave them out of the golden.

Intra-process stability does not prove cross-process stability. Environment variables, working directory, and umask can still leak. Re-run the golden under a clean cwd in CI.

After the golden exists

A coding assistant is optional once the fixture fails locally. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can draft the _relpath extract against that fixture. The committed golden remains the acceptance check, not the model output.

Reject any patch that retouches walk order and path math together. Reject any patch that rewrites error strings to chase a prettier golden. Keep the reporter boring until the helper has its own tests.

Pin walk order and integer byte totals first. Extract one relpath helper second. Treat every extra cleanup as a later ticket with its own golden diff.

Pin Path Sets and Duplicate Winners Before One Glob Extract

Dakota Huang — Sun, 20 Sep 2026 01:19:41 +0000

Live glob order is a flaky golden. Pin path sets, content hashes, and duplicate winners first. Extract one collector only after those contracts are frozen.

Filesystem walk order changes across disks. It also changes across containers. A sorted() cleanup looks like hygiene. It is a behavior change when two files share a key.

Core contract, not the walk

glob.glob does not promise lexical order. Python documents that as platform-dependent. Treat the returned sequence as an implementation accident. Treat membership and collision policy as the product contract.

AI-assisted refactors add sorted() constantly. They also surface swallowed errors. Both edits can be correct later. Neither belongs in the first cut.

Observables: pin or ignore

Score each field before you touch the module. Use the table as a merge gate.

Observable	Pin in golden?	Why
Relative path set	Yes	Membership is the collector contract
sha256 of each file	Yes	Parsers depend on exact bytes
Duplicate-key winner	Yes	Hidden order dependency
Item count and key union	Yes	Detects dropped or extra records
Raw `glob.glob` order	No	Varies by inode layout
mtime, ctime, inode	No	CI checkouts rewrite them
Absolute prefixes	No	Temp roots and hosts differ
Symlink skip policy	Yes	Changes which bytes get read
Decode and JSON errors	Yes	Current code swallows them

Do not freeze wall-clock stamps. Do not freeze device numbers. Do not freeze live walk order on a real disk.

Example messy module

This fixture is a teaching module. It is unlabeled production code. Treat it as unexecuted until you run it.

# inventory.py
from __future__ import annotations

import glob
import json
import os
from typing import Any

ROOT = os.environ.get("INV_ROOT", "data")


def load_items() -> list[dict[str, Any]]:
    pattern = os.path.join(ROOT, "**", "*.json")
    paths = glob.glob(pattern, recursive=True)
    items: list[dict[str, Any]] = []
    for path in paths:
        if os.path.islink(path):
            continue
        try:
            raw = open(path, "rb").read()
            payload = json.loads(raw.decode("utf-8"))
        except (OSError, UnicodeDecodeError, json.JSONDecodeError):
            continue
        if not isinstance(payload, dict):
            continue
        sku = payload.get("sku")
        if not sku:
            continue
        payload["_source"] = path
        items.append(payload)
    return items

Callers often take the first matching SKU. That makes glob order a silent policy. The skipped errors are also a policy. Leave both policies intact for the first extract.

Stub glob, do not golden the disk

Live tmp_path layouts still race on order. Stub glob.glob with a tuple you control. Then the loop contract is deterministic. The filesystem remains unpinned noise.

# test_inventory_char.py
from __future__ import annotations

import hashlib
import json
from pathlib import Path

import inventory

GOLDEN = Path(__file__).with_name("inventory_char.json")


def _write_tree(root: Path) -> dict[str, Path]:
    root.mkdir(parents=True, exist_ok=True)
    a = root / "a.json"
    b = root / "nested" / "b.json"
    dup = root / "nested" / "dup.json"
    bad = root / "nested" / "bad.json"
    txt = root / "nested" / "ignore.txt"
    b.parent.mkdir(parents=True, exist_ok=True)
    a.write_text('{"sku": "A", "qty": 1}', encoding="utf-8")
    b.write_text('{"sku": "B", "qty": 2}', encoding="utf-8")
    dup.write_text('{"sku": "A", "qty": 99}', encoding="utf-8")
    bad.write_text("{not-json", encoding="utf-8")
    txt.write_text("nope", encoding="utf-8")
    link = root / "link.json"
    link.symlink_to(a)
    return {"a": a, "b": b, "dup": dup, "bad": bad, "link": link}


def _sha256(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()


def _snapshot(items: list[dict]) -> dict:
    sources = [i.get("_source", "") for i in items]
    by_sku: dict[str, list[int]] = {}
    for i in items:
        by_sku.setdefault(str(i.get("sku")), []).append(int(i.get("qty", -1)))
    first_qty_for_a = next((i.get("qty") for i in items if i.get("sku") == "A"), None)
    return {
        "count": len(items),
        "skus_sorted": sorted(str(i.get("sku")) for i in items),
        "first_sku": items[0]["sku"] if items else None,
        "first_qty_for_sku_A": first_qty_for_a,
        "qty_by_sku": {k: v for k, v in sorted(by_sku.items())},
        "source_names": sorted(Path(s).name for s in sources),
        "key_union": sorted({k for i in items for k in i.keys()}),
    }


def test_load_items_pins_sets_and_duplicate_winner(tmp_path, monkeypatch):
    files = _write_tree(tmp_path)
    monkeypatch.setenv("INV_ROOT", str(tmp_path))
    ordered = [
        str(files["dup"]),
        str(files["link"]),
        str(files["a"]),
        str(files["bad"]),
        str(files["b"]),
    ]
    monkeypatch.setattr(inventory.glob, "glob", lambda *a, **k: list(ordered))

    items = inventory.load_items()
    snap = _snapshot(items)
    snap["hashes"] = {
        name: _sha256(path)
        for name, path in files.items()
        if path.is_file() and not path.is_symlink()
    }

    if not GOLDEN.exists():
        GOLDEN.write_text(json.dumps(snap, indent=2, sort_keys=True) + "\n")
    expected = json.loads(GOLDEN.read_text(encoding="utf-8"))
    assert snap == expected

Record the golden once on a quiet tree. Commit that file in the same change set. Later runs must match it byte for byte after sort_keys.

The stub list is the experiment knob. link.json stays in the list. The loop still skips it. bad.json stays in the list. The loop still swallows the decode error. sku A appears twice. The first remaining row is the winner for naive callers.

Numbered extract sequence

Follow this order. Do not skip a step.

Commit the messy module without edits.
Add the stubbed characterization test.
Generate inventory_char.json on one machine.
Re-run the test on a second path layout.
Confirm hashes and winners still match.
Extract iter_json_paths() as a one-line wrapper.
Keep load_items() iterating that sequence.
Point the stub at iter_json_paths if needed.
Reject sorts, filters, and new error lists.
Stop. Do not fold in extra cleanup.

The extract should look like this. Anything larger is a second change.

def iter_json_paths(root: str) -> list[str]:
    pattern = os.path.join(root, "**", "*.json")
    return glob.glob(pattern, recursive=True)


def load_items() -> list[dict[str, Any]]:
    items: list[dict[str, Any]] = []
    for path in iter_json_paths(ROOT):
        if os.path.islink(path):
            continue
        try:
            raw = open(path, "rb").read()
            payload = json.loads(raw.decode("utf-8"))
        except (OSError, UnicodeDecodeError, json.JSONDecodeError):
            continue
        if not isinstance(payload, dict):
            continue
        sku = payload.get("sku")
        if not sku:
            continue
        payload["_source"] = path
        items.append(payload)
    return items

Keep the body of the loop untouched. The helper returns the same list object shape. It does not sort. It does not uniquify SKUs. It does not follow symlinks.

What the golden must reject

Add two negative checks after the extract. They document the change you refused.

def test_reject_sorted_collector(tmp_path, monkeypatch):
    files = _write_tree(tmp_path)
    monkeypatch.setenv("INV_ROOT", str(tmp_path))
    ordered = [str(files["dup"]), str(files["a"]), str(files["b"])]
    monkeypatch.setattr(inventory, "iter_json_paths", lambda root: list(ordered))
    items = inventory.load_items()
    assert items[0]["qty"] == 99

    monkeypatch.setattr(
        inventory, "iter_json_paths", lambda root: sorted(ordered)
    )
    sorted_items = inventory.load_items()
    # lexical a.json before dup.json on this layout
    assert sorted_items[0]["qty"] != items[0]["qty"]


def test_reject_surfacing_bad_json(tmp_path, monkeypatch):
    files = _write_tree(tmp_path)
    monkeypatch.setenv("INV_ROOT", str(tmp_path))
    monkeypatch.setattr(
        inventory,
        "iter_json_paths",
        lambda root: [str(files["bad"]), str(files["b"])],
    )
    items = inventory.load_items()
    assert len(items) == 1
    assert items[0]["sku"] == "B"

The first test proves sorted() flips the SKU A winner. The second test proves a raised parse error would be a new contract. Hold both for a later, explicit change.

Free-model pass after goldens only

A model can propose the helper text. It cannot choose the observables. Write the goldens yourself. Then ask the model for a one-function extract that keeps the loop body identical.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can draft that extract against the committed tests. They do not replace the pin step. If the patch adds sorted() or an errors list, the tests fail and the patch is discarded.

Limitations

This harness does not prove thread safety. It does not prove encoding on Windows code pages. It does not prove ** glob behavior on every Python build. glob.glob still ignores some symlink directories depending on platform. Path.rglob is a different iterator and needs its own pins.

Stubbing glob.glob hides real permission errors. Add one integration test on a tiny fixture tree. Assert only the path set and hashes there. Leave order out of that test.

Duplicate winners depend on the stub sequence. Production order remains unstable. If product code needs a stable winner, make that a second change. Use an explicit min(path) or last-write rule. Do not pretend glob order is that rule.

Open file handles use a bare open. That is part of the mess. Do not convert it to a context manager in the same patch. Handle leaks are a separate characterization target.

Who should not use this

Do not use this flow on cryptographic parsers. Do not use it on access-control filters. Do not use it when dropped JSON must become a hard failure. Those cases need designed tests, not captured silence.

Do not use it as a substitute for schema checks. Hashes freeze bytes, not meaning. A valid JSON object with the wrong units will still match.

Skip this extract if the module already has one seam. Skip it if two collectors exist. Join them only after each has goldens.

Stop condition

The change is done when iter_json_paths exists. The loop body must match the prior bytes. Goldens must still pass on the stubbed sequence. No sort. No new logging. No error-channel redesign.

If a later change needs stable order, add a named sort at the call site. Record a new golden for that call site only. Keep the collector unordered. That split is the whole point of the pin.

Pin URL Query Encoding Before One Client Extract

Dakota Huang — Sat, 19 Sep 2026 22:07:07 +0000

Pin URL query encoding before you extract any HTTP client helper. Callers often depend on plus signs, sorted keys, and hidden safe characters. A cleanup extract can change every signed request without failing unit tests.

Messy repos bury urlencode calls across adapters, retries, and debug dumps. Each site passes a slightly different doseq, safe, or quote_via. Characterization tests freeze those contracts before any move.

The contract you actually ship

Query strings are not a cosmetic serialization detail. Signature schemes hash the exact bytes on the wire. Caching layers key on the encoded form, not the dict.

Pin at least five observable fields before you touch production clients.

Space encoding chooses plus signs or percent-twenty sequences.
Key order follows caller order or an implicit sort.
Sequence folding depends on the doseq boolean flag.
Extra unescaped characters come from the safe argument.
Empty values are omitted, set to key-equals, or bare keys.

Skip this list and the extract will look green. Downstream auth will still break on the first replay.

Decision table for one extract

Use this table before you allow a helper to exist. Every row is a frozen contract, not a preference.

Observable	Keep as-is	Forbidden during extract
Space	Current quote_via	Switching plus and percent-twenty
Order	Current pair order	Sorting keys for readability
Sequences	Current doseq	Joining lists with commas
Safe chars	Current safe set	Unescaping slashes or tildes
Empty	Current omission rules	Dropping empty keys

The extract may move code between modules only. It may not change any table cell.

Step 1: inventory every query builder

Search the repo with a boring, exact pattern. Do not start in a generated client tree.

rg -n "urlencode|quote_plus|urlparse|parse_qsl" --type py

Record file, line, and call keywords in a short table. Treat request params kwargs as another query builder. Treat f-string query assembly as a third builder.

Stop if you find more than one encoding policy. Two policies means two helpers later, not one.

Count hits per package before you pick a target. Start with the module that builds signed URLs. Leave generated SDK wrappers for a later pin.

Step 2: record goldens as strings you can diff

The next block is a proposed characterization harness. It writes JSON goldens you can commit.

# tools/pin_query_goldens.py
from __future__ import annotations

import json
from pathlib import Path
from urllib.parse import quote, urlencode

GOLDEN = Path("tests/goldens/query_encoding.json")

SAMPLES = [
    {"id": "spaces", "params": [("q", "hello world"), ("n", "1")]},
    {"id": "unicode", "params": [("q", "café")]},
    {"id": "reserved", "params": [("path", "/a/b"), ("q", "a+b")]},
    {"id": "seq", "params": [("tag", ["a", "b"]), ("q", "x")]},
    {"id": "empty", "params": [("q", ""), ("keep", "1")]},
    {"id": "safe_tilde", "params": [("q", "~user/name")]},
]


def encode_variants(params):
    as_dict = {}
    for key, value in params:
        if key in as_dict:
            prev = as_dict[key]
            as_dict[key] = prev + [value] if isinstance(prev, list) else [prev, value]
        else:
            as_dict[key] = value
    return {
        "urlencode_pairs": urlencode(params),
        "urlencode_doseq": urlencode(params, doseq=True),
        "urlencode_quote_via_quote": urlencode(params, quote_via=quote),
        "urlencode_doseq_quote": urlencode(params, doseq=True, quote_via=quote),
        "urlencode_safe_tilde": urlencode(params, safe="~"),
        "dict_urlencode": urlencode(as_dict),
        "dict_urlencode_doseq": urlencode(as_dict, doseq=True),
    }


def main() -> None:
    rows = []
    for sample in SAMPLES:
        encoded = encode_variants(sample["params"])
        rows.append({"id": sample["id"], "params": sample["params"], **encoded})
    GOLDEN.parent.mkdir(parents=True, exist_ok=True)
    GOLDEN.write_text(json.dumps(rows, indent=2, ensure_ascii=False) + "\n")
    print(f"wrote {GOLDEN} rows={len(rows)}")


if __name__ == "__main__":
    main()

Run it once on the current interpreter. Commit the JSON file as the encoding pin.

python tools/pin_query_goldens.py
git add tests/goldens/query_encoding.json
git commit -m "test: pin urllib.parse query encoding goldens"

Label this run as a local characterization, not a published benchmark. Rerun it only when the interpreter series changes.

Passing a dict with list values stringifies the brackets. Pair lists with doseq true preserve repeated keys. Your goldens must include both input shapes.

Step 3: fail closed on any encoding drift

Add a test that rereads the golden file. Do not regenerate golden files inside the CI job.

# tests/test_query_encoding_goldens.py
from __future__ import annotations

import json
from pathlib import Path

from tools.pin_query_goldens import encode_variants

GOLDEN = Path("tests/goldens/query_encoding.json")


def test_query_encoding_matches_pinned_goldens():
    rows = json.loads(GOLDEN.read_text())
    assert rows, "golden file must not be empty"
    for row in rows:
        got = encode_variants([tuple(item) for item in row["params"]])
        for key, expected in got.items():
            assert row[key] == expected, f"{row['id']}.{key}"

Run the characterization test before any production edit. A red test means your pin is wrong. Fix the golden pin before any refactor.

Store the golden path in one constant. Duplicate golden paths cause silent test splits. Keep the writer and reader on the same helper.

Step 4: pin one production call site

Goldens of urllib.parse alone are not enough. Next, wrap the messiest production builder with tests.

# tests/test_legacy_search_url.py
from urllib.parse import parse_qsl, urlsplit

from payments.client import build_search_url  # existing messy function


def test_legacy_search_url_query_bytes():
    url = build_search_url(
        host="https://example.test",
        q="hello world",
        tags=["a", "b"],
        path="/v1/items",
    )
    parts = urlsplit(url)
    assert parts.scheme == "https"
    assert parts.netloc == "example.test"
    assert parts.path == "/v1/items"
    # Pin the encoded query, not a parsed dict.
    assert parts.query == "q=hello+world&tags=a&tags=b"
    assert parse_qsl(parts.query, keep_blank_values=True) == [
        ("q", "hello world"),
        ("tags", "a"),
        ("tags", "b"),
    ]

Adjust the expected string to whatever the repo already emits. Do not correct that encoded string here.

Parse the query only after the raw string assertion. Parsed dicts hide plus and order bugs. Raw query strings are the real product contract.

Step 5: one extract, same query string

Only now extract one helper with identical defaults. Copy the urlencode kwargs into a named function.

# payments/query.py
from urllib.parse import urlencode


def build_query(params):
    """Proposed extract. Keep urlencode kwargs identical."""
    return urlencode(params, doseq=True)  # doseq=True is the pinned contract

Then change exactly one production call site. Leave every other query builder completely untouched.

# inside payments/client.py — single call site
from payments.query import build_query


def build_search_url(host, q, tags, path):
    query = build_query([("q", q), ("tags", tags)])
    return f"{host}{path}?{query}"

Re-run both test files after the extract. The encoded query string must remain unchanged after extract.

pytest tests/test_query_encoding_goldens.py tests/test_legacy_search_url.py -q

If the assertion on parts.query fails, revert the extract. Do not patch the golden to match new taste.

Put the chosen kwargs in a one-line comment. Future editors should see doseq without code archaeology. Those comments are cheaper than a broken signature.

Step 6: add a byte checksum for signed clients

String equality is enough for most internal apps. Signed clients should also pin UTF-8 digest bytes. The next test is a proposed extra lock.

# tests/test_search_query_digest.py
import hashlib
from pathlib import Path
from urllib.parse import urlsplit

from payments.client import build_search_url

PIN = Path("tests/goldens/search_query.sha256")


def test_search_query_sha256():
    url = build_search_url(
        host="https://example.test",
        q="hello world",
        tags=["a", "b"],
        path="/v1/items",
    )
    query = urlsplit(url).query.encode("utf-8")
    digest = hashlib.sha256(query).hexdigest()
    if not PIN.exists():
        PIN.write_text(digest + "\n")
    assert digest == PIN.read_text().strip()

Keep this checksum next to the query golden. Do not hash parsed dictionaries in this checksum. Hash the exact query component text encoded as UTF-8.

Write the pin file on one local run only. Commit it before reviewers inspect the extract. Later runs must not rewrite the digest file.

Failure analysis: three innocent diffs

The first innocent diff sorts keys for stable logs. Sorting changes cache keys and HMAC bases. Revert that sort during the extract window.

The second innocent diff switches quote_plus to quote. Spaces then stop being encoded as plus signs. Many form endpoints will reject the new client.

The third innocent diff joins list values with commas. The doseq flag instead repeats the same key. Analytics backends treat those shapes as different filters.

Where a free model may help

After the goldens fail closed, a model can draft the extract. It should not choose safe, doseq, or sort order.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode provides free model access and a free server option. Use that workspace to propose the single-function move only.

Paste the failing test and the current function, not the whole monolith. Reject any diff that edits urlencode keyword arguments. Reject any diff that silently sorts query keys.

Reject any diff that touches a second call site. This is a proposed workflow, not a measured product study. No model names are required for the workflow.

Limitations

These pins capture CPython urllib.parse for one interpreter. They do not prove wire compatibility with Go, Java, or browsers. They do not cover IDNA hosts, IPv6 brackets, or fragment handling.

urlencode dict input drops repeated keys unless you pass pairs. Goldens must use the same input type as production.

Compare query strings as text in the golden file. Encode UTF-8 later if signatures hash raw bytes.

Do not treat plus-versus-percent as a style war. RFC 3986 prefers percent-twenty for the space character.

Form encoding prefers plus signs for spaces. Your repository already chose one encoding convention.

Relative URLs, matrix params, and semicolon separators stay out of scope. So do httpx params mutations and requests PreparedRequest paths. Pin those stacks with their own goldens later.

Who should not use this approach

Do not use this extract path for public API query contracts. Ship an explicit versioned encoder for those contracts instead. Do not use it when tests cannot run without network.

Do not use it to standardize several services in one commit. Skip it if the client already has a frozen HTTP cassette. Extend that cassette before extracting any helper.

Skip it for binary query values or non-UTF-8 payloads. Skip it when legal holds require bit-identical traffic replays without tests. In that case, capture packets before any extract.

Close the loop

Keep the golden JSON in the same commit as the extract. Reviewers then see exact bytes, not stated intent.

A later cleanup may still want sorted keys. That work is a second change with a new pin.

The smallest safe change is the one that leaves parts.query identical. If that string changes, the extract was not safe.

Pin Observed Cache-Key Strings Before One Hasher Extract

Dakota Huang — Sat, 19 Sep 2026 18:56:37 +0000

Cache-key extracts fail without pinned observed key strings. Missing fields and explicit None often collide after cleanup. Record the live key bytes first, then extract one hasher.

Messy service modules inline cache keys beside I/O. Coding assistants then rewrite key building during refactors. Hit rates drop while duplicate writes silently appear.

The failure then hides behind otherwise green unit tests. Characterization must pin bytes, not intended cleanliness.

This workflow pins observed key strings before any extract. It then moves exactly one small hasher function. It does not rewrite the whole messy module.

Why observed keys rot during refactors

Inline key builders mix dict order with hidden types. Python 3.7+ preserves insertion order by language rule. Insertion order is still not a public cache contract.

json.dumps without sort_keys follows that insertion order. Equal payloads can therefore emit different key strings.

None and a missing key are not equal values. json.dumps emits null for explicit None fields. A missing key disappears from the serialized object.

Assistants often normalize both cases into None. That merge then changes every downstream cache identity.

Lists remain order-sensitive in most product identities. Sorting tags for stability quietly breaks those identities. Datetime values raise TypeError inside default json.dumps.

Hidden default=str helpers leak timezone-shaped key text. Those leaks become the real on-the-wire cache key.

A prettier canonical JSON dump is a different product change. Do not smuggle it into the hasher extract. Pin the live string first, then decide on canonicalization.

Artifact: a two-column key pin

Treat the harness below as a proposed local pin. Do not treat these fixtures as production telemetry. The extract must preserve the live output column.

Record a diagnostic canonical digest beside live keys. Never replace live keys with that diagnostic digest.

1. Alias the live builder

Do not extract a helper during this step. Find the inline builder in the messy module. Route tests through one thin alias only.

# proposed harness alias; keep the messy implementation in place
from billing_service import _inline_cache_key  # existing code

def build_cache_key(payload: dict) -> str:
    return _inline_cache_key(payload)

Call the real path from every case. Reimplementation here would pin fiction, not behavior.

2. Add a diagnostic canonical dump

The diagnostic dump exists for diffs, not for serving traffic. It makes None-versus-missing visible in review. It must not become the new cache key in the same commit.

import hashlib
import json
from datetime import datetime, timezone


def canonical_dumps(obj) -> str:
    def default(value):
        if isinstance(value, datetime):
            if value.tzinfo is None:
                value = value.replace(tzinfo=timezone.utc)
            return value.astimezone(timezone.utc).strftime(
                "%Y-%m-%dT%H:%M:%SZ"
            )
        raise TypeError(f"unpinned type: {type(value).__name__}")

    return json.dumps(
        obj,
        sort_keys=True,
        separators=(",", ":"),
        ensure_ascii=True,
        default=default,
    )


def diagnostic_digest(obj) -> str:
    blob = canonical_dumps(obj).encode("utf-8")
    return hashlib.sha256(blob).hexdigest()

sort_keys=True applies to dict keys only. Lists keep order under this contract. That split is the whole diagnostic value.

3. Record a case table with two columns

Name every case in a stable dictionary. Store live output and diagnostic JSON together. Fail when either column drifts after the extract.

import json
import os
from datetime import datetime, timezone
from pathlib import Path

GOLDEN = Path("goldens/cache_key_cases.json")
RECORD = os.environ.get("RECORD_GOLDENS") == "1"

CASES = {
    "missing_user": {"order_id": 9},
    "null_user": {"order_id": 9, "user_id": None},
    "reordered": {"user_id": 3, "order_id": 9},
    "list_ab": {"tags": ["a", "b"]},
    "list_ba": {"tags": ["b", "a"]},
    "nested": {"meta": {"b": 1, "a": 2}},
    "aware_dt": {
        "ts": datetime(2026, 9, 20, 12, 0, tzinfo=timezone.utc)
    },
    "naive_dt": {"ts": datetime(2026, 9, 20, 12, 0)},
    "bool_not_int": {"ok": True},
    "int_not_bool": {"ok": 1},
}


def rows_for(current_key_fn):
    rows = {}
    for name, payload in CASES.items():
        live = current_key_fn(payload)
        if not isinstance(live, str):
            raise TypeError(f"{name} emitted {type(live).__name__}")
        rows[name] = {
            "live": live,
            "canonical_json": canonical_dumps(payload),
            "diagnostic_sha256": diagnostic_digest(payload),
        }
    return rows


def record_or_check(current_key_fn):
    rows = rows_for(current_key_fn)
    if RECORD:
        GOLDEN.parent.mkdir(parents=True, exist_ok=True)
        GOLDEN.write_text(json.dumps(rows, indent=2, sort_keys=True) + "\n")
        return
    if not GOLDEN.exists():
        raise FileNotFoundError("set RECORD_GOLDENS=1 once, then commit")
    expected = json.loads(GOLDEN.read_text())
    assert rows == expected, (rows, expected)

The record path is explicit and opt-in. A first-run side effect is too easy to miss. Commit the JSON before any hasher extract.

4. Lock the dangerous inequalities

Keep four direct asserts beside the table. Tables hide intent when they only store blobs. These asserts name the collisions assistants usually create.

def test_none_is_not_missing():
    a = diagnostic_digest({"order_id": 9})
    b = diagnostic_digest({"order_id": 9, "user_id": None})
    assert a != b


def test_dict_order_does_not_matter_in_diagnostic():
    a = diagnostic_digest({"user_id": 3, "order_id": 9})
    b = diagnostic_digest({"order_id": 9, "user_id": 3})
    assert a == b


def test_list_order_does_matter():
    a = diagnostic_digest({"tags": ["a", "b"]})
    b = diagnostic_digest({"tags": ["b", "a"]})
    assert a != b


def test_true_is_not_one():
    assert diagnostic_digest({"ok": True}) != diagnostic_digest({"ok": 1})


def test_live_builder_is_pinned():
    record_or_check(build_cache_key)

The live builder may still depend on insertion order. That fact belongs in the live column. Do not "fix" it during the extract commit.

5. Extract one hasher only

After goldens pass without RECORD_GOLDENS, move one function. Keep the alias name stable for callers. Do not touch I/O, TTL, or key prefixes.

def extract_cache_key(payload: dict) -> str:
    # proposed extract: body copied from _inline_cache_key
    return _inline_cache_key(payload)


def build_cache_key(payload: dict) -> str:
    return extract_cache_key(payload)

Re-run the pin with recording disabled. If the live column changes, stop immediately. Revert the extract and inspect the JSON diff. Do not edit goldens to match a cleaner hasher.

Switching traffic to diagnostic_digest is a second change. It needs a versioned prefix and a TTL plan. It is not part of this extract.

Commands that keep the pin honest

Record once on an unchanged tree. Then assert on a clean second run.

mkdir -p goldens tests
RECORD_GOLDENS=1 python -m pytest tests/test_cache_key_goldens.py -q
python -m pytest tests/test_cache_key_goldens.py -q
git add goldens/cache_key_cases.json tests/test_cache_key_goldens.py
git commit -m "pin observed cache-key strings before hasher extract"

After the extract, diff only the golden file.

python -m pytest tests/test_cache_key_goldens.py -q
git diff -- goldens/cache_key_cases.json

A one-line JSON change is a behavior change. Treat that line as a product decision, not noise.

Decision table: what may share a commit

Use this table before accepting a model patch. If a row says no, split the work.

Change	Pin first	Same commit as extract
Move hasher body to one function	Yes	Yes
Preserve live key strings	Yes	Yes
Add `sort_keys` to live keys	Yes	No
Collapse missing keys into None	Yes	No
Sort list values for stability	Yes	No
Add `default=str` for datetimes	Yes	No
Change key prefix or TTL	Yes	No
Adopt diagnostic SHA-256 as the key	Yes	No

The extract commit has one job. It relocates bytes without changing bytes. Every other row is a keyed migration.

Three patches that usually break goldens

Patch A inserts payload.setdefault("user_id", None). Missing-user and null-user then collide. The diagnostic digest column catches that merge.

Patch B sorts tag lists before hashing. list_ab and list_ba then match. Product identity for ordered tags is lost.

Patch C formats datetimes with str(ts). Naive and aware clocks then depend on the host. The live column drifts across machines.

Reject all three during the extract. File them as separate migrations with prefixes. Do not bargain with the golden file.

Re-running the pin off your laptop

Naive datetime paths still leak host timezone. Default encodings still leak host locale. A second machine is useful after goldens exist.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. Use the free server to replay the same pytest pin, not to invent a prettier key.

After the goldens pass, a free model can draft the one-function extract. You still reject any patch that retouches goldens. Redact user ids from fixtures before any prompt.

Do not paste production payloads into a remote prompt. Cache keys often embed account identifiers. Keep those pins local and stripped.

Limitations

JSON diagnostic dumps are not RFC 8785 JCS. Do not use diagnostic_digest for signatures or tokens. ensure_ascii=True escapes non-ASCII bytes on purpose. Changing that flag rotates every diagnostic row.

Floats remain an open hole in this pin. 1.0 and 1 differ in JSON output. NaN is not legal JSON and must raise. Decimal needs an explicit encoder before it enters cases.

This workflow assumes a single-process Python service. It does not rewrite Redis contents already stored. Old live keys remain until TTL expiry. Plan a v2: prefix if canonicalization must ship later.

default=str stays banned inside the diagnostic dump. str(datetime) is not a stable contract here. The explicit UTC format is the diagnostic point.

The live column may already be unstable across processes. If it is, the pin documents that instability. Stability is a later change, not this extract.

Who should skip this

Do not use this if keys already come from protobuf or msgpack. Do not use this for cryptographic signing material. Do not use this when list order is free and writers already sort.

Do not extract a hasher while changing TTL in one commit. Do not extract a hasher while changing codec in one commit. Do not extract a hasher while adding a prefix in one commit.

Skip the remote re-run when fixtures still contain production payloads. Keep those pins on a redacted local tree. The method is for behavior locks, not for data export.

Teams with an existing canonical codec should pin that codec instead. Do not introduce a second JSON flavor beside it. Two canonicalizers will drift under review.

Close

Pin observed key strings before the hasher extract. Keep None distinct from missing fields. Keep list order unless product code already sorts.

Force UTC only in the diagnostic dump. Move one function and leave goldens frozen. If the JSON file moves, the refactor is no longer small.

Publish the case table with the extract review. Leave model chat transcripts out of the merge.

Pin Header Aliases Before One CSV Exporter Extract

Dakota Huang — Sat, 19 Sep 2026 15:48:48 +0000

Pin header aliases before extracting a CSV exporter.
Mixed dumps hide dialect drift after the cut.

Characterization tests freeze the current byte-for-byte dump.
Only then is a one-function extract actually safe.

The failure this sequence prevents

Messy report modules often build CSV text inline.
They mix banners, aliases, and dialect flags together.

A later extract can look clean during review.
The new helper then changes quoting on empty rows.

Callers still pass the same list of dicts.
The output file hash still changes anyway.

That result is not a style problem.
It is an unpinned contract on disk.

Inline writers also hide header alias drift.
Full Name may become name only sometimes.

Reviewers read the new function as obvious cleanup.
Excel users then see shifted columns and broken imports.

What you must freeze

Freeze four observables before you move any writer.

Freeze the header alias map from keys to columns.
Freeze delimiter, quotechar, lineterminator, and extrasaction together.
Freeze empty-row policy, including records that are all None.
Freeze exact UTF-8 bytes, including the trailing newline.

Do not freeze wall-clock timestamps inside the row cells.
Strip or stub those time fields before pinning bytes.

Do not freeze row order if the source is a dict.
Sort by a stable key inside the harness.

Do not assume None renders like an empty string.
Pin whatever the messy module currently emits.

Decision table

Use this table before you touch production code.

Signal in the dump path	Extract in this commit	Wait or stop
Tests pin aliases, dialect, empty rows, and bytes	Yes	No
Banner lines share the same write function	No	Split observations first
Cell values include local clock or timezone text	No	Stub those fields
Two call sites already share one writer helper	Stop	The extract already exists
Review asks only for renamed locals	No	Names are not the contract
Output may include a UTF-8 BOM for Excel	Pin the BOM	Do not strip it yet

Extract only when the first column reads yes.
Treat every other signal as review noise.

Artifact: a characterization harness

Label this example as a proposed local harness.
It is not a measured production benchmark of speed.

Create tests/test_characterize_csv_dump.py in your tree.
Keep the import path pointed at the messy module.

# proposed harness — unexecuted against your tree
from __future__ import annotations

import io
from pathlib import Path

from reports import messy_dump  # replace with the real module

PIN_DIR = Path(__file__).parent / "pins"
PIN_DIR.mkdir(exist_ok=True)

CASES = [
    {
        "name": "aliases_and_empty",
        "rows": [
            {"Full Name": "Ada", "e-mail": "ada@example.test", "Score": "1"},
            {"Full Name": None, "e-mail": None, "Score": None},
            {"Full Name": "Bob", "e-mail": "", "Score": "0"},
        ],
    },
    {
        "name": "comma_inside_field",
        "rows": [
            {"Full Name": "Lovelace, Ada", "e-mail": "a@b.test", "Score": "9"},
        ],
    },
    {
        "name": "non_ascii_name",
        "rows": [
            {"Full Name": "Søren", "e-mail": "s@example.test", "Score": "2"},
        ],
    },
]


def dump_bytes(rows: list[dict]) -> bytes:
    buf = io.StringIO()
    messy_dump(rows, buf)  # current inline writer
    return buf.getvalue().encode("utf-8")


def test_pin_or_compare() -> None:
    for case in CASES:
        pin = PIN_DIR / f"{case['name']}.csv"
        got = dump_bytes(case["rows"])
        if not pin.exists():
            pin.write_bytes(got)
            continue
        assert got == pin.read_bytes(), case["name"]

Run the harness once to write the pin files.

python -m pytest tests/test_characterize_csv_dump.py -q

Commit the pin files in that same change.
Do not combine that commit with the extract.

Numbered workflow

1. Locate the inline writer

Search for csv.writer and DictWriter in one pass.
Note every keyword argument on those constructor calls.

rg -n "csv\.(writer|DictWriter)|lineterminator|extrasaction|quoting" reports

Record delimiter and quoting in a scratch note.
Do not change those arguments yet.

2. Separate banner bytes from CSV body

Title lines often include a generated report date.
Dates will thrash pin files on every run.

Pin the body separately when a banner exists.
Leave banner formatting outside the extract.

# proposed split — characterization only, not a redesign
def split_banner(text: str) -> tuple[str, str]:
    lines = text.splitlines(keepends=True)
    if lines and lines[0].startswith("REPORT"):
        return lines[0], "".join(lines[1:])
    return "", text

Keep the split inside the test helper for now.
Production code still dumps through one messy function.

3. Record header aliases as input data

Do not decode aliases from memory or comments.
Drive them through the public dump function instead.

ALIASES = {
    "Full Name": "name",
    "e-mail": "email",
    "Score": "score",
}

If the messy code uses a different map, pin that map.
Your extract must reproduce the same column order.

Column order is part of the file contract.
Dict iteration order is not a substitute for pins.

4. Add empty-row and quoting cases

Empty rows expose None versus "" rendering.
Quoted commas expose dialect mistakes after the move.

One happy-path row is not a characterization suite.
Three cases still miss Unicode and trailing newline rules.

Add a non-ASCII name as a fourth pinned case.
Keep the pin files UTF-8 with a known newline.

If Excel clients require a leading BOM, pin the BOM.
Do not strip it during the extract commit.

5. Fail the harness on purpose

Edit quotechar on a throwaway local branch.
Confirm the pin test fails before you trust it.

python -m pytest tests/test_characterize_csv_dump.py -q

If the test still passes, the pin is blind.
Fix the harness before any production extract.

Blind pins usually compare parsed rows, not bytes.
Parsed rows hide quoting, BOM, and terminator drift.

6. Draft the extract only after green pins

A second pair of eyes can draft the extract.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option.
Use that draft only after characterization tests already pass.
Do not let a model edit the messy file first.

Paste the messy function and the committed pin files.
Ask for one writer function that preserves the same bytes.
Reject extra renames of public callers in that draft.

Review the draft as a diff, not as narrative trust.
If any pin fails, discard the draft completely.

7. Apply one extract

Move only the CSV body writer in this commit.
Leave banners, clocks, and I/O in the original function.

# proposed extract — apply only after pins stay green
import csv
from typing import Mapping, Sequence, TextIO


def write_score_csv(
    rows: Sequence[Mapping[str, object]],
    out: TextIO,
    *,
    aliases: Mapping[str, str],
) -> None:
    fieldnames = list(aliases.values())
    writer = csv.DictWriter(
        out,
        fieldnames=fieldnames,
        extrasaction="ignore",
        lineterminator="\n",
        quoting=csv.QUOTE_MINIMAL,
    )
    writer.writeheader()
    for row in rows:
        mapped = {aliases[k]: row.get(k, "") for k in aliases}
        if all(v in (None, "") for v in mapped.values()):
            continue
        writer.writerow({k: "" if v is None else v for k, v in mapped.items()})

Call this helper from the original dump path only.
Do not change flag parsing in the same commit.

Match extrasaction, terminator, and quoting to the pins.
Copy those values from the messy constructor, not from taste.

8. Re-run pins, then stop

python -m pytest tests/test_characterize_csv_dump.py -q

Green means the extract preserved the pinned bytes.
Red means you restore the function and inspect dialect kwargs.

Do not “improve” quoting while the extract is in flight.
Quoting changes belong in a later, explicit commit.

If Windows clients need \r\n, pin that terminator.
Then set lineterminator in both test and helper.

Dialect traps the pins are meant to catch

QUOTE_MINIMAL and QUOTE_ALL can parse to equal rows.
They still fail a byte comparison on commas and blanks.

extrasaction="raise" explodes on unknown dict keys.
extrasaction="ignore" drops those keys without a trace.

Empty-row skipping is often an accidental continue.
Extracting it without a pin changes downstream line counts.

Header aliases are order-sensitive on disk.
A set of names is not enough to reconstruct the file.

Limitations

This workflow does not prove semantic correctness of scores.
It only proves the dump bytes did not move.

It will not catch numeric rounding inside already-built cells.
It will not catch timezone shifts you stubbed out first.

Pin files are fixtures, not product documentation.
They rot when owners change column names on purpose.

Large binary attachments do not belong beside these pins.
CSV text does, because humans can still diff it.

Naive pins break on \r\n versus \n across machines.
Set the terminator explicitly in tests and production code.

Who should not use this

Do not use this on a greenfield CSV writer.
There is no legacy byte contract to freeze.

Do not use this when two teams own competing schemas.
Pins cannot settle a product dispute about columns.

Do not use this as a substitute for schema tests.
If a formal schema already exists, prefer that contract.

Skip model drafts when the dump can include secrets.
Keep credential rows out of any prompt or paste.

Skip the extract when one helper already writes every path.
You would only shuffle names without changing risk.

What done means

Done is a pin commit plus one extract commit.
Done is not a renamed module and a hopeful glance.

If the next change alters empty-row policy, update pins first.
Then change the writer under those new bytes.

If pins already fail on your machine, skip drafts.
A free model on a free server can propose the extract after pins pass.
Keep acceptance on the byte comparison, not the prose.

Workers Should Not Inherit Your Logging Handlers

Dakota Huang — Sat, 19 Sep 2026 12:37:26 +0000

Have you ever watched pool.join() sit there after a logging cleanup that looked boring in review? The workers are not crunching data. They are waiting on a lock the parent copied into them, or they are writing nowhere because the child never got a handler at all.

That split is the whole story. Fork and spawn do not share a logging world. A free coding model will still offer you one setup_logging() import and call it tidy.

The hang is a lock, not a slow map

Stdlib logging wraps every handler in an RLock. A parent thread can hold that lock during emit(). If the process forks at that moment, the child receives a lock that is already taken. Nobody in the child will release it. The next logging.info() in a worker blocks forever, and the pool join waits on that worker.

Spawn does not copy the lock. It also does not copy the handlers. The child starts a fresh interpreter. Your import-time basicConfig() in the parent never ran there. Worker logs vanish, and the job looks “fine” because the parent still prints a summary.

So the same cleanup produces two failures. One machine deadlocks. Another machine goes quiet. Reviewers who only run macOS spawn will miss the Linux fork hang. Reviewers who only run Linux fork will miss silent workers on Windows.

Import time is the accident

The usual “cleanup” is not a new architecture. Someone moves logging.basicConfig(...) into a helpers module. Every worker import then configures the root logger, or the parent configures it before the pool starts, and children inherit whatever was there.

You do not need a messy god module for this to bite. One shared logutil.py imported by the CLI and by the worker is enough. The model sees duplication and wants a single function. That is the wrong merge.

Think of handlers as file descriptors with opinions. You would not let every child inherit an open socket and also open it again. Logging is that socket, plus a lock, plus a format string that finance will one day grep.

A lab that shows both failures

The file below is sample code. It is not a production collector. It installs a slow handler so the fork race is easy to hit, then starts a worker that also logs. On interpreters that still offer fork, the child often never finishes. On spawn, the child usually finishes and prints nothing through that handler.

# lab_logging_pool.py — sample lab, not a service
from __future__ import annotations

import logging
import multiprocessing as mp
import threading
import time


class SlowStreamHandler(logging.Handler):
    def emit(self, record: logging.LogRecord) -> None:
        with self.lock:
            time.sleep(0.4)
            print("PARENT_OR_INHERITED:", self.format(record), flush=True)


def worker(n: int) -> str:
    log = logging.getLogger("job")
    log.info("worker got %s", n)
    handlers = [type(h).__name__ for h in logging.getLogger().handlers]
    return ",".join(handlers) or "NO_ROOT_HANDLERS"


def run_lab(start_method: str) -> None:
    root = logging.getLogger()
    root.handlers[:] = [SlowStreamHandler()]
    root.setLevel(logging.INFO)
    logging.getLogger("job").setLevel(logging.INFO)

    t = threading.Thread(target=lambda: logging.getLogger("job").info("parent emit"), daemon=True)
    t.start()
    time.sleep(0.05)

    ctx = mp.get_context(start_method)
    proc = ctx.Process(target=worker, args=(7,))
    proc.start()
    proc.join(timeout=3.0)
    print(start_method, "alive", proc.is_alive(), "exit", proc.exitcode)


if __name__ == "__main__":
    for method in mp.get_all_start_methods():
        print("---", method, "---")
        run_lab(method)

Run it twice on the same machine. Then run it on a second OS if you have one. You are collecting a map of start methods, not a vibe about “logging best practices.”

python lab_logging_pool.py

If fork stays alive after the timeout, you watched the copied lock. If spawn exits quickly and the returned handler list would have been empty, you watched a silent child. Both are regressions. Neither is a style argument.

QueueListener is a topology, not a rename

The stdlib answer is a queue. The parent owns a QueueListener and a real handler. Workers get a QueueHandler only. After fork or spawn, workers must not keep StreamHandlers that point at the same stdout lock the parent uses.

The snippet below is a proposed pattern. It is unlabeled as production-ready because you still have to decide JSON versus text, and you still have to close the listener on the path that handles SIGTERM.

# proposed_queue_logging.py — proposed pattern, unexecuted in production
from __future__ import annotations

import logging
import logging.handlers
import multiprocessing as mp
from logging.handlers import QueueHandler, QueueListener

_queue = None
_listener = None


def start_parent_logging() -> None:
    global _queue, _listener
    _queue = mp.get_context("spawn").Queue(-1)
    handler = logging.StreamHandler()
    handler.setFormatter(logging.Formatter("%(processName)s %(levelname)s %(message)s"))
    _listener = QueueListener(_queue, handler, respect_handler_level=True)
    _listener.start()
    root = logging.getLogger()
    root.handlers[:] = [QueueHandler(_queue)]
    root.setLevel(logging.INFO)


def worker_logging_init() -> None:
    if _queue is None:
        raise RuntimeError("parent must call start_parent_logging first for spawn pickling")
    root = logging.getLogger()
    root.handlers[:] = [QueueHandler(_queue)]
    root.setLevel(logging.INFO)


def stop_parent_logging() -> None:
    if _listener is not None:
        _listener.stop()

Notice the initializer. Spawn will not remember parent basicConfig. Fork may remember too much. The initializer is how you force a single handler graph in both worlds. A model that deletes the initializer because “the parent already configured logging” is editing the topology.

A review gate that inspects handlers

Do not argue about format strings first. Assert who owns what. The test below is sample pytest. It skips start methods the interpreter does not provide, which is the honest thing to do in 2026 instead of pretending every laptop still defaults to fork.

# test_handler_graph.py — sample harness
from __future__ import annotations

import logging
import multiprocessing as mp

import pytest
from logging.handlers import QueueHandler


def _worker_report(_n: int) -> tuple[str, ...]:
    return tuple(type(h).__name__ for h in logging.getLogger().handlers)


@pytest.mark.parametrize("method", mp.get_all_start_methods())
def test_workers_do_not_keep_stream_handlers(method: str) -> None:
    ctx = mp.get_context(method)
    queue = ctx.Queue(-1)

    def _init() -> None:
        root = logging.getLogger()
        root.handlers[:] = [QueueHandler(queue)]
        root.setLevel(logging.INFO)

    with ctx.Pool(2, initializer=_init) as pool:
        reports = pool.map(_worker_report, [1, 2])

    for names in reports:
        assert names == ("QueueHandler",), names
        assert "StreamHandler" not in names
        assert "SlowStreamHandler" not in names

Run that on your machine before anyone “simplifies” logging. If a patch adds basicConfig to a module imported by workers, this test is the objection. A green explanation from a model is not.

python -m pytest test_handler_graph.py -q

Decision table for the review

Start method you actually use	What the child should hold	Cleanup that looks innocent
`fork`	QueueHandler only, no inherited StreamHandler	Import-time `basicConfig` in a shared helper
`spawn`	QueueHandler installed in an initializer	Assuming parent handlers pickle across
`forkserver`	Explicit reconfigure in the server or initializer	Copying a FileHandler path into every child
mixed CI images	The intersection of the rows above	“It passed on my Mac” as evidence

Read the table as a gate. If CI images disagree on start methods, you do not have one logging setup. You have a matrix. Collapsing that matrix into a single import is how the hang ships.

Where a free model is allowed to talk

After the handler graph test exists, a model is useful for one narrow job: drafting a comment that names the start methods and forbids import-time basicConfig. It is not useful for inventing a new logging framework, swapping to structlog, or “simplifying” the initializer away.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. Those are the only product facts this article relies on.

Feed the failing test and the lab file together. Ask for a comment block above the initializer, nothing else. If you want that comment drafted off to the side of your laptop, the free model access on the free server is enough for that request. Paste the comment back. Run pytest again. The test stays the reviewer.

This lab will not save every logger

It does not prove your JSON field names. It does not prove syslog facility numbers. It does not make print() thread-safe. It does not fix a custom handler that opens a network socket in emit() without its own timeout.

Audit logs that must not travel through an in-memory queue belong on a different path. If a worker crash can drop the last N records, a QueueListener is the wrong sink for that stream. Do not pretend the topology test equals durability.

Python’s default start method has not been a single global truth across OS images. Do not hard-code “Linux means fork” in a runbook you will reread next quarter. Query mp.get_all_start_methods() and mp.get_start_method() in the lab instead of quoting a blog from memory.

Leave this on the shelf when it is the wrong fight

Skip the lab if the process has no children. Skip it if you already run only spawn and workers are thin subprocesses that log to their own files by design. Skip it if you cannot run pytest where the workers actually start.

Skip it for libraries that must not install root handlers at import. That is a different contract, and this article would push you the wrong way. Skip it when the product requirement is “children inherit the parent tty exactly,” because QueueHandler will not give you that.

Teams that want one PR to replace logging, metrics, and tracing should stop. Handler topology is enough work. Formatters can wait for a later argument that is actually about formatters.

Shut the pool down on purpose

pool.join() is not a performance metric. It is a statement that every child finished without sitting on a copied lock. Import-time logging steals that statement and dresses it up as cleanup.

Keep handlers out of the worker import graph. Install a QueueHandler in an initializer you can see in review. If a patch needs a paragraph to justify basicConfig in a shared module, the paragraph is the bug.

Pin Kitchen-Sink Output Before One Pure Extract

Dakota Huang — Sat, 19 Sep 2026 10:16:10 +0000

Messy kitchen-sink modules break under wholesale AI rewrites. Pin observable output with characterization tests before edits. Then extract one pure function and nothing else today.

This order keeps implicit contracts stable during review. It also keeps the resulting diff cheap to inspect.

The failure mode

Kitchen-sink files mix policy, mutation, and shared pools. Callers depend on side effects nobody documented. A full rewrite drops those implicit contracts without warning.

Fluent refactors often look cleaner in the diff. Tests then fail on order, caps, and in-place mutation. Engineering starts when those outputs are pinned in code.

Public debate still mixes fluent rewrites with verified refactors. Characterization tests are the cheaper gate for messy trees. They record behavior before structure changes at all.

Do not ask a model to “clean utils.py.” That prompt invites a taste rewrite. Taste rewrites reorder mutations that callers already ship.

What this tutorial pins

The sample is a credit allocator with residue. It mutates user dicts in list order. It spends a shared pool with a hidden per-user take.

It applies plan bonuses before the hard cap. Unknown plans receive zero bonus today. Inactive users may be skipped while keys appear.

Do not treat this module as a design target. Treat it as production residue you inherited. The tests below record what the function does now.

1. Snapshot the messy module

Save this file as credits.py without renaming symbols. Do not tidy the magic numbers yet.

DEFAULT_BONUS = {"free": 0, "pro": 50, "team": 120}
CAP = 1000


def allocate(users, pool, plan_overrides=None, skip_inactive=True):
    overrides = plan_overrides or {}
    leftover = pool
    touched = []
    for user in users:
        if skip_inactive and not user.get("active", True):
            user["credit"] = user.get("credit", 0)
            continue
        plan = user.get("plan") or "free"
        bonus = overrides[plan] if plan in overrides else DEFAULT_BONUS.get(plan, 0)
        current = user.get("credit", 0)
        granted = bonus
        if leftover > 0:
            take = leftover if leftover < 25 else 25
            granted += take
            leftover -= take
        next_credit = current + granted
        if next_credit > CAP:
            leftover += next_credit - CAP
            next_credit = CAP
        user["credit"] = next_credit
        user["plan"] = plan
        touched.append(user["id"])
    return {"leftover": leftover, "touched": touched, "pool_in": pool}

Three hidden rules matter for later extracts. The pool grants at most twenty-five per active user. Cap overflow returns extra credit into leftover.

Inactive rows still receive a credit key. plan is written back only on the active path. Input order is the allocation order for the pool.

2. Write characterization tests first

Do not extract helpers before the suite is green. Pin return values and in-place mutations together.

Save this file as test_credits_char.py next to the module.

import copy
import unittest
from credits import allocate, CAP


def users(*rows):
    return [dict(r) for r in rows]


class AllocateCharacterization(unittest.TestCase):
    def test_empty_users_returns_full_pool(self):
        out = allocate([], 100)
        self.assertEqual(out["leftover"], 100)
        self.assertEqual(out["touched"], [])
        self.assertEqual(out["pool_in"], 100)

    def test_inactive_skipped_but_credit_key_set(self):
        rows = users({"id": "a", "active": False, "plan": "pro"})
        original = copy.deepcopy(rows)
        out = allocate(rows, 80)
        self.assertEqual(out["touched"], [])
        self.assertEqual(out["leftover"], 80)
        self.assertEqual(rows[0]["credit"], 0)
        self.assertEqual(rows[0]["plan"], original[0]["plan"])

    def test_unknown_plan_gets_zero_bonus(self):
        rows = users({"id": "b", "plan": "enterprise", "credit": 10})
        out = allocate(rows, 0)
        self.assertEqual(rows[0]["credit"], 10)
        self.assertEqual(rows[0]["plan"], "enterprise")
        self.assertEqual(out["leftover"], 0)

    def test_pro_bonus_then_pool_take_of_25(self):
        rows = users({"id": "c", "plan": "pro", "credit": 0})
        out = allocate(rows, 40)
        self.assertEqual(rows[0]["credit"], 75)
        self.assertEqual(out["leftover"], 15)
        self.assertEqual(out["touched"], ["c"])

    def test_cap_overflow_returns_to_leftover(self):
        rows = users({"id": "d", "plan": "team", "credit": 990})
        out = allocate(rows, 50)
        self.assertEqual(rows[0]["credit"], CAP)
        self.assertEqual(out["leftover"], 160)

    def test_override_replaces_default_bonus_only(self):
        rows = users({"id": "e", "plan": "free", "credit": 0})
        out = allocate(rows, 25, plan_overrides={"free": 7})
        self.assertEqual(rows[0]["credit"], 32)
        self.assertEqual(out["leftover"], 0)

    def test_missing_plan_becomes_free_on_active_path(self):
        rows = users({"id": "i", "credit": 0})
        out = allocate(rows, 0)
        self.assertEqual(rows[0]["plan"], "free")
        self.assertEqual(rows[0]["credit"], 0)
        self.assertEqual(out["touched"], ["i"])

    def test_input_is_mutated_in_list_order(self):
        rows = users(
            {"id": "f", "plan": "pro", "credit": 0},
            {"id": "g", "plan": "pro", "credit": 0},
        )
        out = allocate(rows, 30)
        self.assertEqual(rows[0]["credit"], 75)
        self.assertEqual(rows[1]["credit"], 55)
        self.assertEqual(out["touched"], ["f", "g"])
        self.assertEqual(out["leftover"], 0)


if __name__ == "__main__":
    unittest.main()

Run the harness against the untouched module first.

python -m unittest test_credits_char.py -v

Every test must pass on the original file. That green run is the only spec you have. Proposed workflow: commit the tests before any extract.

git add credits.py test_credits_char.py
git commit -m "test: pin allocate mutation and leftover"

3. Record a decision table

Use a table instead of undocumented reviewer memory. Each row freezes one observed rule.

Input signal	Observed rule	Safe to change now
Empty `users`	Pool returned untouched	No
Inactive user	Credit key set, plan unchanged	No
Unknown plan	Bonus is 0	No
Pool remainder	At most 25 per active user	No
Credit above cap	Overflow added to leftover	No
`plan_overrides`	Replaces default bonus only	No
List order	First user takes pool first	No
Missing `plan`	Active path writes `"free"`	No

Keep the last column at “No” until tests exist. After the suite stays green, one cell may flip. Flip only the cell you will extract.

Inactive users keep the original plan field unchanged. A tidy rewrite often writes plan before continue. The inactive test exists to catch that move.

4. Make the smallest safe change

Extract only bonus lookup on this pass. Leave mutation, cap math, and pool takes untouched.

Proposal: add _bonus_for as a pure function. Do not change call order around it.

def _bonus_for(plan, overrides):
    if plan in overrides:
        return overrides[plan]
    return DEFAULT_BONUS.get(plan, 0)

Then replace the inline bonus line with one call.

bonus = _bonus_for(plan, overrides)

Re-run the same characterization file after that single edit. Do not add features and do not rename allocate.

python -m unittest test_credits_char.py -v
git diff -- credits.py

If any test fails, revert the extract immediately. The characterization suite remains the only merge gate. The expected diff is one helper plus one call site.

5. Keep model output inside that gate

Fluent coding assistants often skip this gate. Models propose whole-file cleanups that look tasteful. Whole-file cleanups break mutation contracts on the first run.

A tighter loop works better on messy repositories.

Paste only allocate plus failing test names.
Ask for one pure extract, never a redesign.
Reject patches that alter leftover arithmetic at all.
Re-run the characterization file after every patch.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. Those two pieces fit this characterization loop on messy repos. The model can draft _bonus_for from the pinned tests. The server can run unittest when the local tree is messy.

Do not send the whole repository on the first pass. Send the pinned tests and one function only. Compare the incoming diff against the decision table rows.

This article does not name models, quotas, or hardware. Treat those details as facts to verify in current docs. Availability can change without notice in product copy. Re-check the live docs before you plan a pipeline.

If the extract is trivial, skip the remote path. Local unittest already covers the seam.

6. A second extract only after green tests

Stop after _bonus_for lands and tests pass. The pool slice of twenty-five is still implicit. Cap overflow is still implicit in leftover math.

A later extract might isolate the pool take. Write new characterization rows before that extract.

    def test_pool_take_never_exceeds_25(self):
        rows = users({"id": "h", "plan": "free", "credit": 0})
        out = allocate(rows, 80)
        self.assertEqual(rows[0]["credit"], 25)
        self.assertEqual(out["leftover"], 55)

Add that test on the still-unextracted pool logic. Then extract _take_from_pool(leftover) if the suite stays green.

One behavior per extract remains the whole method. Two extracts in one patch hide the failing seam.

Why these golden values are the artifact

The overflow row is easy to get wrong. Start with credit 990, team bonus 120, and pool 50. The function takes 25 from the pool, not 50.

Granted credit becomes 145 before the cap. 990 plus 145 is 1135, so 135 overflows. Leftover becomes 50 minus 25 plus 135, which is 160.

If a rewrite “simplifies” overflow, leftover drifts first. The characterization test fails before reviewers argue taste. That failure is the engineering signal, not the diff.

Mutation order is the second trap. The first pro user takes 25 pool units. The second pro user only receives the remaining five. A pure map over users would split the pool differently.

Copy the input with dict(r) before each case. Never reuse the same list across tests. Shared fixtures hide mutation bugs that production will show.

pool_in looks redundant until a wrapper appears. Wrappers often mutate the incoming pool name. Pinning the echo catches that rename during later extracts.

Limitations

Characterization tests freeze bugs as if they were law. If leftover overflow is wrong, tests will protect it.

They also miss uncalled branches in the same function. skip_inactive=False has no row in the suite above. Inactive users that already hold credit are untested here.

The suite uses in-process mutation of ordinary dicts. It will not catch file locking around this allocator. It will not catch thread races on the shared pool.

Golden values are literals copied from one run. They need review when the business policy actually changes. Do not confuse pinning current behavior with approving it.

Who should not use this approach

Do not use this flow for greenfield design work. You would pin accidents and call them contracts.

Do not use it when the module is wrong and security-sensitive. Fix the defect first, then pin the corrected output.

Do not use it as a substitute for explicit types. The dict shape is still implicit after _bonus_for.

Do not batch five extracts in one model pass. The point of the method is isolation of blame.

Skip remote model use if the code cannot leave the machine. Run the same tests locally in that case.

Checklist

Copy the kitchen-sink function into place unchanged.
Add tests that read return values and mutations.
Fill a decision table from observed, failing knowledge.
Extract one pure helper and no other seams.
Re-run the same file, not a brand-new suite.
Keep model patches inside that exact diff.

The core conclusion stays simple under pressure. Pin output first, then change one seam. Fluency is not the same artifact as a green characterization file.

Record Goldens First, Then Make One Messy-Repo Edit

Dakota Huang — Sat, 19 Sep 2026 06:17:22 +0000

Do not refactor a messy module until tests pin current behavior. Characterization tests freeze observable outputs before any structural change.

Naive cleanup often changes results that nobody ever documented. A tangled file encodes edge cases inside accidental control flow.

This workflow records those edges before the first edit. Then you apply one mechanical change and re-run the harness.

What the tests must pin

Pin inputs, outputs, and error types, not internal names. Call the public entry points that production already uses.

Record returned structures, stdout bytes, and raised exception classes. Skip private helpers until a later extract has dedicated pins.

Treat current bugs as frozen contracts without an explicit ticket. A refactor that silently fixes rounding is still a behavior change.

Why one change beats a rewrite

One change means one intent, not a weekend rewrite. Rename one symbol, extract one function, or delete one dead branch.

Do not mix formatting, typing, and logic in the same commit. Each commit must show a characterization delta of zero.

Large diffs hide the line that altered a total. Reviewers cannot verify behavior when structure and values move together.

Numbered workflow

1. Isolate one messy entry point

Pick a function that existing callers already import. Avoid starting with a package-wide rename or folder move.

Write the call signature beside the new test path. Keep fixture names stable so later diffs stay small.

2. Capture a golden corpus

Drive the function with fixtures from logs, bugs, and support tickets. Serialize results to JSON with sorted keys and integer money.

Store one JSON file per case under tests/goldens. Commit those files before any production code changes.

3. Assert replay equality

Load each golden file and call the same entry point. Compare with an exact matcher, not a fuzzy snapshot helper.

Fail the run on the first mismatch and print both payloads. That print is the characterization debugger you will use.

4. Make one mechanical edit

Extract a local helper or rename a confusing identifier. Do not change branches, defaults, or numeric literals yet.

Re-run the harness until every golden file still matches. If a file drifts, revert the edit and shrink the change.

5. Only then open a behavior ticket

New behavior needs a new test, not a silent golden rewrite. Update goldens only when a ticket names the old output as wrong.

Normalize before you pin

Raw dumps often include clocks, ids, and unordered containers. Those fields create false diffs that block safe extracts.

Strip or freeze time in the recorder wrapper only. Sort lists that the public contract treats as sets.

Do not sort lists when order is part of the contract. Document each normalization rule next to the recorder.

Artifact: replay harness and recorder

The Python below is a labeled, unexecuted proposal. Adapt the module path and entry name to your tree.

# characterization_harness.py
# Proposal: freeze public compute_totals() output before any extract.

from __future__ import annotations

import json
import importlib
from pathlib import Path

GOLDEN_DIR = Path("tests/goldens/order_totals")
MODULE_UNDER_TEST = "billing.order_totals"
ENTRY = "compute_totals"


def load_cases():
    cases = []
    for path in sorted(GOLDEN_DIR.glob("*.json")):
        payload = json.loads(path.read_text(encoding="utf-8"))
        cases.append((path.name, payload))
    return cases


def call_entry(inputs: dict):
    mod = importlib.import_module(MODULE_UNDER_TEST)
    fn = getattr(mod, ENTRY)
    try:
        result = fn(**inputs)
        return {
            "ok": True,
            "result": result,
            "error_type": None,
            "error_msg": None,
        }
    except Exception as exc:
        return {
            "ok": False,
            "result": None,
            "error_type": type(exc).__name__,
            "error_msg": str(exc),
        }


def replay():
    failures = []
    for name, payload in load_cases():
        observed = call_entry(payload["inputs"])
        expected = payload["output"]
        if observed != expected:
            failures.append((name, expected, observed))
    return failures


if __name__ == "__main__":
    failed = replay()
    if not failed:
        print("characterization: 0 diffs")
        raise SystemExit(0)
    for name, expected, observed in failed:
        print(f"DIFF {name}")
        print("expected", json.dumps(expected, sort_keys=True))
        print("observed", json.dumps(observed, sort_keys=True))
    raise SystemExit(1)

Record goldens with a sibling script that only writes files. Never overwrite golden files during a replay run.

# record_goldens.py
# Proposal: one-time capture. Review each file before commit.

import json
from characterization_harness import GOLDEN_DIR, call_entry

CASES = [
    ("empty_cart", {"items": [], "region": "US", "coupon": None}),
    (
        "single_taxed",
        {
            "items": [{"sku": "A", "qty": 1, "cents": 1999}],
            "region": "US",
            "coupon": None,
        },
    ),
    (
        "coupon_stack",
        {
            "items": [{"sku": "A", "qty": 2, "cents": 500}],
            "region": "EU",
            "coupon": "SAVE10",
        },
    ),
    (
        "unknown_region",
        {
            "items": [{"sku": "B", "qty": 1, "cents": 100}],
            "region": "XX",
            "coupon": None,
        },
    ),
]


def main():
    GOLDEN_DIR.mkdir(parents=True, exist_ok=True)
    for name, inputs in CASES:
        output = call_entry(inputs)
        path = GOLDEN_DIR / f"{name}.json"
        body = json.dumps(
            {"inputs": inputs, "output": output},
            indent=2,
            sort_keys=True,
        )
        path.write_text(body, encoding="utf-8")
        print("wrote", path)


if __name__ == "__main__":
    main()

This example shows shape only, not a measured tax table. Your live module fills result during the record run.

{
  "inputs": {
    "coupon": null,
    "items": [{"cents": 1999, "qty": 1, "sku": "A"}],
    "region": "US"
  },
  "output": {
    "error_msg": null,
    "error_type": null,
    "ok": true,
    "result": {
      "subtotal_cents": 1999,
      "tax_cents": 0,
      "total_cents": 1999
    }
  }
}

Inspect every JSON file after the record script finishes. Commit goldens and the harness in the same change.

python record_goldens.py
python characterization_harness.py
git add tests/goldens/order_totals characterization_harness.py record_goldens.py
git commit -m "Pin order_totals outputs before any extract"

Optional pytest wrapper keeps the same exact compare.

# tests/test_order_totals_characterization.py
# Proposal: pytest entry that fails on any golden drift.

from characterization_harness import replay


def test_order_totals_goldens_match():
    failed = replay()
    assert failed == [], [name for name, _, _ in failed]

python -m pytest tests/test_order_totals_characterization.py -q

Decision table for the first cut

Use exactly one row from the table below. Leave every other row for a later, separate commit.

Signal in the messy file	First allowed change	Forbidden in that commit
Mixed names for one concept	Rename the local aliases	New branches or default values
Long function with one caller	Extract a private helper	Public API shape change
Commented-out block	Delete the dead block	Nearby live logic edits
Magic number used twice	Extract a named constant	Recalculate the number
Duplicate dump calls	Extract one serializer	Encoding or key-order change

The harness must stay green after the chosen row. A red run means the edit was not mechanical.

Using a free model only for fixture drafts

A model can propose extra fixtures from logs and stack traces. It must not become the oracle for correct totals.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode provides free model access and a free server option. Use that pair to draft candidate cases on a throwaway clone.

Paste a redacted log slice and the public function signature. Ask for input dictionaries that hit missing branches, not rewrites.

Review every generated case against real production traces. Drop any fixture you cannot explain in one plain sentence.

Humans commit the golden files after manual review. The harness must reject any later silent drift.

A free server helps when the messy repo will not boot locally. Clone, install, record goldens, then discard the instance.

Do not treat generated tests as coverage proof. They remain drafts until replay matches committed files.

Limitations

This method does not prove the module is correct. It only proves the next edit did not change recorded outputs.

Golden files rot when upstream data formats change. Re-record through a reviewed, ticketed process, never by habit.

Exact JSON compare fails on unordered sets and timestamps. Normalize those fields in the recorder, not in production code.

The harness will not catch unused private helpers. Those need a later extract with their own pins.

Floating-point money will flake if you serialize raw floats. Store integer cents, as the example fixtures do.

Seed-dependent randomness will also break an exact replay. Inject clocks and rng objects before you record anything.

Who should skip this approach

Do not use this on a greenfield module with no callers. Write intent tests instead of freezing accidental behavior.

Do not use this as a substitute for a security review. Characterization will happily freeze a leak or an auth hole.

Do not freeze personal data inside golden files. Redact, hash, or synthesize those fields before recording.

Skip this if you cannot run the module twice deterministically. Non-deterministic I/O needs seams before any golden capture.

Skip this if product owners already rejected the current outputs. Fix the behavior under a ticket, then pin the new contract.

Close

Pin the public outputs before the first structural edit. Change one mechanical thing and re-run the harness.

The run must return zero diffs before you continue. That sequence is the entire method for messy modules.

Run Characterization Tests Before Touching a Tangled Closer

Dakota Huang — Sat, 19 Sep 2026 03:09:28 +0000

Do not start a messy refactor with a rewrite.
Pin current outputs with characterization tests first.
Then change one seam and rerun the harness.

A tangled closer mixes parse, math, files, and prints.
One extract can alter rounding or newline handling.
Golden snapshots catch that drift before callers fail.

The failure mode

Most messy modules have no contract tests.
Engineers extract helpers from names, not outputs.
Return dicts stay stable while file bytes shift.

CSV money fields are a common silent break.
Python banker's rounding surprises many later extract patches.
A clean helper may switch to always-up rounding.

Stdout order can swap after a cosmetic cleanup.
JSON separators can change from spaced to compact.
Callers that hash output files will then fail.

You need pins on every observable channel.
Private function names are not observable contracts.
Do not assert on them during this phase.

What to pin

Characterization tests freeze today's observable behavior only.
They do not claim the behavior is right.
Fix semantics after the module has clear seams.

Record these four channels for a typical closer.

Return payload shape and numeric field values.
Exact output file bytes, including trailing newlines.
Stdout text with any clocks stripped out.
Exception class and message text for bad rows.

Drop wall-clock timestamps from the golden files.
Replace them with a fixed token during capture.
Otherwise every run produces a new snapshot.

Module-level caches are a fifth hidden channel.
Reset them in setUp before each test case.
Leftover keys will poison later golden assertions.

The messy module

The listing below is a stand-in closer.
Treat it as untested production Python code.
Do not clean it before the first pins.

# sales_close.py
from __future__ import annotations

import json
from pathlib import Path

CACHE: dict[str, float] = {}


def close_shift(log_path: str, out_path: str, tax_bps: int = 825) -> dict:
    CACHE.clear()
    gross = 0.0
    n = 0
    lines = Path(log_path).read_text().split("\n")
    for raw in lines:
        if raw.strip() == "" or raw.startswith("#"):
            continue
        parts = raw.split(",")
        sku = parts[0].strip()
        qty = int(parts[1])
        price = float(parts[2])
        line = round(qty * price, 2)
        CACHE[sku] = CACHE.get(sku, 0.0) + line
        gross = round(gross + line, 2)
        n += 1
    tax = round(gross * tax_bps / 10000.0, 2)
    net = round(gross + tax, 2)
    payload = {
        "count": n,
        "gross": gross,
        "tax": tax,
        "net": net,
        "by_sku": dict(CACHE),
    }
    text = json.dumps(payload, indent=2, sort_keys=True)
    Path(out_path).write_text(text + "\n")
    print("CLOSED", n, "lines")
    return payload

Note three landmines before any helper extract.
Blank lines at EOF still split into empties.
The hash prefix test does not skip indented comments.

Built-in round follows binary float rules, not decimal textbooks.
Pin the runtime result instead of a textbook table.
A later Decimal helper would be a behavior change.

Characterization harness

Keep golden files next to the tests, not tmp.
Commit them so reviews can see byte diffs.
Run the suite with a frozen working directory.

# test_sales_close_char.py
from __future__ import annotations

import io
import unittest
from contextlib import redirect_stdout
from pathlib import Path
from tempfile import TemporaryDirectory

import sales_close

FIXTURE = """# morning
A,2,1.25
B,1,10
A,1,3.50

# trailing
"""

GOLDEN_DIR = Path(__file__).parent / "goldens"


class CloseShiftCharacterization(unittest.TestCase):
    def setUp(self) -> None:
        sales_close.CACHE.clear()

    def _run(self, body: str, tax_bps: int = 825) -> tuple[dict, bytes, str]:
        with TemporaryDirectory() as tmp:
            root = Path(tmp)
            log = root / "shift.log"
            out = root / "shift.json"
            log.write_text(body)
            buf = io.StringIO()
            with redirect_stdout(buf):
                result = sales_close.close_shift(str(log), str(out), tax_bps)
            return result, out.read_bytes(), buf.getvalue()

    def test_happy_path_return_and_bytes(self) -> None:
        GOLDEN_DIR.mkdir(exist_ok=True)
        result, data, stdout = self._run(FIXTURE)
        self.assertEqual(result["count"], 3)
        self.assertEqual(stdout, "CLOSED 3 lines\n")
        golden = GOLDEN_DIR / "happy_path.json"
        if not golden.exists():
            golden.write_bytes(data)
            self.fail("wrote golden; rerun to assert bytes")
        self.assertEqual(data, golden.read_bytes())

    def test_indented_comment_is_not_skipped(self) -> None:
        body = "  # not a comment\n"
        with self.assertRaises(IndexError):
            self._run(body)

    def test_exact_binary_price_stays_stable(self) -> None:
        result, _, _ = self._run("A,1,1.25\n")
        self.assertEqual(result["gross"], 1.25)

The first test writes goldens on a cold run.
That write is deliberate for the initial capture.
Never accept a golden you did not open.

Inspect goldens/happy_path.json before the commit.
Confirm count, by_sku, and trailing newline by eye.
Then rerun until the fail-on-write path is gone.

Numbered workflow

Follow this order and skip none of the steps.

Copy one real production log into the fixtures directory.
Strip secrets but keep the original messy whitespace intact.
Add the harness and run once to emit goldens.
Open the golden JSON and check rounding by hand.
Commit tests, fixtures, and goldens in one change.
Extract one helper while keeping the public signature.
Re-run the suite and revert if bytes drift.

Do not extract parse and round in one patch.
One seam per patch keeps the diff reviewable.
Reviewers can then match bytes to the helper.

Smallest safe change

Extract rounding only and leave parse plus I/O.
The public close_shift signature must stay frozen.

def _round_money(value: float) -> float:
    return round(value, 2)


def close_shift(log_path: str, out_path: str, tax_bps: int = 825) -> dict:
    CACHE.clear()
    gross = 0.0
    n = 0
    lines = Path(log_path).read_text().split("\n")
    for raw in lines:
        if raw.strip() == "" or raw.startswith("#"):
            continue
        parts = raw.split(",")
        sku = parts[0].strip()
        qty = int(parts[1])
        price = float(parts[2])
        line = _round_money(qty * price)
        CACHE[sku] = CACHE.get(sku, 0.0) + line
        gross = _round_money(gross + line)
        n += 1
    tax = _round_money(gross * tax_bps / 10000.0)
    net = _round_money(gross + tax)
    payload = {
        "count": n,
        "gross": gross,
        "tax": tax,
        "net": net,
        "by_sku": dict(CACHE),
    }
    text = json.dumps(payload, indent=2, sort_keys=True)
    Path(out_path).write_text(text + "\n")
    print("CLOSED", n, "lines")
    return payload

Re-run the unittest module with verbose output enabled.
The golden bytes must match the committed file.
Stdout must still equal the pinned CLOSED line.

If file bytes change, the extract is already wrong.
Today's round result is the contract, not a wish.
Do not fix rounding inside the same patch.

Decision table

Map each observable signal to a stop rule.

Signal	Continue when	Stop if
Return dict	Keys and numbers match	Extra key or type appears
File bytes	Exact match to golden	Indent, order, or newline shifts
Stdout	Exact string match	Extra space or extra log line
Exception	Same class as before	Message text or class changes
CACHE	`by_sku` matches return	Insertion order or leftover keys

Use the table during review, not after merge.
A red cell means revert the helper extract.
Do not update goldens to hide a surprise.

Using a free model and a free server

A free coding model can draft the first harness.
It should not invent new rounding or skip rules.
Feed it the messy function and the pin list.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option.
Use the model to propose characterization tests only.
Use the free server when local interpreters conflict.

Paste the messy function and ask for pins, not rewrites.
Require the model to assert raw file bytes.
Pretty JSON equality misses separator and newline drift.

Run the emitted tests on the free server.
Download the golden files back into git.
The repo remains the source of truth here.

Reject patches that rename CACHE in the same step.
Reject patches that switch json dumps separators.
Those edits are new behaviors, not safe extracts.

Limitations

Characterization tests lock today's bugs in place.
Indented comments still crash after the rounding extract.
That crash is now a pinned, reviewed contract.

Do not use goldens as a long-term design spec.
They expire the moment you change the format.
Update them in a dedicated, reviewed follow-up commit.

Binary files need a different pin strategy altogether.
A hash plus size can work for binaries.
This closer is text, so full bytes are fine.

A free model can miss hidden side-effect channels.
Module globals and print calls are easy to skip.
The checklist above exists to close that gap.

A free server does not replace project CI jobs.
It only unblocks a dirty local Python toolchain.
Merge checks still run in your normal pipeline.

Who should skip this

Skip this if the module has no callers yet.
Write real unit tests and a clear spec instead.
There is nothing useful to characterize in that case.

Skip this for security-sensitive parsers and decoders.
Pinned crashes are not a useful threat model.
Fuzzing and reviewed grammars belong in that work.

Skip this if you must change rounding now.
Do that in a behavior-change commit first.
Then characterize the new contract, not the old.

Commands

python -m unittest test_sales_close_char.py -v
git add sales_close.py test_sales_close_char.py goldens
git diff --check

Keep the extract diff under one helper function.
If the diff touches parse and I/O, split it.
Characterization only pays off with small seams.

The closer is still messy after one extract.
That outcome is expected for this workflow.
Rewrite later, when the tests already exist.

If you run this harness on a free server, commit goldens locally first.

Characterize First: One Safe Edit in a Messy Repo

Dakota Huang — Sat, 19 Sep 2026 00:02:07 +0000

A messy refactor fails without locked current outputs. Characterization tests freeze those current outputs in place. Only then is one small change actually safe.

Skip the big rewrite on the first pass. Skip the clever rename storm as well. Capture what the module returns today instead.

The failure mode

Legacy modules hide rounding, aliases, and implicit defaults. Callers often depend on those hidden accidents. A clean extract can still change bytes.

Unit tests written after the edit miss that drift. They encode hoped behavior rather than current behavior. The regression then ships disguised as a cleanup.

Hoped tests also skip odd types at the boundary. Strings sneak in where integers were assumed. Gold files make those accidents visible early.

The core rule

Treat today's outputs as the working specification. Do not improve behavior in the first edit. Preserve output bytes, then change only structure.

This is not praise for bad logic. It is only a sequencing rule for edits. Correctness work comes after the gold lock.

Artifact: a gold-file characterization harness

The artifact is a small Python snapshot harness. It calls one public function across fixed cases. It writes JSON gold files you can diff.

Label the listing below as sample code. Adapt names to your real messy module. Keep secrets and live customer payloads out.

1. Isolate one messy entry point

Pick one function that already has real callers. Record its name, arity, and observed argument types. Ignore private helpers until the public lock holds.

# probe_quote.py — sample, not production telemetry
from legacy_quote import quote_total

CASES = [
    {"qty": 1, "unit_cents": 199, "coupon": None, "state": "CA"},
    {"qty": 3, "unit_cents": 199, "coupon": "SAVE10", "state": "CA"},
    {"qty": 0, "unit_cents": 500, "coupon": None, "state": "NY"},
    {"qty": 2, "unit_cents": 50, "coupon": "SAVE10", "state": "TX"},
    {"qty": 9, "unit_cents": 333, "coupon": "", "state": "ca"},
    {"qty": "4", "unit_cents": "100", "coupon": "save10", "state": "Ny"},
]

Six cases are a start, not coverage. Add shapes mined from logs after this lock. Prefer observed argument shapes over invented clean ones.

2. Snapshot outputs to gold files

Run the snapshot against current committed HEAD only. Do not snapshot a dirty worktree mix. The gold file must match what callers already run.

# snapshot.py — sample harness
import json
from pathlib import Path
from probe_quote import CASES
from legacy_quote import quote_total

GOLD = Path("gold")
GOLD.mkdir(exist_ok=True)


def run_case(case):
    value = quote_total(**case)
    return {
        "input": case,
        "output": value,
        "output_type": type(value).__name__,
    }


def main():
    rows = [run_case(c) for c in CASES]
    path = GOLD / "quote_total.json"
    text = json.dumps(rows, indent=2, sort_keys=True) + "\n"
    path.write_text(text)
    print(f"wrote {path} n={len(rows)}")


if __name__ == "__main__":
    main()

Commit the gold file in its own change. That commit is the behavior lock. Later diffs then show output movement clearly.

python snapshot.py
git add gold/quote_total.json snapshot.py probe_quote.py
git commit -m "lock quote_total characterization gold"

3. Replay gold on every later edit

The replay test is intentionally boring. It reloads JSON and calls the same function. Equality plus type name are the whole oracle.

# test_quote_characterization.py — sample
import json
from pathlib import Path
from legacy_quote import quote_total


def test_quote_total_matches_gold():
    rows = json.loads(Path("gold/quote_total.json").read_text())
    assert rows, "gold file must not be empty"
    for row in rows:
        actual = quote_total(**row["input"])
        assert actual == row["output"]
        assert type(actual).__name__ == row["output_type"]

python -m pytest test_quote_characterization.py -q

A failing assertion means behavior moved. Structure changes must keep this test green. Type changes also count as behavior here.

Sample messy module under test

The module below is intentionally awkward. Mixed types and silent defaults live here. Use it only as a teaching stand-in.

# legacy_quote.py — messy on purpose
def quote_total(qty, unit_cents, coupon=None, state="CA"):
    q = int(qty)
    cents = int(unit_cents)
    raw = q * cents
    code = (coupon or "").strip().upper()
    if code == "SAVE10":
        raw = int(raw * 0.9)
    st = str(state).upper()
    if st == "CA":
        raw = int(raw * 1.0725)
    elif st == "NY":
        raw = int(raw * 1.08)
    if raw < 0:
        raw = 0
    return raw

Note the float multiply and truncating int() calls. Callers may depend on that truncation today. A Decimal rewrite would change some stored totals.

Empty coupon strings already differ from missing coupons. Case folding on state is also load-bearing. Gold files catch those details without a narrative spec.

Grow the case list from traces

Static cases miss the ugly production shapes. Add a log miner after the first lock. Keep the miner read-only and sampled.

# mine_cases.py — sample, synthetic logs only
import ast
import json
from pathlib import Path


def parse_line(line):
    # expected: quote_total kwargs as a Python dict literal
    return ast.literal_eval(line.strip())


def main():
    seen = []
    for line in Path("synthetic_quote.log").read_text().splitlines():
        if not line.strip():
            continue
        case = parse_line(line)
        if case not in seen:
            seen.append(case)
    Path("mined_cases.json").write_text(json.dumps(seen, indent=2) + "\n")


if __name__ == "__main__":
    main()

Re-snapshot after merging mined rows into CASES. Do not edit gold by hand after that merge. Hand-edited gold files hide accidental churn.

Cap the miner on unique argument tuples. Huge gold files slow reviews and hide signal. A few dozen diverse rows beat thousands of clones.

Numbered workflow

Follow this order on every messy entry point. Do not start at the extract. The lock is the first deliverable.

List every caller of the chosen messy function.
Collect live argument shapes from logs and tests.
Snapshot gold JSON from current committed code.
Commit those gold files before any refactor starts.
Wire a replay test into the default CI job.
Perform one structural edit with no behavior pitch.
Re-run the replay test on the same cases.
Halt the change if gold output drifts unexpectedly.

Keep each change reviewable in one diff. Do not mix a bugfix with an extract. Bugfixes update gold in a separate commit.

Smallest safe change

Safe means gold stays byte-for-byte identical. Extract a local helper and stop. Rename an internal variable and stop.

Here is one safe extract against the sample module. Public results must still match the gold file. Coupon parsing moves; tax branches stay put.

def _apply_coupon(raw, coupon):
    code = (coupon or "").strip().upper()
    if code == "SAVE10":
        return int(raw * 0.9)
    return raw


def quote_total(qty, unit_cents, coupon=None, state="CA"):
    q = int(qty)
    cents = int(unit_cents)
    raw = _apply_coupon(q * cents, coupon)
    st = str(state).upper()
    if st == "CA":
        raw = int(raw * 1.0725)
    elif st == "NY":
        raw = int(raw * 1.08)
    if raw < 0:
        raw = 0
    return raw

Re-run pytest after this extract lands. If gold matches, the extract is finished. Do not fix California rounding in that same patch.

If gold fails, revert the extract first. Debug the helper in isolation after revert. Never “while we are here” on failing gold.

Decision table

Use the table as a merge gate. Structure work and behavior work stay split. Gold updates always need an explicit reason.

Proposed edit	Gold first	Same commit as extract	Verdict
Extract `_apply_coupon`	Required	No extra edits	Safe if gold holds
Rename `raw` to `subtotal`	Required	Local only	Safe if gold holds
Replace `int()` with `round()`	Required	Never with extract	Behavior change
Switch totals to `Decimal`	Required	Never with extract	Behavior change
Add a new coupon code	Required	Own commit	Intentional delta
Delete the `qty==0` path	Required	Never silent	Likely break

Read each row before opening the editor. If the verdict is behavior change, stop. Write a new gold commit with the intended delta.

Intentional deltas still need a before snapshot. The before snapshot proves the old contract. The after snapshot documents the new contract.

Optional model help, with a hard gate

A free coding model can draft extra cases from traces. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option.

Feed the model the function signature and gold JSON. Ask only for additional input rows, not new logic. Run every proposed row through quote_total on current code.

Keep rows that match a fresh snapshot. Drop rows the live function cannot accept. Drop rows that describe desired future math.

The free server can execute snapshot.py and pytest. Do not upload customer quotes or keys. Gold files should contain synthetic inputs only.

The model is a case generator, not a referee. The gold file remains the only referee. Human review remains the last merge gate.

What this workflow does not claim

It does not prove functional correctness. It proves stability against a recorded sample. Coverage still depends on the case list quality.

It does not replace partner contract tests. It does not replace property tests for parsers. It is a lock on one messy entry point.

Float tax math remains a later problem. Characterization makes that later change visible. Visibility is the point of the gold file.

Who should not use this approach

Do not use this as a substitute for design. Greenfield modules need real specs, not gold accidents. Do not freeze known security bugs as gold.

Do not snapshot functions that hit production networks. Do not store PII inside JSON fixtures. Do not run untrusted model output without pytest.

Skip this when the function already has tight tests. Skip this when you cannot execute the module. Skip this when the public result is nondeterministic.

Nondeterministic code needs clocks and seeds first. Freeze time before you freeze outputs. Randomness without seeds makes gold files lie.

Limits of the sample harness

The harness compares Python equality only. It will miss log side effects. It will miss file writes and extra queries.

Extend the snapshot if those channels matter. Capture stdout, SQL, and filesystem diffs next. Still change one concern per commit after that.

JSON also struggles with NaN and raw bytes. Prefer explicit output_type as shown above. Add a hex encoding step when bytes appear.

CI should run the replay test on every pull request. A sample job is a single pytest node. Do not hide gold failures behind allowlists.

# sample CI snippet, unexecuted
steps:
  - run: python -m pytest test_quote_characterization.py -q

If the job is red, the extract is not done. Comments about “equivalent refactor” do not override gold. Bytes are the review language here.

Closing sequence

Lock outputs. Extract one helper. Re-run gold. That loop is the whole method.

Keep gold files in version control. Review every model-drafted extra case. Ship one behavior-preserving edit at a time.

Record Entry-Point Goldens Before the Smallest Repo Cut

Dakota Huang — Fri, 18 Sep 2026 20:55:16 +0000

Record current outputs before you touch a tangled tree. Characterization tests turn today's quirks into a contract. The first edit should be the smallest seam that still compiles.

A messy repo rarely lacks tests by accident. Package imports still open files during module import. One cleanup pass can change bytes callers already store.

This article is a method, not a war story. Examples below are labeled proposals, not production patches. Run them on a copy of your tree.

1. Map the blast radius first

List every public entry before you rename a file. Scripts, CLI parsers, and package exports all count. Private helpers are not the first extraction target.

Do the mapping in this exact order.

Search the tree for script entry points and console wrappers.
Note modules imported by more than one package path.
Record environment keys read during import time.
Freeze that inventory in a committed text file.

rg -n "if __name__" -g "*.py"
rg -n "os.environ|Path.cwd|open\(" -g "*.py"
rg -n "entry_points|console_scripts" -g "*.{py,toml,cfg,in}"

Do not refactor during this grep pass. The map is the first artifact you keep. Missing an entry point is the usual silent failure.

Count call sites in the inventory file, not in chat. A number you can re-run beats a remembered list. Re-run the same three commands after every later cut.

2. Give each entry a case directory

Each case is a directory, not a hidden pytest soup. Directories keep stdin, argv, and workdirs obvious. Humans can diff them without a fixture DSL.

Use one layout and do not freelance it.

cases/invoice_csv/
  argv.json
  env.json
  stdin.bin
  target.txt
  work/
    input.csv

Keep target.txt as a relative module path only. Keep work/ as the cwd for that run. Never point the harness at live credentials or customer files.

argv.json should look like a real process vector.

["invoice.py", "--format", "csv", "input.csv"]

env.json should be a full map, not a delta. Partial env hides locale, timezone, and home-path bugs. Dump the process env once, then delete secrets by hand.

Add stdin.bin only when the entry reads standard input. Empty stdin still needs an explicit empty file. Implicit emptiness makes later diffs harder to trust.

3. Freeze bytes with a golden harness

The harness below is a proposal, not a measured benchmark. It writes goldens on the first run. Later runs fail on any byte-level drift.

Label: unexecuted example. Adapt paths before you run it.

# char_harness.py
from __future__ import annotations

import hashlib
import json
import os
import runpy
import sys
from pathlib import Path
from typing import Any

ROOT = Path(__file__).resolve().parent
GOLDEN = ROOT / "goldens"
CASES = ROOT / "cases"


def _digest(payload: bytes) -> str:
    return hashlib.sha256(payload).hexdigest()


def _run_case(case_dir: Path) -> dict[str, Any]:
    stdin_path = case_dir / "stdin.bin"
    argv = json.loads((case_dir / "argv.json").read_text(encoding="utf-8"))
    extra_env = json.loads((case_dir / "env.json").read_text(encoding="utf-8"))
    target = (case_dir / "target.txt").read_text(encoding="utf-8").strip()
    stdin_data = stdin_path.read_bytes() if stdin_path.exists() else b""

    old_argv = sys.argv[:]
    old_env = os.environ.copy()
    old_cwd = Path.cwd()
    try:
        sys.argv = argv
        os.environ.clear()
        os.environ.update(extra_env)
        os.chdir(case_dir / "work")
        ns = runpy.run_path(str(ROOT / target), run_name="__main__")
        return {
            "ok": True,
            "result": ns.get("RESULT"),
            "stdin_sha": _digest(stdin_data),
        }
    except Exception as exc:  # pin the type name, not a traceback
        return {
            "ok": False,
            "error_type": type(exc).__name__,
            "error": str(exc),
            "stdin_sha": _digest(stdin_data),
        }
    finally:
        sys.argv = old_argv
        os.environ.clear()
        os.environ.update(old_env)
        os.chdir(old_cwd)


def main() -> int:
    update = "--update" in sys.argv
    GOLDEN.mkdir(exist_ok=True)
    failed = 0
    for case_dir in sorted(p for p in CASES.iterdir() if p.is_dir()):
        observed = _run_case(case_dir)
        blob = json.dumps(observed, sort_keys=True, default=str).encode()
        golden_path = GOLDEN / f"{case_dir.name}.json"
        if update or not golden_path.exists():
            golden_path.write_bytes(blob)
            print(f"WROTE {golden_path}")
            continue
        expected = golden_path.read_bytes()
        if blob != expected:
            failed += 1
            print(f"DRIFT {case_dir.name}")
            print(f"  expected_sha={_digest(expected)}")
            print(f"  observed_sha={_digest(blob)}")
    return 1 if failed else 0


if __name__ == "__main__":
    raise SystemExit(main())

Run the harness in two steps, never one mixed step.

python char_harness.py --update
python char_harness.py
echo $?

The first command writes goldens from a cold tree. The second command must exit zero on that same tree. Commit cases and goldens in the same review.

Pin four fields for every entry you keep.

Canonical input bytes or parsed argument objects stay fixed.
Return value, output bytes, and raised type stay fixed.
Working directory, env keys, and argv stay fixed.
Files created, updated, or deleted stay fixed.

Skip private helpers on this first pass. Private names move when you extract later. Public results are the only contract that callers already have.

4. Treat drift as a stop sign

A failed hash is not a style debate. Open the two JSON blobs side by side. Confirm whether the change was intended at all.

If the drift is accidental, revert the edit immediately. If the drift is the point, update that one golden. Never batch-update goldens after a wide rewrite.

Print both hashes in the review notes. Reviewers can reject a hash they cannot explain. Unexplained goldens are how bugs become fixtures.

5. Cut one seam, then stop

Only one behavior-preserving edit follows a green harness. Extract a function the goldens already cover. Keep the old import path as a re-export.

Follow these change rules in order.

Extract one function the goldens already exercise end to end.
Keep the old import path as a one-line re-export.
Do not rename keyword arguments on this same pass.
Re-run the harness after that single edit only.

# proposed re-export, unexecuted example
from app.billing.csv_out import render_invoice as render_invoice

Stop if any golden moves after the extract. The seam is wrong or a case is missing. Restore the file, add a case, and only then retry.

Do not count deleted lines as progress here. Progress is a green harness plus one moved symbol. Architecture cleanup waits for a later, intentional golden change.

Where a free model belongs

A model is useful after traces exist, not before. It can suggest missing argv combinations from case files. It must not replace the harness or the revert rule.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode provides free model access and a free server option. That pair fits throwaway harness runs on a dirty tree. Paste traces in, take candidate cases out, and keep the revert decision local.

Do not ask the model to rewrite the messy package. Ask it to list inputs the goldens never cover. You still add those case directories by hand.

Reject invented APIs from chat by default. The messy module already has an API, however ugly. Characterization work records that API; it does not replace it.

Decision table for the next hour

Observation	Next step	Do not
Goldens missing for an entry	Add one case directory	Extract that entry
One function, goldens green	Extract and re-export	Rename callers
Drift on `error_type` only	Pin the exception class	Swallow the error
Drift on timestamps	Freeze time inside the case	Update goldens blindly
Model proposes a new API	Reject the names	Accept names from chat
Import opens a file	Pin that file in `work/`	Mock after the extract

Read the table left to right before each commit. The middle column is the only allowed action. The right column is how this method usually dies.

Limitations you should budget for

Characterization tests will lock bugs in place. That lock is the point of the first cut. Plan a later pass that changes goldens on purpose.

Golden JSON will rot when serializers change whitespace. sort_keys=True and default=str are part of the contract. Do not pretty-print in one run and compact in the next.

runpy.run_path is not a full process boundary. Shared process state can leak across case directories. Split to subprocesses if order-dependent failures appear.

# proposed subprocess pin, unexecuted example
import subprocess
import sys

proc = subprocess.run(
    [sys.executable, str(ROOT / target), *argv[1:]],
    cwd=case_dir / "work",
    env=extra_env,
    input=stdin_data,
    capture_output=True,
    check=False,
)
payload = {
    "returncode": proc.returncode,
    "stdout_sha": _digest(proc.stdout),
    "stderr_sha": _digest(proc.stderr),
}

Count stream bytes when you switch to subprocesses. stdout, stderr, and returncode belong in the golden. In-process RESULT shortcuts miss stream data that callers already parse.

Network, clocks, and RNG do not belong in raw goldens. Fake those seams before the first --update. Otherwise the harness trains you to ignore red runs.

Who should not use this method

Do not use this method on unread crypto or auth code. Pinning output can freeze a vulnerability as expected. Get a review before any golden update in that code.

Do not use it when you cannot reproduce a run. Flaky time, network, and RNG belong in fakes first. A drifting harness is worse than no harness.

Do not outsource the revert decision to a model. The harness is the authority for this pass. Chat text is not a test, and it is not a commit.

Skip the method if you already have contract tests. Those tests already name the behavior you would pin. Adding goldens on top only duplicates failure noise.

What done means on this pass

Done is a green harness plus one moved function. It is not a cleaned architecture. It is not a new framework around the mess.

Commit in this order only, with no extras.

Commit case directories and the harness with empty goldens omitted.
Commit goldens produced from one cold --update run.
Commit the single extract and the old-path re-export.
Commit harness output that proves zero drift on that extract.

If step four fails, revert step three before anything else. Add a case that names the missed branch. Repeat from the last green tree, not from memory.

This sequence is slow on purpose. Speed is how messy repos lose production behavior. Pin first, cut once, then stop for the day.

Characterize a Messy Module Before One Safe Change

Dakota Huang — Fri, 18 Sep 2026 17:51:42 +0000

Characterization tests beat a clever refactor every time. Pin observable output before you touch the module.

A messy file hides sort order and truncation rules. One extract without a baseline will shift those rules.

The failure mode

Developers often rewrite a helper that looks messy. The rewrite drops an implicit sort or default name.

Callers then fail on empty jobs or long errors. The refactor looked clean and still shipped a bug.

Core rule

Write tests first, then change one pure seam. Leave mixed formatting and defaults alone until later.

This article uses a small jobs summary module. The same sequence applies to any tangled helper.

Artifact: a messy jobs summary

The module below is intentionally compact and tangled. It formats job dicts into a stable text report.

Do not clean it before the tests exist. Copy it into jobs_summary.py as the subject.

# jobs_summary.py
import time


def summarize(jobs, now=None):
    if now is None:
        now = time.time()
    lines = []
    failed = 0
    for job in jobs:
        name = job.get("name") or "unnamed"
        status = job.get("status", "unknown")
        started = job.get("started_at")
        err = job.get("error") or ""
        if len(err) > 40:
            err = err[:37] + "..."
        dur = 0
        if started is not None:
            dur = int(now - started)
        if status == "failed":
            failed += 1
        lines.append((status, name, dur, err))
    order = {"failed": 0, "running": 1, "ok": 2, "unknown": 3}
    lines.sort(key=lambda row: (order.get(row[0], 9), row[1]))
    header = f"jobs={len(jobs)} failed={failed}"
    body = []
    for status, name, dur, err in lines:
        extra = f" err={err}" if err else ""
        body.append(f"{status} {name} {dur}s{extra}")
    if not body:
        return header
    return header + "\n" + "\n".join(body)

What the tests must pin

Characterization tests record behavior, including ugly parts. They do not judge whether the behavior is ideal.

Inject now so duration math stays deterministic. Never call time.time() inside a characterization test.

Decision table

Input shape	Locked observation
`[]`	header only, `jobs=0 failed=0`
missing `name`	label is `unnamed`
`name` is `""`	also `unnamed`, because of `or`
missing `status`	status is `unknown`
`error` longer than 40	slice 37 chars, then `...`
empty `error`	no `err=` suffix on the line
mixed statuses	`failed`, then `running`, then `ok`
unknown status label	sort key falls through to 9
`started_at` present	duration is `int(now - started_at)`
`started_at` missing	duration is `0s`
`status == "failed"`	counted in the header only

Each row is one assertion, not a redesign prompt. If production depends on a quirk, the test keeps it.

Workflow

1. Freeze the public function only

Test summarize only, not imagined private helpers. Private extracts come after the baseline is green.

Name the file test_jobs_summary.py. Keep it next to the messy module.

2. Cover the edges in the table

Empty input is a first-class characterization case. Missing keys and long errors belong in the same file.

Add the empty-string name case before any extract. That or chain is a frequent silent change.

3. Run the suite until it is boring

Green tests are the lock, not a style win. Re-run after every edit, including comment-only edits.

4. Make the smallest safe change

Extract one function that the tests already imply. Stop when the suite is green and the diff is tiny.

Characterization tests

The tests below freeze strings, not object graphs. Exact text is the contract for this helper.

# test_jobs_summary.py
import unittest
from jobs_summary import summarize

NOW = 1_700_000_040


class SummarizeCharacterization(unittest.TestCase):
    def test_empty_jobs_header_only(self):
        self.assertEqual(summarize([], now=NOW), "jobs=0 failed=0")

    def test_missing_name_becomes_unnamed(self):
        text = summarize([{"status": "ok"}], now=NOW)
        self.assertIn("ok unnamed 0s", text)

    def test_empty_name_also_unnamed(self):
        text = summarize([{"name": "", "status": "ok"}], now=NOW)
        self.assertIn("ok unnamed 0s", text)

    def test_missing_status_is_unknown(self):
        text = summarize([{"name": "build"}], now=NOW)
        self.assertIn("unknown build 0s", text)

    def test_error_truncated_at_forty(self):
        err = "x" * 41
        text = summarize(
            [{"name": "sync", "status": "failed", "error": err}],
            now=NOW,
        )
        self.assertIn("err=" + ("x" * 37) + "...", text)
        self.assertNotIn("x" * 41, text)

    def test_empty_error_omits_suffix(self):
        text = summarize(
            [{"name": "sync", "status": "ok", "error": ""}],
            now=NOW,
        )
        self.assertEqual(text, "jobs=1 failed=0\nok sync 0s")

    def test_sort_failed_running_ok_then_name(self):
        jobs = [
            {"name": "b", "status": "ok"},
            {"name": "a", "status": "running"},
            {"name": "c", "status": "failed"},
            {"name": "a", "status": "failed"},
        ]
        text = summarize(jobs, now=NOW)
        lines = text.splitlines()[1:]
        self.assertEqual(
            lines,
            ["failed a 0s", "failed c 0s", "running a 0s", "ok b 0s"],
        )

    def test_unknown_status_sorts_after_known(self):
        jobs = [
            {"name": "z", "status": "weird"},
            {"name": "a", "status": "ok"},
        ]
        lines = summarize(jobs, now=NOW).splitlines()[1:]
        self.assertEqual(lines, ["ok a 0s", "weird z 0s"])

    def test_duration_uses_injected_now(self):
        jobs = [{"name": "etl", "status": "ok", "started_at": NOW - 9}]
        text = summarize(jobs, now=NOW)
        self.assertEqual(text, "jobs=1 failed=0\nok etl 9s")

    def test_failed_count_ignores_other_statuses(self):
        jobs = [
            {"name": "a", "status": "failed"},
            {"name": "b", "status": "ok"},
            {"name": "c", "status": "failed"},
        ]
        header = summarize(jobs, now=NOW).splitlines()[0]
        self.assertEqual(header, "jobs=3 failed=2")


if __name__ == "__main__":
    unittest.main()

Commands

Run the suite from the repo root with unittest.

python -m unittest test_jobs_summary.py -v

A useful check is a deliberate mutation of truncation. Change the slice to 30 characters and rerun tests.

The long-error case must fail if the pin is real. Restore the original slice before the extract commit.

# temporary mutation inside jobs_summary.py, then:
python -m unittest test_jobs_summary.py -v
# expect test_error_truncated_at_forty to fail, then revert

Record the failure name in the review notes. A pin that never fails is not a pin.

Smallest safe change

After the pin, extract truncation and nothing else. Do not retouch sort keys in the same diff.

def _truncate_error(err):
    if len(err) > 40:
        return err[:37] + "..."
    return err

Call _truncate_error(err) from the existing loop. Keep summarize as the only public entry point.

Do not rename keys or change sort order yet. Do not replace or with a None check yet.

Re-run unittest. The output strings must match exactly. If a line drifts, revert and shrink the extract.

A second later commit can extract duration. That split keeps git blame honest.

git add test_jobs_summary.py jobs_summary.py
git commit -m "test: characterize summarize output"
# then, after the extract:
git add jobs_summary.py
git commit -m "refactor: extract error truncation only"

Two commits make a later revert cheap and obvious. Mixed commits hide which edit broke a caller.

Using a model without skipping the pin

A coding model can suggest extra edge cases quickly. It cannot replace a committed, locally executed baseline.

MonkeyCode offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Paste the messy function and the decision table only. Ask for candidate cases, not a rewritten module.

Keep generated tests out of main until they pass locally. Discard any case that asserts a behavior you did not observe.

The free server does not change the review duty. You still own the exact strings in the characterization file.

Reject a generated extract that also “fixes” empty names. That is a behavior change hiding in a refactor.

What this method does not do

It does not fix product bugs you intend to keep. Characterization will freeze a truncation bug if you allow it.

Split that work: pin first, then add a behavior-change test. Do not mix a bugfix and an extract in one diff.

Non-deterministic clocks, networks, and maps will flake. Inject time, and sort any set-driven output explicitly.

Full-file snapshots also drift on header wording. Prefer line-level asserts for the rows you care about.

Who should skip this

Skip this if the module has no callers yet. Greenfield code can use ordinary TDD instead.

Skip this if you need a new report format now. That is a behavior change, not a safe refactor.

Skip this if output includes raw timestamps from the wall clock. Stabilize time first or the snapshots will thrash.

Skip this if legal copy must change with the extract. Frozen strings would block a required wording update.

Failure analysis

The usual break is an unstable sort of equal keys. The sample module sorts by status, then by name.

Two failed jobs named alike will keep input order. Python’s sort is stable; pin that if callers depend on it.

Add a duplicate-name case when the real log shows ties. Do not assume uniqueness from the happy path.

Another break is or versus missing empty strings. job.get("name") or "unnamed" treats empty as unnamed.

Add a case with "name": "" before any extract. Models often “fix” that line without a test.

Integer truncation of duration is another silent edit. int(9.9) is 9; a round() swap will fail the pin.

Checklist

Copy the messy function with no cosmetic edits.
Inject clocks, paths, and other non-deterministic inputs.
Fill a decision table from real caller traces.
Lock public output with exact string asserts.
Mutate one rule and confirm a named test fails.
Restore the mutation, then extract a single helper.
Re-run the suite and stop if any line drifts.

Close

Messy repos reward patience more than tasteful helpers. Lock the text, extract one seam, then stop.

Commit the characterization file before the extract commit. If a free model drafts cases, keep the assertions yours.