Characterize a Messy Merger First, Then Ship One Hunk

Characterization tests freeze today's outputs before any extract. A one-hunk diff is the only follow-up that stays reviewable. Invert that order and silent defaults will move.

The failure this protocol stops

Assistants now rewrite helpers without a behavior freeze. Reviewers praise names and miss coercion changes. The protocol below treats the current dump as law.

A messy module usually hides three leak types. Defaults fill missing keys with surprising sentinels. Coercion turns bad inputs into silent zeros.

Insertion order becomes a public contract through JSON. You cannot see those leaks in a rename diff. You can see them in a frozen output trace.

What you pin, and what you skip

Pin return values, exception types, and caller-visible side effects. Skip clocks, random salts, and live network bodies. Skip log wording unless a parser consumes it.

Signal	Pin?	Reason
Returned tuple of key and qty	Yes	Callers iterate this exact shape
Exception class	Yes	Upstream branching depends on type
Exception message string	Rarely	Wording is incidental in most trees
UUID and wall time	No	Every run would churn the goldens
Dict insertion order	Yes if dumped	JSON often preserves that order

Use the table as a pre-extract filter only. Do not treat those rows as coverage proof.

Artifact: dump traces, then freeze them

The worked example is a messy CSV merger. Duplicate SKUs sometimes sum across adjacent rows. Sometimes the last quantity wins for that SKU. Types coerce in place during the loop.

Label: this listing is an unexecuted copy-paste example. Run the listing locally on your own tree. Do not read the fixtures as production metrics.

Step 1. Isolate the messy function

Keep the legacy module untouched during the dump. Resist cleanup comments and local renames here. The dump must see the current production path.

# merger.py — freeze first, extract later
from decimal import Decimal, InvalidOperation

def merge_rows(rows):
    acc = {}
    order = []
    for row in rows:
        key = str(row.get("sku") or "").strip() or "UNKNOWN"
        raw = row.get("qty", 0)
        try:
            qty = Decimal(str(raw))
        except (InvalidOperation, TypeError):
            qty = Decimal("0")
        mode = (row.get("mode") or "sum").lower()
        if key not in acc:
            acc[key] = qty
            order.append(key)
            continue
        if mode == "last":
            acc[key] = qty
        else:
            acc[key] += qty
    return [(k, acc[k]) for k in order]

Empty SKU values collapse into the UNKNOWN key. Invalid qty values collapse into decimal zero. Missing mode values sum rather than overwrite.

Step 2. Dump traces from fixtures, not guesses

Build fixtures from logs when you can. Redact tokens before any fixture hits disk. Cap the sample so the suite stays readable. Convert each sample into one list of rows.

# dump_traces.py
import json
from merger import merge_rows

FIXTURES = [
    [{"sku": "A", "qty": "1.5"}, {"sku": "A", "qty": 2, "mode": "sum"}],
    [{"sku": "  ", "qty": None, "mode": "last"}],
    [{"sku": "B", "qty": "x"}, {"sku": "B", "qty": 3, "mode": "LAST"}],
    [{"sku": "C", "qty": 1, "mode": "last"}, {"sku": "C", "qty": 9}],
    [{"sku": "D", "qty": "4.00"}, {"sku": "d", "qty": 1, "mode": "sum"}],
]

def serialize(pairs):
    return [(k, format(v, "f")) for k, v in pairs]

if __name__ == "__main__":
    traces = []
    for i, rows in enumerate(FIXTURES):
        traces.append({
            "id": i,
            "input": rows,
            "output": serialize(merge_rows(rows)),
        })
    print(json.dumps(traces, indent=2, sort_keys=True))

Run this exact command and keep the file.

python dump_traces.py > traces.json
git add traces.json dump_traces.py merger.py
git commit -m "freeze merger traces before any extract"

Do not pretty-edit traces.json after the dump. Hand edits break the recorded output contract. Re-run the dumper if fixtures later change.

Case five is an intentional case-fold trap. SKU "D" and "d" remain two distinct keys. The merger does not lowercase SKU strings. That fact belongs inside the frozen traces.

Step 3. Load traces in a characterization suite

# test_merger_char.py
import json
from pathlib import Path
from merger import merge_rows

TRACES = json.loads(Path("traces.json").read_text())

def serialize(pairs):
    return [(k, format(v, "f")) for k, v in pairs]

def test_each_trace_stays_stable():
    for case in TRACES:
        got = serialize(merge_rows(case["input"]))
        want = [tuple(p) for p in case["output"]]
        assert got == want, case["id"]

Run the suite with a quiet pytest invocation.

python -m pytest test_merger_char.py -q

Name the test after stability, not business intent. Intent tests belong in a later dedicated file. This file exists only to detect output drift.

Step 4. Pin exception types without pinning prose

Legacy None input may already raise TypeError. Pin the class and leave the message free.

import pytest
from merger import merge_rows

def test_none_rows_raises_type_error():
    with pytest.raises(TypeError):
        merge_rows(None)

Skip this test if None currently returns UNKNOWN. Characterization copies reality and does not invent raises.

Step 5. Ship one hunk, then stop

Extract one private helper for quantity coercion. Touch exactly one call site in merge_rows. Keep every public trace identical after the move.

def coerce_qty(raw):
    from decimal import Decimal, InvalidOperation
    try:
        return Decimal(str(raw))
    except (InvalidOperation, TypeError):
        return Decimal("0")

Replace only the inline try and except block. Leave key folding and mode defaults untouched.

python -m pytest test_merger_char.py -q
git diff --stat

Diffstat should list merger.py after tests already landed. If traces fail, revert the hunk immediately. Do not stack a rename on a red extract.

Follow this numbered extract protocol without extra cleanup.

1. Confirm traces.json is committed on the main branch.
2. Run the characterization suite and record the pass line.
3. Extract one private helper and one call site.
4. Re-run the same suite against the same traces.
5. Open a new branch for the next helper later.

There is no sixth opportunistic cleanup step. "While we are here" is how silent leaks land.

Failure taxonomy after a red suite

Read the assertion id before you edit code. Then classify the drift into one bucket.

Output length changed after the helper extract. A key collapsed or a new UNKNOWN appeared. Decimal text changed in the serialized pairs. The serializer used repr or dropped trailing scale. Mode last failed on a mixed default row. You altered defaulting while moving the helper. Key case merged D with d unexpectedly. You lowercased SKUs during the extract by accident.

Fix the hunk or revert it completely. Do not patch traces.json to match new bugs.

Where free model access belongs

Fixture lists grow slowly on a messy module. A model can suggest extra rows from traces. It must not edit merger.py before tests exist.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. Point the model at dump_traces.py and traces.json. Ask for additional FIXTURES that hit empty keys. Run dump_traces.py and pytest on the free server. Keep merger.py read-only until the suite stays green.

The model is a fixture generator in this protocol. It is not an unattended refactor agent here. Local pytest still decides whether the hunk ships. Remove the product and the protocol remains usable.

Limits

A green characterization suite does not prove correctness. Bad totals stay pinned if fixtures include them. You still owe intent tests after the freeze.

Five fixtures miss unicode SKUs and extra blanks. They miss mixed blank keys with last mode. Harvest more rows from redacted logs first. Do that before any further helper extracts.

Decimal formatting hides scientific notation edge cases. format(v, "f") is the chosen serialization pin. Do not switch to str(v) in midstream commits. Python minor versions can change object repr text. They should not change this format call.

Who should not use this approach

Do not freeze a module with zero callers. Delete it instead of pinning dead output. Do not freeze cryptographic or auth code this way. A characterization pin can preserve a real weakness.

Do not substitute this method for parser property tests. Characterization is a freeze, not a grammar spec. Skip it when real contract tests already lock shape. Extra golden files then duplicate maintenance noise.

Skip it when outputs contain secrets or tokens. traces.json would leak those values into git. Redact first or refuse to dump traces.

Merge checklist

Use this list as a gate, not decoration.

1. Tests and traces landed in a prior commit.
2. Public signature of merge_rows stayed fully unchanged.
3. git diff --stat shows one helper plus one call.
4. pytest used committed traces.json, not a regenerated guess.
5. No extra file was cleaned in the same branch.

A messy repo does not need a rewrite this week. It needs a freeze, then one reviewable hunk. Characterization tests are that freeze on disk. The next extract waits for a green run. Keep traces and the extract hunk in two commits.