Write Characterization Tests Before the First Extract

A checkout helper in a brownfield shop repository has no tests, three authors, and a comment that says do not touch. The function mixes coupon stacking, tax rounding, and a loyalty multiplier that only fires on Thursdays. An agent can rewrite the file in one pass, and the diff will look clean while the totals drift. The cheaper path is to pin today's outputs first, then change one small piece.

This walkthrough treats characterization tests as the first refactor, not leftover coverage after a rewrite. It uses a small, intentionally messy Python module as the working artifact. The method remains useful if every coding-agent mention is removed from the process. No customer metrics are claimed here; the module is a labeled teaching fixture.

Cheap rewrites do not cheapen understanding

Generated edits have made large cleanups feel inexpensive compared with reading caller history. A messy module still encodes rounding rules, implicit defaults, and quirks that live only in production traces. When those quirks are untested, a cleanup is a behavior change that happens to look like formatting. The useful response is a smaller surface that you can prove did not move.

A nearby discussion in developer communities is what happens to technical debt when generated code gets cheap. The practical answer for a messy repo is not a wider rewrite window. It is a golden file for one function, followed by a single mechanical extract. Anything larger is a product change and should be named as one.

Inventory the boundary before any generation

Work from a frozen tree and a single entry point rather than a whole-package rewrite. Do not start by asking a model to clean the file. The inventory is the first artifact reviewers can audit without executing a prompt log.

1. Record the module path, public functions, and known callers.
2. Capture three to five argument sets from logs, fixtures, or a staging replay.
3. Note side effects such as files, clocks, environment variables, and network calls.
4. Name the characterization boundary as one function, not the package.

A short charter file stops later prompts from inventing callers that do not exist.

# CHARTER.md
module: shop/pricing.py
entry: compute_total(items, coupon, weekday, tax_region)
callers: checkout.py, admin/reprice.py
side_effects: reads LOYALTY_THU from the environment
known_quirk: negative coupons currently clamp after tax, not before
do_not: rewrite compute_total in the first change

A messy module you can actually run

The listing below is a teaching fixture, not production code from a named company. Dead branches and clamp-after-tax order are part of the contract until a later, explicit change. Characterization work records that contract, including the parts the team already dislikes.

# shop/pricing.py
from __future__ import annotations

import os
from typing import Iterable


def compute_total(
    items: Iterable[dict],
    coupon: str | None,
    weekday: str,
    tax_region: str,
) -> dict:
    subtotal = 0.0
    for item in items:
        qty = item.get("qty") or 1
        subtotal += float(item["price"]) * qty
        if item.get("fragile"):
            subtotal += 2.5

    if coupon == "SAVE10":
        subtotal = subtotal * 0.9
    elif coupon == "SAVE10":
        subtotal = subtotal * 0.8  # unreachable; keep until a named cleanup
    elif coupon and coupon.startswith("FLAT"):
        try:
            subtotal -= float(coupon[4:])
        except ValueError:
            pass

    if weekday == "Thu" and os.environ.get("LOYALTY_THU") == "1":
        subtotal *= 0.97

    if tax_region == "CA":
        tax = round(subtotal * 0.0825, 2)
    elif tax_region == "CA":
        tax = round(subtotal * 0.0725, 2)
    else:
        tax = round(subtotal * 0.0, 2)

    if subtotal < 0:
        subtotal = 0.0

    return {
        "subtotal": round(subtotal, 2),
        "tax": tax,
        "total": round(subtotal + tax, 2),
    }

Build the golden table from observed calls

Do not invent inputs that the module never sees in callers. Pull argument tuples from logs or from a one-off tracer around the live function. Commit the JSON that the frozen code produced, not the totals someone considers more reasonable.

# tools/trace_pricing.py
from shop.pricing import compute_total

SAMPLES = [
    ([{"price": 20, "qty": 2}], "SAVE10", "Mon", "CA"),
    ([{"price": 20, "qty": 2, "fragile": True}], "FLAT5", "Thu", "CA"),
    ([{"price": 3.33, "qty": 3}], None, "Thu", "NY"),
    ([{"price": 10}], "FLATX", "Fri", "CA"),
    ([{"price": 1, "qty": 1}], "FLAT999", "Mon", "CA"),
]

if __name__ == "__main__":
    import json
    import os

    os.environ["LOYALTY_THU"] = "1"
    rows = []
    for items, coupon, weekday, region in SAMPLES:
        out = compute_total(items, coupon, weekday, region)
        rows.append(
            {
                "items": items,
                "coupon": coupon,
                "weekday": weekday,
                "tax_region": region,
                "out": out,
            }
        )
    print(json.dumps(rows, indent=2))

Run the tracer once on a clean tree and store the output beside the tests. That file is the artifact another engineer can replay without trusting a narrative about expected money.

git switch -c char-pricing
python tools/trace_pricing.py > tests/fixtures/pricing_golden.json
git add CHARTER.md shop/pricing.py tools/trace_pricing.py tests/fixtures/pricing_golden.json

Turn the fixture into characterization tests

The tests assert today's JSON, not the totals the shop wishes it had charged. If a later refactor changes a cent, the suite must fail even when the new number looks more correct. Assert return values, not local variable names, so a later extract does not fail for cosmetic reasons.

# tests/test_pricing_characterization.py
from __future__ import annotations

import json
import os
from pathlib import Path

import pytest

from shop.pricing import compute_total

FIXTURE = Path("tests/fixtures/pricing_golden.json")
ROWS = json.loads(FIXTURE.read_text())


@pytest.mark.parametrize("row", ROWS, ids=[str(i) for i in range(len(ROWS))])
def test_compute_total_matches_golden(row):
    os.environ["LOYALTY_THU"] = "1"
    got = compute_total(
        row["items"],
        row["coupon"],
        row["weekday"],
        row["tax_region"],
    )
    assert got == row["out"]

LOYALTY_THU=1 pytest -q tests/test_pricing_characterization.py

If the suite is green on the frozen tree, you have a behavioral pin. You do not yet have a better design, and you should not delete unreachable branches in the same step.

Decision table: add a row, or change one thing

Use the table before any agent edit lands on compute_total. Expanding the golden file is usually cheaper than touching production logic. The default action is another row, not a rewrite.

Observation	Action	Stop condition
A caller passes a key the fixture never includes	Add one golden row from that caller	New row reproduces on the frozen tree
Output depends on the clock or the network	Wrap that dependency before more tests	Tests run without live services
Two dead branches exist but no caller hits them	Leave them; do not clean yet	No production stack frame points here
Golden suite is green and callers are listed	Extract one pure helper	Diff adds one function and keeps assertions
A test fails after a style-only edit	Revert; the edit changed behavior	git diff -- shop/pricing.py is empty

The smallest safe change

After the suite is green, extract rounding into a helper without changing call order. Do not delete dead branches in the same commit. Do not fix clamp-after-tax while renaming helpers, because that mixes a product decision with a mechanical move.

# shop/money.py
def money(value: float) -> float:
    return round(value, 2)

# only the return site in shop/pricing.py changes
from shop.money import money

return {
    "subtotal": money(subtotal),
    "tax": tax,
    "total": money(subtotal + tax),
}

Run the same pytest command against the golden file. If any assertion fails, the extraction was not mechanical, and the commit should not proceed. A second change, such as clamping before tax, needs a new test that states the desired total and a review of historical invoices.

pytest -q tests/test_pricing_characterization.py
git add shop/money.py shop/pricing.py tests
git commit -m "Extract money() helper; keep compute_total outputs pinned"

Guard the diff size

Teams that let agents edit freely still need a hard stop on blast from the first extract. The script below is a local check, not a benchmark of any product. It fails when shop/pricing.py grows a large hunk after the golden tests already exist.

# tools/assert_small_pricing_diff.py
import subprocess
import sys

result = subprocess.run(
    ["git", "diff", "--numstat", "--", "shop/pricing.py"],
    check=True,
    capture_output=True,
    text=True,
)
line = result.stdout.strip()
if not line:
    sys.exit(0)
added, deleted, _path = line.split("\t", 2)
if int(added) + int(deleted) > 12:
    raise SystemExit(
        f"pricing.py diff too large for a first extract: +{added} -{deleted}"
    )

python tools/assert_small_pricing_diff.py

Where a free model and a free server actually help

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A model is useful for proposing extra golden rows from grep'd call sites, not for rewriting compute_total on the first pass. Free model access is enough for that narrow drafting job if you still run the tracer and keep only rows that reproduce. A free server option matters when the laptop tree is dirty, because characterization must execute against the frozen module rather than half-applied local edits.

The workflow is freeze the branch, generate candidate samples, run the tracer and pytest in isolation, then accept only JSON the frozen code actually produced. If a suggestion cannot be replayed, discard it. The product does not replace the golden file; it only lowers the cost of a disposable run.

Limitations

Characterization locks bugs and quirks with the same strength that it locks intended rules. Teams that need a correct total, not a stable total, still have to write a second, explicit specification later. The method also assumes a deterministic function at the chosen boundary.

Hidden clocks, unordered iteration on older runtimes, and networked tax services will make the golden file flap. Generated tests that assert internal local names instead of return values are a false pin. They break on extraction even when customer-visible behavior is stable. A green suite is not permission to expand scope in the same pull request.

Who should not use this approach

Skip this protocol during an active billing incident, when the current output is already known to be wrong and time-boxed. Skip it when the module is not invocable without production secrets or a full monolith boot. Skip it when a regulator requires the new behavior immediately and historical totals must change under a tracked migration.

Frontend-only work with visual output and no pure function may need HTML or screenshot goldens instead of JSON. Concurrent code with races needs a different harness. If you cannot list callers, you do not yet have a characterization boundary, and an extract will only relocate the uncertainty.

Close the loop

Keep the charter, the fixture, and the one-function diff in the same pull request. Reviewers should replay python tools/trace_pricing.py and pytest without depending on a narrative. The next refactor starts only after this pin is merged, and the following change should name the behavior it intends to alter.

Isolation from a dirty working tree is part of the method, whether that isolation is a second clone or a disposable server. The smallest safe change is still one helper, one green golden file, and a diff small enough to read in a single review pass.