Pin the Bugs Before You Move the Function: A Two-Way Behavior Ledger

A refactor that fixes a bug is a failed refactor. It hides an intended change inside a mechanical move.

Your tests go green, so nobody looks. Six weeks later someone asks why refunds changed sign.

This post covers a small ledger that pins current behavior and intended behavior separately. Then it makes a move that touches neither without a loud failure.

The failure mode: green suite, changed contract

Characterization tests describe what the code does today. That includes behavior you consider wrong.

Two accidents follow from that gap.

1. The move silently changes a pinned value. Easy to catch, if the test asserts exact values.
2. The move silently fixes a pinned bug. Hard to catch, because a fixed bug usually makes assertions pass or disappear.

The second one is the expensive one. It lands a semantic change inside a commit titled move: extract pricing.

Step 1: Write the ledger, not the tests

Start with a data file. One row per behavior you refuse to lose.

The example below is illustrative. Your capture run produces your numbers, not mine.

# refactor/behavior_ledger.yml
seam: billing.legacy:compute_invoice
cases:
  - id: int_qty_rounding
    inputs: {qty: 3, unit_cents: 333, rate: "0.0825", currency: USD}
    current: {total_cents: 1081}
    intended: {total_cents: 1081}
    status: preserve

  - id: jpy_zero_decimals
    inputs: {qty: 2, unit_cents: 500, rate: "0.10", currency: JPY}
    current: {total_cents: 1100}
    intended: {total_cents: 1100}
    status: preserve

  - id: negative_qty_sign
    inputs: {qty: -1, unit_cents: 500, rate: "0", currency: USD}
    current: {total_cents: 500}
    intended: {total_cents: -500}
    status: known_bug
    note: "Refund path keeps a positive sign. Pinned so the move cannot fix it quietly."

Three fields carry the weight. current is observed. intended is a human decision. status is the admission you usually skip.

The seam line is the only line you edit between the before-run and the after-run. That makes the review diff one line long.

Step 2: Reject contradictory ledgers

A ledger you can fudge is not a gate. Validate it before any code moves.

# refactor/validate_ledger.py
import pathlib, sys, yaml

spec = yaml.safe_load(pathlib.Path("refactor/behavior_ledger.yml").read_text())
bad, seen = [], set()

for c in spec["cases"]:
    if c["id"] in seen:
        bad.append(f"duplicate id: {c['id']}")
    seen.add(c["id"])
    if c["status"] == "preserve" and c["current"] != c["intended"]:
        bad.append(f"{c['id']}: preserve row disagrees with intended")
    if c["status"] == "known_bug" and c["current"] == c["intended"]:
        bad.append(f"{c['id']}: marked known_bug but current == intended")
    if c.get("current") is None:
        bad.append(f"{c['id']}: no baseline captured")

print("\n".join(bad) or "ledger ok")
sys.exit(1 if bad else 0)

Run it with python refactor/validate_ledger.py. The check is cheap and it kills the lazy escape hatch.

You cannot label a suspicious value known_bug unless you also state what it should be.

Step 3: Capture baselines append-only

Capture runs against the code as it exists now. It must never overwrite an existing baseline.

# refactor/capture.py
import importlib, pathlib, sys, yaml

LEDGER = pathlib.Path("refactor/behavior_ledger.yml")

def call(seam, inputs):
    module, _, func = seam.partition(":")
    return getattr(importlib.import_module(module), func)(**inputs)

def main(argv):
    update = "--update" in argv
    spec = yaml.safe_load(LEDGER.read_text())
    bad = 0
    for case in spec["cases"]:
        observed = call(spec["seam"], case["inputs"])
        recorded = case.get("current")
        if recorded is None:
            if not update:
                print(f"MISSING {case['id']}: capture with --update first")
                bad += 1
            else:
                print(f"CAPTURED {case['id']}: {observed}")
            continue
        if observed != recorded:
            print(f"DRIFT {case['id']}: {recorded} -> {observed}")
            bad += 1
    return 1 if bad else 0

sys.exit(main(sys.argv[1:]))

Note what --update does not do. It fills missing rows. It never rewrites a recorded value.

If a value is wrong, you delete the row and write down why in the commit message. Re-baselining must cost effort.

Step 4: Replay before the move

The suite reads the ledger directly, so there is one source of truth. No duplicated expected values in test files.

# refactor/test_ledger.py
import importlib, pathlib, pytest, yaml

SPEC = yaml.safe_load((pathlib.Path(__file__).parent / "behavior_ledger.yml").read_text())

def call(case):
    module, _, func = SPEC["seam"].partition(":")
    return getattr(importlib.import_module(module), func)(**case["inputs"])

@pytest.mark.parametrize("case", SPEC["cases"], ids=lambda c: c["id"])
def test_ledger(case):
    observed = call(case)
    if case["status"] == "preserve":
        assert observed == case["current"], f"drift: {case['current']} -> {observed}"
        return
    if observed == case["intended"]:
        pytest.fail("the move fixed a pinned bug; flip the ledger in its own commit")
    assert observed == case["current"], f"pinned bug changed: {case['current']} -> {observed}"

The known_bug branch is the tripwire. It fires in both directions.

Same buggy value means pass. Correct value means fail, with a message telling you what to do next.

Run python refactor/capture.py and pytest refactor/test_ledger.py -q. Both must pass before you touch the module.

If they do not pass pre-move, your ledger is wrong. Fix the ledger, not the code.

Step 5: Make the smallest change that keeps the ledger green

Smallest is a measurable rule, not a feeling. Use three constraints.

1. Every hunk cites at least one case id in the commit trailer: Ledger: int_qty_rounding, jpy_zero_decimals.
2. No hunk that no case needs. If you cannot name the case, drop the hunk.
3. No ledger edits in the move commit. Zero. The seam line lands in a separate commit you can see.

git log -1 --format=%B | grep -qE '^Ledger: ' || { echo 'missing Ledger trailer'; exit 1; }

Constraint 2 is where the size actually shrinks. Extraction tends to drag along formatting, renaming, and a helper split.

Most of those hunks protect nothing. Deleting them is how the diff becomes reviewable.

For the move itself, copy-then-switch beats in-place editing. Copy the body, point the seam at the new module, run the suite.

Step 6: Replay after the move

Change one line in the ledger: the seam value. Run both commands again.

Read the failures by class:

Result	Meaning	Action
preserve fails	Behavior drifted	Revert the offending hunk
known_bug fails on current	Pinned bug changed shape	Understand it, then decide
known_bug fails on intended	The move fixed the bug	Flip status in its own commit
All green	Mechanical move	Merge

The third row is the whole point. The fix is now a deliberate, separate change with its own ledger diff.

It gets its own review, its own changelog line, and its own rollback path.

If you want the classic split, do the move commit first. Then a fix: commit that flips known_bug to preserve and sets current == intended.

The validator enforces that second commit is not fudged. It has to be internally consistent.

Where free model access and a free server fit

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I use two operator-supplied capabilities here, and nothing more. MonkeyCode offers free model access and a free server option.

Drafting ledger rows: paste the legacy function and one real call site, then ask for candidate case ids and intended values. Treat every row as a proposal, never as a baseline.

The validator only catches contradictions. It cannot catch a confidently wrong rounding rule, so a human signs off on each intended value.

The free server option solves a narrower problem. Capture and replay must import the same code twice.

Local state breaks that: stale .pyc files, PYTHONPATH leftovers, a stray .env. Running both passes in a clean environment removes the ambiguity about what you measured.

That is the entire role. If you strip the product from this post, the ledger and tripwire still work.

Limitations and who should skip this

This works best on pure functions with serializable inputs and outputs. Money math, parsing, and formatting qualify.

It struggles with time, randomness, sockets, and concurrency. Pin a clock or inject a seed first, or the baselines are noise.

Twenty cases is usually enough. A ledger with 300 rows is a second test suite nobody maintains.

Skip this approach if:

• the seam needs six collaborators to construct;
• the team does not enforce commit trailers already;
• the "bug" is contested and nobody can name the intended value;
• you want a tool to decide intent for you. It will not.

One more caveat. The ledger pins what you thought to capture. Missing input classes stay missing.

Treat coverage as a separate problem from preservation. This gate protects known behavior, not unknown behavior.

Keep the ledger file next to the code it protects. When a future move trips the tripwire, the failure message is the artifact worth sharing with your team.