Record Observed Outputs Before One Safe Refactor

A messy function should not be rewritten on day one. Lock today's outputs with characterization tests before any edit. The smallest safe change is the edit those tests still accept.

This workflow is a proposal with unexecuted examples. It does not report a measured production run. Use the workflow to separate observation from cleanup.

Why a wide rewrite fails review

A large rewrite hides several behavior changes in one diff. Reviewers then cannot name the line that moved an output. A one-branch patch keeps that cause easy to see.

Characterization tests record current behavior, including awkward results. They do not prove the behavior is right for users. They only block accidental output changes during a refactor.

Michael Feathers called this kind of lock a characterization test. The term comes from his legacy-code testing work. This article does not quote or copy that book.

What the ledger stores

Store inputs the code already receives in real calls. Store the output the current function returns for each input. Store error type and message when the function raises.

Do not store the output you wish the function returned. That wish belongs in a later specification test. Mixing wishes into the ledger hides the refactor risk.

Do not wait for a fixed row quota before the edit. Cover the branches the planned patch can touch. Extra rows can wait for the next small patch.

Step 1. Name one target function

Pick one function with several branches and unclear outputs. Leave the rest of the module outside this patch. A narrow target keeps the ledger small enough to read.

Write the function name, file path, and caller list. Stop if you cannot name a single caller. Unknown callers make a supposedly safe edit unsafe.

Step 2. Collect inputs that already exist

Pull inputs from fixtures, logs, and old bug notes. Add a boundary only when current code already branches on it. Do not invent new product rules during collection.

Give each row a stable id and a source note. The source note shows where the input came from. Reviewers can reject rows that lack a source.

Step 3. Record the observed outputs

Run the current function once for every collected row. Save the return value next to the input fields. Save the exception class name if the call fails.

Sort mapping keys before comparison when order is not contractual. Keep list order when callers depend on that order. Write that choice into the test, not into a comment only.

Step 4. Freeze the replay test

Commit the ledger file and the replay test together. Fail the test when any stored field changes. Ignore fields that no current caller actually reads.

Run the test on the unchanged code before you edit. A red result means the ledger is wrong, not the code. Fix the ledger until the unchanged code is green.

The first green run is the baseline, not a refactor result. Save the command output beside the ledger commit. A later red run is then compared against that baseline.

Step 5. Apply the smallest safe change

Change one branch, one extraction, or one local name. Leave every other branch byte-for-byte in that patch. Rerun the full ledger before you request review.

If the ledger goes red, revert that edit. Do not fix forward by editing the stored outputs. A red ledger means the change was not behavior-preserving.

Step 6. Reject a wide edit

Treat this walkthrough as hypothetical, not as a recorded run. A wide edit changed void handling and the hold check together. The ledger would fail on two rows, so the cause would be unclear.

Reverting both lines and extracting only the hold check would stay green. That split is the smallest safe change in this example. The void branch stays untouched until a later patch.

Decision table

Use this table before you touch product code. The middle column is the only edit allowed. The right column is a change to reject in this patch.

Signal	Allowed edit	Reject in this patch
Replay fails on unchanged code	Fix the ledger or harness	Product logic edits
One branch is copied twice	Extract that branch only	File-wide restyle
Key order is not contractual	Sort at the return edge	Parser rewrite
Desired output is still unknown	Keep the ledger and stop	Cleanup of live branches
Caller contract is already written	Schedule a second patch	Mixed contract and cleanup
Formatter touched unrelated lines	Restore those lines	Style mixed with extraction

Proposal code

The samples below are proposals, not captured production results. Run them in a scratch repository before you trust them. Replace the sample rows with inputs from your own module.

The example uses pytest because the replay is a plain assertion. Any runner that fails on mismatch can host the same ledger.

# Proposal. Unexecuted until you run it in a scratch repo.
import json
from pathlib import Path

LEDGER = Path("fixtures/order_status_ledger.json")

def load_rows():
    return json.loads(LEDGER.read_text(encoding="utf-8"))

def observe(status_for, row):
    try:
        value = status_for(row["order"], row["flags"])
        return {"observed": value, "error": None}
    except Exception as exc:
        return {"observed": None, "error": type(exc).__name__}

def test_status_matches_ledger(status_for):
    for row in load_rows():
        got = observe(status_for, row)
        assert got["observed"] == row["observed"]
        assert got["error"] == row["error"]

[
  {
    "id": "paid-no-hold",
    "source": "sample-seed",
    "order": {"paid": true, "void": false},
    "flags": {"hold": false},
    "observed": "paid",
    "error": null
  },
  {
    "id": "paid-with-hold",
    "source": "sample-seed",
    "order": {"paid": true, "void": false},
    "flags": {"hold": true},
    "observed": "hold",
    "error": null
  },
  {
    "id": "void-wins",
    "source": "sample-seed",
    "order": {"paid": true, "void": true},
    "flags": {"hold": true},
    "observed": "void",
    "error": null
  }
]

# Before: one function, four returns, no helper.
def status_for(order, flags):
    if order.get("void"):
        return "void"
    if flags.get("hold") and order.get("paid"):
        return "hold"
    if order.get("paid"):
        return "paid"
    return "open"

# After: smallest safe change is one extraction.
def hold_blocks(order, flags):
    return bool(flags.get("hold") and order.get("paid"))

def status_for(order, flags):
    if order.get("void"):
        return "void"
    if hold_blocks(order, flags):
        return "hold"
    if order.get("paid"):
        return "paid"
    return "open"

# Proposal writer. Run only on the unchanged function.
# Adjust the import to your module path.
import json
from pathlib import Path

from src.status_for import status_for

SEEDS = Path("fixtures/order_status_seeds.json")
OUT = Path("fixtures/order_status_ledger.json")

def main():
    rows = []
    for seed in json.loads(SEEDS.read_text(encoding="utf-8")):
        item = {
            "id": seed["id"],
            "source": seed["source"],
            "order": seed["order"],
            "flags": seed["flags"],
        }
        try:
            item["observed"] = status_for(seed["order"], seed["flags"])
            item["error"] = None
        except Exception as exc:
            item["observed"] = None
            item["error"] = type(exc).__name__
        rows.append(item)
    OUT.write_text(json.dumps(rows, indent=2) + "\n", encoding="utf-8")

if __name__ == "__main__":
    main()

[
  {
    "id": "paid-no-hold",
    "source": "sample-seed",
    "order": {"paid": true, "void": false},
    "flags": {"hold": false}
  }
]

Run the writer only against the unchanged function. Review the generated file before you commit it. A generated row without a source note should be dropped.

Commands to keep the patch small

python -m pytest tests/test_order_status_ledger.py -q --tb=line
git diff --numstat -- src/status_for.py
git diff -U0 -- src/status_for.py

Count changed product lines before you approve the patch. A behavior-preserving extraction should touch only a few lines. A jump in changed lines means the edit grew past the plan.

Read the numstat added count and deleted count. Stop when both counts exceed the planned extraction. Split the extra edits into a later patch.

Where optional tooling fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. Those two availability claims are the only product facts used here.

No model name, quota, hardware, or duration is assumed. A free model can suggest extra input rows from the function text. You still must run the current function to fill observed outputs.

A model suggestion is not an observed output. A free server can run the same pytest file on a clean checkout. Use it when the laptop is busy or the environment should stay disposable.

The server run does not replace reading the diff. Keep fixtures in the repository, not only on the server. Free access can change, so the ledger must stay portable.

Pin your own Python version in the project file you already trust. Re-run the ledger after any runner or dependency change. A green result on one machine is not a portable proof by itself.

Limitations

The ledger freezes bugs beside the intended results. A product fix must edit the ledger in a separate reviewed change. Otherwise the replay test will block the correction.

Row count is not coverage of every branch. Empty flags, missing keys, and null orders need their own rows. Add those rows only after you observe the current result.

This flow does not measure speed, memory, or load. Do not claim a performance win from a green ledger. Add a separate benchmark if speed is the actual goal.

Who should skip this flow

Skip this flow when the bug is already a known wrong output. Write a failing specification test and fix that output. A characterization test would freeze the bug in place.

Skip it when inputs are not stable enough to record. Time, randomness, and remote calls must be stubbed first. An unstable row will fail for reasons unrelated to the edit.

Skip it when the module has no tests and no runner yet. Build a minimal runner before you extract any helper. Extraction without a replay test is only a move.

Close

Start with one function and a short ledger. Make one behavior-preserving edit, then rerun the replay test. Stop the patch when the ledger turns red.

Do not edit stored facts to match a larger rewrite. Run this ledger on MonkeyCode's free server before you merge.