The Refactor Gate Is a Runtime Return-Shape Ledger

A messy module is not ready for a refactor. Freeze every public return shape before any edit. Then change one callable and re-run the ledger.

Why shapes, not payloads

Shape means keys, types, nullability, and exception class names. Exact strings and numeric payloads stay out of the ledger. The oracle fails only when the observable contract moves.

Value snapshots rot on timestamps, request ids, and locale text. Those flakes block refactors that should remain safe. A shape oracle ignores literals and still catches dropped fields.

This workflow targets in-process public functions and nothing else. It does not replace HTTP checks or command tests. Keep those oracles on the seams they already cover.

Why this order

Most failed refactors change structure rather than arithmetic. A renamed key ships as a silent production break. A new None branch often ships as a new exception.

Characterization tests should lock that public surface first. Implementation cleanup comes after the lock stays green. One callable per patch keeps the blame set small.

Artifact: a JSONL return-shape ledger

The harness below is a proposed local tool. It is not a measured production field report. Treat the sample module as an illustrative fixture only.

Each ledger line stores one fixture call site. The shape walker stays deterministic and free of time. Fixture arguments must not call datetime or uuid helpers.

Record schema

{
  "callable": "messy_billing.quote_line",
  "fixture_id": "tax_exempt_member",
  "ok": true,
  "exc_type": null,
  "shape": {
    "kind": "dict",
    "keys": {
      "sku": {"kind": "str"},
      "cents": {"kind": "int"},
      "tax": {"kind": "none"}
    }
  }
}

Illustrative module under test

# messy_billing.py — illustrative only, not a field case study

def quote_line(item):
    if "sku" not in item:
        raise KeyError("sku")
    qty = item.get("qty", 1)
    if not isinstance(qty, int):
        raise TypeError("qty")
    cents = qty * 199
    tax = None if item.get("tax_exempt") else round(cents * 0.08)
    return {"sku": item["sku"], "cents": cents, "tax": tax}

Fixture file

[
  {
    "callable": "messy_billing.quote_line",
    "fixture_id": "tax_exempt_member",
    "args": [{"sku": "SKU-1", "qty": 2, "tax_exempt": true}],
    "kwargs": {}
  },
  {
    "callable": "messy_billing.quote_line",
    "fixture_id": "missing_sku",
    "args": [{"qty": 1}],
    "kwargs": {}
  },
  {
    "callable": "messy_billing.quote_line",
    "fixture_id": "qty_type_mismatch",
    "args": [{"sku": "SKU-1", "qty": "2"}],
    "kwargs": {}
  }
]

Harness

# shape_ledger.py — proposed local oracle, unexecuted until you run it
from __future__ import annotations

import importlib
import json
import sys
from pathlib import Path
from typing import Any

LEDGER = Path("ledgers/return_shapes.jsonl")
FIXTURES = Path("fixtures/calls.json")
FOCUS = ("ok", "exc_type", "shape")


def shape_of(value: Any) -> dict[str, Any]:
    if value is None:
        return {"kind": "none"}
    if isinstance(value, bool):
        return {"kind": "bool"}
    if isinstance(value, int):
        return {"kind": "int"}
    if isinstance(value, float):
        return {"kind": "float"}
    if isinstance(value, str):
        return {"kind": "str"}
    if isinstance(value, bytes):
        return {"kind": "bytes", "len_bucket": len(value).bit_length()}
    if isinstance(value, list):
        if not value:
            return {"kind": "list", "empty": True}
        first = shape_of(value[0])
        return {
            "kind": "list",
            "empty": False,
            "len_bucket": len(value).bit_length(),
            "item": first,
            "uniform": all(shape_of(v) == first for v in value),
        }
    if isinstance(value, dict):
        items = sorted(value.items(), key=lambda kv: str(kv[0]))
        return {
            "kind": "dict",
            "keys": {str(k): shape_of(v) for k, v in items},
        }
    return {"kind": "other", "type": type(value).__name__}


def invoke(dotted: str, args: list[Any], kwargs: dict[str, Any]) -> dict[str, Any]:
    mod_name, func_name = dotted.rsplit(".", 1)
    func = getattr(importlib.import_module(mod_name), func_name)
    try:
        result = func(*args, **kwargs)
        return {"ok": True, "exc_type": None, "shape": shape_of(result)}
    except Exception as exc:
        return {
            "ok": False,
            "exc_type": type(exc).__name__,
            "shape": {
                "kind": "exc",
                "msg_len_bucket": len(str(exc)).bit_length(),
            },
        }


def load_fixtures() -> list[dict[str, Any]]:
    data = json.loads(FIXTURES.read_text(encoding="utf-8"))
    if not isinstance(data, list):
        raise SystemExit("fixtures/calls.json must be a JSON array")
    return data


def keyset(shape: dict[str, Any]) -> set[str]:
    if shape.get("kind") != "dict":
        return set()
    return set(shape.get("keys", {}))


def classify_drift(expected: dict[str, Any], got: dict[str, Any]) -> str:
    if expected["ok"] != got["ok"]:
        return "ok_flip"
    if expected["exc_type"] != got["exc_type"]:
        return "exc_type_change"
    if expected["shape"] == got["shape"]:
        return "none"
    removed = keyset(expected["shape"]) - keyset(got["shape"])
    added = keyset(got["shape"]) - keyset(expected["shape"])
    if removed:
        return "key_removed"
    if added:
        return "key_added"
    exp_list = expected["shape"] if expected["shape"].get("kind") == "list" else {}
    got_list = got["shape"] if got["shape"].get("kind") == "list" else {}
    if exp_list.get("uniform") != got_list.get("uniform"):
        return "list_uniformity"
    return "kind_change"


def record() -> None:
    LEDGER.parent.mkdir(parents=True, exist_ok=True)
    lines = []
    for fx in load_fixtures():
        observed = invoke(fx["callable"], fx.get("args", []), fx.get("kwargs", {}))
        row = {
            "callable": fx["callable"],
            "fixture_id": fx["fixture_id"],
            **observed,
        }
        lines.append(json.dumps(row, sort_keys=True))
    LEDGER.write_text("\n".join(lines) + "\n", encoding="utf-8")


def check() -> int:
    if not LEDGER.exists():
        print("MISSING_LEDGER")
        return 1
    expected_rows = [
        json.loads(line)
        for line in LEDGER.read_text(encoding="utf-8").splitlines()
        if line.strip()
    ]
    by_id = {(row["callable"], row["fixture_id"]): row for row in expected_rows}
    failures = 0
    for fx in load_fixtures():
        key = (fx["callable"], fx["fixture_id"])
        if key not in by_id:
            print(f"MISSING_ROW {key}")
            failures += 1
            continue
        observed = invoke(fx["callable"], fx.get("args", []), fx.get("kwargs", {}))
        prior = {k: by_id[key][k] for k in FOCUS}
        got = {k: observed[k] for k in FOCUS}
        if prior != got:
            failures += 1
            label = classify_drift(prior, got)
            print(f"SHAPE_DRIFT {label} {key}")
            print(json.dumps({"expected": prior, "got": got}, indent=2, sort_keys=True))
    return failures


if __name__ == "__main__":
    cmd = sys.argv[1] if len(sys.argv) > 1 else "check"
    if cmd == "record":
        record()
        raise SystemExit(0)
    raise SystemExit(check())

Save the harness as shape_ledger.py beside your fixtures. Save the sample module only if you lack a real target.

Step 1 — Inventory public callables

List every function imported by other packages first. Skip helpers whose names start with an underscore. Write dotted names into fixtures/calls.json before recording.

python -c "import messy_billing, inspect; print([n for n,o in inspect.getmembers(messy_billing, inspect.isfunction) if not n.startswith('_')])"

Do not start inside nested closures or local lambdas. Public importers define the contract that callers already rely on. File length is a weak proxy for refactor risk here.

Step 2 — Build fixtures for return and error paths

Each fixture needs a stable fixture_id string. Cover success, missing key, and type mismatch first. Those three paths catch most silent shape shifts early.

Keep arguments free of current time and random bytes. Keep generated identities out of structures under test. If identity leaks, wrap that boundary before you record.

Step 3 — Record the ledger on known-good behavior

mkdir -p ledgers fixtures
python shape_ledger.py record
git add ledgers/return_shapes.jsonl fixtures/calls.json shape_ledger.py
git commit -m "chore: freeze runtime return shapes for messy_billing"

Record only from a known-good revision of the module. Do not record shapes from a dirty working tree. A tainted ledger would encode the current bug as law.

Step 4 — Fail closed in check mode

python shape_ledger.py check
echo $?

A non-zero process exit means return structure moved. Read the SHAPE_DRIFT block before you write a patch. Update the ledger only with an explicit product decision.

Step 5 — Change one callable, then stop

Edit one function body from the inventoried public list. Do not rename exports inside that same patch. Do not move modules inside that same patch either.

Re-run check immediately after the single-function edit. A green ledger means the observable contract still holds. A red ledger means revert the edit or split work.

python shape_ledger.py check && git add messy_billing.py && git commit -m "refactor: simplify quote_line arithmetic only"

Step 6 — Optional model draft against the gate

A coding model can propose that single-function edit. Feed it fixtures/calls.json and the ledger file path. Keep the harness as the merge gate, not the model.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. Use those only after the ledger exists and check mode is green.

The model output remains a candidate diff, nothing more. The ledger decides whether that candidate can ship. Discard any patch that also touches a second callable.

Drift classifier

Classify each failure before you debate the refactor. The table below maps observation to a concrete action. Do not treat every drift as a license to rewrite goldens.

Observation	Meaning	Action
ok_flip	Success became an exception, or reverse	Revert the callable or split the patch
exc_type_change	Handler contract moved	Keep the old class or version it
key_removed	Attribute access on callers can break	Treat as breaking and do not ship
key_added	Extra field appeared	Ship only if callers ignore unknowns
kind_change	Field type moved	Revert or add an adapter layer
list_uniformity	Mixed element shapes appeared	Inspect None inserts and hidden unions
MISSING_ROW	Fixture has no frozen record	Record on known-good code, then continue

Failure analysis notes

Key removal is usually a breaking caller defect. Key addition may be compatible if callers ignore extras. Kind changes are breaks even when tests still pass elsewhere.

An ok flip from true to false is a new exception path. An exception class rename is a break for handlers. List uniformity flips often hide mixed None insertions.

Length buckets are coarse on purpose for stability. Lists of length three and four can share one bucket. Value regressions can hide inside an unchanged shape tree.

Limitations

The walker stringifies dict keys before the comparison step. Non-string keys may collide after that string conversion. Custom objects collapse to a type name and nothing else.

Exception records store class name and message length bucket. Exact message text is not part of this oracle. Two different defects can share one exception shape.

The harness imports the live module on every check. Import-time side effects still run during that import. Isolate import writes before the first ledger record.

Who should not use this

Do not use this while redesigning a public wire format. Shape drift is the intended result of that redesign. Versioned adapters belong in that project instead of this ledger.

Do not use this on functions that have no fixtures. An empty ledger encodes no behavior at all. Check mode would stay green through accidental deletion.

Do not use this ledger as a performance suite. Length buckets are not timings or query counts. Extra I/O will not appear in the shape tree.

Teams without review time should not auto-accept ledger rewrites. Rewriting recorded shapes is a public contract change. It needs the same review as an intentional API bump.

Local counts only

Count fixtures, callables, and drift failures in your tree. Do not publish those counts as industry-wide proof. They describe one repository and one recording session.

wc -l ledgers/return_shapes.jsonl fixtures/calls.json
python shape_ledger.py check

A useful local rule is one callable per commit. Larger batches hide the failing shape inside noise. Blame then points at the whole mixed patch.

Closing

Map the return surface before you touch the mess. Freeze shapes rather than full payloads or guesses. Change one callable, then let the ledger accept or reject.

Copy the harness onto one hot module this week. Stop after the first green single-function extract.