A tangled pricing function still mixes tax rules, discounts, and rounding in one seven-hundred-line Python module. An agent then opens a pull request that rewrites helpers, renames locals, and claims the cleanup is behavior-preserving. The existing tests remain green because they only assert a final integer total for two happy-path invoices. This walkthrough freezes one entry point as a contract tape, then allows only the smallest extract that keeps that tape identical.
The method is intentionally narrow. It does not certify the whole service, and it does not bless a large rewrite just because unit tests still pass.
Why green tests still miss the extract
Messy modules usually have tests that pin outcomes, not contracts. An agent can change control flow, drop a rare branch, or replace None with {} while those outcome tests stay green. Reviewers then debate naming while the silent behavior change hides in a helper that used to skip missing keys. A useful freeze therefore records more than the final number.
Record four things for a single entry point, not for the entire package:
- argument type trees and sorted mapping keys
- return type trees, including
Noneversus empty containers - exception class names on known failure inputs
- a short digest of the canonical JSON result
That combination is a contract tape. It is cheaper than a full-suite dump and stricter than one assertion on a total. The tape is the gate; the extract is allowed only after the gate is red-green on the current tree.
The lab fixture
The example below is a proposed fixture, not a production service. Treat every snippet as unexecuted sample code for this walkthrough.
# messy_pricing.py — proposed lab fixture
from decimal import Decimal, ROUND_HALF_UP
def price_invoice(payload):
items = payload.get("items") or []
subtotal = Decimal("0")
for item in items:
qty = Decimal(str(item.get("qty") or 0))
unit = Decimal(str(item.get("unit") or 0))
subtotal += qty * unit
discount = Decimal(str(payload.get("discount") or 0))
if payload.get("kind") == "wholesale" and subtotal > 100:
discount += Decimal("5")
taxable = subtotal - discount
if taxable < 0:
taxable = Decimal("0")
rate = Decimal("0.08") if payload.get("region") == "west" else Decimal("0.06")
tax = (taxable * rate).quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)
total = (taxable + tax).quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)
return {
"subtotal": str(subtotal),
"discount": str(discount),
"tax": str(tax),
"total": str(total),
"flags": payload.get("flags") or {},
}
The function looks small, yet it mixes defaults, regional tax, wholesale extras, and stringified decimals. An agent can split it into several helpers and still keep total stable on the two cases a sparse test file already covers. The missing risk is a changed flags default, a dropped wholesale bonus, or a different empty-items path.
Build the contract tape
The recorder walks JSON-like values, stores a shape tree, and stores a short digest. Run it against one function only. Do not start by taping every private helper, because that freeze would block the extract you actually want.
# contract_tape.py — proposed walkthrough code
from __future__ import annotations
import hashlib
import json
from pathlib import Path
from typing import Any
TAPE_DIR = Path("tapes")
def shape_of(value: Any) -> Any:
if value is None:
return {"kind": "none"}
if isinstance(value, bool):
return {"kind": "bool"}
if isinstance(value, int) and not isinstance(value, bool):
return {"kind": "int"}
if isinstance(value, float):
return {"kind": "float"}
if isinstance(value, str):
return {"kind": "str", "len": len(value)}
if isinstance(value, (list, tuple)):
return {
"kind": "list",
"len": len(value),
"items": [shape_of(v) for v in list(value)[:8]],
}
if isinstance(value, dict):
keys = sorted(value.keys(), key=lambda k: str(k))
return {
"kind": "dict",
"keys": [str(k) for k in keys],
"fields": {str(k): shape_of(value[k]) for k in keys},
}
return {"kind": type(value).__name__}
def canonical(value: Any) -> Any:
if isinstance(value, dict):
return {str(k): canonical(value[k]) for k in sorted(value, key=lambda x: str(x))}
if isinstance(value, (list, tuple)):
return [canonical(v) for v in value]
return value
def digest(value: Any) -> str:
blob = json.dumps(canonical(value), separators=(",", ":"), ensure_ascii=True)
return hashlib.sha256(blob.encode("utf-8")).hexdigest()[:16]
def record_call(fn, payload):
row = {"input_shape": shape_of(payload)}
try:
result = fn(payload)
except Exception as exc:
row["exception"] = type(exc).__name__
row["result_shape"] = None
row["digest"] = None
return row
row["exception"] = None
row["result_shape"] = shape_of(result)
row["digest"] = digest(result)
return row
Seed the tape with cases that miss the happy path, not only the two invoices already in unit tests. Wholesale bonus, missing items, and a discount that drives taxables below zero are the usual silent diffs.
# record_tape.py — proposed walkthrough code
import json
import sys
from pathlib import Path
from contract_tape import TAPE_DIR, record_call
from messy_pricing import price_invoice
CASES = [
{
"name": "retail_east_empty_flags",
"payload": {
"items": [{"qty": 2, "unit": "10.00"}],
"discount": "1.00",
"region": "east",
},
},
{
"name": "wholesale_west_bonus",
"payload": {
"items": [{"qty": 12, "unit": "9.50"}],
"kind": "wholesale",
"region": "west",
"flags": {"rush": True},
},
},
{
"name": "negative_after_discount",
"payload": {"items": [{"qty": 1, "unit": "3"}], "discount": "9.00"},
},
{
"name": "missing_items",
"payload": {"region": "west"},
},
]
def build_tape():
return {
"entry": "price_invoice",
"cases": [
{"name": case["name"], **record_call(price_invoice, case["payload"])}
for case in CASES
],
}
def main(mode: str) -> int:
TAPE_DIR.mkdir(exist_ok=True)
path = TAPE_DIR / "price_invoice.json"
fresh = build_tape()
if mode == "--write":
path.write_text(json.dumps(fresh, indent=2, sort_keys=True) + "\n")
print(f"wrote {path}")
return 0
if not path.exists():
print("missing tape; run with --write first", file=sys.stderr)
return 2
pinned = json.loads(path.read_text())
if pinned != fresh:
print("contract tape drift")
print(json.dumps({"pinned": pinned, "fresh": fresh}, indent=2))
return 1
print("contract tape matched")
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1] if len(sys.argv) > 1 else "--check"))
Commands stay boring on purpose. Write once from the known-messy tree, then check after every extract. If the check is not in CI yet, run it locally before you even open the diff.
python record_tape.py --write
python record_tape.py --check
git add tapes/price_invoice.json contract_tape.py record_tape.py
git commit -m "Pin price_invoice contract tape before extract"
A matched tape means the entry point still accepts the same shapes and still emits the same canonical result. It does not mean the internals are pretty, and it does not mean every caller is covered.
Allow only the smallest safe change
After the tape is pinned, the next move is one extract, not a module rewrite. The candidate in this fixture is the discount block, because it is a closed rule with one extra wholesale branch. Keep price_invoice as the public entry so callers do not move in the same commit.
def apply_discount(subtotal, payload):
discount = Decimal(str(payload.get("discount") or 0))
if payload.get("kind") == "wholesale" and subtotal > 100:
discount += Decimal("5")
return discount
That is the whole change budget for the first patch. Do not rename flags, do not switch Decimal to float, and do not introduce a pricing class in the same diff. If an agent returns a four-file cleanup, reject it before reading the prose in the pull request.
Use a diff gate so the budget is mechanical. The script below is proposed local tooling, not a required platform hook.
# extract_gate.py — proposed walkthrough code
import subprocess
import sys
MAX_FILES = 2
MAX_NET_LINES = 40
def main() -> int:
raw = subprocess.check_output(
["git", "diff", "--numstat", "HEAD"],
text=True,
).strip()
if not raw:
print("no unstaged diff against HEAD")
return 0
files = 0
net = 0
for line in raw.splitlines():
added, deleted, path = line.split("\t", 2)
if path.startswith("tapes/"):
continue
files += 1
if added != "-" and deleted != "-":
net += abs(int(added) - int(deleted)) + min(int(added), int(deleted))
if files > MAX_FILES or net > MAX_NET_LINES:
print(f"extract budget exceeded: files={files} net_lines~={net}")
return 1
print(f"extract budget ok: files={files} net_lines~={net}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
Accept or reject the agent patch
| Observation | Decision | Next step |
|---|---|---|
| Tape matches and the diff touches one helper plus the entry function | Accept | Commit, then pick the next closed block |
| Tape matches but three unrelated files moved | Reject | Ask for a single-function extract |
Tape drifts on missing_items only |
Reject | Restore the empty-items default before any rename |
| Tape drifts on digest but not on shapes | Reject | A value changed; do not treat it as a style cleanup |
| Tests pass while the tape is missing | Reject | The suite is too coarse to review an agent rewrite |
The table is the review script. It keeps the discussion on contracts and budgets instead of on whether the generated names look tidy.
Where a disposable coding environment fits
Once the tape and the gate exist, an agent is useful only as a proposer of the next one-function extract. It should not be the source of truth for behavior. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can host that record-and-check loop when you want a scratch machine, without turning the tape into a marketing demo.
Keep the workflow local-first either way:
- Write the tape from the messy tree.
- Ask for one extract that preserves
price_invoice. - Run
python record_tape.py --checkandpython extract_gate.py. - Reject any patch that fails either command, even when unit tests pass.
The product mention is optional. The tape still works if you run the same commands on a laptop and ignore every coding agent.
Limitations, and who should skip this
The tape hashes canonical JSON of return values, so unordered sets, timestamps, and randomly allocated identifiers will thrash the digest. Do not use this recorder on those outputs without first stripping volatile fields. Shape trees also stop at eight list items, which is enough for invoice lines in this fixture and too weak for bulk imports.
Skip the method when the entry point is a long-lived process, a GUI loop, or a network client with live clocks. Skip it when you do not own the module, because pinning a tape is still a behavior freeze and can conflict with an active feature branch. Skip it when the real bug is numeric policy, such as rounding mode, because a digest match can still hide a business change if your cases never hit that branch.
The approach also fails closed on purpose. A missing tape is a failed gate, not a reason to trust a large agent rewrite. If the team cannot name four cases that miss the happy path, the extract is not the current problem; the missing cases are.
What this walkthrough actually settles
A messy module becomes safer to touch when one entry point has a replayable contract, not when an agent restyles the file. The smallest safe change is then a single closed helper, reviewed against a tape and a diff budget. If those two checks pass, you earned the next extract. If they fail, the rewrite was a story about cleanliness, not a proof about behavior.
Top comments (0)