A Hashed Output Corpus Makes a Transformer Refactor Auditable

A checkout team inherited a four-hundred-line transform_row helper that still mutated its input dictionary in place. The helper mixed tax rounding, currency labels, coupon stacking, and CSV escaping without a single assertion. An agent then proposed a cleanup that renamed locals, extracted helpers, and quietly changed discounted-tax rounding. Reviewers almost merged the tidy diff because the file looked smaller, not because behavior had been pinned.

That near-miss is the usual shape of a messy-repo refactor in 2026. Coding agents now emit fluent extracts on demand, and public threads argue they already outwrite most humans. Fluency still does not tell a reviewer whether the transformer emits the same bytes. This article treats the agent as an optional proposer and treats a hashed output corpus as the actual gate.

Why a tidy extract is not evidence

A transformer is a bad target for unconstrained rewrite because its contract lives in combined output, not in helper names. Rounding mode, blank-versus-missing keys, stable key order, and timezone defaults rarely show up in a linter. An extract that “only moves code” can still change 1.005 versus 1.00, or swap "" for a dropped field.

Human characterization tests help, but they usually encode the cases a person remembered. Batch files contain the cases the team actually ships. A hashed corpus records what the current mess already produces for a frozen sample set. The corpus is not a claim of correctness. It is a freeze of present behavior, including the ugly parts you are not ready to change.

Use this loop when all of the following are true:

• The module is a pure-ish function from fixtures to bytes, JSON, or CSV.
• You can collect a few dozen real-looking inputs without production secrets.
• You want one structural change, not a redesign of rates or tax law.
• Reviewers will reject the change unless old outputs replay bit-for-bit.

Skip the loop when the transformer talks to live clocks, networks, or unseeded randomness that you cannot seam. Skip it when the business already wants different numbers, because a golden master will fight a deliberate behavior change.

A freeze-then-extract loop

The working rule is narrow. Mine fixtures first, pin output hashes, allow one extract, then replay the same corpus. Anything larger is a second change and needs a second pin.

1. Inventory samples, not opinions

Collect files the batch already understands: truncated production-like rows, empty carts, stacked coupons, and one hostile Unicode name. Store them as fixtures, not as comments in a chat transcript. Label every file with a stable identifier so later diffs name the failing sample instead of a line number inside a blob.

fixtures/transform/
  001_empty_cart.json
  002_coupon_stack.json
  003_half_up_tax.json
  004_unicode_name.json
  005_missing_currency.json

Do not redact by rewriting numbers by hand if you can strip secrets with a dedicated scrubber. Hand-edited fixtures silently become a second, undocumented transformer.

2. Render through today’s mess and hash the bytes

Call the current function once per fixture and hash the exact bytes it already emits. Prefer canonical JSON or explicit CSV settings so key order and newline policy are part of the pin. The script below is an illustrative harness, not a measured production run.

# illustrative harness — unexecuted in this article
from __future__ import annotations

import hashlib
import json
from pathlib import Path
from transform_row import transform_row  # the messy module under freeze

FIXTURES = Path("fixtures/transform")
LOCK = Path("corpus.lock.json")

def render(path: Path) -> bytes:
    payload = json.loads(path.read_text(encoding="utf-8"))
    result = transform_row(payload["row"], payload["rates"], payload["flags"])
    text = json.dumps(result, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
    return (text + "\n").encode("utf-8")

def digest(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()

def freeze() -> None:
    entries = []
    for path in sorted(FIXTURES.glob("*.json")):
        blob = render(path)
        entries.append({"id": path.name, "sha256": digest(blob), "nbytes": len(blob)})
    LOCK.write_text(json.dumps({"entries": entries}, indent=2) + "\n", encoding="utf-8")

def replay() -> int:
    lock = json.loads(LOCK.read_text(encoding="utf-8"))
    failures = []
    for entry in lock["entries"]:
        path = FIXTURES / entry["id"]
        actual = digest(render(path))
        if actual != entry["sha256"]:
            failures.append(entry["id"])
            print(f"MISMATCH {entry['id']}: {actual} != {entry['sha256']}")
    print(f"replayed={len(lock['entries'])} mismatches={len(failures)}")
    return 1 if failures else 0

if __name__ == "__main__":
    import sys
    sys.exit(freeze() or 0) if sys.argv[1:] == ["freeze"] else sys.exit(replay())

Run the freeze only against the unpatched tree. After that, treat corpus.lock.json as reviewable source, the same way you treat a lockfile for dependencies.

python harness.py freeze
git add fixtures/transform corpus.lock.json harness.py
git commit -m "Pin transformer golden-master corpus before extract"

3. Name the smallest safe change

Write the intended extract in one sentence that names a seam and forbids extra work. Example: extract _format_money(amount, currency) and keep rounding inside transform_row. If the sentence needs an “and”, you are proposing two changes. Two changes need two corpus pins or an explicit behavior waiver.

A useful seam is a function that turns already-computed values into bytes or labels. A dangerous seam is a function that reorders discounts, talks to today’s date, or reads environment variables. Keep I/O and policy on the original side of the cut until the corpus stays green.

4. Replay before you argue about style

After the extract, replay the lockfile on the same fixtures. A green replay means the public bytes did not move. A red replay means the extract was not an extract. Do not “fix” a mismatch by rewriting the lockfile unless the mismatch is the product change you meant to ship, and that change has its own review.

python harness.py replay
# expected: replayed=5 mismatches=0

If a single fixture fails, print both JSON documents with sorted keys and diff them as text. Hash mismatches without a document diff train people to shrug and re-freeze. The corpus only works when a mismatch is cheaper to inspect than to ignore.

Decision table for the extract itself

Use the table as a review checklist, not as a scoring rubric. Any “no” in the reject column stops the patch, even when the agent’s names look nicer than yours.

Question	Accept the extract	Reject and restore
Does corpus.lock.json still match every fixture?	Yes, bit-for-bit	Any hash moved
Did public function signatures stay in place?	Yes	Callers now need adapters
Did rounding, sorting, or defaulting stay in the old function?	Yes	Policy moved with the helper
Is there a single new function or module?	One seam	Drive-by renames elsewhere
Are tests, fixtures, and lockfile in the same patch?	Yes	“Cleanup” without replay
Did clocks, locales, or network calls gain a new read?	No new reads	Hidden non-determinism

The table is deliberately boring. Boring gates are the point, because agent diffs optimize for local readability, not for byte stability.

An illustrative extract that should survive replay

The original function below is labeled illustrative. It is a compressed stand-in for the usual mess: mutation, string building, and rounding in one place.

# illustrative messy baseline — not production code
from decimal import Decimal, ROUND_HALF_UP

def transform_row(row, rates, flags):
    row = dict(row)  # later agents often drop this copy
    subtotal = Decimal(str(row.get("subtotal") or 0))
    rate = Decimal(str(rates.get(row.get("region"), "0")))
    if flags.get("coupon"):
        subtotal -= Decimal(str(flags["coupon"]))
    tax = (subtotal * rate).quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)
    total = subtotal + tax
    currency = row.get("currency") or "USD"
    row["tax_label"] = f"{currency} {tax}"
    row["total_label"] = f"{currency} {total}"
    row["tax"] = str(tax)
    row["total"] = str(total)
    return row

The smallest safe change moves only label formatting. Rounding stays put, because rounding is policy, not presentation.

# illustrative extract — still unexecuted here
def _format_money(currency, amount):
    return f"{currency} {amount}"

def transform_row(row, rates, flags):
    row = dict(row)
    subtotal = Decimal(str(row.get("subtotal") or 0))
    rate = Decimal(str(rates.get(row.get("region"), "0")))
    if flags.get("coupon"):
        subtotal -= Decimal(str(flags["coupon"]))
    tax = (subtotal * rate).quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)
    total = subtotal + tax
    currency = row.get("currency") or "USD"
    row["tax_label"] = _format_money(currency, tax)
    row["total_label"] = _format_money(currency, total)
    row["tax"] = str(tax)
    row["total"] = str(total)
    return row

If replay fails after this cut, the extract is lying about being presentational. Common liars are str(Decimal) format changes, implicit float, and dropping the defensive dict(row) copy. Restore the tree, tighten the sentence that named the seam, and try a smaller cut.

Where a constrained agent runtime belongs

The corpus and the replay belong on your machine or in CI, next to the fixtures. The agent, if you use one at all, belongs after corpus.lock.json exists, and it should receive the one-sentence seam as a constraint, not a vague “please clean this up”. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode’s free model access and free server option are relevant only for that second half of the loop: proposing the extract inside a disposable runtime while the golden master stays under your version control. They do not replace the lockfile, and they do not prove the extract is correct. If you try that split, keep secrets out of the fixtures and treat the remote session as untrusted compute, the same way you would treat any other shared box.

A practical prompt to paste after the freeze looks like the block below. It is a constraint list, not a request for a rewrite of tax policy.

Propose one extract in transform_row.py.
Do not change rounding, defaults, key names, or return shapes.
Do not edit fixtures/ or corpus.lock.json.
After the edit, I will run: python harness.py replay
If you cannot keep the replay green, return no diff.

Reject patches that expand the prompt into extra files, extra helpers, or “while we are here” import sorting across unrelated modules. Those extras are how byte-stable extracts become unreviewable novellas.

Limitations, cost, and who should not bother

Golden masters freeze bugs as faithfully as they freeze intended behavior. If 003_half_up_tax.json already encodes a rounding error, the corpus will punish the fix until you replace that fixture in a dedicated behavior-change commit. That is a feature when you wanted a structural extract. It is a hazard when the team thought it was fixing math.

The method also fails closed on non-determinism. Unsorted dicts on older runtimes, local timezone stamps, and iterating set objects will thrash hashes. If you cannot seam those reads, you do not have a transformer you can golden-master yet. Build a clock or locale seam first, then freeze, then extract.

Do not use this approach for:

• Cryptographic or financial code that needs proof, not historical bytes.
• Modules whose output is streaming UI, not a finite document.
• Refactors whose whole point is changing rates, copy, or legal rounding.
• Samples you cannot store without customer secrets or credentials.

There is no benchmark in this article because the claim is not speed. The claim is auditability: a reviewer can say what stayed the same. If you cannot name the fixtures, you are still negotiating with an agent’s confidence instead of with a lockfile.

The next time a fluent diff offers to rescue transform_row, pin the bytes it already prints. Extract one presentation seam. Replay the corpus. Only then discuss whether the names are nicer.