Record the Ordered Side-Effect Ledger Before You Move One Function

Byte-identical output does not mean identical behavior. A refactor can produce the same final files and the same test pass rate, while silently reordering two writes. Pin the order of side effects, not only the bytes they leave behind.

That is the whole thesis. This post shows a small gate for it.

The failure a diff gate misses

Consider a save-then-audit pipeline. The refactor calls audit first, then save. Final rows are identical. The audit timestamp is now stale by a few milliseconds, and one downstream report drifts.

A golden-file test on the output JSON stays green. A coverage report stays green. Nothing catches it except something that watches the call order.

So record the order. Compare it. Keep it as a file you can read in review.

Step 1 — Census the I/O seams first

Count call sites before writing any test. This number is your scope budget.

rg -n "requests\.(get|post|put)|httpx\.|open\(|sqlite3\.connect|cursor\.execute|boto3\.client|session\.commit" \
  --type py app/ | tee seams.txt

Group the output by module. If one module owns more than a handful of distinct seams, split the work. A ledger for a 40-seam module will be unreadable.

Write down the count. You will compare against it later.

Step 2 — Wrap the seams, skip the magic

Do not monkeypatch the standard library. Route calls through thin adapters you control.

# tests/ledger.py
import hashlib, json, os

LEDGER = []


def _digest(value) -> str:
    blob = json.dumps(value, sort_keys=True, default=str).encode()
    return hashlib.sha256(blob).hexdigest()[:12]


def record(kind: str, target: str, payload) -> None:
    if os.environ.get("LEDGER") != "1":
        return
    LEDGER.append({"kind": kind, "target": target, "payload": _digest(payload)})

The adapter adds one line and stays boring.

# app/io_fs.py
from tests.ledger import record


def write_file(path, data):
    record("fs.write", path, {"len": len(data)})
    with open(path, "w") as fh:
        fh.write(data)

Same shape for HTTP and SQL. Record the target and a payload digest, never the raw body.

Step 3 — Canonicalize before you compare

Raw ledgers are noisy. Apply four rules and no more.

1. Drop wall-clock timestamps entirely.
2. Replace temp directories with a fixed token.
3. Hash payloads above a size threshold.
4. Keep sequence exactly as recorded; never sort.

Rule four is the point of the exercise. Sorting destroys the signal you built the recorder for.

Each ledger line becomes kind|target|digest. One line per event, ordered.

Step 4 — Freeze a golden ledger on current code

Run the pipeline once with LEDGER=1. Write the lines to tests/golden/ledger.txt. Then read that file in code review.

A human reading 30 ordered lines catches wrong-target bugs fast. That review step is worth more than the assertion.

# tests/test_ledger.py
import os
import pytest
from tests.ledger import LEDGER
from app.pipeline import run_pipeline

GOLDEN = open("tests/golden/ledger.txt").read().strip()


def test_ledger_order(tmp_path, monkeypatch):
    monkeypatch.setenv("LEDGER", "1")
    LEDGER.clear()
    run_pipeline(out_dir=tmp_path, fixture="case_07")
    got = "\n".join(f"{e['kind']}|{e['target']}|{e['payload']}" for e in LEDGER)
    assert got == GOLDEN

Use one deterministic fixture. A ledger built on random input is a flake generator.

Step 5 — Prove the gate can fail

An untested assertion is decoration. Break the order on purpose.

git stash list >/dev/null
python - <<'PY'
import pathlib
p = pathlib.Path("app/pipeline.py")
s = p.read_text()
a = "    save(record)\n    audit(record)\n"
b = "    audit(record)\n    save(record)\n"
assert a in s, "pattern not found; edit by hand"
p.write_text(s.replace(a, b))
PY
pytest tests/test_ledger.py -q   # expect FAIL

Revert immediately with git checkout -- app/pipeline.py. If the test stays green, your recorder is not on that seam. Fix the wiring before continuing.

Step 6 — Move one boundary per commit

Now do the actual refactor. Extract one function. Keep every seam call in its original position relative to the others.

Commit the move alone. Run the ledger test. Then, in a separate commit, change behavior if you still need to.

Keep the seam count from Step 1 in the commit message. "12 seams, order unchanged" is a reviewable claim.

Step 7 — Read the ledger diff, not just the red

When the test fails after a move, print both sides.

LEDGER=1 pytest tests/test_ledger.py -q 2>&1 | diff - tests/golden/ledger.txt

Most failures fall into three buckets. Missing events mean a seam got bypassed. Extra events mean a retry loop appeared. Swapped lines mean the move changed ordering.

Each bucket has a different fix. Do not paper over them with a reordered golden file.

When this gate earns its cost

Situation	Use the ledger gate?
Module writes to more than one external system	Yes
Pure functions, no side effects	No, a byte pin is enough
Ledger exceeds roughly 200 events per run	Narrow scope first
Ordering is not observable downstream	Skip it
Async fan-out with no ordering contract	Pin the event set, not the sequence

Two rows matter most. Pure code does not need this. Async code with no ordering guarantee will produce false alarms.

Where a hosted model and server fit

The tedious part is Step 3, not Step 2. Normalization rules need to cover UUIDs, temp paths, and connection strings. A model can draft those rules from a sample ledger, and you approve each one.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free model access is a reasonable fit for drafting normalization candidates, since the task is short and reviewable. The free server option matters when your CI runner cannot reach the repo, which is common on locked-down networks.

Neither replaces the human read of ledger.txt. That read is where the real bug gets caught.

Limitations and who should skip this

A ledger is a proxy for behavior. It is not proof of correctness, and it will not catch a wrong value inside a single event.

Nondeterministic payloads inflate diffs until you suppress them. Slow pipelines make the test painful to run locally. Repos without adapter layers need Step 2 work first, which can be larger than the refactor itself.

Skip this approach if your codebase is a pure-logic library. Skip it if the "messy" part is naming and layout, not I/O order. Skip it if no one will read the golden file during review.

The gate is small. The discipline of reading an ordered event list is the actual product.