Pin the Call Log, Not the Return Value: Choosing the Smallest Safe Refactor Move

Return values can match exactly while effect order silently changes.

That is the refactor that passes CI and corrupts data in production. Golden-file characterization tests compare outputs. They rarely compare the order in which side effects fire. So the smallest safe change is not the fewest lines. It is the change that does not move an effect across a reader.

This post builds one artifact for that: a canonical effect trace, recorded before the move and diffed after.

Why output-only characterization tests miss the dangerous case

Consider a refund path. It writes a refund row, then publishes an event.

A tidy-looking extract flips those two lines. The returned value is identical. The golden file is identical. But now a consumer can read an event for a refund row that is not committed yet.

Output pinning cannot see this. Order pinning can. The difference is one recorded line position.

The same failure shape appears in cache-then-commit, file-then-index, and lock-then-validate refactors.

The artifact: a canonical effect trace

Wrap every effect seam. Record four fields per call: sequence number, seam name, an arguments digest, and a result digest.

Normalize before hashing. Strip addresses, timestamps, and UUIDs. Otherwise the trace is noise.

The harness below is a template. Treat it as unexecuted scaffolding until it runs green on your repo.

# trace_gate.py — template; not yet run against your codebase
import contextlib, functools, hashlib, json, os, re

TRACE_PATH = os.environ.get("TRACE_PATH", "trace.log")
_lines: list[str] = []

_NOISE = [
    (re.compile(r"0x[0-9a-fA-F]{6,}"), "0xADDR"),
    (re.compile(r"\d{4}-\d{2}-\d{2}T[\d:.]+Z?"), "<TS>"),
    (re.compile(r"[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"), "<UUID>"),
]

def _digest(value, limit: int = 200) -> str:
    try:
        blob = json.dumps(value, sort_keys=True, default=repr)
    except TypeError:
        blob = repr(value)
    blob = blob[:limit]
    for pattern, replacement in _NOISE:
        blob = pattern.sub(replacement, blob)
    return hashlib.sha256(blob.encode()).hexdigest()[:12]

def record(seam_name: str):
    def deco(fn):
        @functools.wraps(fn)
        def wrapper(*args, **kwargs):
            result = fn(*args, **kwargs)
            line = f"{len(_lines):04d}|{seam_name}|{_digest([args, kwargs])}|{_digest(result)}"
            _lines.append(line)
            return result
        return wrapper
    return deco

@contextlib.contextmanager
def trace_run(path: str = TRACE_PATH):
    _lines.clear()
    try:
        yield _lines
    finally:
        with open(path, "w", encoding="utf-8") as fh:
            fh.write("\n".join(_lines) + "\n")

One trap matters here. Patch the lookup site, not the definition. If a caller did from app.gateway import charge_card, wrapping app.gateway.charge_card changes nothing.

Step 1 — Enumerate effect seams, not functions

Do not wrap every function. Wrap writes, network calls, clock reads, lock acquisition, and event publication.

Start with a discovery pass over the module you plan to touch.

grep -rEn '\b(open|write_text|unlink|rename|requests\.(post|put|delete)|session\.(add|commit|flush)|publish|emit|send_|acquire|time\.time|datetime\.now)\b' \
  app/refunds --include='*.py' | wc -l

Sort what comes back into three buckets: durable writes, outbound messages, and clock or random reads. Durable writes and outbound messages belong in the trace. Clock and random reads belong in it too, but as frozen values.

If the count is under five, you may not need this gate. Use a plain golden file and move on.

Step 2 — Prove the trace is stable before you trust it

An unstable baseline is not a gate. It is a random number generator with extra steps.

Record the same scenario twenty times on unchanged code. Compare digests.

mkdir -p traces
for i in $(seq 1 20); do
  TRACE_PATH="traces/run-$i.log" pytest -q tests/test_refund_trace.py
done
sha256sum traces/run-*.log | awk '{print $1}' | sort -u | wc -l

One unique digest means the log is deterministic. More than one means you must relax the comparison before proceeding. Do not edit the code to fix this. Edit the comparison.

The relaxation ladder has three rungs:

1. Exact order across all seams. Strongest. Use it whenever it holds.
2. Per-seam multiset. Same calls, order within a seam ignored. Use it when retries legitimately reorder.
3. Phase barriers. Declare named phases and compare only crossings between them. Use it for concurrent workers.

Each rung down buys stability and gives up precision. Say which rung you chose in the test name.

Step 3 — Pick the smallest move with the decision table

Line count is a bad proxy for risk. Effect-order exposure is the real measure.

Planned move	Order risk	Gate required	Revert radius
Rename a local, inline a constant	None	Output golden	1 file
Extract a pure helper (no seams)	None	Output golden	1 file
Wrap one call site in a function	Low	Trace, single seam	1–2 files
Move a write across a read	High	Full trace + 20-run stability	One module
Move a module or package	High	Trace + import-time probe	Whole package

Work down the table. Stop at the first move that unblocks the next one.

Most refactors that feel like one big change are three medium ones. The table tells you which of the three is actually dangerous.

Step 4 — Make one move, then diff the trace

Commit the golden trace before the move. Never commit the move and the golden update together.

pytest -q tests/test_refund_trace.py || git diff --no-index golden/refund.trace traces/run-1.log

Read the diff by position, not by content. A shifted sequence number is the signal.

Three outcomes are common. Same lines, same order: safe move. Same lines, new order: investigate. New or missing lines: the extraction changed call count, which is almost always real.

Step 5 — Decide whether the order change is benign

Some reorderings are harmless. Batching two writes into one transaction is usually fine, if no reader interleaves.

Two questions settle most cases. Can any consumer observe the intermediate state? Is every consumer idempotent against a replay?

Two yeses means you can accept the change and update the golden in a separate commit. One no means revert the move and slice smaller.

Never widen the comparison to make the gate pass. That converts a real bug into a permanent blind spot.

Where a free model and a free server actually fit

The recorder is boring, repetitive work. You need one wrapper per seam, plus a digest normalizer per argument shape. That is mechanical pattern-matching, not design.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The account operator states that MonkeyCode offers free model access and a free server option. If that holds for you, the wrapper boilerplate and the normalizer regexes are a reasonable thing to draft there. The judgment calls — which seams to pin, which order changes are benign — stay with you. Model-drafted wrappers need review, because a wrapper around the wrong lookup site silently records nothing.

The twenty repeat runs are the other half. They are cheap per run but tedious to babysit. A free server option can host that loop and keep it off a laptop. Use a throwaway key, and check the terms before sending proprietary source anywhere. If your repo is private or regulated, skip the hosted option and run the loop in your own CI.

Limitations and who should not use this

Trace pinning is over-strict by design. It forbids reorderings that are technically safe. That cost is real, and you pay it in review time.

Normalizers rot. Every new argument shape adds a rule. Budget for it, or the digests churn and the gate gets disabled.

Instrumentation changes timing. A recorder on a hot path can hide the race it was meant to catch. Keep it in tests, not production.

Skip this approach if any of these hold:

• The script is under a few hundred lines and you plan to delete it.
• Effects already sit behind an outbox or idempotency layer with explicit ordering.
• The code is heavily concurrent with no identifiable phase boundaries.
• There is no CI and no habit of reverting on red.

The rule worth keeping

The smallest safe change is defined by effect order, not by diff size.

Pin the call log. Prove it stable across repeats. Move one thing at a time. Read the position diff before you touch the golden file.