Hash the Side-Effect Ledger Before You Accept a Cleanup Refactor

Messy modules rarely break because a pure helper returns the wrong integer on a tidy fixture. They break because three functions share a temporary CSV path, an environment flag, and a cache nobody named. A coding agent then proposes a cleanup that deletes dead branches, renames locals, and still satisfies every existing assertion. The next production export fails because the implicit file layout moved while the return payload stayed identical.

That failure mode is the reason this workflow exists, and it is not a style problem. The first commit should freeze a ledger of hidden couplings and store a hash beside it. Only after that hash is in source control should you allow one structural change. The cleanup is legitimate only when the recorded hash remains identical.

Cleanup diffs fail differently than feature diffs

Feature work usually changes an observable on purpose, so reviewers know which assertions must move. Cleanup work is sold as behavior-preserving, which trains people to trust deletions and rename-only hunks. Coding agents amplify that bias because they optimize for shorter files, conventional names, and green unit tests. Reviewers then accept large deletions that would look suspicious inside a feature pull request.

Return-value tests are the wrong gate for that class of change. The public function can still return {"ok": true, "rows": 12} while the working directory quietly shifts. Downstream jobs that glob files or catch a named exception will fail after merge. Those hidden couplings remain part of the contract even when no unit test mentions them.

Build a side-effect ledger instead of another unit test

Treat the messy module as a black box that emits more than a return value. A ledger is a canonical JSONL file with one record per fixture and fully sorted keys. Side-effect entries need stable ordering so the serialized bytes stay deterministic across reruns. The SHA-256 digest of that file is the only number that must remain constant.

Each record should capture the following fields and omit anything that varies by machine:

• public callable name and a short digest of the arguments
• result kind: return, raise, or timeout
• exception class name when the result kind is raise
• environment keys the module actually read during the call
• files created, removed, or appended, relative to a sandbox root
• working directory relative to that same sandbox root

Canonicalize every path against the sandbox root before you serialize the record. Wall-clock timestamps and absolute home-directory prefixes make the hash flaky on contact. A flaky gate teaches the team to skip the protocol, which is worse than having no gate.

Keep the first change inside a six-step loop

1. Choose one public entrypoint rather than the entire package, and list fixtures that replay without network access.
2. Wrap the process in a sandbox with a temporary root, a copied fixture tree, and an environment allowlist.
3. Run the recorder and commit ledger.jsonl plus ledger.sha256 with no production code changes.
4. Declare a file-touch budget: the cleanup commit may edit one implementation file and one test file.
5. Permit a single structural change, such as extracting a path helper or passing the cache as an argument.
6. Re-run the recorder and accept the diff only when the hash matches and the touch budget holds.

If the hash changes, the cleanup is not a cleanup and should not keep that label. Treat the diff as a behavior change, add an intentional test, and restart the protocol. Agents that continue after a mismatch are doing product work under a refactor heading.

Example recorder, labeled as a proposal

The script below is an unexecuted example you can adapt to one entrypoint. It does not claim production coverage numbers, and it will miss native writes outside the sandbox. Read it as a starting template rather than as a library you vendor unchanged.

# ledger_recorder.py — proposal: pin hidden couplings for one entrypoint
from __future__ import annotations

import hashlib
import json
import os
import sys
import traceback
from pathlib import Path
from typing import Any, Callable

SANDBOX = Path(os.environ["LEDGER_SANDBOX"]).resolve()
LEDGER_PATH = Path(os.environ.get("LEDGER_PATH", "ledger.jsonl"))


def canonicalize(path: Path) -> str:
    try:
        return str(path.resolve().relative_to(SANDBOX))
    except ValueError:
        return f"<outside>/{path.name}"


def digest_args(args: tuple[Any, ...]) -> str:
    blob = json.dumps(args, default=str, sort_keys=True).encode()
    return hashlib.sha256(blob).hexdigest()[:12]


def record_call(name: str, args: tuple[Any, ...], fn: Callable[..., Any]) -> dict[str, Any]:
    env_reads: set[str] = set()
    before = {
        canonicalize(p): p.stat().st_mtime_ns
        for p in SANDBOX.rglob("*")
        if p.is_file()
    }
    real_getenv = os.getenv

    def wrapped_getenv(key: str, default: Any = None) -> Any:
        env_reads.add(key)
        return real_getenv(key, default)

    os.getenv = wrapped_getenv  # type: ignore[assignment]
    try:
        value = fn(*args)
        result = {"kind": "return", "value": value, "error": None}
    except Exception as exc:
        result = {
            "kind": "raise",
            "value": None,
            "error": type(exc).__name__,
            "trace_tail": traceback.format_exc().splitlines()[-1],
        }
    finally:
        os.getenv = real_getenv  # type: ignore[assignment]

    after = {
        canonicalize(p): p.stat().st_mtime_ns
        for p in SANDBOX.rglob("*")
        if p.is_file()
    }
    files: set[tuple[str, str]] = set()
    for rel in sorted(set(before) | set(after)):
        if rel not in before:
            files.add(("create", rel))
        elif rel not in after:
            files.add(("remove", rel))
        elif before[rel] != after[rel]:
            files.add(("append", rel))

    return {
        "callable": name,
        "args_digest": digest_args(args),
        "kind": result["kind"],
        "error": result["error"],
        "value": result["value"],
        "env_reads": sorted(env_reads),
        "files": sorted(files),
        "cwd": canonicalize(Path.cwd()),
        "sys_path_heads": [
            canonicalize(Path(p)) if Path(p).exists() else p for p in sys.path[:3]
        ],
    }


def write_ledger(rows: list[dict[str, Any]]) -> str:
    canonical = [json.dumps(row, sort_keys=True, default=str) for row in rows]
    canonical.sort()
    LEDGER_PATH.write_text("\n".join(canonical) + "\n", encoding="utf-8")
    digest = hashlib.sha256(LEDGER_PATH.read_bytes()).hexdigest()
    Path("ledger.sha256").write_text(digest + "\n", encoding="utf-8")
    return digest


def main() -> None:
    # Proposal: replace with the real entrypoint and replayable fixtures.
    from app.export import run_export  # labeled example import

    os.chdir(SANDBOX)
    fixtures = [
        ("run_export", ("2026-09-08",)),
        ("run_export", ("2026-09-09",)),
    ]
    rows = [record_call(name, args, run_export) for name, args in fixtures]
    print(write_ledger(rows))


if __name__ == "__main__":
    main()

A matching check belongs in continuous integration as a command, not as a comment on the pull request. The commands assume a clean sandbox and a hash file committed on the characterization branch.

export LEDGER_SANDBOX="$(pwd)/.ledger-sandbox"
rm -rf "$LEDGER_SANDBOX"
mkdir -p "$LEDGER_SANDBOX/fixtures"
cp -R tests/fixtures/. "$LEDGER_SANDBOX/fixtures/"
python ledger_recorder.py
test "$(tr -d '[:space:]' < ledger.sha256)" = "$(sha256sum ledger.jsonl | awk '{print $1}')"
git diff --name-only origin/main...HEAD | awk 'END { if (NR > 2) { print "touch budget exceeded"; exit 1 } }'

The final command is the file-touch budget for the cleanup commit itself. Diffs that rewrite six files while claiming no behavior change should fail even when the hash matches. Reviewers cannot audit accidental protocol shifts across that much surface in one sitting.

Negotiate the first change with a decision table

Use the table as the only negotiation surface with the agent session. Anything outside the selected row is a new task, not a continuation of the cleanup. Put the selected row in the prompt and omit future style goals that would invite extra edits.

Observed coupling	Safe first change	Hash must stay	Touch budget
Shared temp CSV path	Extract export_path(sandbox, date) and call it from one site	Yes	1 impl + 1 test
Process-global cache dict	Pass the cache into the entrypoint; do not rename keys	Yes	1 impl + 1 test
os.getenv("EXPORT_MODE")	Read the flag once at the edge; keep the default identical	Yes	1 impl + 1 test
Catches ValueError by name	Leave the type; do not switch to a custom hierarchy yet	Yes	0 impl if the type would change
Absolute /tmp writes	Relocate writes under the sandbox root only	Yes	1 impl + 1 test

If two rows look tempting, pick the coupling that already causes production incidents and defer the rest. Parallel cleanups destroy the meaning of the hash because a mismatch cannot be attributed to one seam. The agent should see the selected row and the current ledger hash, then stop.

Run the ledger on a pinned runner

Local laptops pollute characterization hashes through leftover files, extra environment variables, and home-directory prefixes. A dedicated runner with a known sandbox root removes that noise from the digest. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access can draft the single-seam patch from the decision table, and the free server option can execute the recorder so the hash is not a function of your laptop.

That split is more useful than another chat transcript about naming style. The model proposes one change inside the touch budget, and the server replays the fixtures. You accept the diff only when both gates pass on the same job log. Skip any run that cannot pin the Python version, working directory, and fixture tree.

Limitations

The ledger does not see network I/O you forgot to stub or threads that flush after the recorder returns. Native extensions that write outside Path.rglob will also slip past the file list. Hash stability still depends on canonical JSON and on redacting values that embed timestamps.

This protocol is slower than asking an agent to clean the module in one pass. It will reject useful renames that change an exception type or a filename pattern on purpose. Those edits are product changes and need an explicit test update, not a cleanup label.

Who should skip this protocol

Do not use a side-effect ledger on greenfield code where the public contract still moves every day. Do not use it as a substitute for a typed interface when you already have a stable API module. Do not point an agent at the whole repository and then widen the touch budget until the gate becomes theater.

If the module's only consumers are humans clicking a button, return-value checks may already be enough. The ledger earns its keep when hidden couplings are the real API for downstream jobs. Matching hashes plus a two-file budget is a boring gate, and boring gates are how messy modules survive a first structural change.