Build an Environment Ledger Before Cleaning a Tangled CLI

A cleanup pull request looked harmless until the nightly invoice job wrote files into the wrong directory. The refactor had replaced os.getcwd() with a packaged resource path, and every relative open() followed it. Tests still passed because they ran from the repository root, which matched the old accidental contract. The failure appeared only on a scheduler whose working directory was /var/lib/batch rather than the checkout.

Messy repositories accumulate these environmental contracts for years without documenting them as interfaces. Coding agents and human cleanups both treat cwd, locale, timezone, and temp directories as leftover implementation details. They are not details when a batch job, a cron entry, or a container WORKDIR already depends on the old accident. The practical sequence is to characterize those surfaces first, then apply the smallest safe change.

Hidden environment contracts in tangled checkouts

Cleanup diffs usually chase names, helpers, and folder structure rather than observable input and output. That bias is reasonable when the module is already a pure function of explicit arguments. Most long-lived scripts are not pure, because they read os.environ, format dates, and write beside the process working directory. An agent that merely modernizes paths can preserve unit tests while changing production file placement.

Three failure modes show up repeatedly in invoice, ETL, and report jobs:

• Relative file I/O that silently follows cwd instead of an explicit base directory.
• Locale-sensitive decimal separators, month names, and CSV headers that differ under C versus en_US.UTF-8.
• Temporary files that land in $TMPDIR on a laptop and in /tmp inside a scheduler image.

None of these outcomes require a malicious model or an unusually large rewrite. They require an unpinned environment plus a cleanup that looks locally smaller than its production effect. Current public debate about agents outgrowing tests is adjacent here: the tests did not shrink, they never observed cwd, locale, or temp placement at all.

What to record before anyone rewrites a module

Treat the process environment as a public API even when the original authors never named it that way. A side-effect ledger is a JSON document that captures the surfaces a later diff is not allowed to change without an explicit decision. The ledger is not a substitute for domain tests; it is a fence around accidental behavior that unit tests rarely exercise.

Record at least the following fields for each entry-point script:

• process cwd and the resolved path of every relative open, mkdir, and glob
• TZ, LC_ALL, LANG, LC_TIME, LC_NUMERIC, and the locale.getlocale() tuple
• tempfile.gettempdir(), TMPDIR, TMP, TEMP, and a sample NamedTemporaryFile prefix
• umask, file-creation mode of one artifact, and whether output paths are absolute
• a hash of stdout, stderr, and the sorted list of written file paths

Keep secrets out of the ledger on purpose. Redact tokens, connection strings, and customer payloads before the file is committed or copied onto any shared machine.

A proposed characterization harness

The following Python example is a proposed, unexecuted sketch for a small CLI module. It is meant to run against a frozen fixture directory, not against production data. Adjust the command list and fixture layout for the messy repository under review.

#!/usr/bin/env python3
"""Proposed characterization harness for environment-sensitive CLIs."""
from __future__ import annotations

import hashlib
import json
import locale
import os
import subprocess
import sys
import tempfile
from pathlib import Path

LEDGER_VERSION = 1


def read_umask() -> str:
    current = os.umask(0)
    os.umask(current)
    return oct(current)


def snapshot_env(cwd: Path) -> dict:
    return {
        "cwd": str(cwd.resolve()),
        "tz": os.environ.get("TZ"),
        "lang": os.environ.get("LANG"),
        "lc_all": os.environ.get("LC_ALL"),
        "lc_time": os.environ.get("LC_TIME"),
        "lc_numeric": os.environ.get("LC_NUMERIC"),
        "locale": list(locale.getlocale()),
        "tmpdir": tempfile.gettempdir(),
        "tmp_env": {
            "TMPDIR": os.environ.get("TMPDIR"),
            "TMP": os.environ.get("TMP"),
            "TEMP": os.environ.get("TEMP"),
        },
        "umask": read_umask(),
        "python": sys.version.split()[0],
    }


def run_subject(command: list[str], cwd: Path, env: dict) -> dict:
    proc = subprocess.run(
        command,
        cwd=cwd,
        env=env,
        capture_output=True,
        text=True,
        check=False,
    )
    written = sorted(
        str(p.relative_to(cwd)) for p in cwd.rglob("*") if p.is_file()
    )
    stdout_hash = hashlib.sha256(proc.stdout.encode("utf-8")).hexdigest()
    stderr_hash = hashlib.sha256(proc.stderr.encode("utf-8")).hexdigest()
    tree_src = "\n".join([str(proc.returncode), stdout_hash, stderr_hash, *written])
    return {
        "returncode": proc.returncode,
        "stdout_sha256": stdout_hash,
        "stderr_sha256": stderr_hash,
        "tree_sha256": hashlib.sha256(tree_src.encode("utf-8")).hexdigest(),
        "written": written,
    }


def main() -> None:
    fixture = Path("fixtures/invoice_batch").resolve()
    cwd = fixture / "work"
    out_dir = Path("ledgers")
    out_dir.mkdir(exist_ok=True)
    env = os.environ.copy()
    env.update(
        {
            "TZ": "UTC",
            "LC_ALL": "C",
            "LANG": "C",
            "TMPDIR": str(fixture / "tmp"),
        }
    )
    ledger = {
        "version": LEDGER_VERSION,
        "env": snapshot_env(cwd),
        "run": run_subject(
            [sys.executable, "export_invoices.py", "--date", "2026-09-16"],
            cwd=cwd,
            env=env,
        ),
    }
    path = out_dir / "export_invoices.v1.json"
    generated = out_dir / "export_invoices.generated.json"
    generated.write_text(json.dumps(ledger, indent=2, sort_keys=True))
    if path.exists():
        previous = json.loads(path.read_text())
        if previous != ledger:
            raise SystemExit("characterization drift: environment contract changed")
        print("ledger unchanged")
        return
    path.write_text(json.dumps(ledger, indent=2, sort_keys=True))
    print(f"wrote {path}")


if __name__ == "__main__":
    main()

Pin the process environment from the shell as well, because a developer laptop often leaks LANG and TMPDIR into otherwise careful Python. The command below is a proposed isolation wrapper, not a production scheduler definition.

mkdir -p fixtures/invoice_batch/work fixtures/invoice_batch/tmp ledgers

env -i \
  PATH="$PATH" \
  HOME="$HOME" \
  TZ=UTC \
  LC_ALL=C \
  LANG=C \
  TMPDIR="$(pwd)/fixtures/invoice_batch/tmp" \
  python3 tools/env_ledger.py

python3 tools/env_ledger.py
git diff --stat -- ledgers export_invoices.py

A tiny proposed pytest check can fail the build when a cleanup regenerates a different ledger. Keep that test on the committed JSON, and do not overwrite the committed file from CI.

# proposed: tools/test_env_ledger.py
import json
from pathlib import Path


def test_ledger_matches_committed_snapshot():
    committed = json.loads(Path("ledgers/export_invoices.v1.json").read_text())
    generated = json.loads(Path("ledgers/export_invoices.generated.json").read_text())
    assert generated == committed

If the ledger is unchanged and git diff --stat stays inside the intended files, the cleanup remains a candidate. If either check moves, stop and decide whether the new behavior is an intended contract change. Do not refresh the JSON to make a large rewrite look green.

Decision table for the smallest safe change

Use the table below before accepting an agent diff or a human cleanup on a tangled script.

Observation	Allowed next step	Disallowed next step
Ledger matches; diff touches only path constants in one file	Replace cwd-relative opens with an explicit --base-dir	Rename modules, extract helpers, or reformat adjacent jobs
Ledger matches; diff also reorders imports and comments	Keep the path change; revert the noise	Bundle style and behavior in one review
Ledger drifts on tmp paths only	Pin TMPDIR in the job spec, then retry	Accept "we use tempfile correctly now" without a pin
Ledger drifts on stdout hashes because dates shifted	Freeze TZ=UTC and LC_TIME=C, then retry	Patch the golden hash to match the new local clock
Ledger drifts on written relative paths	Treat cwd as an API; add a test that chdir()s	Continue extracting functions on top of the drift
Diff spans more than one job or shared util	Split the change; re-run the ledger per entry point	"While we are here" refactors across the batch folder

The smallest safe change is the diff that preserves the ledger and touches one contract at a time. Extracting a helper is a second change, even when the helper looks obviously right in isolation. That split is the entire method: characterize, change one surface, re-read the ledger, then stop.

Where a free coding agent belongs in this loop

An agent is useful after the ledger exists, not before the first characterization run. It can propose additional relative-open sites, draft a --base-dir flag, or generate a second fixture that starts in a different cwd. It should not be the process that decides the ledger is stale or that a golden hash deserves a rewrite.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option, which is enough to run the harness in a throwaway workspace instead of on a laptop that already holds production exports. That separation matters because characterization fixtures can include path layouts that resemble real jobs even after payloads are redacted.

A practical loop on a free server looks like the following numbered sequence.

1. Copy a redacted fixture tree, not the production working directory.
2. Run the ledger command until it writes a stable JSON document.
3. Ask the model only for the smallest path-explicit change in one file.
4. Re-run the ledger and inspect git diff --stat before any further prompt.

If the model returns a multi-file rewrite, reject the whole patch and restate the file budget in the next request. Free capacity does not change the review rule; it only keeps the experiment off the machine that mounts customer disks. A throwaway workspace is useful here when the alternative is running fixture trees beside real export directories.

Limitations

This harness does not prove functional correctness of invoice math, tax rounding, or SQL results. It only detects environmental drift that unit tests often miss because they share the developer's cwd and locale. Hashing stdout will fail on intentional logging changes, which then requires a split between semantic output and diagnostic streams. Parallel jobs that race on temporary names need extra pinning, such as a deterministic prefix, or the ledger will flap without a real regression. Container images that reset LANG at entry can disagree with a laptop run even when the Python file itself is unchanged.

The umask field is easy to mis-snapshot if another thread mutates the mask during the run. Treat that field as advisory unless the job creates files whose mode is part of an operational agreement. Do not upload ledgers that still contain customer identifiers in written path names or in captured standard output.

Who should not use this approach

Skip this workflow when the program is already a pure function with injected clocks, paths, and locales. Skip it when the repository cannot provide a redacted fixture and the alternative is copying live data onto a shared server. Skip it for cryptographic code, access-control changes, or anything where a matching stdout hash is the wrong safety property. Teams that already maintain formal acceptance tests should keep those tests, because the ledger is a fence rather than a specification.

Operators who cannot inspect git diff --stat before merge should not delegate cleanup to any model, paid or free. The method assumes a human still owns the environmental contract after the agent returns a patch.

Cleanup diffs fail in production because messy repositories encode cwd, locale, and temp paths as silent APIs. Pin those surfaces with a ledger, allow one contract-preserving change, and only then consider a larger rewrite. A free remote workspace can host the fixture run, but it cannot replace the decision table that keeps the first diff small.