Refactor One Function After Characterization Holds

Messy repository refactors fail on hidden behavior, not names. Freeze today's output bytes before moving any helper. Then change exactly one production seam after that lock.

Why cleanup sweeps lose

Large generated refactors look tidy in the patch viewer. They still drop implicit contracts no test ever named. Patch size is not a safety metric here.

Hidden file writes are the usual first casualty. Cache keys and cwd-relative paths follow close behind. Lint will not tell you those contracts moved.

Do not start from architecture taste or naming style. Start from observed bytes of one entry point. Taste comes after the harness stays green.

The working rule

Characterization tests record what the code does now. They do not certify that this behavior is correct. They only freeze the mess you still ship.

A later correctness pass can change the goldens on purpose. That pass needs a reviewed message and a new fixture. It is not the first refactor commit.

Step 1 — Bound one entry point

Do not inventory the entire monorepo on day one. Pick a single CLI handler or HTTP worker. Write inputs, outputs, and side channels on paper.

Include argv, environment variables, cwd, and clock reads. Name every file the handler reads or writes. Stop when that list covers one job only.

Sample envelope for a dusty report builder:

• Input file: fixtures/job-a/job.json
• Stdout: one summary line
• Stderr: warning lines
• Side files: .cache/report.bin and out/latest.csv
• Exit code: integer status

Paper envelopes beat generated call graphs at this stage. Graphs hide process-level I/O that tests must pin. Static fan-in analysis can wait until after the lock.

Step 2 — Capture a golden run

Run the real module against a fixture directory. Skip mocks during this first lock. Save stdout, stderr, exit code, and output files.

mkdir -p fixtures/job-a expected/job-a
cp samples/messy-job.json fixtures/job-a/job.json
export TZ=UTC LANG=C LC_ALL=C
python -m reports.cli fixtures/job-a/job.json \
  > expected/job-a/stdout.txt \
  2> expected/job-a/stderr.txt
echo $? > expected/job-a/exit.txt
cp .cache/report.bin expected/job-a/report.bin
cp out/latest.csv expected/job-a/latest.csv
git add expected/job-a fixtures/job-a
git commit -m "lock report job-a characterization bytes"

Commit expected files before any production edit. That commit is the lock, not a comment. Reviewers should reject refactors that land without this lock.

Pin locale and timezone during capture. Unpinned clocks will churn CSV timestamps. Recapture only with an explicit review note.

Step 3 — Encode the golden as a test

The test should fail only when observed bytes change. It must not interpret CSV columns yet. Domain meaning waits until the mess is pinned.

The following harness is a template, not a measured run.

# tests/test_characterize_report.py
from pathlib import Path
import os
import subprocess
import sys

ROOT = Path(__file__).resolve().parents[1]
EXP = ROOT / "expected" / "job-a"
FIX = ROOT / "fixtures" / "job-a"


def _run_job(tmp_path: Path) -> dict:
    work = tmp_path / "work"
    work.mkdir()
    (work / ".cache").mkdir()
    (work / "out").mkdir()
    job = work / "job.json"
    job.write_bytes((FIX / "job.json").read_bytes())
    env = os.environ.copy()
    env.update({"TZ": "UTC", "LANG": "C", "LC_ALL": "C"})
    proc = subprocess.run(
        [sys.executable, "-m", "reports.cli", str(job)],
        cwd=work,
        env=env,
        capture_output=True,
        check=False,
    )
    return {
        "exit": f"{proc.returncode}\n".encode(),
        "stdout": proc.stdout,
        "stderr": proc.stderr,
        "bin": (work / ".cache" / "report.bin").read_bytes(),
        "csv": (work / "out" / "latest.csv").read_bytes(),
    }


def test_job_a_bytes_match(tmp_path):
    got = _run_job(tmp_path)
    assert got["exit"] == (EXP / "exit.txt").read_bytes()
    assert got["stdout"] == (EXP / "stdout.txt").read_bytes()
    assert got["stderr"] == (EXP / "stderr.txt").read_bytes()
    assert got["bin"] == (EXP / "report.bin").read_bytes()
    assert got["csv"] == (EXP / "latest.csv").read_bytes()

Run the template once on the untouched tree. The first pass must stay green. A red first pass means the capture script is wrong.

python -m pytest tests/test_characterize_report.py -q

Prefer subprocess over in-process runpy for messy modules. Import-time I/O will poison later tests in the same session. Isolation is part of the lock, not a style choice.

Step 4 — Classify every mismatch

A failing characterization test is not a rewrite license. Classify the byte delta before touching production code. Use one decision table for every failure.

Delta class	Example	Next action
Capture bug	Path differs from golden cwd	Fix the harness, recapture
Intentional product change	New CSV column required	New golden, separate commit
Accidental drift	Cache header flipped	Revert the production edit
Locale or clock noise	Timestamp shifted by TZ	Pin env, recapture once
Extra cleanup	Two helpers moved together	Split the commit, retest

Do not mix classes in a single commit. Mixed commits hide which byte actually moved. Reviewers cannot reconstruct the cause later.

Normalize only fields that cannot be pinned. UUIDs and subsecond stamps are the usual candidates. Document each strip inside the envelope notes.

Step 5 — Apply the smallest safe change

Touch one function in one commit only. Re-run the full harness after that single edit. If two modules must move, the seam is wrong.

Allowed edits after the lock holds:

1. Extract a pure helper the test already covers.
2. Rename a local symbol with no byte change.
3. Add a guard that preserves current output bytes.
4. Delete dead code only if goldens still match.

Forbidden until a later, explicit cycle:

1. Framework swaps inside the same commit.
2. Cache layout redesigns.
3. Concurrency or process-model changes.
4. Drive-by naming sweeps across the package.

Counter-example: moving write_cache and render_csv together. That pair looks related in the same file. It is two seams and two failure modes.

Keep a one-line change budget in the commit message. Example: extract _format_row from render_csv. If the message needs "and", split the work.

git add reports/csv.py tests/test_characterize_report.py
# Do not stage tests unless the golden class changed on purpose.
git diff --stat
python -m pytest tests/test_characterize_report.py -q
git commit -m "extract _format_row from render_csv"

Stage production and tests together only for intentional golden updates. Those updates are product changes, not refactors. Keep that distinction in the log.

Artifact: go / no-go checklist

Use this checklist before you open the editor. Print it beside the paper envelope.

1. One entry point is named in the PR body.
2. Golden bytes exist for stdout, stderr, and files.
3. Locale and timezone are pinned in the test.
4. First harness run is green on main.
5. Proposed edit lists one function path.
6. Tests and production are not changed together.
7. Failures are classified with the table above.

If any box is unchecked, stop. The next edit would be a guess. Guesses are how messy trees get messier.

Add a second fixture only after the first job stays green. Job-b should exercise a different branch or file shape. Two goldens beat one wide snapshot of mixed jobs.

# Template for a second job, after job-a is stable
mkdir -p fixtures/job-b expected/job-b
cp samples/messy-job-empty.json fixtures/job-b/job.json
TZ=UTC LANG=C LC_ALL=C python -m reports.cli fixtures/job-b/job.json \
  > expected/job-b/stdout.txt 2> expected/job-b/stderr.txt

Where free model access belongs

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A coding model can propose the next seam. It cannot replace locked golden bytes. Feed it the envelope, the green test, and a one-function limit.

MonkeyCode offers free model access and a free server option. Run the characterization harness on that server when the laptop is busy. Keep the local tree unchanged until remote bytes match expected/.

Prompt with a constraint, not a cleanup vibe:

The repository is messy. Do not clean it.
Keep tests/test_characterize_report.py green.
Propose a patch that touches one function only.
Return a diff. Do not add features or new tests.

Reject any patch that edits tests and production together. That pattern hides behavior drift. Re-run the harness after applying the diff by hand.

If a free server slot is already available, run the harness there before merge.

Limitations

This method freezes bugs as well as features. That freeze is intentional on day one. Correctness changes need a later, labeled golden update.

Golden files rot when clocks or locales drift. They also rot across dependency upgrades that change print formatting. Recapture is a product decision, not a refactor.

The harness does not prove thread safety. It does not prove authorization or injection resistance. It only proves today's bytes for one job.

Free model text remains untrusted. A free server is not a production SLA. Do not upload secrets, customer dumps, or private keys.

Byte equality is brittle for floats and random fields. Strip or normalize those fields during capture if they churn. Write the strip rules next to the envelope.

Network calls inside the handler will still flake. Characterization does not replace a fake clock or a fake HTTP layer. Add those seams only after the local file bytes are locked.

Who should skip this

Skip this workflow when no entry point can run. Skip it when a pure function already has unit tests. Skip it when policy forbids sending code to a hosted model.

Greenfield services with a written spec do not need this lock. Write acceptance tests against the spec instead. Characterization is for behavior you inherited and cannot yet explain.

Do not use the smallest-change rule as a stall tactic. If the envelope is wrong, redraw it. Then lock new goldens in their own commit.

Teams without fixture storage should not start here. Golden binaries belong in Git LFS or a reviewed artifact store. Untracked expected/ directories are not a lock.

Close

Lock the mess with bytes you can re-run. Change one function. Re-run the harness. Repeat until the envelope is small enough to name. The refactor ends when the next edit needs new goldens. File prettiness is not a stop condition here.