Treat a Tangled File as a Black Box First

Treat a tangled file as a black box first. Characterization tests pin today's mixed side effects. Only then extract one pure function.

Large model diffs often look like real progress. They rewrite mixed I/O without a local oracle. Tests stay green while receipts and files drift.

This article shows a pinned extract workflow. The sample is labeled and stays unexecuted.

Adapt the commands to your own tree. Do not treat snippets as measured production data.

Why the wide diff fails

A god module mixes compute, I/O, and formatting. Callers quietly depend on every mixed surface.

A cleanup pull request can shift two surfaces. Receipts change while inventory bytes also change.

Free coding models can amplify that exact risk. They propose wide extracts across mixed I/O. They lack a local oracle unless you supply one.

Where a model belongs in this workflow

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option.

Use those tools only after characterization pins exist. They do not replace characterization tests at all.

What to pin

Pin observable behavior rather than preferred file structure. Structure can wait until one extract lands.

Surface	Pin method	Fail signal
stdout text	exact bytes or parsed fields	extra lines, reordered tokens
return value	deep equality on stable keys	dropped fields, type changes
inventory file	sha256 of canonical JSON	indent drift, key-order drift
append-only log	full text after injected clock	extra lines, timestamp jitter

Skip private helper names in the pin suite. Skip comment wording unless imports are a public contract.

Artifact: a tangled seat-hold module

The next file is a teaching example only. It does not come from a production repository.

It holds event seats and writes JSON files. It also prints a receipt line to stdout.

# hold_seat.py — tangled teaching example, not production code
from pathlib import Path
import json
from datetime import datetime, timezone

INVENTORY = Path("inventory.json")
LOG = Path("holds.log")

def hold_seat(event_id, seat, customer, now=None):
    now = now or datetime.now(timezone.utc)
    data = json.loads(INVENTORY.read_text())
    event = data["events"][event_id]
    if seat not in event["open"]:
        print(f"HOLD_FAIL {event_id} {seat}")
        prior = LOG.read_text() if LOG.exists() else ""
        LOG.write_text(prior + f"{now.isoformat()} fail {seat}\n")
        return {"ok": False, "reason": "taken", "at": now.isoformat()}
    event["open"].remove(seat)
    event["held"][seat] = customer
    INVENTORY.write_text(json.dumps(data, indent=2, sort_keys=True) + "\n")
    print(f"HOLD_OK {event_id} {seat} {customer}")
    prior = LOG.read_text() if LOG.exists() else ""
    LOG.write_text(prior + f"{now.isoformat()} ok {seat}\n")
    return {
        "ok": True,
        "event": event_id,
        "seat": seat,
        "customer": customer,
        "at": now.isoformat(),
        "open_count": len(event["open"]),
    }

This function moves several surfaces in one call. Stdout, the log file, and inventory all change.

The return dict changes in the same breath. Any later extract must keep those surfaces stable.

1. Freeze the workspace

Work on a throwaway branch for this pin. Do not mix formatters with behavior pins yet.

git checkout -b pin/hold-seat
python -V
mkdir -p fixtures/hold_seat

Record interpreter version in the pull request body. Characterization tests remain sensitive to runtime details.

A JSON spacing change can look like a regression. Pin the runtime before you pin the bytes.

2. Build a fixture, not a redesign

Seed inventory with one event only. Keep the fixture tiny and boring.

One open seat and one held seat suffice. Extra seats hide the first mismatch.

{
  "events": {
    "talk-9": {
      "open": ["A1", "A2"],
      "held": {"B1": "ada"}
    }
  }
}

Save that blob under fixtures/hold_seat/inventory.json. Copy it into a temp path per test.

Never reuse a dirty inventory path across cases. Failed holds must not leak into later asserts.

3. Capture mixed surfaces in one harness

The test below is a characterization harness. It does not score design quality.

It asserts today's bytes and today's dict. Inject clock, paths, and stdout capture.

# test_hold_seat_pins.py — unexecuted teaching harness
import json
import hashlib
from datetime import datetime, timezone
from pathlib import Path
import hold_seat as hs

FIXED = datetime(2026, 9, 3, 12, 0, tzinfo=timezone.utc)
FIXTURE = Path("fixtures/hold_seat/inventory.json")

def sha256(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()

def seed(tmp_path: Path):
    inv = tmp_path / "inventory.json"
    log = tmp_path / "holds.log"
    inv.write_bytes(FIXTURE.read_bytes())
    log.write_text("")
    hs.INVENTORY = inv
    hs.LOG = log
    return inv, log

def test_hold_success_pins_surfaces(tmp_path, capsys):
    inv, log = seed(tmp_path)
    result = hs.hold_seat("talk-9", "A1", "lin", now=FIXED)
    out = capsys.readouterr().out
    assert result["ok"] is True
    assert result["open_count"] == 1
    assert result["at"] == "2026-09-03T12:00:00+00:00"
    assert out == "HOLD_OK talk-9 A1 lin\n"
    assert log.read_text() == "2026-09-03T12:00:00+00:00 ok A1\n"
    data = json.loads(inv.read_text())
    assert "A1" not in data["events"]["talk-9"]["open"]
    assert data["events"]["talk-9"]["held"]["A1"] == "lin"

def test_hold_taken_leaves_inventory_untouched(tmp_path, capsys):
    inv, log = seed(tmp_path)
    before = sha256(inv)
    result = hs.hold_seat("talk-9", "B1", "lin", now=FIXED)
    out = capsys.readouterr().out
    assert result == {
        "ok": False,
        "reason": "taken",
        "at": "2026-09-03T12:00:00+00:00",
    }
    assert out == "HOLD_FAIL talk-9 B1\n"
    assert log.read_text() == "2026-09-03T12:00:00+00:00 fail B1\n"
    assert sha256(inv) == before

Run the suite once on the untouched module. Store expected hashes only after a human glance.

pytest -q test_hold_seat_pins.py
sha256sum fixtures/hold_seat/inventory.json

If stdout includes timestamps you did not inject, stop. Inject time first, then pin bytes.

Do not pin a moving clock and call it safety. That fixture will flake on the next run.

4. Add a decision gate before any extract

Do not extract yet. Score the candidate change against the pins.

Question	If yes	If no
Are stdout, files, and returns pinned?	continue	write more tests
Would the extract change stdout bytes?	reject	continue
Would the extract change inventory hashes?	reject	continue
Is the extract one pure function?	continue	shrink the diff
Does the prompt ask for a rewrite?	reject	continue

The gate is mechanical on purpose. "Looks cleaner" is not a pass condition.

5. Extract the smallest pure core

The only safe extract here is seat math. File I/O stays in the original function.

Printing stays in that facade as well. Pure code should not touch Path objects.

def apply_hold(event, seat, customer):
    """Pure extract. Example only. Callers still own I/O."""
    if seat not in event["open"]:
        return False, {
            "open": list(event["open"]),
            "held": dict(event["held"]),
        }
    open_seats = [s for s in event["open"] if s != seat]
    held = dict(event["held"])
    held[seat] = customer
    return True, {"open": open_seats, "held": held}

Wire it with a narrow swap inside hold_seat. Keep the original function as the facade.

ok, updated = apply_hold(event, seat, customer)
if not ok:
    print(f"HOLD_FAIL {event_id} {seat}")
    prior = LOG.read_text() if LOG.exists() else ""
    LOG.write_text(prior + f"{now.isoformat()} fail {seat}\n")
    return {"ok": False, "reason": "taken", "at": now.isoformat()}
event["open"] = updated["open"]
event["held"] = updated["held"]

Re-run the pin suite after that swap. Any mismatch means the extract is too large.

pytest -q test_hold_seat_pins.py
git diff --stat

Expect the module file, and maybe the test file. A twenty-file diff fails the gate.

List names in the diff and reject surprise edits. Format-only churn is still behavior risk here.

6. Ask a model only after the pins hold

After pins exist, a model can draft apply_hold. It should not invent new log formats.

It should not move files onto new paths. It should not rename the public facade yet.

Feed the model the test file first. Then ask for the smallest extract that stays green.

A free server can run pytest against those pins. Keep that run on fixtures, never on live inventory.

Do not paste secrets into that prompt. Do not point tools at production seat files.

This is not a benchmark article. No latency or quality numbers are claimed here.

7. Stop after one extract

Resist the second helper in the same patch. Resist renaming hold_seat in this pass.

Resist JSON pretty-print tweaks as well. Each extra change mixes the failure signal.

When pins fail, you need one suspect. Two extracts in one diff hide the cause.

Open the next pull request only after this merges. Characterization tests travel with the module.

Limitations

Characterization tests freeze today's bugs in place. They will defend a bad receipt line.

That freeze is intended for inherited modules. Fix the bug later with an explicit assertion change.

Hash pins break on any byte change. Indent, key order, and trailing newlines all count.

Use sort_keys=True and injected clocks anyway. Still expect brittle fixtures on purpose.

This workflow does not prove thread safety. It does not prove disk-full behavior either.

It does not prove concurrent holds on one seat. Add those cases as separate, named tests.

A free model can still emit a rewrite. The pins are the reject mechanism only.

They are not a guarantee of good design. JSON equality is not semantic equality either.

open_count can stay correct while seat order shifts. Add an order pin if callers depend on it.

Who should not use this

Do not use this on greenfield modules with no callers. Write intent tests instead of pins.

Characterization is for inherited behavior you cannot restated yet. New code deserves explicit contracts.

Do not use this as a substitute for typed contracts. If you can specify the domain, specify it.

Pins are a bridge, not an architecture document. Do not run a free server against customer data.

Fixtures only. Synthetic seats only. No copied production dumps.

Do not skip the human glance at hashes. A pinned wrong file is a durable wrong file.

Checklist

1. Branch. Freeze tooling. Record the interpreter version.
2. Seed a tiny fixture. Inject clock and filesystem paths.
3. Pin stdout, return value, inventory, and log text.
4. Gate the extract with the decision table above.
5. Extract one pure function. Re-run the pin suite.
6. Merge. Do not stack a second extract in this patch.

The god module can stay ugly for one more week. Ugly plus pinned is safer than pretty plus guessed.

If you already have free model access, attach the pin file before any extract prompt. The model should chase the suite, not a vibe.