A Four-File Behavior Tape Before One Parser Split

Characterization tests belong in front of every extract. A messy exporter still has observable outputs today. Freeze those outputs, then change one parser call.

Untested extracts invent hidden contracts from developer memory. Memory is not a durable regression suite. Three output channels usually drift during a later split.

Those channels are payloads, process streams, and files. A unit test on one dict can miss the rest. A behavior tape should record all three together.

Why a tape beats a partial test

Most legacy scripts print a summary and write files. Callers depend on both sides of that pair. A green unit test can still break the nightly job.

Feathers named this pattern characterization testing for legacy code. The pattern pins current behavior, including known bugs. It does not certify that behavior as correct.

This article proposes an unexecuted local harness for exporters. It uses a tiny invoice exporter as the example. No production timings or customer counts appear here.

The artifact: four gold files

The tape is a directory, not a mock maze. Each recorded run writes exactly four gold files. Git then treats that directory as the contract.

The four files are exit.code, stdout.txt, stderr.txt, and files.sha256. Together those four files describe the command surface. They do not describe any internal helper names.

Step 1: Isolate one public command

Pick the command that operators already type in shells. Do not start inside a private 400-line helper. The public command is the real contract surface.

# proposed example only; not a production run
python tools/export_invoices.py --in fixtures/in --out /tmp/export

That command prints one summary line to stdout. It writes one JSON file per invoice. Both of those outputs belong in the tape.

Step 2: Freeze a tiny fixture tree

Copy two anonymized invoices into the fixtures/in tree. Strip secrets, live URLs, and wall-clock timestamps from them. Inject a fixed report date through the environment.

mkdir -p fixtures/in fixtures/gold
export REPORT_DATE=2026-01-15
export PYTHONHASHSEED=0
export TZ=UTC

Two invoices open more branches than a single invoice. Thousands of invoices hide mismatches inside raw volume. Keep that fixture small, legal, and fully deterministic.

Step 3: Record the tape once

The recorder below is proposed example code only. It launches the messy exporter as a subprocess. It then hashes every file under --out.

# record_tape.py — proposed example, unexecuted in this article
from __future__ import annotations

import argparse
import hashlib
import os
import subprocess
import sys
from pathlib import Path


def sha256_file(path: Path) -> str:
    digest = hashlib.sha256()
    with path.open('rb') as handle:
        for chunk in iter(lambda: handle.read(65536), b''):
            digest.update(chunk)
    return digest.hexdigest()


def write_manifest(out_dir: Path, manifest: Path) -> None:
    lines = []
    for path in sorted(out_dir.rglob('*')):
        if path.is_file():
            rel = path.relative_to(out_dir).as_posix()
            lines.append(sha256_file(path) + '  ' + rel)
    manifest.write_text('\n'.join(lines) + '\n', encoding='utf-8')


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument('--gold-dir', default='fixtures/gold')
    parser.add_argument('--out-dir', default='fixtures/out')
    args = parser.parse_args()

    gold = Path(args.gold_dir)
    out = Path(args.out_dir)
    gold.mkdir(parents=True, exist_ok=True)
    out.mkdir(parents=True, exist_ok=True)

    cmd = [
        sys.executable,
        'tools/export_invoices.py',
        '--in',
        'fixtures/in',
        '--out',
        str(out),
    ]
    env = os.environ.copy()
    env.setdefault('REPORT_DATE', '2026-01-15')
    env.setdefault('PYTHONHASHSEED', '0')
    env.setdefault('TZ', 'UTC')

    proc = subprocess.run(cmd, capture_output=True, env=env, check=False)
    (gold / 'exit.code').write_text(str(proc.returncode) + '\n', encoding='utf-8')
    (gold / 'stdout.txt').write_bytes(proc.stdout)
    (gold / 'stderr.txt').write_bytes(proc.stderr)
    write_manifest(out, gold / 'files.sha256')
    return 0


if __name__ == '__main__':
    raise SystemExit(main())

Run the recorder on a clean branch once. Commit the gold tape beside the messy script. Do not hand-edit gold files after that commit.

Step 4: Fail the build on tape drift

A second make target replays the same command. It writes a fresh tape under fixtures/got. diff -u must stay silent on all four files.

# proposed Makefile fragment; unexecuted in this article
export REPORT_DATE=2026-01-15
export PYTHONHASHSEED=0
export TZ=UTC

.PHONY: tape-record tape-check

tape-record:
    rm -rf fixtures/out
    python record_tape.py --gold-dir fixtures/gold --out-dir fixtures/out

tape-check:
    rm -rf fixtures/out fixtures/got
    python record_tape.py --gold-dir fixtures/got --out-dir fixtures/out
    diff -u fixtures/gold/exit.code fixtures/got/exit.code
    diff -u fixtures/gold/stdout.txt fixtures/got/stdout.txt
    diff -u fixtures/gold/stderr.txt fixtures/got/stderr.txt
    diff -u fixtures/gold/files.sha256 fixtures/got/files.sha256

Keep the Makefile boring, explicit, and easy to read. Hidden command flags create extra hidden output contracts. The command line in gold must match CI.

Byte comparison is the point of this tape. Pretty printers can hide real drift from reviewers. Do not normalize JSON unless callers also normalize it.

CI can call the same target without extra wrappers. A red tape is a failed job, not a warning. Do not allow skip flags on that job.

Step 5: Extract one parser only

Stop after the tape is green on mainline behavior. The smallest safe change is one parse function. File writes stay in the original exporter path.

# tools/export_invoices.py — proposed before/after sketch
from pathlib import Path
import json
import os
import sys


def parse_invoice(raw: str) -> dict:
    """Proposed extract. Keep side effects out of this function."""
    data = json.loads(raw)
    data['report_date'] = os.environ['REPORT_DATE']
    return data


def export_one(src: Path, dest_dir: Path) -> None:
    parsed = parse_invoice(src.read_text(encoding='utf-8'))
    dest = dest_dir / (parsed['id'] + '.json')
    dest.write_text(json.dumps(parsed, sort_keys=True) + '\n', encoding='utf-8')


def main() -> None:
    in_dir = Path(sys.argv[sys.argv.index('--in') + 1])
    out_dir = Path(sys.argv[sys.argv.index('--out') + 1])
    out_dir.mkdir(parents=True, exist_ok=True)
    count = 0
    for src in sorted(in_dir.glob('*.json')):
        export_one(src, out_dir)
        count += 1
    print('exported ' + str(count) + ' invoices')


if __name__ == '__main__':
    main()

Do not extract the writer in the same patch. Do not rename JSON keys in the same patch. One behavior-preserving split is the whole change.

Using sort_keys=True is itself a contract choice. Add it only if gold already has sorted keys. Otherwise the file hashes will fail on purpose.

Step 6: Replay the tape before merge

Run tape-check on the extract branch before merge. Matching hashes mean the parser split preserved outputs. A stdout mismatch means the summary line changed.

Do not rewrite gold to silence a failed tape. Restore the old call path instead of editing gold. Gold files move only when product intent moves.

Commit the extract only after the four files match. Keep the tape commit separate from the extract commit. Reviewers can then see behavior did not move.

Decision table

Observation after extract	Tape signal	Next action
Summary line gained a space	stdout.txt differs	Restore print; do not touch gold
JSON key order changed	files.sha256 differs	Match old serializer settings
Warning went to stderr	stderr.txt differs	Keep stderr empty or pin the warning
Process now returns 1	exit.code differs	Abort the extract
All four files match	no drift	Merge the single parser split
Dates shifted by one day	hashes and stdout differ	Pin REPORT_DATE and TZ
Only helper names changed	no drift	Safe; names are not the contract

Read the table left to right during review. The tape signal names the failing channel. The next action should be smaller than the last patch.

What the tape does not prove

Green gold files do not prove invoices are correct. They prove this edit did not change outputs. That is a smaller and more honest claim.

Locale, hash seed, and iteration order still leak. Pin all three in the recorder environment. Skip live HTTP in the same patch as the extract.

Property tests can wait until the parser is pure. Characterization comes first on these messy command surfaces. Mixing both in one pull request hides the cause.

Who should skip this method

Skip it when inputs cannot be made deterministic. Skip it when outputs contain secrets or personal data. Skip it when a tight unit suite already covers the command.

Skip it when the goal is a redesign, not a split. Freezing a bad tape cements the bug in CI. Delete the tape after a real specification exists.

Do not use this method to justify a large rewrite. Four gold files cannot supervise a 20-file move. Split one parser, then stop for the day.

A second reader on the tape diff

After the extract, the remaining question is narrow. Did stdout or hashes change in a caller-visible way? A second reader on the unified diff can help.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode provides free model access and a free server option. Either can review a redacted tape diff after the extract. They should not invent the parser or rewrite gold.

Redact invoice amounts and paths before any paste. Keep the model off the fixture tree. The four gold files remain the source of truth.

If a suggestion says to update gold, discard that suggestion. If a suggestion says to extract the writer too, stop. The next patch starts only after this tape is green.

This workflow still needs a human merge decision. No accuracy, quota, or model name is claimed here. The tape either matches or it does not.