Build an Outcome Grid, Then Relocate a Single Function

Characterize public outcomes before you change any structure. Relocate one function only after every grid cell matches. A structural edit without that grid remains an untested guess.

This workflow targets one messy Python module with mixed returns. It does not replace integration tests or production traffic replay. It freezes observable outcomes so a tiny move stays honest.

What an outcome grid records

An outcome grid is a table of public results. Each row binds one callable to one fixture. Each cell stores kind, stable repr, and error prefix.

Golden files freeze bytes on disk after a command. CLI tapes freeze argv and stdout for a whole process. This grid freezes in-process return paths and exception types instead.

Use it when helpers hide inside a module that callers already import. Skip it when the only contract is a CLI or a file format. You should pick one characterization style per seam. Do not stack three oracles on the same change.

The fixture contract

Keep fixtures local, tiny, and fully deterministic. Ban wall-clock timestamps and all live network calls. Ban unordered set iteration when order affects the repr.

Label every fixture with the input class it represents. Empty path, relative path, missing key, and duplicate key are four classes. Cover at least one raising class per public function.

Store fixtures as JSON, not as Python literals in tests. JSON keeps the corpus editable without importing the messy module. It also avoids executing module import side effects during corpus review.

1. Inventory public callables

List the names that external callers already import. Ignore underscore helpers until a later focused pass. Write the list by hand if the module is small.

# proposal: inventory.py — unexecuted example
PUBLIC = [
    "normalize_path",
    "merge_meta",
    "render_status",
]

Do not auto-discover every function with dir(). Auto-discovery will pin private helpers you plan to delete. A handwritten list is the public contract you keep.

2. Encode fixture rows

Each row names a function, args, and kwargs. Keep every fixture value strictly JSON-serializable for the corpus. Use strings for paths inside each row. Use JSON null for any omitted kwargs.

[
  {"id": "np-empty", "fn": "normalize_path", "args": [""], "kwargs": {}},
  {"id": "np-rel", "fn": "normalize_path", "args": ["./tmp/../x"], "kwargs": {}},
  {"id": "mm-dup", "fn": "merge_meta", "args": [{"a": 1}, {"a": 2}], "kwargs": {}},
  {"id": "rs-none", "fn": "render_status", "args": [null], "kwargs": {}}
]

Four fixture rows are enough to start. Add rows only when a cell stays ambiguous. An ambiguous cell shows the same repr for two different behaviors.

3. Run the characterization harness

The harness imports the target module only once. It calls each named function with the row payload. It captures return, exception type, message prefix, and stdio.

# proposal: outcome_grid.py — example harness
from __future__ import annotations

import importlib
import io
import json
from contextlib import redirect_stderr, redirect_stdout
from pathlib import Path
from typing import Any

MAX_PREFIX = 80


def stable_repr(value: Any) -> str:
    text = repr(value)
    if len(text) > 240:
        return text[:240] + "...<truncated>"
    return text


def run_row(mod, row: dict) -> dict:
    fn = getattr(mod, row["fn"])
    stdout = io.StringIO()
    stderr = io.StringIO()
    try:
        with redirect_stdout(stdout), redirect_stderr(stderr):
            result = fn(*row["args"], **row["kwargs"])
        return {
            "id": row["id"],
            "fn": row["fn"],
            "kind": "return",
            "type": type(result).__name__,
            "repr": stable_repr(result),
            "err_prefix": "",
            "stdout": stdout.getvalue()[:MAX_PREFIX],
            "stderr": stderr.getvalue()[:MAX_PREFIX],
        }
    except Exception as exc:
        return {
            "id": row["id"],
            "fn": row["fn"],
            "kind": "raise",
            "type": type(exc).__name__,
            "repr": "",
            "err_prefix": str(exc)[:MAX_PREFIX],
            "stdout": stdout.getvalue()[:MAX_PREFIX],
            "stderr": stderr.getvalue()[:MAX_PREFIX],
        }


def write_grid(module_name: str, fixtures_path: Path, out_path: Path) -> None:
    mod = importlib.import_module(module_name)
    rows = json.loads(fixtures_path.read_text(encoding="utf-8"))
    cells = [run_row(mod, row) for row in rows]
    out_path.write_text(
        json.dumps(cells, indent=2, sort_keys=True),
        encoding="utf-8",
    )


if __name__ == "__main__":
    write_grid("messy", Path("fixtures.json"), Path("grid.before.json"))

Redirect stdio so print debugging does not leak into cells. Truncate prefixes so message noise does not fail the diff. Sort JSON keys so the file is stable in git.

4. Freeze the before grid

Commit grid.before.json together with the fixture corpus. Do not edit frozen cells by hand later. If a cell contains a timestamp, delete that fixture class.

Run the harness twice on an unchanged tree. The two output files must be byte identical. A mismatch here means the module is not yet characterizable.

python outcome_grid.py
cp grid.before.json grid.repeat.json
python -c "from pathlib import Path; a=Path('grid.before.json').read_bytes(); b=Path('grid.repeat.json').read_bytes(); raise SystemExit(0 if a==b else 1)"

Stop if that exit code is 1. Then find the non-deterministic field in the cell. Remove it from the cell schema or from the fixture.

5. Relocate one function only

Pick the function with the fewest public cells. Move its body into a new dedicated module. Re-export the same name from the old module.

# proposal: paths.py — new home
def normalize_path(raw: str) -> str:
    if raw == "":
        raise ValueError("empty path")
    # existing body copied verbatim
    return raw


# messy.py — keep the import surface
from paths import normalize_path

Do not rename any arguments during this pass. Do not fix edge cases in this pass. Do not reformat unrelated functions in this pass. That single relocation is the change under test.

6. Diff cells, not source trees

Re-run the harness and write a grid.after.json file. Compare cells by id, not by file order. Fail on kind, type, repr, or err_prefix drift.

# proposal: diff_grid.py — example checker
import json
from pathlib import Path

KEYS = ("kind", "type", "repr", "err_prefix", "stdout", "stderr")


def load(path: str) -> dict:
    rows = json.loads(Path(path).read_text(encoding="utf-8"))
    return {row["id"]: row for row in rows}


def diff(before: str, after: str) -> int:
    left = load(before)
    right = load(after)
    failed = 0
    for key in sorted(set(left) | set(right)):
        if key not in left or key not in right:
            print(f"MISSING {key}")
            failed += 1
            continue
        for field in KEYS:
            if left[key][field] != right[key][field]:
                print(f"DRIFT {key}.{field}")
                print(f"  before={left[key][field]!r}")
                print(f"  after={right[key][field]!r}")
                failed += 1
    return failed


if __name__ == "__main__":
    raise SystemExit(diff("grid.before.json", "grid.after.json"))

A green diff means callers still see the same outcomes. A red diff names the exact fixture class that broke. Named cells are the entire point of this grid.

How to read a red cell

Kind drift from return to raise is a behavior change. Type drift from dict to list is a behavior change. Repr drift with the same type is often formatting or key order.

Err_prefix drift is usually a rewritten exception message. Callers that match on message text will notice. Callers that match on exception type may not notice. Record both so you can choose later.

Stdout drift after a pure extract means a print snuck in. Remove the print or accept it as a new contract. Do not ignore stdio cells during an extract.

Decision table

Seam you must keep	Prefer	Avoid on this pass
Public in-process functions	Outcome grid	Whole-repo formatters
Files written to disk	Content hashes	Exception message prefixes
CLI argv and stdout	Process tape	Importing internals
HTTP JSON bodies	Schema snapshots	Relocating two handlers

Use one row from that table per change. Mixing oracles hides which contract actually broke down. The smallest safe change has one oracle and one edit.

After the grid is green

A locked grid is a cheap review surface for a coding model. Paste only the function body and the green cells. Ask for a relocation plan, not a rewrite.

MonkeyCode offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The free server can run the harness while you inspect drifting cells. The free model can draft the re-export after the before-grid is committed.

Do not ask any model to invent fixtures. Models guess input classes and miss raising paths. You still own the corpus and the cell schema. Treat model output as a patch candidate, then re-run the diff.

If you try this split, keep the grid in git. The committed cells are the only durable record. The chat log is not a record.

Limitations

This harness cannot freeze live object identity at all. Live sockets and open files will not round-trip through repr. Replace those returns with a summary record before characterization.

Hash randomization can shuffle dict order in older runs. Prefer sort_keys on JSON and explicit key lists in repr helpers. Do not pin PYTHONHASHSEED as your only control. Use it only if the module already requires that seed.

Float formatting and timezone names will drift across platforms. Exclude those fields or normalize them in stable_repr. A grid that fails on two laptops is not yet a contract.

Import-time side effects will poison the inventory step. If import messy writes files, wrap the import in a temp cwd. If it reads env vars, set them in the harness. Do not hide those values in your shell profile.

This method does not prove any thread safety. It also does not prove any runtime performance claims. It does not prove that private helpers remain correct. It only proves that listed public outcomes stayed stable.

Who should not use this

Do not use this grid to redesign behavior and structure together. Split those two jobs into separate patches. Behavior changes need new fixtures and an explicit cell update. Structure changes need a green and unchanged grid.

Skip this approach when the module has no known callers. There is no public list to inventory. Write a real unit test for the new API instead.

Skip it for binary protocols and GUI event loops. Repr cells will only be noise there. Prefer byte hashes or recorded event traces there.

Skip it if you cannot run the module twice with identical results. Fix that determinism problem before you write fixtures. Characterization of a moving target wastes the corpus.

Close

Lock outcome cells for the functions you still export. Then relocate only one function body in git. Re-run the outcome grid after that single move. Anything larger is a second change, and it needs a second freeze.