Lock Output Contracts Before You Rewrite One Path

Blind refactors fail on brownfield code without contracts. Characterization of output shapes must precede any rewrite. Change one path only after those contracts stay green.

The actual failure mode

Messy modules mix parsing, I/O, and business rules. A cleanup often shifts exception types by accident. Callers then break without any compile-time error.

Exact byte snapshots also rot in practice. Timestamps and unordered maps create false failures. You need stable contracts, not frozen bytes.

This workflow targets one brownfield Python module. It does not require a full rewrite. It also does not require a new framework.

What you freeze

Freeze three observables on the public surface. Keep public function names under explicit test. Record output key sets and value types.

Record exception classes on documented failure inputs. Skip wall-clock fields during every comparison. Skip full tracebacks because they are not contracts.

Property-style checks survive ordinary formatting churn well. Byte-for-byte snapshots usually fail that test. That distinction is the entire method here.

Proposed contract matrix

The matrix below is a labeled proposal. Rename the functions to match your module. Treat the table as a planning artifact.

Probe	Public call	Fixture	Allowed keys	Value types	Exception
P1	load_report(path)	valid.json	id, rows, total	str, list, int	none
P2	load_report(path)	missing file	—	—	FileNotFoundError
P3	load_report(path)	truncated.json	—	—	ValueError
P4	summarize(report)	two-row dict	count, total	int, int	none
P5	summarize(report)	empty rows	count, total	int, int	none
P6	summarize(report)	missing total	—	—	KeyError

Six probes are enough for a first pass. Add probes only when a caller depends on them. Do not encode private helpers in this matrix.

Proposed harness

The following code is an unexecuted example. Copy it into tests/test_contracts.py after review. Adjust imports to match your package layout.

# Unexecuted example: tests/test_contracts.py
from __future__ import annotations

from pathlib import Path
from typing import Any, Callable

import pytest

from reports import load_report, summarize

FIXTURES = Path(__file__).parent / "fixtures"

MATRIX = [
    {
        "id": "P1",
        "call": lambda: load_report(FIXTURES / "valid.json"),
        "keys": ["id", "rows", "total"],
        "types": {"id": str, "rows": list, "total": int},
        "exc": None,
    },
    {
        "id": "P2",
        "call": lambda: load_report(FIXTURES / "nope.json"),
        "keys": None,
        "types": None,
        "exc": FileNotFoundError,
    },
    {
        "id": "P3",
        "call": lambda: load_report(FIXTURES / "truncated.json"),
        "keys": None,
        "types": None,
        "exc": ValueError,
    },
    {
        "id": "P4",
        "call": lambda: summarize(
            {"id": "a", "rows": [{}, {}], "total": 10}
        ),
        "keys": ["count", "total"],
        "types": {"count": int, "total": int},
        "exc": None,
    },
    {
        "id": "P5",
        "call": lambda: summarize(
            {"id": "a", "rows": [], "total": 0}
        ),
        "keys": ["count", "total"],
        "types": {"count": int, "total": int},
        "exc": None,
    },
    {
        "id": "P6",
        "call": lambda: summarize({"id": "a", "rows": []}),
        "keys": None,
        "types": None,
        "exc": KeyError,
    },
]


@pytest.mark.parametrize("probe", MATRIX, ids=lambda p: p["id"])
def test_contract(probe: dict[str, Any]) -> None:
    fn: Callable[[], Any] = probe["call"]
    expected_exc = probe["exc"]
    if expected_exc is not None:
        with pytest.raises(expected_exc):
            fn()
        return
    result = fn()
    assert isinstance(result, dict)
    assert sorted(result) == sorted(probe["keys"])
    for key, expected_type in probe["types"].items():
        assert isinstance(result[key], expected_type)

Store the happy-path fixture beside the test. Keep it small and committed. A proposed tests/fixtures/valid.json looks like this.

{
  "id": "r1",
  "rows": [{"sku": "a", "n": 2}, {"sku": "b", "n": 3}],
  "total": 5
}

Add truncated.json as a cut-off object. Leave nope.json uncreated on purpose. Missing files are a first-class probe.

Run the harness before any production edit. A red suite means the matrix is wrong. Fix the probes before you touch production code.

Numbered workflow

Step 1 — Inventory the public surface

List every function imported by other packages. Ignore private helpers with a leading underscore. Write that list into the matrix first column.

If the module has no clear public surface, stop. Define the exported API in one file. Characterization needs a boundary you can name.

Step 2 — Capture shapes, not payloads

Call each probe with a fixture from version control. Record sorted result keys for every dict return. Record the runtime type name for each key.

For sequences, record element type and length bounds. Do not record full row contents in this pass. Contents belong in unit tests after the refactor.

Step 3 — Encode exceptions as probes

Failure paths are the usual regression site. Pass the bad fixture into the same harness. Assert the exception class, not the message text.

Messages change during cleanup without semantic change. Classes usually represent the real caller contract. Keep that rule consistent across the matrix.

Step 4 — Prove red, then prove green

Break one probe on purpose before trusting it. Confirm the test runner reports that probe. Restore the probe and demand a green run.

A harness that cannot fail is not a harness. This check is mandatory before any edits. Skip it and you will trust noise.

Step 5 — Choose the smallest safe change

Use the decision table in the next section. Touch one production file when that is possible. Touch one function when the file is huge.

Do not rename public functions in this pass. Do not reorder raised exception types either. Those changes are new contracts, not refactors.

Step 6 — Optional model pass after green tests

A frozen matrix lets a model edit under constraint. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. Use that option only after the harness is green.

Ask the model to rewrite one internal path. Forbid public signature changes in the same prompt.

Keep the prompt short and strictly mechanical. Paste the matrix and the single target function. Reject any diff that touches extra files.

A proposed prompt follows. It is not a vendor script. Swap the function name for yours.

Rewrite only _parse_rows in reports.py.
Do not change load_report or summarize signatures.
Do not change exception classes from the contract matrix.
Leave tests/test_contracts.py untouched.
Return a diff limited to reports.py.

If you need a proposer, try that free access on a throwaway branch. This model step is optional, not required.

The harness works with a careful human edit. The model is a proposer, not an oracle.

Step 7 — Re-run, then inspect the diff

Re-run the six probes after the patch. Any exception-class drift is an automatic reject. Any missing output key is an automatic reject.

Then inspect the git diff stat output next. Extra files mean the change was not small. Reset and narrow the edit or the prompt.

Decision table: is the change small enough?

Read the table before you open a pull request. Small is a checklist, not a feeling. If two signals say stop, split the change.

Signal	Proceed	Stop and split
Public function names unchanged	yes	no
Exception classes unchanged	yes	no
Output keys are a superset	yes	no
Output keys lost a field	no	yes
Only one production file changed	yes	no
Tests outside the matrix also failed	no	yes
New I/O calls appeared	inspect	usually split

Superset keys are allowed in this method. Callers that ignore unknown keys keep working. Lost keys are a breaking change in this method.

Worked command sequence

These commands assume a local virtualenv only. They are a proposed sequence for the harness. They are not measured performance benchmarks.

python -m venv .venv
source .venv/bin/activate
pip install pytest
pytest tests/test_contracts.py -q
# edit one function in reports.py
pytest tests/test_contracts.py -q
git diff --stat
git diff -- reports.py

If pytest already exists, skip the install step. Do not add new test frameworks for this pass. One runner keeps the characterization signal cheap.

A second useful check is path scope. Limit git diff to the intended file. Anything else is out of contract.

git diff --name-only | grep -v '^reports.py$' && echo OUT_OF_SCOPE

What this does not prove

Green contracts do not prove functional correctness. They prove the public shape did not drift. Behavior inside a value can still be wrong.

They also miss most concurrency bugs entirely. They miss filesystem races on real disks. Add specialized tests if those risks matter.

Numeric totals can keep a type and still lie. Add arithmetic unit tests after the shape is stable. Do not overload the contract matrix with math.

The matrix also ignores log lines and metrics. Those streams drift during cleanup work. Keep them out of the first freeze.

Who should not use this approach

Do not use this on a greenfield module. Write real unit tests there from day one. Characterization is for inherited, under-tested code.

Do not use this on security-sensitive parsers alone. Shape tests will not catch injection flaws. Use dedicated security review for those paths.

Do not use this if you must change the public API. That work needs an explicit migration plan. This method assumes the contract stays put.

Skip the model pass on proprietary code without review. A free server is still a remote machine. Keep secrets and customer data off that path.

Closing constraint

The core rule is small and strict. Freeze output contracts before you rewrite one path. Reject any patch that moves the matrix.

Keep the harness in charge after every edit. The tests decide the outcome, not the diff. That split is the only safety signal.