Freeze the Error Contract Before One except Change

Mixed None-and-raise APIs fail under model rewrites. The error taxonomy is the real product surface. Pin that taxonomy with characterization tests first, always.

Change one except clause only after the pin stays green. Happy-path unit tests will not save this refactor. Callers already branch on None, dict keys, and types.

Collapse any one of those shapes and production breaks. The defect is an unrecorded taxonomy, not missing types.

Why model rewrites miss the contract

AI coding threads keep offering full-file cleanups. Those cleanups prefer one error type. They also prefer raising over returning None.

That preference is style, not evidence. A messy dispatcher often has three outcomes. Some inputs raise ValueError for bad payloads.

Some inputs return None after a send failure. Some inputs return {"ok": False, "status": 429}. Downstream code already checks all three shapes.

Loop-style agent edits make the same cut. They unify handlers because duplication looks sloppy. Duplication here is the published contract.

Four observables to freeze

Record these four fields for every fixture. Skip prose messages on the first pass. Message strings drift across harmless edits.

1. Escaping exception type, or a sentinel if none.
2. Return shape: None, mapping keys, or other.
3. Integer status when a mapping returns.
4. Count of WARNING-or-higher log records.

Types and keys usually stay stable. Status integers also stay stable. Log counts need a named logger, not the root.

Decision table for one dispatcher

The table below is the spec. It is a labeled example. It is not a production trace.

Fixture	Escapes	Return	Status	WARN+
empty body	RuntimeError	n/a	n/a	0
invalid JSON	ValueError	n/a	n/a	0
JSON list	ValueError	n/a	n/a	0
missing id	none	None	n/a	1
send TypeError	none	None	n/a	1
send TimeoutError	none	None	n/a	1
downstream 429	none	dict	429	1
downstream 200	none	dict	200	0

Keep the send TypeError row. That row is the trap. A cleaner except often drops it.

Artifact: characterization harness

The code is a worked example. Run it locally before any extract. Do not treat it as measured field data.

# events.py — labeled example, not a live service
from __future__ import annotations

import json
import logging
from typing import Any, Callable

log = logging.getLogger("events")

SendFn = Callable[[dict], tuple[int, Any]]


def dispatch_event(raw: str | None, send: SendFn):
    """Messy contract: None, dict, or raise."""
    if raw is None or raw == "":
        raise RuntimeError("empty body")
    try:
        payload = json.loads(raw)
    except Exception:
        raise ValueError("bad json")
    if not isinstance(payload, dict):
        raise ValueError("bad json")
    if "id" not in payload:
        log.warning("missing id")
        return None
    try:
        status, body = send(payload)
    except Exception:
        log.warning("send failed")
        return None
    if status >= 400:
        log.warning("downstream %s", status)
        return {"ok": False, "status": status, "body": body}
    return {"ok": True, "status": status, "body": body}

# test_events_contract.py — worked example
import json
import logging

import pytest

from events import dispatch_event


def _records(caplog):
    return [r for r in caplog.records if r.name == "events" and r.levelno >= logging.WARNING]


def _run(raw, send, caplog):
    caplog.set_level(logging.WARNING, logger="events")
    try:
        value = dispatch_event(raw, send)
        return None, value, _records(caplog)
    except Exception as exc:
        return type(exc), None, _records(caplog)


def test_empty_body_raises_runtime_error(caplog):
    exc, value, recs = _run("", lambda p: (200, "ok"), caplog)
    assert exc is RuntimeError
    assert value is None
    assert len(recs) == 0


@pytest.mark.parametrize("raw", ["{", "[]", "null"])
def test_bad_json_raises_value_error(raw, caplog):
    exc, value, recs = _run(raw, lambda p: (200, "ok"), caplog)
    assert exc is ValueError
    assert value is None
    assert len(recs) == 0


def test_missing_id_returns_none(caplog):
    exc, value, recs = _run('{"name": "x"}', lambda p: (200, "ok"), caplog)
    assert exc is None
    assert value is None
    assert len(recs) == 1


def test_send_type_error_returns_none(caplog):
    def send(_payload):
        raise TypeError("broken client")

    exc, value, recs = _run('{"id": 1}', send, caplog)
    assert exc is None
    assert value is None
    assert len(recs) == 1


def test_send_timeout_returns_none(caplog):
    def send(_payload):
        raise TimeoutError("late")

    exc, value, recs = _run('{"id": 1}', send, caplog)
    assert exc is None
    assert value is None
    assert len(recs) == 1


def test_downstream_429_is_dict(caplog):
    exc, value, recs = _run('{"id": 1}', lambda p: (429, "slow"), caplog)
    assert exc is None
    assert value == {"ok": False, "status": 429, "body": "slow"}
    assert len(recs) == 1


def test_downstream_200_is_dict(caplog):
    exc, value, recs = _run('{"id": 1}', lambda p: (200, {"n": 1}), caplog)
    assert exc is None
    assert value == {"ok": True, "status": 200, "body": {"n": 1}}
    assert len(recs) == 0

Run the file before editing handlers. Use a short traceback during the first pin.

python -m pytest test_events_contract.py -q --tb=short

Prove the suite catches a collapsed taxonomy

A green suite with no mutation check is weak. You must see a failed unified-error fork. Keep that fork out of the merge.

# test_events_mutation.py — must fail against a unified raise rewrite
import events


def test_unified_raise_would_break_send_type_error(monkeypatch, caplog):
    def rewritten(raw, send):
        payload = __import__("json").loads(raw)
        status, body = send(payload)  # TypeError now escapes
        return {"ok": status < 400, "status": status, "body": body}

    monkeypatch.setattr(events, "dispatch_event", rewritten)
    with pytest.raises(TypeError):
        events.dispatch_event('{"id": 1}', lambda p: (_ for _ in ()).throw(TypeError("x")))

Expect this mutation test to fail on the rewrite. Restore the messy handler after that check. The failure is the evidence, not a vibe.

Numbered workflow

1. Copy the messy module into a branch. Do not edit handlers yet.
2. Inventory callers that check is None or catch types.
3. Fill the four-column table from those callers.
4. Encode each row as one test function.
5. Add one mutation test that unifies errors.
6. Confirm the mutation test fails on purpose.
7. Restore the original handler body.
8. Apply one extract or one except edit.
9. Re-run the same characterization file.
10. Revert if any row changes shape.

Step two is not optional. Tests invented from the callee miss caller branches. Caller branches are the contract.

rg -n "dispatch_event\(" -g "*.py"
rg -n "is None|except ValueError|except RuntimeError" -g "*.py"

Record each hit as a fixture name. Missing hits become missing rows. Missing rows make unsafe extracts look safe.

The smallest safe change

Do not narrow except Exception on send first. The table says TypeError becomes None. Narrowing that clause raises TypeError instead.

That raise is a contract break. Extract the block without narrowing. Keep the same log line and None return.

def _send_or_none(payload, send):
    try:
        return send(payload)
    except Exception:
        log.warning("send failed")
        return None


def dispatch_event(raw, send):
    if raw is None or raw == "":
        raise RuntimeError("empty body")
    payload = _parse_object(raw)
    if "id" not in payload:
        log.warning("missing id")
        return None
    result = _send_or_none(payload, send)
    if result is None:
        return None
    status, body = result
    if status >= 400:
        log.warning("downstream %s", status)
        return {"ok": False, "status": status, "body": body}
    return {"ok": True, "status": status, "body": body}

That extract is boring. Boring is the point. The taxonomy stays in the table.

A later change can narrow exceptions. Do that only with a version note. Add a row that expects TypeError to escape. Tell callers before you merge.

Parse-side except narrowing

The JSON branch is different. json.loads should raise json.JSONDecodeError. Re-raising ValueError is the published type.

Narrowing except Exception to except json.JSONDecodeError can be safe. Prove it with the invalid JSON row. Prove it with the JSON list row too.

def _parse_object(raw):
    try:
        payload = json.loads(raw)
    except json.JSONDecodeError:
        raise ValueError("bad json") from None
    if not isinstance(payload, dict):
        raise ValueError("bad json")
    return payload

Both bad-JSON fixtures must still raise ValueError. A leaked JSONDecodeError is a taxonomy change. Do not ship that leak without a caller audit.

from None also drops __cause__. Add a cause fixture if any caller reads it. Skip that fixture when no caller inspects causes.

Where a free coding model belongs

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A model can draft the extract. It must not invent a new taxonomy. MonkeyCode provides free model access and a free server option.

Use either only after the characterization file is green. Feed the model the table and the test file. Ask for one extract, not a rewrite.

Reject a patch that removes a row. Reject a patch that changes None into a raise. This is not an agent loop.

One prompt. One diff. Same tests.

Limitations

The harness does not prove semantic equality. It pins types, keys, status, and log counts. It misses timing, retry storms, and byte identity.

Log counts break if a library logs extra warnings. Pin the logger name events. Cap propagation on that logger.

Do not pin the root logger. Root pins go red on unrelated imports. That noise hides a real taxonomy drift.

The table is only as good as the fixtures. One unlisted caller path is an untested shape. Untested shapes are how "safe" extracts land in incident channels.

Who should not use this

Do not use this flow for greenfield APIs. Design one error shape there. Do not preserve None plus raises on purpose.

Do not use this flow for security boundaries. Characterization will pin insecure behavior. Pinning is not hardening.

Do not use this flow without tests you can run offline. A model server cannot replace the table. If pytest cannot collect locally, stop.

Teams with a published OpenAPI error schema may skip dict-key rows. Use the schema as the table instead. Still pin process-local exception escapes.

HTTP schemas often omit those local raises. Omitting them is how ValueError turns into a 500. Keep the escape column anyway.

Checklist before merge

1. Characterization file is the only new proof.
2. Diff touches one handler extract or one except.
3. Mutation test still fails against a unified-error fork.
4. No fixture row changed shape.
5. None callers still exist, or a version note ships.

If a model patch violates any line, drop the patch. Rewrite pressure is not evidence. The table is.

If the characterization file is already green, one extract prompt on the free server is enough. Skip that prompt when the table still has empty cells.