Pin JSON Bytes and Default Handlers Before One Serializer Extract

Messy repos hide json.dumps flags in dozens of call sites. A later helper extract then changes wire bytes without a failing test. Pin those bytes first, then extract one serializer function.

Reviewers rarely catch ensure_ascii flipping from True to False. sort_keys and separators also rewrite objects that look equal in Python. default handlers change datetime and Decimal encoding on the first deploy.

This workflow freezes dumps() outputs as UTF-8 bytes. It then allows one function extract and nothing else. Parsed dict equality is not a pin and must not gate the change.

The failure mode in a tangled module

Inline dumps() calls look harmless until a client parses key order. Some gateways hash the raw body and reject reordered objects. Some logs treat escaped Unicode as a new event class.

A typical messy module mixes three dumps dialects in one file. One call sorts keys for cache stability. Another omits spaces for a compact queue payload. A third ships ensure_ascii=True for an old HTTP stack.

Extracting to_json(data) without a byte pin merges those dialects. The merge often lands as a silent default of json.dumps. Downstream tests still pass because they decode JSON and compare Python objects.

What the pin must freeze

A useful pin records exact UTF-8 bytes, not parsed dicts. It also records the exception type dumps() raises on bad values. It records whether default= was present at each call site.

Capture these fields for every representative payload. Skip fields that the call site never observed in production traffic.

Field	Why it drifts	Pin as
sort_keys	Dict order is not JSON order	bytes
ensure_ascii	é versus \\u00e9	bytes
separators	Compact versus spaced bodies	bytes
default handler	datetime, Decimal, set	bytes or error type
allow_nan	NaN becomes non-JSON text	bytes or ValueError
skipkeys	Non-str keys vanish or raise	bytes or TypeError

Do not pin pretty-print indent unless a call site uses it. Do not pin Python dict equality after json.loads. Do not pin wall-clock timestamps inside payloads.

Artifact: a dumps pin harness

The harness below is a local example, not a measured production run. Place it next to the messy module. Keep fixtures in a committed directory so diffs stay reviewable.

# pin_json_bytes.py
from __future__ import annotations

import json
from dataclasses import dataclass
from datetime import datetime, timezone
from decimal import Decimal
from pathlib import Path
from typing import Any, Callable

FIXTURE_DIR = Path(__file__).parent / "json_pins"


@dataclass(frozen=True)
class DumpCase:
    name: str
    payload: Any
    dumps_kwargs: dict[str, Any]


def _default(value: Any) -> Any:
    if isinstance(value, datetime):
        return value.isoformat()
    if isinstance(value, Decimal):
        return str(value)
    raise TypeError(f"unpinned type: {type(value)!r}")


CASES = [
    DumpCase(
        "cache_key_sorted",
        {"b": 1, "a": 2},
        {"sort_keys": True, "separators": (",", ":")},
    ),
    DumpCase(
        "queue_compact_ascii",
        {"title": "café", "ok": True},
        {"ensure_ascii": True, "separators": (",", ":")},
    ),
    DumpCase(
        "audit_spaced",
        {"n": Decimal("1.50"), "at": datetime(2026, 9, 16, tzinfo=timezone.utc)},
        {"ensure_ascii": False, "default": _default},
    ),
]


def dump_bytes(case: DumpCase) -> bytes:
    text = json.dumps(case.payload, **case.dumps_kwargs)
    return text.encode("utf-8")


def write_pins() -> None:
    FIXTURE_DIR.mkdir(exist_ok=True)
    for case in CASES:
        path = FIXTURE_DIR / f"{case.name}.json.bin"
        path.write_bytes(dump_bytes(case))


def assert_pins(dumps_fn: Callable[..., str]) -> None:
    for case in CASES:
        path = FIXTURE_DIR / f"{case.name}.json.bin"
        expected = path.read_bytes()
        kwargs = dict(case.dumps_kwargs)
        got = dumps_fn(case.payload, **kwargs).encode("utf-8")
        if got != expected:
            raise AssertionError(
                f"{case.name}: pin drift {got!r} != {expected!r}"
            )

Record pins once from the current call sites. Commit the .json.bin files as binary fixtures. Later extracts must match those bytes with no whitespace drift.

# test_json_pins.py
import json
from pin_json_bytes import assert_pins, write_pins


def test_write_pins_is_manual_only() -> None:
    # Run write_pins() from a shell when capturing, not in CI.
    assert callable(write_pins)


def test_current_dumps_matches_committed_pins() -> None:
    assert_pins(json.dumps)

Capture command for the first pin set:

python -c "from pin_json_bytes import write_pins; write_pins()"
pytest test_json_pins.py -q
xxd json_pins/queue_compact_ascii.json.bin | head

The xxd check exists to catch UTF-8 versus escaped ASCII by eye. Do not trust a terminal print of the decoded object. Two payloads can loads() equal and still differ in bytes.

Numbered workflow

Follow the steps in order. Stop when a step fails. Do not extract during inventory.

1. Inventory dumps() call sites

Search the messy module with a single pattern. Record kwargs, not only the function name. Note any wrapper that already calls dumps().

rg -n "json\.dumps\(|dumps\(" -g "*.py" app/

Group sites that share identical kwargs into one candidate extract. Leave mixed-kwargs sites out of the first extract. Mixed kwargs are a second change and a second pin set.

2. Build one payload per dialect

Pick the smallest object that still trips each flag. Include Unicode, Decimal, datetime, True, and empty dict. Exclude live secrets and customer records from fixtures.

Name each case after the caller, not after the flag. Caller names survive later file moves. Flag names hide which product path broke.

3. Commit binary pins before any edit

Run write_pins() on the current tree only. Commit fixtures in the same branch as the tests. Do not regenerate pins after the extract lands.

If a pin file changes in git, the extract is too large. Revert the helper and split the dialect instead. Byte drift is a failed gate, not a fixture update.

4. Prove the pin fails on a known dialect mix

Break one flag on purpose before the real extract. This is a labeled probe, not a production patch.

# labeled probe: expect test_current_dumps_matches_committed_pins to fail
import json
from pin_json_bytes import assert_pins

def mixed_dumps(payload, **kwargs):
    kwargs.pop("sort_keys", None)
    kwargs["ensure_ascii"] = True
    return json.dumps(payload, **kwargs)

assert_pins(mixed_dumps)

The probe must fail on cache_key_sorted or queue_compact_ascii. If it stays green, the pin is comparing decoded objects. Fix the harness before touching production code.

5. Extract one serializer for one dialect

Move a single kwargs set into one function. Keep the function in the same module for the first patch. Do not rename keys inside payloads during this step.

def dumps_cache_key(payload: dict) -> str:
    return json.dumps(payload, sort_keys=True, separators=(",", ":"))

Point only matching call sites at dumps_cache_key. Leave queue and audit sites on raw json.dumps. Re-run pytest and the xxd spot check.

6. Diff the branch against the pin set

The allowed diff is the new function plus call-site swaps. Fixture files must stay binary-identical. Test files may grow assertions but must not rewrite pins.

git diff --stat
git diff -- json_pins/
pytest test_json_pins.py -q

A non-empty diff under json_pins/ means the extract changed bytes. Restore the helper and reduce the move. Do not refresh pins to match the new helper.

Where a free remote runner fits

Laptop Python builds can hide dumps() drift across versions. A second runtime is useful after the pin suite exists. It is not a substitute for committed fixtures.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option, which can draft the one-dialect helper after pins are green and run the same pytest suite off the laptop. Skip both if local pytest already isolates dumps() bytes.

Do not ask a model to regenerate pin files. Do not ask a model to merge dialects in one patch. Feed only the green pin tests and the single-kwargs extract goal.

Limits of a byte pin

Byte pins do not prove the JSON schema is correct. They only prove this extract did not change encodings. Schema drift needs a separate contract test.

They also fail on intentional pretty-print changes. If a human-readable admin dump must gain indent=2, that is a new dialect. Give it a new case name and a new extract.

Floating time fields will thrash binary fixtures. Freeze clocks in payloads before recording pins. Naive datetime objects are a dialect, not an accident to ignore.

Python version gaps can change nothing except implementation details. json.dumps output for these flags is stable on current CPython for the cases above. Still rerun pins when the runtime changes.

Who should not use this approach

Do not use byte pins for streaming JSON lines with timestamps. Do not use them when the payload includes unordered set iteration. Do not use them as a substitute for an HTTP contract test.

Skip this extract if every dumps() site already shares one kwargs dict. Skip it if the module ships only debug logs and no wire format. Skip it if legal review forbids committed payload shapes.

Teams without pytest or another byte-level runner should not extract yet. Install the runner and record pins first. An untested helper extract is still a dialect merge.

Close

Wire clients consume bytes, not Python dicts. Pin dumps() bytes for one dialect, then extract that dialect only. Leave every other json.dumps call untouched until its own pin exists.