Spend a Freeze Slot Only After Property and Fixture Both Fail

A skip list is not a test strategy. If an agent patch can add checks faster than reviewers can read them, the merge gate needs a freeze budget and a promotion path, not another unmarked skip.

The rule is operational. Every new or newly flaky test must try to become a property, then a pinned fixture, and only then a time-boxed freeze. Two dated failures of that path are the price of one freeze slot.

Why generated suites lose signal

Agent patches often ship extra tests. Some encode a real invariant. Others pin a snapshot the next patch will rewrite. A third group fails for causes the diff does not own: clocks, shared tempdirs, sandbox 429s.

CI that treats those three as equal failures trains reviewers to freeze the noisy ones. The freeze list then grows faster than the suite. Coverage stays high. Signal does not.

Cheap generation makes the labor mismatch worse. A model can emit thirty tests in one pass. Review time does not scale with that output. The bottleneck is classification work, not inference cost.

Three lanes, one repository budget

Define lanes before you score the patch. Caps belong on the repository, not on a single diff.

1. Property — a predicate over a family of inputs. It does not read the patch text, the agent transcript, or a golden file the model just wrote.
2. Fixture — one accepted input/output pair behind a lock hash. The lock changes only in a reviewed update. The agent cannot write it.
3. Freeze — a named exemption with owner, reason, and expiry. It is residual capacity. It is not the default lane.

A starting budget for a medium service is a proposal, not a measured benchmark: 12 properties, 40 fixtures, 3 freezes. Tune from CI wall-clock and review queue length. Do not tune from a vendor slide.

Symptom	Lane	Merge rule
Fails on random input, same predicate	Property	Must pass. No skip.
Fails on one golden blob the patch owns	Fixture	Pass, or update the lock in the same diff.
Fails after two promotion attempts, cause outside the diff	Freeze	Occupies one budget slot until expiry.
Fails once, no promotion log	Unclassified	Block merge.
Freeze with no owner or no expiry	Invalid	Block merge.

Promotion protocol

Run this sequence on every test that is new, renamed, or newly flaky. Do not start at freeze.

1. Attempt a property. Rewrite the check against a generator. The predicate must not import the diff.
2. Record the result. If it holds for the agreed example count, keep the property and delete the brittle original.
3. Attempt a fixture pin. If the behavior is a single accepted tuple, hash input and output. Store the hash where the agent cannot write.
4. Record isolation work. If output still drifts after pinning time, RNG, and network, write down the steps that were tried.
5. Spend a freeze slot. Only after two dated promotion records may a freeze occupy the ledger. Owner is mandatory. Expiry is mandatory.

Two is a policy constant. One attempt is usually a shrug. Three delays merges without extra signal. Change the constant in the lane file, not in a review comment.

Artifact: a ledger gate

The harness below is a proposed merge check. It is not fleet telemetry. Keep lane_gate.py and the budget map on a path the agent is not allowed to edit.

#!/usr/bin/env python3
"""lane_gate.py — classify agent-patch tests and cap the freeze lane."""

from __future__ import annotations

import json
import sys
from datetime import date, datetime
from pathlib import Path
from typing import Any

LANES = {"property", "fixture", "freeze"}
BUDGET = {"property": 12, "fixture": 40, "freeze": 3}
MIN_PROMOTIONS = 2


def parse_day(value: str) -> date:
    return datetime.strptime(value, "%Y-%m-%d").date()


def load_ledger(path: Path) -> dict[str, Any]:
    data = json.loads(path.read_text(encoding="utf-8"))
    if "tests" not in data or not isinstance(data["tests"], list):
        raise ValueError("ledger must contain a tests array")
    return data


def validate(entry: dict[str, Any], today: date) -> list[str]:
    errors: list[str] = []
    name = entry.get("name") or "<unnamed>"
    lane = entry.get("lane")
    if lane not in LANES:
        errors.append(f"{name}: lane must be property|fixture|freeze")
        return errors

    if lane == "property":
        if not entry.get("predicate"):
            errors.append(f"{name}: property requires a predicate id")
        if entry.get("reads_diff"):
            errors.append(f"{name}: property must not read the patch diff")

    if lane == "fixture":
        if not entry.get("lock_hash"):
            errors.append(f"{name}: fixture requires lock_hash")
        if entry.get("lock_writable_by_agent"):
            errors.append(f"{name}: fixture lock must be agent-unwritable")

    if lane == "freeze":
        promotions = entry.get("promotions") or []
        if not entry.get("owner"):
            errors.append(f"{name}: freeze requires owner")
        if not entry.get("reason"):
            errors.append(f"{name}: freeze requires reason")
        expiry = entry.get("expiry")
        if not expiry:
            errors.append(f"{name}: freeze requires expiry")
        else:
            try:
                if parse_day(expiry) <= today:
                    errors.append(f"{name}: freeze expired on {expiry}")
            except ValueError:
                errors.append(f"{name}: expiry must be YYYY-MM-DD")
        dated = [p for p in promotions if p.get("at") and p.get("lane_attempted")]
        if len(dated) < MIN_PROMOTIONS:
            errors.append(
                f"{name}: freeze needs {MIN_PROMOTIONS} dated promotion attempts, got {len(dated)}"
            )
        attempted = {p.get("lane_attempted") for p in dated}
        if "property" not in attempted or "fixture" not in attempted:
            errors.append(f"{name}: promotions must include property and fixture")
    return errors


def enforce(ledger: dict[str, Any], today: date) -> int:
    errors: list[str] = []
    counts = {lane: 0 for lane in LANES}
    seen: set[str] = set()

    for entry in ledger["tests"]:
        name = entry.get("name")
        if not name:
            errors.append("test entry missing name")
            continue
        if name in seen:
            errors.append(f"{name}: duplicate ledger entry")
        seen.add(name)
        errors.extend(validate(entry, today))
        lane = entry.get("lane")
        if lane in counts:
            counts[lane] += 1

    for lane, cap in BUDGET.items():
        if counts[lane] > cap:
            errors.append(f"{lane} lane over budget: {counts[lane]} > {cap}")

    for line in errors:
        print(f"LANE_GATE: {line}", file=sys.stderr)
    if errors:
        print(f"LANE_GATE: blocked ({len(errors)} error(s); counts={counts})", file=sys.stderr)
        return 1
    print(f"LANE_GATE: ok counts={counts}")
    return 0


def main() -> int:
    if len(sys.argv) != 2:
        print("usage: python lane_gate.py tests/lane_ledger.json", file=sys.stderr)
        return 2
    path = Path(sys.argv[1])
    try:
        ledger = load_ledger(path)
    except (OSError, ValueError, json.JSONDecodeError) as exc:
        print(f"LANE_GATE: cannot read ledger: {exc}", file=sys.stderr)
        return 2
    return enforce(ledger, date.today())


if __name__ == "__main__":
    raise SystemExit(main())

Sample ledger. The freeze row is valid only because both promotion lanes were attempted and dated.

{
  "tests": [
    {
      "name": "test_balance_non_negative",
      "lane": "property",
      "predicate": "balance >= 0",
      "reads_diff": false
    },
    {
      "name": "test_invoice_json_v3",
      "lane": "fixture",
      "lock_hash": "sha256:4f2c9a1b",
      "lock_writable_by_agent": false
    },
    {
      "name": "test_rates_endpoint_occasional_timeout",
      "lane": "freeze",
      "owner": "payments-oncall",
      "reason": "sandbox 429s; isolation attempted with a cassette",
      "expiry": "2026-09-17",
      "promotions": [
        {
          "at": "2026-09-01",
          "lane_attempted": "property",
          "result": "predicate depends on remote clock"
        },
        {
          "at": "2026-09-02",
          "lane_attempted": "fixture",
          "result": "payload hash drifts across sandbox seeds"
        }
      ]
    }
  ]
}

Run the gate against HEAD, not against a ledger the agent rewrote in the same patch without a human lock.

python lane_gate.py tests/lane_ledger.json
echo $?

List freeze expiry without opening the file in an editor:

python - <<'PY'
import json
from pathlib import Path
ledger = json.loads(Path("tests/lane_ledger.json").read_text(encoding="utf-8"))
for row in ledger["tests"]:
    if row.get("lane") != "freeze":
        continue
    print(f"{row['name']}\t{row.get('owner')}\t{row.get('expiry')}")
PY

A collector guard can refuse anonymous tests. Skipping them is the wrong default for a high-assurance repo. Fail collection instead, or mark unknown names as strict xfail. The invariant is the same: no unlaned test reaches green.

# conftest.py — proposed collector guard
import json
from pathlib import Path
import pytest

LEDGER = Path(__file__).with_name("lane_ledger.json")

def pytest_collection_modifyitems(config, items):
    ledger = json.loads(LEDGER.read_text(encoding="utf-8"))
    known = {row["name"] for row in ledger["tests"]}
    unknown = []
    for item in items:
        short = item.nodeid.split("::")[-1]
        if item.name not in known and short not in known:
            unknown.append(item.nodeid)
    if unknown:
        pytest.exit("unlaned tests: " + ", ".join(unknown), returncode=1)

Where generation belongs

Candidate properties still need a draft. That draft can come from a local template or from a model. The gate must not call the model.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. Those two facts are useful only for emitting candidate predicates, fixture-hash requests, or freeze reasons without standing up paid inference for the brainstorming step. A model may propose a property id. It may not increment BUDGET, stretch expiry, or set lock_writable_by_agent to true.

If you already review agent diffs, keep the ledger in the same PR as the patch. Generate candidates on a free server if that is what you have. Merge only what the gate accepts.

Limitations

This protocol assumes stable test names and a human owner for every freeze. It will not help a tree of one-off scripts with random filenames. It will fight you if properties secretly read golden files through a helper.

Do not use a freeze budget on code that cannot tolerate known-failing checks. A freeze is residual risk with a date. If the domain forbids residual risk, the freeze cap is zero.

Do not treat the ledger as a substitute for hermetic CI. If flakes come from shared mutable state, promotion to property will keep failing for the right reason. Fix the environment first.

The sample caps are policy seeds, not public benchmarks. If python lane_gate.py is green while production is red, the ledger is lying. Treat that as a gate bug. Do not raise BUDGET["freeze"] to hide it.

Who should skip this

Solo prototypes with a handful of tests do not need a freeze ledger. Teams that already pin every behavior as a reviewed fixture and never skip can ignore the freeze lane. Shops that let agents edit CI config should not adopt this until those paths are locked.

The conclusion does not change with the generator. Promote first. Pin second. Freeze last, and only inside a number you can explain.