Merge-Lane Tests Vote. Witness and Freeze Do Not.

A green pytest process is the wrong merge object for an agent patch. The suite mixed three kinds of evidence and then collapsed them into one exit code. That collapse is the defect, not the model that wrote the diff.

Split every test into a lane before the patch can land. Merge-lane tests vote. Witness tests corroborate a hash-locked fixture. Freeze tests keep a flake visible without granting a pass. If the merge lane is empty, the gate fails closed.

Why one exit code lies

Agent patches arrive with tests they influenced. Some of those tests restate the implementation. Some pin a golden file the agent also wrote. Some disappear behind an unconditional skip after a flake embarrassed a run. None of that is visible in pytest -q.

The object you need is a vote tally, not a process status. Count who is allowed to speak. Then count who is only allowed to record.

A skip deletes evidence. A freeze must not.

Three lanes, three rights

Assign every collected test exactly one marker. Refuse unmarked tests. The rights are not equal.

1. Merge lane — property checks and human-authored oracles. These may fail the patch. They are the only tests that can approve it.
2. Witness lane — hash-locked fixtures and characterization bytes. These may fail if the bytes move. They cannot be the only green signal.
3. Freeze lane — known flakes with a replay capsule. These run and record. They never vote.

The rule is mechanical. Merge count must be at least one. Witness count may be large. Freeze count is a queue length, not a quality score. If freeze outgrows merge, the suite is parking failures instead of specifying behavior.

Decision table

Use the table as the review comment, not as folklore.

Signal	Lane	May fail the patch?	May approve the patch?	Required attachment
Invariant over a generated domain	merge	yes	yes	domain + shrinking note
Human-written input/output pair	merge	yes	yes	owner id in the test docstring
Golden file / snapshot bytes	witness	yes, on digest change	no	SHA-256 of the file
Characterization of current behavior	witness	yes, on digest change	no	lock date
Intermittent failure with captured seed	freeze	no (non-voting)	no	seed, input digest, expiry
Unconditional skip	illegal	n/a	n/a	reject the job

Unmarked tests are illegal in this workflow. So are skips that hide a flake without a capsule.

Proposed workflow

The steps below are a procedure, not a measured production study. Label it that way in the PR template.

1. Ban @pytest.mark.skip on agent-patch CI. Map every skip to freeze or delete it.
2. Mark remaining tests merge, witness, or freeze. Fail collection if a test has zero or two of those markers.
3. Lock witness files by digest. The lock file is the oracle, not the filename.
4. Require a freeze ledger row for every freeze test: seed, input digest, expiry date, last replay job URL.
5. Run the merge lane on an isolated runner. Do not let the laptop clock or local entropy mint the vote.
6. Tally votes. Approve only when merge passed, witness digests matched, and freeze did not exceed the budget.

The order matters. Properties first. Fixtures second. Freeze last. The agent does not write freeze tickets.

Marker contract

Keep the contract in conftest.py. Collection is the right layer. Runtime is too late.

# conftest.py
import pytest

LANES = frozenset({"merge", "witness", "freeze"})

def pytest_collection_modifyitems(config, items):
    unmarked = []
    doubled = []
    for item in items:
        present = [m for m in LANES if item.get_closest_marker(m)]
        if len(present) == 0:
            unmarked.append(item.nodeid)
        elif len(present) > 1:
            doubled.append(item.nodeid)
    if unmarked or doubled:
        lines = ["lane markers are mandatory and exclusive"]
        if unmarked:
            lines.append("unmarked:\n  " + "\n  ".join(unmarked))
        if doubled:
            lines.append("multiple lanes:\n  " + "\n  ".join(doubled))
        pytest.exit("\n".join(lines), returncode=2)

That exit is fail-closed. A missing marker is not a default merge vote.

Merge-lane property, not a tautology

A merge test must be able to fail a plausible wrong patch. Round-trip, inverse, and monotonicity checks usually can. Equality against the function under test usually cannot.

# tests/test_invoice_merge.py
import pytest
from hypothesis import given, settings
from hypothesis import strategies as st

from invoice import apply_credit, invert_credit

amounts = st.decimals(min_value="0.01", max_value="1000000", places=2)

@pytest.mark.merge
@settings(max_examples=80, deadline=None)
@given(total=amounts, credit=amounts)
def test_credit_round_trip(total, credit):
    applied = apply_credit(total, credit)
    restored = invert_credit(applied, credit)
    assert restored == total
    assert applied <= total

If invert_credit is implemented as return total, the test dies. That is the point. A test that cannot die is not a merge voter.

Keep the domain explicit. Infinite floats and wall-clock time do not belong in the merge lane unless you pin the generator and the clock.

Witness lane: lock bytes, not filenames

Witness tests are allowed to be large. They are not allowed to be the only gate. Hash the fixture. Commit the digest next to the file.

# tests/test_invoice_witness.py
import hashlib
from pathlib import Path

import pytest

from invoice import render_statement

FIXTURE = Path(__file__).parent / "fixtures" / "statement_v3.txt"
LOCK = Path(__file__).parent / "fixtures" / "statement_v3.sha256"

@pytest.mark.witness
def test_statement_bytes_locked():
    digest = hashlib.sha256(FIXTURE.read_bytes()).hexdigest()
    assert digest == LOCK.read_text().strip()
    rendered = render_statement(customer_id="c-1044")
    assert hashlib.sha256(rendered.encode()).hexdigest() == digest

Locking is a two-step command, not a hope.

sha256sum tests/fixtures/statement_v3.txt | awk '{print $1}' > tests/fixtures/statement_v3.sha256

If the agent rewrites both the renderer and the fixture, the digest still moves only when a human updates the lock. Treat lock edits as a separate review surface. Do not batch them with the behavior patch unless the PR description says why the bytes changed.

Freeze lane: a queue with a capsule

A freeze test runs. It must not vote. It must carry enough state to replay without the original laptop.

{
  "test_id": "tests/test_invoice_freeze.py::test_fx_table_stutters",
  "seed": 1742219,
  "input_sha256": "9f3c0c1a2b8e44d0c6a1f0b7d2e91c88a1b0c4d5e6f708192a3b4c5d6e7f8091",
  "expires_on": "2026-10-02",
  "last_replay_job": "https://ci.example.invalid/jobs/placeholder"
}

# tests/test_invoice_freeze.py
import json
from pathlib import Path

import pytest

LEDGER = Path(__file__).parents[1] / "freeze_ledger.json"

@pytest.mark.freeze
def test_fx_table_stutters(monkeypatch):
    row = json.loads(LEDGER.read_text())[0]
    # Replay machinery belongs here. The test records, it does not approve.
    assert row["seed"] is not None
    assert len(row["input_sha256"]) == 64

Expiry is part of the contract. A freeze past expires_on is a failed job, even if the test body passed. Frozen evidence that never comes back is a skip with extra paperwork.

Budget checker

The checker is a linter for voting rights. Run it after collection data is written, or parse markers from disk if you do not want a pytest plugin yet.

# tools/lane_budget.py
from __future__ import annotations

import ast
import json
import sys
from datetime import date
from pathlib import Path

LANES = {"merge", "witness", "freeze"}

def markers_for(path: Path) -> list[tuple[str, str]]:
    tree = ast.parse(path.read_text(), filename=str(path))
    found = []
    for node in ast.walk(tree):
        if not isinstance(node, ast.FunctionDef):
            continue
        for dec in node.decorator_list:
            name = None
            if isinstance(dec, ast.Attribute) and isinstance(dec.value, ast.Name):
                if dec.value.id == "pytest":
                    name = dec.attr
            elif isinstance(dec, ast.Name):
                name = dec.id
            if name in LANES:
                found.append((f"{path}::{node.name}", name))
    return found

def main() -> int:
    tests = Path("tests")
    rows = []
    for py in tests.rglob("test_*.py"):
        rows.extend(markers_for(py))
    counts = {lane: 0 for lane in LANES}
    for _, lane in rows:
        counts[lane] += 1
    ledger = json.loads(Path("freeze_ledger.json").read_text())
    today = date.fromisoformat("2026-09-18")
    expired = [r for r in ledger if date.fromisoformat(r["expires_on"]) < today]
    errors = []
    if counts["merge"] < 1:
        errors.append("merge lane empty: fail closed")
    if counts["freeze"] > counts["merge"]:
        errors.append(
            f"freeze {counts['freeze']} exceeds merge {counts['merge']}"
        )
    if counts["witness"] > 0 and counts["merge"] == 0:
        errors.append("witness cannot approve without merge voters")
    if expired:
        errors.append(f"expired freeze tickets: {len(expired)}")
    if len(ledger) != counts["freeze"]:
        errors.append("ledger rows must match freeze tests one-to-one")
    report = {"counts": counts, "errors": errors}
    Path("lane_budget.json").write_text(json.dumps(report, indent=2) + "\n")
    print(json.dumps(report, indent=2))
    return 1 if errors else 0

if __name__ == "__main__":
    sys.exit(main())

Wire it ahead of the test command.

python tools/lane_budget.py && pytest -q --strict-markers

--strict-markers stops unknown markers from becoming a fourth unofficial lane.

Run the merge lane off the laptop

Local entropy is a merge-lane bug. Laptop timezones, cached .pyc, and developer PYTHONHASHSEED do not travel with the patch. Isolate the voters.

export PYTHONHASHSEED=0
pytest -q -m merge --strict-markers
pytest -q -m witness --strict-markers
pytest -q -m freeze --strict-markers

Run those three invocations as three job steps. Do not or the exit codes. Witness failure is still a failure. Freeze expiry is still a failure. Only freeze assertion flakes are non-voting, and even those must write a replay artifact.

A remote runner is the cheaper fix than another retry loop. MonkeyCode's free model access and free server option can host that split: the model may draft candidate properties from a written spec, and the server runs the merge lane away from the laptop. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The model does not author freeze tickets. The server does not enlarge the vote. Do not invent a quota, a model name, or a duration the product did not state.

If you already have an isolated runner, point the merge lane at that. The lane rules do not require a particular vendor.

Candidate properties from a model are drafts. A human accepts them into tests/ and into the merge marker. Until that happens they are comments.

What this gate still cannot see

The budget script counts markers. It does not prove a property is non-tautological. Pairwise oracles, harvested failures, and overlap scores are separate gates. This article does not replace them.

Witness digests do not encode meaning. A perfectly locked wrong statement still witnesses. Freeze tickets do not diagnose root cause. They only prevent the flake from voting.

The date used in the sample checker is 2026-09-18. Replace it with the job's clock, pinned, not with the developer's laptop clock.

Who should not use this

Do not use lane voting as the sole control on safety-critical code if you have no human-owned merge oracle. Do not use it on a repository whose tests are all UI screenshots. Those repos have no merge lane yet. Build one property before you install the linter.

Do not use it as a way to hide an unbounded freeze queue. If freeze exceeds merge, the correct action is to delete tests or to promote a replayed flake into a property. Do not raise the freeze cap.

Single-file scripts with no CI also gain little. The value is the fail-closed collection rule under an isolated runner.

Close

Stop asking whether the suite was green. Ask which lane spoke. Merge-lane tests vote. Witness tests corroborate locked bytes. Freeze tests wait in a queue with a seed. If you need an isolated place to run the voters, a free server is enough to keep that vote off the laptop.