A Property Counterexample Is Not a Flake You Can Freeze

A flake freeze is an exemption record, not a passing result. An agent patch may carry one only when three conditions hold at once: the skipped assertion has a named nondeterminism class, the property lane is green on a pinned fixture generation, and the failure is not a shrunk counterexample. Miss any one of those, and the patch stays closed.

Retries hide the distinction that matters. A behavioral miss shrinks to a small input that should become a fixture. A flake misses because order, clock, or an unbound resource moved between runs. One green retry cannot tell those cases apart, so the gate must store the classification instead of the badge.

Two stores, not one status bit

Keep the property report and the exemption record in different files. Mixing them lets a later edit rewrite history. The property report lists predicates, pass counts, and any shrunk input. The exemption record lists the assertion id, the failure class, the fixture-generation pin, and an expiry.

Neither file is a substitute for the other. The pin names one input pack. It is not a hash of the whole repository, and it does not authorize a freeze by itself. If the pack changes, mint a new pin and drop every freeze that still cites the old one.

A closed failure-class set

Open-ended labels turn into a junk drawer. Use a closed set, and treat an unrecognized string as unknown.

Class	Signature a reviewer can check	Freeze?	Companion control
order_dependent	Passes after iteration order is fixed	Yes, short window	Same generation pin
clock_bound	Fails only when wall time crosses a boundary	Yes, short window	Clock injected by the fixture
resource_blip	Timeout or empty upstream body, then a pass	Yes, only beside a green property lane	Status code and body length
counterexample	A predicate shrank an input that breaks it	No	Input enters the fixture pack
unknown	No signature match	No	Investigate; do not exempt

resource_blip looks like a retry problem. Collect the retry as evidence. Do not treat it as authorization. An HTTP 200 with an empty body is a data point about that call, not a clear for the patch under review.

When two classes match, keep the stricter one. counterexample outranks clock_bound. unknown outranks every allow-class, because a partial match is not a match.

Numbered workflow

1. Build the touched set before any run

List the functions the patch edits. A property that never mentions a touched function is reporting on untouched code, and a green result there does not support an exemption. Drop those predicates from the lane report, or mark the lane incomplete.

This step is cheap, and it removes a common false credit. An agent can add a test that asserts a constant after calling an unrelated helper. That test is not evidence about the edit.

2. Pin the fixture generation, then run

Choose the generation id before the process starts. Do not regenerate inputs between predicates. A moving pack makes the later witness unverifiable, even if every predicate prints pass.

export FIXTURE_GENERATION=gen-2026-10-12-a
python -m gate.property_lane \
  --generation "$FIXTURE_GENERATION" \
  --patch patch.diff \
  --report property-report.json

The module path is a proposed interface for this workflow. It is not a shipped tool, and the generation id is an example label. No measured run of that command is claimed here.

3. Sink every counterexample into the pack

A failed predicate must shrink, then land in the fixture pack. That file is a regression case. It is ineligible for a freeze record, including a freeze of a nearby assertion that failed in the same job.

# Proposal only. Not executed against a live suite.
from dataclasses import dataclass

@dataclass(frozen=True)
class Counterexample:
    predicate: str
    generation: str
    payload: dict

def sink(example: Counterexample, pack: dict) -> None:
    row = {
        'predicate': example.predicate,
        'generation': example.generation,
        'payload': example.payload,
    }
    pack.setdefault('counterexamples', []).append(row)

If the shrinker emits an input that does not replay, keep the raw failure log and refuse both the shrink and any freeze for that assertion. A non-replaying shrink is a tool defect, not a waiver.

4. Classify what is still red

Match failure text plus run metadata against the closed set. Require the companion control from the table. A clock_bound label without an injected clock is unknown. An order_dependent label without a fixed-order rerun is unknown.

Do the classification in code, not only in a review comment. Comments drift. The witness file is diffable, so a later edit can be reviewed instead of remembered.

5. Admit the freeze only through the decision function

Callers may format a message. They may not add a side path that returns true for an empty witness. The function is the policy.

# Proposal: unexecuted decision function.
from datetime import datetime, timezone

ALLOWED = {'order_dependent', 'clock_bound', 'resource_blip'}

def freeze_allowed(witness: dict, now: datetime) -> tuple[bool, str]:
    kind = witness.get('failure_class')
    if kind == 'counterexample':
        return False, 'counterexample must enter the fixture pack'
    if kind not in ALLOWED:
        return False, 'unnamed or unknown class'
    if not witness.get('property_lane_green'):
        return False, 'property lane is not green'
    if witness.get('generation') != witness.get('pin'):
        return False, 'fixture generation drifted'
    expiry = datetime.fromisoformat(witness['expiry'])
    if expiry.tzinfo is None:
        return False, 'expiry must be timezone-aware'
    if expiry <= now.astimezone(timezone.utc):
        return False, 'exemption window already closed'
    return True, 'exemption recorded'

Exercise the deny path with a witness document, not with a live agent patch. The negative case is what protects the gate. The dates below are example values for a review check dated 2026-10-12. They are not observations from a production queue.

{
  "assertion_id": "tests/test_order.py::test_stable_keys",
  "failure_class": "counterexample",
  "property_lane_green": true,
  "generation": "gen-2026-10-12-a",
  "pin": "gen-2026-10-12-a",
  "expiry": "2026-10-13T00:00:00+00:00",
  "companion": "shrunk input retained"
}

python -m gate.freeze_check --witness witness.json --now 2026-10-12T00:00:00+00:00

Expect a non-zero exit when failure_class is counterexample, even if property_lane_green is true. That command is part of the same proposal. It is not a published package, and it is not a performance result.

Expiry length is a team policy field, stored in the witness. This workflow does not recommend a universal duration. A missing timezone is a hard deny, because a naive timestamp cannot be compared with a CI clock.

6. Mint a new pin after the pack grows

Adding a counterexample changes the generation. Re-run the property lane on the new pin. Discard every freeze that still names the previous pin, even if the assertion text is identical. Skipping this step is how a one-off miss becomes a standing blind spot.

Review table

Read left to right. Any deny overrides the rest. There is no mostly-green row.

Property lane	Shrunk input stored	Failure class	Pin match	Decision
green	no	order_dependent	yes	allow, with expiry
green	yes	counterexample	yes	deny
green	no	unknown	yes	deny
red	no	clock_bound	yes	deny
green	no	resource_blip	no	deny
green	no	clock_bound	yes, but no injected clock	deny; relabel unknown

A rehearsal runner is not the authority

Drafting predicates is slow if every sketch waits on the merge queue. A separate rehearsal surface can propose sketches and replay classification. It must not sign the witness.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option fit that split, and only that split. A free model can propose predicate sketches and failure-class labels. A free server can host a non-gating replay of those sketches.

This draft relies on those two availability claims as supplied by the operator. It does not state a model name, quota, hardware shape, time limit, or score. Model text stays untrusted. The authoritative property lane still runs in the team's own CI, against the pinned generation.

Apply four checks before a sketch leaves rehearsal.

1. Reject a predicate that does not mention a touched symbol.
2. Reject a label outside the closed set.
3. Treat an empty rehearsal response as resource_blip for that call only.
4. Promote nothing until the authoritative lane writes the same pin and the same class.

Do not copy a rehearsal resource_blip onto a product assertion that failed differently. Empty-body handling in a client is a separate problem from exemption policy. Keep that client fix out of the witness file.

If a witness is missing a field, ask for that field. Another retry will not create a pin, a class, or an expiry. Predicate sketches can be redrafted on the free rehearsal server, then accepted only when freeze_allowed returns true on CI.

Limitations

The closed set will mislabel a novel flake. A green property lane can miss a bug that no predicate mentions. Shrinkers fail, and a failed shrink is not a pass.

Expiry bounds how long a wrong allow-class can hide. It does not detect the wrong class. A rehearsal model can emit a tautology that always passes. The touched-set check reduces that risk, and it does not remove it.

Clock injection does not catch filesystem races or device-level nondeterminism. Add a class only when both the signature and the companion control are nameable. Until then the label is unknown, and unknown does not freeze.

The workflow assumes the patch diff is the unit of review. It fits poorly when a generated tree is so large that the touched set cannot be listed. Split that tree first. Do not weaken the touched-set rule to fit the tree.

Who should not use it

Skip the exemption path when the patch touches authentication, authorization, or deletion. Those edits do not get a window. Skip it when the repository has no properties that mention real behavior, because a freeze stacked on placeholder tests conceals the missing lane.

Skip it when fixture generations cannot be pinned. A later reviewer would be unable to replay the witness. Skip it during incident response as well. A production miss needs a reproduction owned by a person, not an exemption record filed to clear a queue.

What a merge may claim

Merge when the property lane is green on the current pin, and every remaining red assertion is either fixed or covered by a witness that freeze_allowed accepts. The commit message may note that an exemption exists. It may not say the skipped assertion passed.

Retain the counterexample in the pack. Name the flake in the witness. Let the pin and the expiry retire the excuse together.