Shrink the Failing Input Before a Flake Freeze Can Be Written

A flake freeze is a quarantine row, not a pardon. Write one only after a property failure shrinks to a minimal fixture, a fixed replay set splits into completed assert outcomes, and the record stores seed, fixture digest, runner id, and expiry. A timeout is an infrastructure result. It does not enter the freeze ledger.

CI logs collapse three different failures into one red line. An assertion miss means the patch broke a stated property on a completed run. A timeout means the runner stopped waiting. A harness error means the check never produced a verdict. Those classes need different owners, because a single flaky bucket trains reviewers to skip real regressions and runner defects together.

The workflow below is a pre-merge classifier. It does not admit the patch. It also does not replace a later comparison against the base branch. It answers a narrower question: is this failure even eligible to be frozen?

Keep the property clock-free

A property check here is a pure function of the pinned input and the patch output. It may compare sets, bounds, and required keys. It may not read wall time, process memory, socket state, or model latency. If the predicate samples the clock, two honest runs can disagree while the patch stays identical.

Latency still matters, but it belongs on a separate budget probe with its own ledger. Mixing that probe into the property replay creates false freeze candidates whenever the runner is busy. The freeze boundary therefore uses four labels and no others.

Label	Completed run?	Can it authorize a freeze alone?
assert_pass	Yes	No
assert_fail	Yes	No
timeout	No	Never
harness_error	No	Never

Eligibility is a property of the whole replay list. The list must contain both assert_pass and assert_fail, and it must contain nothing else. Uniform failure blocks the patch. Uniform success means the miss was not reproduced on the minimal fixture. Any timeout or harness error sends the case to an infra queue and stops the write.

Reject incomplete freeze records

The fields below are a proposal for the gate, not a schema copied from a live service. A missing field is a hard reject, not a default.

• seed is the integer passed to the generator before the first draw.
• fixture_digest is the SHA-256 of the canonical minimal input.
• origin_digest is the hash of the pre-shrink failing input, so a size-dependent bug is not discarded.
• property_id is a stable predicate name, independent of the patch diff.
• replays is the label list, with length fixed before the run starts.
• runner_id is local, or the label of the server that actually executed the set.
• expires_at is an explicit timestamp chosen by the team, not an open-ended skip.
• patch_id identifies the candidate under test, not the base branch.

Without a seed, another engineer cannot replay the case. Without a fixture digest, the same record can be aimed at a different input and still look authorized. Without an expiry, the row becomes a permanent waiver. Refuse the write before a reviewer is asked to interpret it.

Steps

1. Pin the generator seed and persist it before any example is drawn.
2. Run the property as a pure predicate and map the outcome to one of the four labels. Do not catch a timeout inside the predicate and store it as an assertion miss.
3. On one assert_fail, shrink in a fixed order: drop optional keys from the end, shorten strings, then move integers toward zero. Stop at the smallest input that still fails once.
4. Canonicalize the minimal input and hash it. Hash the original failing input under the same canonical rule.
5. Replay only the minimal fixture. This draft uses five replays. That count is an unexecuted proposal, not a tuned or published threshold.
6. If any replay is timeout or harness_error, append the case to the infra queue. Do not start another model generation from that branch of the decision.
7. If every replay is assert_fail, block the patch. Do not write a freeze row.
8. If the labels mix assert_pass and assert_fail only, write the record with seed, both digests, labels, runner id, patch id, and expiry. Leave the merge decision unset.

Proposal code

This module is unexecuted proposal code. It does not call a model, and it does not open a socket. Change the expiry only through review. On interpreters that reject builtin generic annotations, delete the list[str] and dict hints. The control flow does not depend on them.

import hashlib
import json
from datetime import datetime, timedelta, timezone

REPLAYS = 5  # proposal, not a tuned threshold
FREEZE_DAYS = 7  # visible proposal, not a vendor policy

def digest(fixture: dict) -> str:
    blob = json.dumps(fixture, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(blob.encode()).hexdigest()

def classify(runs: list[str]) -> str:
    allowed = {"assert_pass", "assert_fail", "timeout", "harness_error"}
    if len(runs) != REPLAYS or any(r not in allowed for r in runs):
        return "reject_malformed"
    if any(r in {"timeout", "harness_error"} for r in runs):
        return "infra_queue"
    fails = sum(r == "assert_fail" for r in runs)
    if fails == REPLAYS:
        return "block_patch"
    if fails == 0:
        return "not_reproduced"
    return "freeze_eligible"

def freeze_record(seed, fixture, origin, runs, patch_id, runner_id, property_id):
    if classify(runs) != "freeze_eligible":
        raise ValueError("freeze write refused")
    now = datetime.now(timezone.utc)
    return {
        "seed": seed,
        "property_id": property_id,
        "fixture_digest": digest(fixture),
        "origin_digest": digest(origin),
        "replays": list(runs),
        "patch_id": patch_id,
        "runner_id": runner_id,
        "expires_at": (now + timedelta(days=FREEZE_DAYS)).isoformat(),
    }

def test_timeout_never_freezes():
    runs = ["assert_fail", "timeout", "assert_fail", "assert_pass", "assert_fail"]
    assert classify(runs) == "infra_queue"

A shrinker may live outside this file. The write path still hashes the minimal fixture, so an edit after the hash is a new identity rather than a silent rewrite of the old row.

Commands

The commands below check the classifier. They do not start an agent, and they do not need a model endpoint.

python -m pytest tests/test_freeze_eligibility.py -q
python - <<'PY'
from freeze_gate import classify, digest
print(classify(["assert_fail"] * 5))
print(classify(["assert_fail", "assert_pass", "assert_fail", "assert_pass", "assert_fail"]))
print(classify(["assert_fail", "timeout", "assert_fail", "assert_pass", "assert_fail"]))
print(digest({"user_id": 0, "role": ""}))
PY

If the file matches the proposal, the classifier lines are block_patch, freeze_eligible, and infra_queue, followed by the SHA-256 of the canonical fixture. Execute the commands before treating those strings as a gate. An expected line that was never run is not evidence.

Walkthrough of one fixture

Consider a minimal fixture with user_id set to 0 and role set to an empty string. The predicate requires a non-negative user_id to carry a role from a fixed allow-list. A patch that emits the empty role fails once. Shrinking removes generator-added keys. The empty role stays, because deleting it turns the miss into a different error and no longer explains the original failure.

Under the proposal, five completed labels of fail, pass, fail, fail, pass are freeze_eligible. Five fails block the patch. One timeout anywhere in the five routes the case to infra, even when the other four disagree. This is a rule walkthrough. It is not a measured flake rate from a suite.

Why the shrink happens first

A wide generated payload hides the field that failed. Freezing the wide payload pins incidental keys, so the next review spends time on noise. Freezing the minimal input pins a disagreement a reviewer can read in one screen. Store the shrink path next to the seed: which key was dropped, which string was shortened, which integer moved. The next reader should rebuild the fixture without guessing.

Do not shrink a timeout. There is no completed output to minimize, and the shrinker would invent a cause. That gap is another reason a timeout cannot be a freeze key.

Draft predicates off to the side

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access fits a single preparatory step. Use it to draft candidate predicates from a saved failing trace, then have a person accept or reject each draft. Copy the accepted text into the pure checker and review that diff on its own. Do not call the model during replay. A predicate that asks a model for the verdict will disagree with itself across runs, and the freeze ledger will store that noise as if the product were flaky.

The free server option is the other availability claim supplied for this draft. Treat it as an alternate runner only when current access actually executes this harness there. Nothing here establishes a quota, a hardware shape, a duration, or permanence, and nothing here assumes the free server is a CI product. If that cannot be confirmed from current product documentation, set runner_id to local and run the five replays on one machine. Compare labels only within a single runner id. A timeout on one host beside a pass on another is runner drift, not a candidate for quarantine.

When failing traces are already saved, free model access is a practical place to draft the next predicate before a paid model call has to be justified. Check the current terms first. Free access can change, and this gate should not depend on an unverified endpoint.

Limitations

Shrinking can drop a failure that appears only above a size threshold. Keep origin_digest, and keep the original example bytes. Writing the minimal row is not permission to delete the wide one.

Five replays will miss rare splits. Adding replays increases the chance of observing a disagreement and increases cost. Pick the count from your own suite. Do not quote the five in this draft as a benchmark or as a default for every repository.

Sorted JSON hides key-order bugs and removes false drift from unordered objects. If order carries meaning, do not sort. Record the choice, such as canonicalization: json-sort-keys, so the next runner hashes the same bytes.

A candidate that edits the predicate and the implementation in one diff can make a broken output look lawful. Review the predicate diff separately. This writer does not perform that review.

The classifier is not a production flake study, and it is not a safety proof for the patch. Its job is narrower: keep runner failures out of the freeze ledger.

Who should not use this

Do not use the workflow if the assertion must call a live model or a shared network service. That check cannot stay pure, and a freeze would turn outages into standing policy.

Do not use it if nobody owns expiry. A timestamp does not notify a person. An unowned freeze is a skip with extra fields.

Do not use it as a latency or load gate. Those checks are budget probes. Placing them in the property list marks ordinary queue delay as flake.

Do not use it when the goal is automatic merge. A complete quarantine record is not an approval to ship.

After the run

Retain the seed, both digests, the shrink path, the label list, the runner id, the patch id, and the expiry. Leave merge unset so a later gate can read the row without inheriting a hidden approval. Uniform assertion failures still block the patch. Timeouts remain with the runner owner. Only a shrunk fixture that both passes and fails, under the same seed and the same runner, with an expiry attached, may occupy a freeze row.