Property Checks, Fixture Leases, and Flaky Freezes: A Decision Matrix for Agent Patches

An agent patch is a claim. The code claims to fix an issue, and the tests bundled with that patch claim to prove it. Both claims need independent checking.

After running patch gates for a while, I settled on three instruments:

1. Property checks for logic that should never violate an invariant.
2. Fixture leases for tests that touch shared state.
3. Flaky freezes for tests that are known to be unreliable but cannot be fixed immediately.

Each instrument has a different job. Using the wrong one creates false confidence. Here is how to choose, plus a minimal implementation you can run on a free server.

Why the agent's own tests are not enough

The agent writes a test, and the test passes on the agent's run. That tells you the agent and its test agree with each other. It does not tell you the system is still correct.

Your gate needs properties that did not come from the patch. My rule: the gate should contain at least one assertion the agent never saw.

Instrument 1: Property checks for invariants

Property checks are perfect for functions with clear preconditions and postconditions. They generate many inputs and verify that the invariant holds every time.

Example with hypothesis:

from hypothesis import given, strategies as st

def apply_discount(price, discount):
    if discount < 0:
        raise ValueError("discount must be non-negative")
    return price * (1 - min(discount, 1))

@given(
    st.floats(min_value=0, max_value=1e6),
    st.floats(min_value=0, max_value=1),
)
def test_discount_never_negative(price, discount):
    assert apply_discount(price, discount) >= 0

Notice that test_discount_never_negative knows nothing about a specific bug report. It encodes the contract. If the agent's patch changes apply_discount to return a negative value for some input, the property check catches it even when the agent's own test still passes.

Use property checks when you can name the invariant in one sentence: "the result is never null," "the list stays sorted," "the sum remains the same."

Instrument 2: Fixture leases for shared state

Property checks do not help when the patch touches a database, the file system, or another process. In those cases the test depends on state that other tests may mutate.

The fix is not a global lock. A global lock serializes everything and makes your gate slow. A lease gives the test exclusive access to a specific resource, and only that resource.

import fcntl
import os
from contextlib import contextmanager

@contextmanager
def leased_fixture(lock_path):
    fd = os.open(lock_path, os.O_CREAT | os.O_RDWR)
    try:
        fcntl.flock(fd, fcntl.LOCK_EX)
        yield lock_path
    finally:
        fcntl.flock(fd, fcntl.LOCK_UN)
        os.close(fd)

You can use this for a test database directory or a temporary port file:

def test_order_creation_with_leased_db():
    with leased_fixture("/tmp/orders.lock"):
        db = open_database("/tmp/orders.db")
        result = create_order(db, customer_id="c1")
        assert result.status == "pending"

If the agent patch uses a different connection string or a different file path, the lease will not help. That is intentional: the instrument protects the state you know about, not the state you forgot.

Instrument 3: Flaky freezes with an expiration date

Flaky tests are the most dangerous member of the trio. They pass on one run and fail on the next, so they teach your gate to ignore failures. Many teams simply delete or disable them.

A freeze is a better simfile: it holds the flaky test in quarantine, records why it was frozen, and forces a backfill before the freeze expires.

import json
import time
from pathlib import Path

class FlakyFreezeStore:
    def __init__(self, path):
        self.path = Path(path)
        self._freezes = self._load()

    def _load(self):
        if not self.path.exists():
            return {}
        return json.loads(self.path.read_text())

    def freeze(self, test_id, reason, expires_at, backfill):
        self._freezes[test_id] = {
            "reason": reason,
            "expires_at": expires_at,
            "backfill": backfill
        }
        self.path.write_text(json.dumps(self._freezes, indent=2))

    def expired(self):
        now = time.time()
        return {tid: f for tid, f in self._freezes.items() if now >= f["expires_at"]}

The freeze is not a skip. The original test still runs in the background if you want it to, but the gate does not treat its failure as a regression. Instead, the gate checks the store after the run:

def gate_report():
    expired = store.expired()
    if expired:
        for tid, freeze in expired.items():
            print(f"FAIL: {tid} freeze expired, backfill: {freeze['backfill']}")
        return 1
    return 0

Every freeze must have a backfill task. For example, "reproduce the flake with a seeded random generator" or "reduce the fixture to a single record and add an assertion about the exact sequence." If the freeze expires and the backfill is still not merged, the gate fails.

The decision matrix

Choose the instrument based on what the patch touches:

Patch characteristic	Instrument	Why
Pure function with a clear invariant	Property check	Many inputs, no shared state
Database or file mutation	Fixture lease	Exclusive access to the resource
Test that fails intermittently	Flaky freeze	Stops gate noise without hiding the problem forever
Agent's own test only	Reject or add your own	The test is not independent evidence

Do not stack all three on every patch. A property check on a database test is often too slow. A fixture lease on a pure function adds nothing. The matrix is a filter, not a coat rack.

A minimal gate script

Here is a gate that ties all three together. I run it as a scheduled script on MonkeyCode's free server option, and I used MonkeyCode's free model access to draft the property-check skeleton. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The server cost is zero, so the gate can run every hour without worrying about budget. The script is deliberately small:

#!/usr/bin/env bash
set -euo pipefail

# 1. Run property checks
pytest tests/properties -q

# 2. Run fixture-leased integration tests
pytest tests/integration -q --with-fixture-leases

# 3. Check for expired freezes
python check_freezes.py

If any step fails, the gate exits non-zero and the agent patch is rejected. The patch can be resubmitted with fixes, but it cannot be merged by bypassing the gate.

What this approach does not do

Property checks are not proofs. They generate random inputs, but they can miss an edge case that the generator never produces. Fixture leases prevent cross-test pollution, not logic errors. Freezes are escape hatches, not fixes.

The gate is only as good as the invariants you encode. If you cannot state the invariant, a property check will be a meaningless round trip.

Who should not use this

If your agent patches are always small and your existing CI finishes in seconds, adding a freeze store and a lease layer may be overhead you do not need. If your team deletes tests to make CI green, a permanent skip-list is more honest than a freeze with an expiration date. A freeze is only useful when the team actually commits to the backfill.

Start with one instrument per patch. Add the others only when you see the specific failure mode: a contract violation, a state race, or a flaky test that keeps blocking the gate. That is the difference between a strategy and a stack of rituals.