Run a Patch-Verification Gate on a Free Server: Property Checks, Fixture Locks, and Flaky Quarantine

A green test suite is not a contract. Agent-generated patches often pass existing tests while silently breaking edge cases that no one thought to encode. The fix isn't more unit tests; it's a three-layer gate that runs property checks, locks fixtures to a known-good state, and quarantines flaky tests with a TTL. This post shows a working implementation you can host on a free server.

AI coding agents are getting faster, and so is the damage they can do in a merge request. The interesting failures are no longer syntax errors; they're patches that satisfy every existing assertion while violating an unstated invariant. If you're reviewing such patches by hand, you're the bottleneck. A repeatable, automated gate turns that review into a checklist.

Why a plain test run is not enough

Most test suites are written before the patch exists. They encode the old implementation's assumptions, not the new behavior's promises. When an agent modifies a function, it will often keep the old tests green by avoiding the exact inputs that would expose a semantic shift. That's not malice; it's gradient descent on the test suite.

Property checks catch what example-based tests miss by generating inputs from a declared specification. Instead of asking "does this specific input work?", you ask "does the function hold for this class of inputs?". The agent can't easily game a property without understanding it.

Layer 1: Property checks

Start with pure functions and parse-ready modules. A simple Hypothesis test is enough to expose exceptions, wrong branches, or off-by-one errors.

from hypothesis import given, strategies as st
from my_lib import parse_config

@given(st.dictionaries(st.text(), st.text()))
def test_parse_config_never_raises_for_string_values(data):
    parsed = parse_config(data)
    assert parsed is not None

This test doesn't care about a specific config file. It generates hundreds of dictionaries and verifies that parse_config always returns a value. A surprising exception means the patch broke an assumption the old unit tests never checked.

Add properties that match your domain. If the function should be idempotent, test f(f(x)) == f(x). If it parses a date, test that the output always has year, month, and day keys. Each property is a contract the agent must honor.

Layer 2: Fixture locks

Fixtures drift. An agent patch shouldn't be verified against a database that changed last night, or an API fixture that one teammate regenerated manually. Fixture locks pin the gate to a known-good input digest.

import hashlib
import json

def lock_fixture(name, payload, expected):
    digest = hashlib.sha256(
        json.dumps(payload, sort_keys=True).encode()
    ).hexdigest()
    return {"name": name, "digest": digest, "expected": expected}

Store these locks in a locks.json file. Before running the test suite, compare the current payload's digest against the stored one. If they differ, the gate fails with a clear message: the fixture changed underneath the agent's patch, not the patch itself.

import json

def assert_fixture_locked(name, payload, expected):
    current = lock_fixture(name, payload, expected)
    with open("locks.json") as f:
        locks = json.load(f)
    assert locks[name]["digest"] == current["digest"], \
        f"Fixture {name} changed. Review the data first."

This kills the classic "works on my machine" failure mode. It also forces you to update the lock deliberately when a fixture legitimately changes, which is a good audit trail.

Layer 3: Flaky quarantine

Flaky tests shouldn't block the gate forever. If a test fails once, it could be a race condition unrelated to the patch. Failing twice in a row is a stronger signal. The quarantine pattern gives each failing test a 24-hour expiry, then forces a verdict.

from datetime import datetime, timedelta

QUARANTINE = {}

def run_test(name, fn):
    try:
        fn()
        QUARANTINE.pop(name, None)
        return "pass"
    except Exception:
        if name in QUARANTINE and QUARANTINE[name] > datetime.utcnow():
            return "quarantined"
        QUARANTINE[name] = datetime.utcnow() + timedelta(hours=24)
        return "fail"

A test that fails twice in 24 hours is quarantined, not sent to the developer. After 24 hours, the quarantine expires, and the test runs again. If it fails then, it's likely a real regression and the gate reports the full traceback.

This stops the "one flaky test blocks five agents" cascade. It also makes flakiness visible as a metric: you can log how often each test is quarantined. If a test becomes a permanent resident, you know to rewrite it.

Putting the gate on a free server

All of this logic is just a script, but you need something to watch for new patches. MonkeyCode's free server option gives you an always-on place to run this gate. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I won't quote model quotas or benchmarks, because I haven't measured them; what matters here is that the free tier is enough for a lightweight verification service.

Here's a minimal FastAPI endpoint that receives a patch description and returns the gate verdict:

from fastapi import FastAPI, Request

app = FastAPI()

@app.post("/check_patch")
async def check_patch(request: Request):
    payload = await request.json()
    result = run_gate(payload)
    return {"verdict": result}

You can point your CI webhook to this endpoint. When an agent pushes a patch, the server runs the property checks, validates fixture locks, and applies the flaky quarantine. The free server doesn't need GPU power; it's a small HTTP process that invokes pytest or a custom checker.

The optional part: MonkeyCode's free model access can be used to generate additional property templates from a function's docstring, or to summarize the gate's failure logs. That's a helpful addition, but the gate itself is fully useful without it.

What this gate will not do

• It won't validate UI flows that need a real browser. Property checks and lock files don't replace Playwright or Cypress.
• It can't fix a test suite that already has a flaky rate above 10%. The quarantine would fill up before real failures get through.
• It doesn't replace code review. A pass means the patch respects the checked properties, not that the design is sound.

A practical starting point

If you're already running an agent patch workflow, start with the property layer. Add fixture locks once you see a fixture drift incident. Add flaky quarantine when you feel the pain of random disconnections. Each layer is independent, so you can adopt them slowly.

The free server is a low-risk place to experiment—if the gate dies, you haven't lost CI credits. Run it for a week, collect the failure logs, and compare them to your manual review notes. You'll likely find one or two regressions that the old test suite missed.

Patches that survive this gate are not guaranteed correct. But they've been checked against a wider space than your old suite ever covered, on a server you didn't have to pay for. That's a trade worth making.