The Selective Gate: Property Checks, Fixture Leases, and a Flaky Freeze for a Free Server

Why a free server changes your test strategy

An agent patch can be wrong in a thousand ways. The usual response is to throw a bigger CI at it. But when the only server you have is free, CPU and time are your real constraints. Running every test on every patch will:

• Wait minutes for feedback.
• Drain your free quota before noon.
• Train the agent to ignore slow, noisy failures.

The solution is a selective gate. Instead of executing the entire test suite, you run only the layers that the current diff could plausibly break. The three layers remain the same: property checks, fixture leases, and flaky freeze. But now they run on demand, based on a diff map.

The three layers in one sentence each

• Property checks generate arbitrary inputs and verify invariants. They catch dead code, missing branches, and accidental assertion-removal.
• Fixture leases isolate tests that touch shared state. They catch cross-test contamination that a green single-test run cannot reveal.
• Flaky freeze records every test result. If a test fails after a pass, it blocks the patch and asks a human why.

If you've used these before, you know they each catch a different class of regression. The problem is that running all three on every patch is wasteful. On a free server, you need a price tag for each layer.

The cost map

Here is a decision table that maps patch types to required layers:

Patch affects...	Property	Fixture	Flaky
Pure function, no I/O	✅	⬜	⬜
Shared database schema	⬜	✅	✅
Test files only	⬜	✅	✅
Timing/network call	⬜	✅	✅

The table is heuristic, not gospel. But it saves CPU. A patch that touches only a pure function does not need a database lease. A patch that alters a schema does not need a property check on a sorting routine.

A selector that actually works

Save this script as layer_selector.py:

#!/usr/bin/env python3
import subprocess, json, re

PATTERNS = {
    "property": r"/(pure|algorithms|validate)/",
    "fixture": r"/(db|storage|state)/",
    "flaky": r"/tests/",
}

def changed_files():
    out = subprocess.check_output(["git", "diff", "--name-only", "HEAD"])
    return out.decode().splitlines()

def layers_for(files):
    layers = set()
    for path in files:
        for layer, pattern in PATTERNS.items():
            if re.search(pattern, path):
                layers.add(layer)
    return sorted(layers)

if __name__ == "__main__":
    files = changed_files()
    print(json.dumps(layers_for(files)))

Then a runner that uses pytest markers:

#!/usr/bin/env bash
set -eu

SELECTED=$(python layer_selector.py)
echo "Running: $SELECTED"

if [[ "$SELECTED" == *"property"* ]]; then
  pytest --maxfail=1 tests/test_properties.py
fi

if [[ "$SELECTED" == *"fixture"* ]]; then
  pytest --maxfail=1 tests/test_fixtures.py
fi

if [[ "$SELECTED" == *"flaky"* ]]; then
  python freeze_check.py
fi

With this script, a change to sort.py triggers only property checks. A change to database.py triggers fixture and flaky layers. The agent gets fast feedback, and you conserve your free tier's compute.

Flaky freeze without a database

The freeze layer can be a simple JSON file. This snippet keeps the last five results for each test and blocks when a test becomes inconsistent:

import json, time
from pathlib import Path

STATE = Path("freeze_state.json")

def record_test(test_id, passed):
    data = json.loads(STATE.read_text()) if STATE.exists() else {}
    seq = data.get(test_id, [])
    seq.append(1 if passed else 0)
    data[test_id] = seq[-5:]
    STATE.write_text(json.dumps(data))
    return data

def check_freeze(test_id, passed):
    data = record_test(test_id, passed)
    seq = data[test_id]
    if len(seq) >= 3 and 0 < sum(seq) < len(seq) and not passed:
        print(f"Frozen: {test_id}")
        # block the patch here

The freeze is not a ban on flaky tests. It is a statement that "we do not know why this changed". A human must unblock it.

Putting it on a free server

Assuming you're running this on a free server—MonkeyCode's free server option is one example—the selector becomes a gateway cron job. Every time the agent creates a patch, you trigger the runner. If it passes, you merge. If it fails, you send the failure report back to the agent.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free model access on MonkeyCode matters here because generating agent patches is compute-heavy. The server's job is not to generate the patch, but to verify it. That division of labor makes free-tier CPU usable.

End-to-end workflow

1. Agent works on a local checkout using MonkeyCode's free model access to produce a patch.
2. The patch is pushed to a branch on the free server.
3. The server computes git diff --name-only and runs the layer selector.
4. Only the selected test groups execute. Each failure is annotated with the layer that caught it.
5. If the flaky freeze triggers, the patch is rejected and the agent receives a human-readable reason.
6. A human reviews the freeze history and either clears it or files a bug.

Limitations of the selective approach

• The pattern map is a guess. If a change to utils.py breaks a fixture, your patterns may miss it.
• A flaky freeze is only as good as the history you store. On a fresh server, there is no history—so the freeze will do nothing until you've run tests a few times.
• Property checks can fail on legitimate edge cases if your generators are too aggressive. Calibrate them on known-good inputs first.

Who should not use this

If your project is small enough that the full suite finishes in under a minute, the selector is overengineering. Also, if you have a hard requirement to block every unknown failure, a selective flaky freeze is the wrong gate. It exists to make the agent go slower, not to give it a pass.