Free Compute First: A Cost-Capped Triage Gate for Agent Patches

Most agent patches fail in boring ways. Wrong field names. Inverted conditions. A deleted edge case. You do not need a frontier model to catch those — you need a cheap classifier and fast local checks.

The core conclusion: spend free compute first, and escalate only when the cheap signals disagree. That turns the review problem from "how smart is the reviewer" into "how disciplined is the escalation rule."

Why the gate needs a budget

Every gate has a cost ceiling, whether you admit it or not. If reviewing one patch costs more than the patch is worth, developers route around the gate. It becomes theater.

Two levers keep the cost near zero:

1. Free model endpoints handle classification.
2. A local server runs the verification loop with zero API calls.

This is where MonkeyCode's free model access and free server option fit. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Both remove the usual excuse for skipping triage: "I didn't want to burn tokens on a trivial patch."

A classification call is a labeling task, not a reasoning task. Asking "which of these four buckets is this diff?" survives a free model much better than asking "is this algorithm correct?" Save reasoning for the exceptions.

Step 1 — Bucket the diff

The prompt asks one question and expects one word back. The four buckets:

Bucket	Example	Verification depth
test-only	added fixtures, new assertions	fixture mutation only
refactor	renames, reordering	CI only
logic	conditionals, loops, state	full property suite
config	dependency bumps, env vars	smoke test

Treat a failed classification as escalate, never as accept. Free endpoints return empty bodies sometimes. An empty answer is not a low-risk answer.

Step 2 — Verify locally on the free server

The patch runs on your own hardware. No API call in the feedback loop, so no rate limit, no latency, and no cost per trial. The loop generates seeds, runs the old and new code on the same inputs, and compares results.

The old implementation is the oracle. Agent patches often reinterpret a written spec in their own favor. Comparing old vs. new behavior is the only way to catch that without a human in the loop.

Step 3 — Mutate the fixtures

The agent has seen the existing fixtures. Give it data it has never seen: a zero, an empty list, a max-int. These mutations are fixed, so they are deterministic. Deterministic checks do not add flakiness.

Step 4 — Freeze the flakes

A flaky test is a veto. The first time a test fails, quarantine it. Remove it from the accept path and report it. A patch cannot pass while a quarantined test is in its set.

This rule is what keeps the gate honest. Without it, flakiness becomes a silent merge permit: run again until green.

Step 5 — Escalate on disagreement

The only expensive step is the last one. Escalate when the cheap signals disagree:

Signals	Outcome
all pass, low risk	auto-accept
property divergence	reject with the failing seed
fixture mutation fails	reject with the mutated fixture
classifier and tests disagree	escalate to the stronger model
any flaky test	freeze, do not accept

Note the asymmetry: disagreement goes up, never down. A stronger model is asked to review, not to rubber-stamp.

The artifact: a minimal triage loop

# triage.py — cost-capped acceptance for agent patches
import random
import subprocess
import sys

PATCH_PATH = sys.argv[1]
BUCKETS = ("test-only", "refactor", "logic", "config")
FAILURE_HISTORY = "failures.jsonl"  # append-only log of rejected seeds


def classify(diff: str) -> str:
    # Calls a free model endpoint with a short labeling prompt.
    # On timeout or empty body: return "escalate" and let the
    # rejection path handle it. Never default to "test-only".
    ...


def run(cmd: str, timeout: int = 20) -> tuple[bool, str]:
    try:
        result = subprocess.run(
            cmd, shell=True, capture_output=True, timeout=timeout
        )
        return result.returncode == 0, result.stdout.decode(errors="replace")
    except subprocess.TimeoutExpired:
        return False, "timeout"


def baseline_ok() -> bool:
    ok, _ = run("python solution.py --self-test")
    return ok


def property_divergence(trials: int = 500) -> tuple[bool, str]:
    # Both commands must expose a --seed flag and stay deterministic.
    for _ in range(trials):
        seed = random.randrange(2**32)
        old_ok, old_out = run(f"python solution.py --seed {seed}")
        new_ok, new_out = run(f"python patch.py --seed {seed}")
        if old_ok != new_ok or (old_ok and old_out != new_out):
            return False, f"seed {seed}"
    return True, ""


def fixture_failure(fixtures: list[str]) -> tuple[bool, str]:
    for fixture in fixtures:
        ok, _ = run(f"python patch.py --fixture {fixture}")
        if not ok:
            return False, fixture
    return True, ""


def main() -> None:
    if not baseline_ok():
        print("FREEZE: baseline broken; reject without review")
        sys.exit(1)

    bucket = classify(open(PATCH_PATH).read())
    if bucket not in BUCKETS:
        print(f"ESCALATE bucket={bucket or 'empty'} (cannot classify)")
        sys.exit(1)

    if bucket == "test-only":
        ok, name = fixture_failure(["zero", "empty", "max-int"])
        print(f"REJECT fixture={name}" if not ok else "ACCEPT bucket=test-only")
        sys.exit(0 if ok else 1)

    if bucket == "logic":
        ok, seed = property_divergence()
        if not ok:
            print(f"REJECT property seed={seed}")
            sys.exit(1)
        ok, name = fixture_failure(["zero", "empty", "max-int"])
        if not ok:
            print(f"REJECT fixture={name}")
            sys.exit(1)

    print(f"ACCEPT bucket={bucket}")


if __name__ == "__main__":
    main()

Two details matter. The failure history goes to a JSONL file so the next escalation request includes past counterexamples. And the baseline check runs first: if the old code cannot pass its own self-test, no patch gets reviewed. That prevents the gate from approving patches on top of a broken foundation.

Limitations

Free model endpoints are best-effort. No SLA, no uptime guarantee, occasional empty responses. The gate treats every one of those as escalation, but escalation still costs a developer's time.

The old-vs-new oracle assumes determinism. Patches that intentionally change output — new messages, reordered logs — must be bucketed as refactor, or they will be rejected forever.

This pattern also assumes your test commands are honest. If the suite takes ten minutes, 500 seeds take unreasonably long. Start with 50 seeds and a tight timeout, then widen once you trust the speed.

Who should not use this

Teams under compliance regimes that require named-human review on every change should skip automated acceptance entirely. An audit trail of "the gate said accept" is not a substitute for a signature.

Also skip it for irreversible data operations. A merge that drops a column is not something a flaky-test policy can apologize for. Keep a human in the path for migrations and destructive commands.

The gate works because it spends the free compute first and the scarce compute last. Classification on a free model, verification on your own server, escalation only on disagreement. If you want a version of this harness with the free-model classification wired in, the next post covers the prompt and the failure-history log. The loop above is enough to start measuring where your patches actually fail.