Route Agent Patches to the Cheapest Gate That Can Catch Them

The cheapest reliable verification for an agent patch is a function of the diff's shape, not of who wrote it. Most review pipelines miss this and run the entire matrix on everything. That is not rigor. It is a way of postponing a decision until the queue decides for you.

Patch volume went up. Reviewer attention did not. The only sustainable response is to spend expensive gates where they can actually fail, and cheap gates everywhere else.

The failure mode: uniform verification

A one-line docstring edit and a lockfile bump currently cost the same wall-clock time. Both wait behind the same suite. Both get the same treatment.

The result is predictable. The suite gets slower, the queue gets longer, and people start merging on red because "it's probably the flaky one." Any of that sound familiar? Uniform verification does not buy safety — it buys the habit of ignoring failures.

A patch router fixes the ordering, not the tests. It answers one question before anything runs: what is the minimum set of gates that could plausibly catch a defect in this diff?

Four signals that actually change the gate set

Diff size alone is a weak proxy. These signals are stronger, and all four are extractable with git diff alone.

Signal	Why it changes the tier
Production files changed with no test file changed	Nothing in the diff demonstrates the new behaviour was exercised
Assertions added vs. removed	Tests edited without a new assertion are often tests edited to pass
Dependency or CI surface touched	Blast radius leaves the repo; local gates cannot see it
Change size on production files	A 4-line change and a 400-line change fail differently

Note what is not on the list: commit message, model name, author confidence. None of those are evidence.

The router

Reference implementation below, standard library only, no benchmark claims attached. It reads a diff range and prints the tier plus the gate list.

#!/usr/bin/env python3
"""triage.py - emit the cheapest verification tier that can catch this diff.

Usage: python3 triage.py [base_ref]   (default: HEAD~1)
Output: JSON on stdout.
"""
import json
import re
import subprocess
import sys

TEST_PATH = re.compile(
    r"(^|/)(tests?|spec|__tests__)(/|$)"
    r"|(_test\.py$|^test_.*\.py$|\.test\.[jt]sx?$|\.spec\.[jt]sx?$)"
)
SRC_PATH = re.compile(r"\.(py|ts|tsx|js|jsx|go|rs|java|kt|rb|php|cs)$")
DEP_PATH = re.compile(
    r"(^|/)(package-lock\.json|pnpm-lock\.yaml|poetry\.lock"
    r"|requirements.*\.txt|pyproject\.toml|go\.sum|Cargo\.lock)$"
)
CI_PATH = re.compile(
    r"(^|/)(\.github/workflows/|\.gitlab-ci\.yml|Makefile|tox\.ini|noxfile\.py)"
)
ASSERT_LINE = re.compile(r"\+(?!\+\+).*\b(assert\b|expect\s*\(|require\.\w+\()")

GATES = {
    0: ["lint"],
    1: ["lint", "unit-suite"],
    2: ["lint", "unit-suite", "property-checks", "fixture-lock"],
    3: ["lint", "unit-suite", "property-checks", "fixture-lock",
        "order-permutations", "freeze-audit", "human-signoff"],
}


def git(*args):
    return subprocess.run(("git",) + args, capture_output=True,
                          text=True, check=True).stdout


def numstat(base):
    rows = []
    for line in git("diff", "--numstat", base, "HEAD").splitlines():
        add, dele, path = line.split("\t", 2)
        rows.append({"path": path,
                     "add": 0 if add == "-" else int(add),
                     "del": 0 if dele == "-" else int(dele)})
    return rows


def signals(rows, base):
    src = [r for r in rows
           if SRC_PATH.search(r["path"]) and not TEST_PATH.search(r["path"])]
    tst = [r for r in rows if TEST_PATH.search(r["path"])]
    added = removed = 0
    for line in git("diff", "-U0", base, "HEAD", "--").splitlines():
        if ASSERT_LINE.match(line):
            added += 1
        elif line.startswith("-") and not line.startswith("---") \
                and ASSERT_LINE.match("+" + line[1:]):
            removed += 1
    return {
        "prod_files": len(src),
        "test_files": len(tst),
        "src_churn": sum(r["add"] + r["del"] for r in src),
        "asserts_added": added,
        "asserts_removed": removed,
        "deps_changed": any(DEP_PATH.search(r["path"]) for r in rows),
        "ci_changed": any(CI_PATH.search(r["path"]) for r in rows),
    }


def tier(s):
    if s["deps_changed"] or s["ci_changed"]:
        return 3, "dependency or CI surface moved"
    if s["prod_files"] and not s["test_files"]:
        return (2 if s["src_churn"] > 30 else 1), "production change, no test change"
    if s["prod_files"] and s["test_files"]:
        if s["asserts_added"] == 0:
            return 3, "tests edited but no assertion added"
        if s["asserts_removed"] > s["asserts_added"]:
            return 3, "more assertions removed than added"
        return 2, "production and test change together"
    if s["test_files"]:
        return 1, "test-only change"
    return 0, "docs or comments only"


if __name__ == "__main__":
    base = sys.argv[1] if len(sys.argv) > 1 else "HEAD~1"
    sig = signals(numstat(base), base)
    t, why = tier(sig)
    print(json.dumps({"tier": t, "reason": why,
                      "gates": GATES[t], "signals": sig}, indent=2))

Run it before you run anything else:

python3 triage.py origin/main

Sample output for an agent patch that edited three tests and removed two assertions without adding any:

{
  "tier": 3,
  "reason": "tests edited but no assertion added",
  "gates": ["lint", "unit-suite", "property-checks", "fixture-lock", "order-permutations", "freeze-audit", "human-signoff"],
  "signals": {"prod_files": 0, "test_files": 3, "src_churn": 0, "asserts_added": 0, "asserts_removed": 2, "deps_changed": false, "ci_changed": false}
}

That path escalates on purpose. A patch that only touches tests and only removes assertions is the exact shape you want a human to look at.

Wire it in, in five steps

1. Land triage.py in scripts/ and make it a one-liner in your CI job, before the test step.
2. Make the runner read the JSON and short-circuit. Tier 0 runs lint and stops.
3. Keep tier 3 as a blocking human gate. Do not let it auto-merge on green.
4. Log tier and reason next to every build. After two weeks you will know whether the thresholds match your repo.
5. Tune the 30 in src_churn > 30 from that log, not from intuition.

The runner itself is boring, which is the point:

#!/usr/bin/env bash
set -euo pipefail
python3 scripts/triage.py "${1:-HEAD~1}" > /tmp/plan.json
cat /tmp/plan.json
tier=$(python3 -c 'import json;print(json.load(open("/tmp/plan.json"))["tier"])')
case "$tier" in
  0) ./scripts/lint.sh ;;
  1) ./scripts/lint.sh && ./scripts/test.sh ;;
  2) ./scripts/lint.sh && ./scripts/test.sh \
       && ./scripts/properties.sh && ./scripts/fixture-lock.sh ;;
  *) ./scripts/full-matrix.sh ;;
restart_placeholder
esac

Where a free model tier actually fits

One step in this workflow is high-volume, low-stakes, and wrong a lot: drafting the candidate property list for a tier-2 patch. That is a good job for a free model tier, because every draft is validated by an executable check before it counts.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The second fit is the runner. Tier 0 and tier 1 gates are cheap and should never queue behind the main CI fleet, so an always-on free server running the fast path is a reasonable use of one. Both availability claims — the free model access and the free server option — come from the project operator; check the current terms on the project page yourself before you plan capacity around them.

What must stay outside the model's hands: the tier decision, the pass/fail verdict, and the human sign-off. Those are the parts where a plausible wrong answer is expensive.

Property checks by patch shape

A property that cannot be executed is not a property. Keep the list short, executable, and tied to the diff.

Patch shape	Property to try first	Cheap check
Pure transform	Round-trip: decode(encode(x)) == x	200 generated inputs
Parser or tokenizer	Never raises on arbitrary bytes; returns a result or a typed error	1k fuzz iterations
State machine	No transition lands outside the declared state enum	Enumerate all edges
Adapter around an external API	Every timeout path returns the documented error type	Stub the client, inject failure
Cache or memo layer	Read-after-write never returns a stale value for the same key	Handler-level test

If tier 2 cannot produce at least one executable property for the changed file, that is itself a signal. It usually means the patch boundary is unclear, not that the code is safe.

Freezes need an expiry the router can read

An unexpiring flaky-test freeze is just a deleted test with extra ceremony. Give each record an owner and a date, and let the router see it.

{
  "test": "tests/test_billing.py::test_proration_rounding",
  "owner": "finley",
  "reason": "upstream FX feed returns stale rates after 22:00 UTC",
  "frozen_at": "2026-10-30",
  "expires": "2026-11-13"
}

Add one rule on top of the router: if any freeze record whose test path overlaps the diff has an expires in the past, force tier 3. Expired freezes are not noise. They are unverified claims that nobody has revisited.

Limitations and who should not use this

The heuristics read diff shape, not semantics. A four-line change to an auth check can be more dangerous than a 400-line refactor, and this router will happily assign it tier 1. Reviewers are still the last gate.

Path regexes break on monorepos with co-located tests (foo.test.ts next to foo.ts in the same directory is fine; __tests__ with a custom convention is not). Adjust the patterns before you trust the tier. The assertion counter is a lexical match and will miss custom assertion helpers.

Skip this approach if you have a mandatory full-matrix policy you cannot change, if your suite is fast enough that running everything is genuinely cheap, or if you work in a domain where every diff needs full review regardless of shape. In those cases the router adds a layer without removing work.

Start narrow

Run the router in log-only mode for a week. Compare the tier it would have assigned against what you actually ran. You will find two or three misroutes, fix the regexes, and then the short-circuit is safe to turn on.

If you want to try it with a free model drafting the property lists and a small always-on machine running the tier-0 and tier-1 path, the project page for MonkeyCode lists the current free model access and free server option — verify the terms there first, and start with tier 1 on a repository you already trust.