Bound the Diff Before You Spend a Model Call on an Agent Patch

An agent patch is a cost event before it is a correctness event. If the diff is unbounded, a model-backed property check spends a remote call on a change that a local probe could have rejected. The working rule is narrow: probe on the checkout that already has the repo, price the property lane, and park a flaky test in quarantine with merge credit held at zero.

This is a testing workflow for unattended patches. It is not a benchmark, and it does not report a personal pass rate. The snippets are proposals. They were not executed against a live service for this draft.

What each lane is allowed to emit

Agent patches fail in three ways. Those ways should not share one status bit.

1. The patch is too wide, edits fixtures and tests together, or deletes a check. That is a probe failure. A remote call is not justified.
2. The patch is narrow, fixtures are stable, and the touched files map to named properties. That is a property-lane candidate.
3. A required test is already on the flaky list and the freeze date is still live. That is quarantine. It can delay a signal. It cannot create one.

A single green bit hides which of the three happened. Split the bit, or the cheapest lane will be blamed for a decision the expensive lane never made.

Decision table

Apply this table before any remote call. The numbers are policy defaults for a small service repo. They are not physical constants. Change them in config and keep the change in review.

Signal	Probe	Property lane	Quarantine	Merge credit
More than 400 lines added or removed	Reject	Do not start	Not applicable	0
Fixture path and test path in one diff	Reject	Do not start	Not applicable	0
An assertion or prop_ id removed	Reject	Do not start	Not applicable	0
Narrow diff, fixtures unchanged, properties mapped	Pass	Eligible	Not applicable	Pending human review
Named test is on the live flaky list	Independent	Other properties may run	Open a freeze row	0 from that row
Property result is not hold or fail	Probe stands	Unresolved	Do not convert	0
Every selected property returns hold	Probe stands	Hold	None required	Still pending human review

No row promotes a quarantine record into merge credit. Pending human review is also not a merge.

Step 1. Bound the diff on the checkout

The probe needs git. It does not need a model key.

git diff --numstat origin/main...HEAD
git diff --name-only origin/main...HEAD

Sum the added and deleted columns from --numstat. Treat a missing base ref as a reject. A silent skip would push unknown size into the next lane.

Classify paths with local rules. A fixture path contains fixtures/ or ends in .fixture.json. A test path contains test_ or /tests/. An assertion deletion is a removed line that contains assert, expect(, or the prefix prop_. If classification data is absent, fail closed.

Wide renames will trip the line cap. That is acceptable for an agent lane. A human can raise the cap in the same pull request that introduces the rename, with the reason written next to the constant.

Step 2. Price the property lane from a map

A property is a named statement already stored in the repo. Do not accept a property invented in the patch description.

For a small data service the map can start with three rows.

• store.py selects prop_round_trip and prop_idempotent_put.
• api.py selects prop_reject_empty.
• Any other touched file selects nothing, and nothing means stop.

prop_round_trip says serialize-then-parse returns the same record. prop_idempotent_put says a second put of the same key does not change the stored body. prop_reject_empty says an empty payload remains a client error. The sentences are the spec. The model, if used later, does not get to rewrite them.

Attach a local integer budget to each id, for example 1 unit per property, with a cap of 3 units per patch. These units are an internal planning currency. They are not a statement about any vendor allowance. If the sum exceeds the cap, split the patch. Do not raise the cap to fit the diff.

Step 3. Quarantine a flaky test without paying it

Keep the flaky list in the repo and review it like code.

flaky:
  - id: test_export_race
    expires: 2026-10-22
    owner: ci-team
    reason: shared clock in the export fixture

Compare expires to the job clock. After that date the row is dead, and a dead row must not suppress the test. A live row writes status: quarantined and merge_credit: 0. Other properties may still run. Their results stay on their own ids. Copying a quarantine id into a hold field is a defect, not a shortcut.

The sample expiry is an illustration anchored to an example clock of 2026-10-08. It is not a recommended lifetime for every suite. Pick a date an owner will actually revisit.

Step 4. Encode the table in a router

The module below is a proposal. It does not import an SDK and it does not open a socket. Use it as a contract the CI job can call after it has collected patch facts.

from dataclasses import dataclass, field
from datetime import date

LINE_CAP = 400

@dataclass
class PatchFacts:
    lines_changed: int
    files: list[str]
    assertion_deletions: int
    flaky_ids: list[str]

@dataclass
class Plan:
    probe: str
    properties: list[str] = field(default_factory=list)
    quarantine: list[str] = field(default_factory=list)
    merge_credit: int = 0
    model_call_allowed: bool = False

FIXTURE_MARKERS = ('fixtures/', '.fixture.json')
TEST_MARKERS = ('test_', '/tests/')
PROPERTY_MAP = {
    'store.py': ['prop_round_trip', 'prop_idempotent_put'],
    'api.py': ['prop_reject_empty'],
}

def _is_fixture(path: str) -> bool:
    return any(marker in path for marker in FIXTURE_MARKERS)

def _is_test(path: str) -> bool:
    return any(marker in path for marker in TEST_MARKERS)

def route(facts: PatchFacts, today: date, freeze_expiry: dict[str, date]) -> Plan:
    plan = Plan(probe='reject')
    mixed = any(_is_fixture(p) for p in facts.files) and any(_is_test(p) for p in facts.files)
    if facts.lines_changed > LINE_CAP or facts.assertion_deletions or mixed:
        return plan
    selected: list[str] = []
    for path in facts.files:
        selected.extend(PROPERTY_MAP.get(path, []))
    if not selected:
        return plan
    plan.probe = 'pass'
    plan.properties = selected
    for test_id in facts.flaky_ids:
        expiry = freeze_expiry.get(test_id)
        if expiry is not None and today <= expiry:
            plan.quarantine.append(test_id)
    plan.model_call_allowed = True
    plan.merge_credit = 0
    return plan

model_call_allowed can be true while merge_credit stays 0. That split is intentional. Promotion to a non-zero credit, if you allow it at all, belongs in a later human review step. This function does not perform that promotion.

Step 5. Lock the split with two local tests

These tests do not need a server. They only need the router.

def test_live_freeze_keeps_credit_at_zero():
    facts = PatchFacts(
        lines_changed=40,
        files=['store.py'],
        assertion_deletions=0,
        flaky_ids=['test_export_race'],
    )
    expiry = {'test_export_race': date(2026, 10, 22)}
    plan = route(facts, date(2026, 10, 8), expiry)
    assert plan.probe == 'pass'
    assert plan.properties == ['prop_round_trip', 'prop_idempotent_put']
    assert plan.quarantine == ['test_export_race']
    assert plan.model_call_allowed is True
    assert plan.merge_credit == 0

def test_mixed_fixture_edit_never_arms_a_call():
    facts = PatchFacts(
        lines_changed=20,
        files=['fixtures/order.fixture.json', 'tests/test_order.py'],
        assertion_deletions=0,
        flaky_ids=[],
    )
    plan = route(facts, date(2026, 10, 8), {})
    assert plan.probe == 'reject'
    assert plan.model_call_allowed is False
    assert plan.merge_credit == 0

If a test fails, change the router. Do not relax the assertion to match a convenient result.

Step 6. Read a week of lane logs

Write one JSON object per attempt. Four fields are enough: probe, properties, quarantine, and merge_credit. Add model_call_allowed so spend can be counted without guessing.

python3 - <<'PY'
import json
from pathlib import Path

rows = [json.loads(line) for line in Path('lane-log.jsonl').read_text().splitlines() if line.strip()]
summary = {
    'attempts': len(rows),
    'probe_rejects': sum(r.get('probe') == 'reject' for r in rows),
    'calls_armed': sum(bool(r.get('model_call_allowed')) for r in rows),
    'credit_bugs': sum(bool(r.get('merge_credit')) and bool(r.get('quarantine')) for r in rows),
}
print(json.dumps(summary))
PY

Alert when credit_bugs is non-zero. Do not alert merely because quarantine is non-empty. A live freeze is an expected row, not an incident. The incident is a freeze that moved the merge bit.

The log reader is also unexecuted here. Point it at a real file only after the job writes that shape.

Where a free model lane and a free server fit

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The first legal moment for a remote model is after model_call_allowed is true. MonkeyCode's free model access can execute the already selected property ids. The free server option can host the runner that applies the patch and invokes those ids. Both are availability claims supplied for this draft. They are not a measured quota, a hardware shape, a duration, or a promise that the free lane will stay in place.

Send the property id, a fixture hash, and the patch blob. Require a result token of hold or fail. Anything else, including prose, is unresolved and carries no merge credit. Do not ask the model to invent a new property at call time. That request reopens the unbounded case the probe just closed.

If you already have that free model lane and a spare server, put this router in front of them on one branch and compare probe rejects against armed calls for a week. Keep the product mention out of the merge rule. The merge rule is still the table.

Limitations

• A 400-line cap misclassifies mechanical renames and generated files. Exclude generated paths explicitly rather than lifting the cap for the whole repo.
• Marker strings miss fixtures stored under an unusual directory. Update the markers in the same change that moves the files.
• The property map fails closed on unmapped files. Useful patches will stall until a maintainer adds a row. That stall is cheaper than an improvised check.
• Freeze ownership is part of the design. A row without an owner should be rejected by review, or the list becomes a permanent skip.
• No latency, token, or pass-rate figure is claimed. Nothing in this draft was timed against a hosted endpoint.
• Free access can be withdrawn or reshaped. Do not pin a release gate to a lane you do not control.

Who should not use this

Skip the router when the repo has no named properties and no owner for the flaky list. The table cannot price an empty map. Skip the remote step when policy forbids sending patch contents to a hosted model. Local probe and local tests still stand. Skip the free server when you need a contractual uptime target. A release gate does not belong on an unmetered host.

Also skip the extra lanes for a three-line human fix that already has a reviewer. The process pays for itself on unattended agent diffs. On a reviewed human change it is ceremony.

Bound the diff first. Arm a model call only for a priced property list. Leave flaky tests in quarantine, and keep their merge credit at zero until a person, not a freeze row, says otherwise.