Green CI Is Not a Boundary Test: Partition Tables for Agent Patches

An agent patch that only re-runs the suite's happy path is not tested at the edges it changed. Green CI reports that known examples still pass. It does not report that the agent's implicit preconditions hold.

This article proposes a three-part gate: an assumption register, an input partition table with contract predicates, and flake isolation by (test_id, input_hash) instead of skipping a whole file. The workflow is a method, not a measured incident report. Every code block below is a proposed harness, labeled as such, and is not a production dump.

Why the existing suite is the wrong oracle

Agent patches tend to preserve the examples a human already wrote. They also tend to introduce preconditions the suite never named: sorted input, unique keys, non-empty batches, timezone-naive datetimes, a single-tenant cache.

A passing run then means "the recorded examples still match," not "the module still honors its contract." Those are different claims. Treating them as the same claim is how a green bar ships a narrower function.

The failure mode is consistent. The agent specializes production code to the fixtures. The fixtures were never a specification. Boundary classes stay unstated, so they stay untested.

Three artifacts, one gate

The gate does not replace unit tests. It adds a layer that the agent cannot satisfy by cloning the happy path.

1. Assumption register — a checked-in YAML list of preconditions the patch appears to rely on. Each row is accepted by a reviewer or rejected and turned into a failing contract.
2. Partition table — equivalence classes plus boundary rows. Each row carries an input, a relation (not always an exact output), and a contract predicate.
3. Flake isolator — when a row is non-deterministic, freeze that row's hash, not the test file. A frozen row consumes a budget. It must be replaced by a deterministic partition or dropped before merge.

Property checks live in the table, not as unbounded generators. Fixtures are schema-locked rows, not ad-hoc dicts in test bodies. Flakes are isolated, not ignored.

Proposed workflow

The following steps are a procedure you can run on a branch. They are not a claim about a specific outage, latency number, or model ranking.

Step 1 — Diff the patch, do not prompt from memory

Collect the changed symbols and the tests that import them. Limit the register to those symbols. A global "write more tests" prompt invents assumptions the code does not have.

git diff --name-only origin/main...HEAD
git diff origin/main...HEAD -- '*.py'
git log --oneline origin/main..HEAD

If the patch only reformats files or regenerates bindings, stop. An empty partition table is a signal to skip the ritual, not a signal to pad it.

Step 2 — Draft an assumption register from the diff

A free coding model is useful here as a proposer, not as an authority. Feed the diff. Ask only for candidate preconditions, each tied to a file and a symbol. Discard any row that cannot cite both.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access can draft the register from a diff, and its free server option can run the gate in a separate pool from main CI. Those are availability claims. This article does not name models, quotas, hardware, or runtimes, and it does not rank products.

Keep the prompt narrow. One workable shape:

From this diff only, list preconditions the new code appears to assume.
Each item must include: id, symbol, precondition, evidence (quote the diff).
Do not invent symbols. Do not write tests. Status stays "proposed".

Proposed register schema:

# assumptions.yaml — proposed schema, not a live dump
version: 1
patch_ref: "origin/main...HEAD"
items:
  - id: A001
    symbol: "intervals.merge_ranges"
    precondition: "ranges arrive already sorted by start"
    evidence: "loop uses prev_end without an initial sort"
    status: proposed   # proposed | accepted | rejected
  - id: A002
    symbol: "intervals.merge_ranges"
    precondition: "ranges are half-open [start, end) and end >= start"
    evidence: "overlap test is start < prev_end only"
    status: proposed
  - id: A003
    symbol: "intervals.merge_ranges"
    precondition: "all ranges share one timezone-naive epoch"
    evidence: "subtracts datetimes directly"
    status: proposed

A reviewer marks accepted or rejected. Rejected rows become contracts the patch must not rely on. Accepted rows become partition constraints. Unreviewed proposed rows fail the gate. Silence is not assent.

Step 3 — Build the partition table from accepted assumptions

Equivalence partitioning is older than agents. It still fits, because agent failures cluster at class boundaries: empty, one, many, duplicate, unordered, touching, nested, null, mixed types.

The subject under test in this example is a range merger. The example is synthetic. It exists to make the table concrete.

# subject.py — synthetic function under test, not production code
def merge_ranges(ranges):
    """Merge overlapping [start, end) pairs. Proposed example only."""
    if ranges is None:
        raise TypeError("ranges is required")
    ordered = sorted(ranges, key=lambda r: r[0])
    out = []
    for start, end in ordered:
        if end < start:
            raise ValueError("end < start")
        if not out or start > out[-1][1]:
            out.append([start, end])
        elif end > out[-1][1]:
            out[-1][1] = end
    return [tuple(x) for x in out]

A weak suite would store one sorted, already-merged fixture and stop. The partition table names the classes the agent is likely to assume away.

# partitions.py — proposed harness, unexecuted example
from dataclasses import dataclass
from typing import Any, Callable, Tuple

@dataclass(frozen=True)
class Row:
    id: str
    cls: str
    kind: str  # representative | boundary | invalid
    inp: Any
    relation: str
    contract: Callable[[Any], bool]
    assumption_ids: Tuple[str, ...]

def is_merged(result):
    pairs = list(result)
    for start, end in pairs:
        if end < start:
            return False
    for i in range(1, len(pairs)):
        if pairs[i][0] <= pairs[i - 1][1]:
            return False
    return True

def same_cover(result, inp):
    if not inp:
        return result == []
    lo, hi = min(s for s, _ in inp), max(e for _, e in inp)
    return result[0][0] == lo and result[-1][1] == hi and is_merged(result)

PARTITIONS = [
    Row("P001", "empty", "boundary", [],
        "empty input yields empty output",
        lambda r: r == [], ("A001",)),
    Row("P002", "single", "representative", [(0, 3)],
        "a single range is identity",
        lambda r: r == [(0, 3)], ("A002",)),
    Row("P003", "disjoint_sorted", "representative", [(0, 1), (4, 6)],
        "disjoint ranges stay two intervals",
        lambda r: r == [(0, 1), (4, 6)], ("A001", "A002")),
    Row("P004", "disjoint_unsorted", "boundary", [(4, 6), (0, 1)],
        "order of input must not change cover",
        lambda r: r == [(0, 1), (4, 6)], ("A001",)),
    Row("P005", "touching", "boundary", [(0, 2), (2, 5)],
        "half-open touching ranges do not merge",
        lambda r: r == [(0, 2), (2, 5)], ("A002",)),
    Row("P006", "overlap", "representative", [(0, 4), (3, 7)],
        "overlap collapses to one range",
        lambda r: r == [(0, 7)], ("A002",)),
    Row("P007", "nested", "boundary", [(0, 9), (2, 3)],
        "nested range does not extend cover",
        lambda r: r == [(0, 9)], ("A002",)),
    Row("P008", "invalid_none", "invalid", None,
        "None is rejected, not coerced to []",
        lambda r: False, ("A003",)),
    Row("P009", "invalid_inverted", "invalid", [(5, 1)],
        "inverted range raises, not silently swapped",
        lambda r: False, ("A002",)),
]

P004, P005, and P009 are the rows a happy-path suite usually omits. Agents that "simplify" a merger often drop the sort, treat touching as overlap, or swap inverted bounds. The table makes those choices visible.

Optional metamorphic column: if merge_ranges(X) == Y, then merge_ranges(reversed(X)) == Y and merge_ranges(Y) == Y. Idempotence and permutation-invariance are cheap relations. They do not replace the invalid class. They catch specialization to fixture order.

Step 4 — Lock fixtures to a schema, not to a filename

A fixture that is a loose JSON blob will drift. Lock the shape on load. Refuse the gate if a row cannot name at least one assumption. Untethered tests are how the happy path sneaks back in.

# fixture_schema.py — proposed
FIXTURE_SCHEMA = {
    "type": "object",
    "required": ["id", "cls", "kind", "inp", "relation", "assumption_ids"],
    "properties": {
        "id": {"type": "string", "pattern": "^P[0-9]{3}$"},
        "cls": {"type": "string", "minLength": 1},
        "kind": {"enum": ["representative", "boundary", "invalid"]},
        "inp": {},
        "relation": {"type": "string", "minLength": 8},
        "assumption_ids": {
            "type": "array",
            "items": {"type": "string", "pattern": "^A[0-9]{3}$"},
            "minItems": 1,
        },
    },
    "additionalProperties": False,
}

Coverage of the table is not line coverage. Count classes, not statements:

classes_hit = unique row.cls where status == pass
require: every accepted assumption_id appears in at least one boundary or invalid row

If A001 is accepted ("input is sorted") and no unsorted boundary row exists, the register is decorative. Fail the gate for that reason alone.

Step 5 — Run contracts as properties over bound rows

Each row is a property check with a fixed input. That is weaker than a fuzzer and stronger than a single example. It is cheap enough to run on every agent commit.

# gate.py — proposed runner, unexecuted example
import hashlib
import json
import traceback
from pathlib import Path

FLAKE_BUDGET = 2  # rows, not files
RERUNS = 5        # starting point, not a measured optimum
FREEZE_PATH = Path(".gate/frozen_rows.json")

def row_hash(row_id: str, inp) -> str:
    blob = json.dumps({"id": row_id, "inp": inp}, sort_keys=True, default=str)
    return hashlib.sha256(blob.encode()).hexdigest()[:16]

def load_freezes() -> dict:
    if not FREEZE_PATH.exists():
        return {}
    return json.loads(FREEZE_PATH.read_text())

def invoke(fn, row):
    try:
        out = fn(row.inp)
        ok = bool(row.contract(out))
        return "pass" if ok else "fail", out, None
    except Exception as exc:
        if row.kind == "invalid":
            return "pass", None, type(exc).__name__
        return "error", None, traceback.format_exc()

def run_row(fn, row, freezes: dict) -> dict:
    h = row_hash(row.id, row.inp)
    if h in freezes:
        return {"id": row.id, "status": "frozen", "hash": h}
    statuses = []
    last = None
    for _ in range(RERUNS):
        status, out, exc = invoke(fn, row)
        statuses.append(status)
        last = (status, out, exc)
    if len(set(statuses)) > 1:
        return {"id": row.id, "status": "flake", "hash": h, "seen": statuses}
    status, out, exc = last
    rec = {"id": row.id, "status": status, "hash": h}
    if out is not None:
        rec["out"] = out
    if exc:
        rec["exc"] = exc
    return rec

def summarize(results: list[dict]) -> int:
    frozen = [r for r in results if r["status"] in {"frozen", "flake"}]
    failed = [r for r in results if r["status"] in {"fail", "error"}]
    if len(frozen) > FLAKE_BUDGET:
        print(f"flake budget exceeded: {len(frozen)} > {FLAKE_BUDGET}")
        return 2
    for r in failed:
        print(f"{r['status'].upper()} {r['id']} hash={r['hash']}")
    for r in results:
        if r["status"] == "flake":
            print(f"FLAKE {r['id']} hash={r['hash']} seen={r['seen']}")
    return 1 if failed or any(r["status"] == "flake" for r in results) else 0

Run it as a separate job. Keep the command boring.

mkdir -p .gate
python -m gate --fn intervals.merge_ranges --partitions partitions.py
echo $?

A non-zero status is the only merge signal this gate should emit. Pretty logs are optional. The process exit code is not.

A thin pytest adapter, still proposed:

# test_partitions.py — proposed
import pytest
from gate import load_freezes, run_row
from partitions import PARTITIONS
from subject import merge_ranges

@pytest.mark.parametrize("row", PARTITIONS, ids=lambda r: r.id)
def test_partition_row(row):
    rec = run_row(merge_ranges, row, load_freezes())
    if rec["status"] == "frozen":
        pytest.xfail(f"frozen {rec['hash']}")
    assert rec["status"] == "pass", rec

Parametrize by row id. Do not wrap the whole module in a skip marker. That is the behavior this gate exists to prevent.

Step 6 — Isolate flakes by hash; freeze the row, not the file

Flaky tests are usually treated as file-level skips. That hides every other row in the file. Isolate instead.

1. Re-run the failing row in-process (RERUNS, default 5).
2. If outcomes differ, write {hash, row_id, reason, replacement_row} to .gate/frozen_rows.json.
3. Count frozen and flaking rows against FLAKE_BUDGET.
4. Block merge when the budget is exceeded, or when a freeze still has "replacement_row": null after one accepted patch cycle.

{
  "a1b2c3d4e5f60789": {
    "row_id": "P004",
    "reason": "nondeterministic order when sort was removed by the patch",
    "replacement_row": null
  }
}

The freeze is a debt record. It is not a skip annotation. A patch that adds a freeze without a replacement partition has not finished. Raising FLAKE_BUDGET to ship is the same failure mode as @pytest.mark.skip on the file, with extra ceremony.

Decision matrix

Signal	Action	Do not
Happy-path example still passes	Ignore as proof of the patch	Treat green CI as validation
New assumption in the diff	Add a register row	Prompt the model to "write more tests"
Boundary row fails	Fail the gate	Widen the fixture until it passes
Invalid-class row raises	Pass if kind == invalid	Bare except Exception: pass
Row flakes across reruns	Freeze that hash	Skip the file
Frozen rows exceed budget	Block merge	Raise the budget to ship
Model-proposed row with no symbol	Discard	Accept because the prose sounds sure
Accepted assumption with no boundary row	Fail the gate	Count line coverage instead
Format-only patch	Skip the ritual	Invent partitions to look busy

Limitations, and who should not use this

This harness does not measure model accuracy. It does not claim a reduction in incidents, review time, or flake rate. It does not replace type checkers, load tests, or security review. Unbounded property generation is out of scope. If a domain needs random well-formed documents, add a generator later, still bound by the same schema and flake budget.

Who should not use this approach:

• Repos with no reviewer willing to accept or reject assumption rows. The register becomes fiction.
• Teams that need file-level skips to keep CI green. The budget will only get raised.
• Patches that are pure formatting or generated bindings. The table will be empty and the ritual is waste.
• Safety-critical systems that already have a qualified process this article does not meet.

Time-sensitive product limits, model names, and hardware are omitted on purpose. Verify hosted-runner details against current primary docs before depending on any vendor's free tier.

If a write-protected test plane already stops agents from editing tests, this gate sits beside it. The plane protects the suite. The partition table stops production code from specializing to that suite.

A reasonable next step is to run the proposed harness against one pure function and keep the assumption register in the same PR as the agent patch. The method does not depend on a particular proposer or runner; it depends on naming the assumptions the happy path never wrote down.