Every New Branch in an Agent Diff Needs a Decision Row

A green test suite does not prove that an agent patch is tested. It proves that existing tests still pass. When the patch adds a predicate, a match arm, or an except clause, the untested outcome is a new failure domain. Coverage totals hide that gap. A high line-coverage report can still miss the false side of a freshly inserted if.

Gate the merge on a decision table extracted from the AST diff. Every new control-flow outcome gets a row. Every row names a collected test. If a row is empty, the patch is incomplete.

Why the green check misses new control flow

Agent patches concentrate edits in conditionals. They insert retries, null checks, feature flags, and fallback parsers. Those edits compile. They often leave the happy path intact. CI stays green.

Incidental coverage is not a row. A test that walks the true branch while asserting an unrelated value does not lock the false branch. Flip the predicate and the suite can still pass.

The artifact below is a local gate. It does not call a model. It compares two Python trees, emits required rows, and fails when tests do not claim them. Treat the scripts as a reference sketch, not as a published package.

1. Collect the Python files in the diff

Run the gate on changed .py files. Exclude generated code if you have a path convention for it. The policy is a closed list of new outcomes, not a coverage percentage.

# illustrative local commands
BASE=$(git merge-base origin/main HEAD)
git diff --name-only --diff-filter=AM "$BASE"...HEAD -- '*.py'

Store both blobs. A three-dot range against the merge base keeps stacked commits honest.

git show "$BASE:$path" > /tmp/base.py   # empty file if the path is new
git show "HEAD:$path" > /tmp/head.py

If the file is new, treat the base as an empty module. Every branch in a new file is a new branch. Do not fingerprint line numbers. Line numbers move.

2. Extract control-flow outcomes from each AST

Walk statements that change control flow: If, Try handlers, Match cases, and loop-else. Skip ordinary and/or inside expressions. Those explode the table without telling a reviewer which path production will take.

# tools/branch_table.py — reference sketch, not executed for this article
from __future__ import annotations

import ast
import hashlib
from dataclasses import dataclass


@dataclass(frozen=True)
class Outcome:
    path: str
    kind: str
    pred: str
    side: str  # true | false | handler | case | else
    lineno: int
    fingerprint: str


def _fp(kind: str, pred: str, side: str) -> str:
    raw = f"{kind}|{pred}|{side}".encode()
    return hashlib.sha256(raw).hexdigest()[:12]


def _pred_src(node: ast.AST | None, src: str) -> str:
    if node is None:
        return "bare"
    return ast.get_source_segment(src, node) or ast.dump(node, include_attributes=False)


class Outcomes(ast.NodeVisitor):
    def __init__(self, path: str, src: str) -> None:
        self.path = path
        self.src = src
        self.rows: list[Outcome] = []

    def _add(self, kind: str, pred: str, side: str, lineno: int) -> None:
        self.rows.append(
            Outcome(self.path, kind, pred, side, lineno, _fp(kind, pred, side))
        )

    def visit_If(self, node: ast.If) -> None:
        pred = _pred_src(node.test, self.src)
        self._add("if", pred, "true", node.lineno)
        self._add("if", pred, "false", node.lineno)
        self.generic_visit(node)

    def visit_Try(self, node: ast.Try) -> None:
        for handler in node.handlers:
            typ = _pred_src(handler.type, self.src)
            self._add("except", f"except {typ}", "handler", handler.lineno)
        if node.orelse:
            self._add("try_else", "try", "else", node.lineno)
        self.generic_visit(node)

    def visit_Match(self, node: ast.Match) -> None:
        subj = _pred_src(node.subject, self.src)
        for case in node.cases:
            pat = _pred_src(case.pattern, self.src)
            self._add("match", f"match {subj} case {pat}", "case", case.pattern.lineno)
        self.generic_visit(node)

    def visit_For(self, node: ast.For) -> None:
        if node.orelse:
            self._add("for_else", _pred_src(node.target, self.src), "else", node.lineno)
        self.generic_visit(node)

    def visit_While(self, node: ast.While) -> None:
        if node.orelse:
            pred = _pred_src(node.test, self.src)
            self._add("while_else", pred, "else", node.lineno)
        self.generic_visit(node)


def outcomes_for(path: str, src: str) -> dict[str, Outcome]:
    visitor = Outcomes(path, src)
    visitor.visit(ast.parse(src))
    return {row.fingerprint: row for row in visitor.rows}


def new_outcomes(path: str, base_src: str, head_src: str) -> list[Outcome]:
    base = outcomes_for(path, base_src) if base_src.strip() else {}
    head = outcomes_for(path, head_src)
    added = [head[key] for key in head.keys() - base.keys()]
    return sorted(added, key=lambda row: (row.path, row.lineno, row.side))

Fingerprint predicate text plus side, not identifiers. An agent can rename a helper and keep the untested branch. The hash then still points at the same outcome.

3. Emit the decision table the patch must fill

Write YAML a reviewer can read. JSON is enough for CI. The table is the contract, not a comment in the pull request.

# tools/emit_table.py — reference sketch
import json
from pathlib import Path


def emit(rows: list[Outcome], dest: Path) -> None:
    payload = [
        {
            "id": row.fingerprint,
            "path": row.path,
            "kind": row.kind,
            "predicate": row.pred,
            "side": row.side,
            "test": None,
        }
        for row in rows
    ]
    dest.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")

A filled table looks like this:

[
  {
    "id": "a1b2c3d4e5f6",
    "path": "src/billing/retry.py",
    "kind": "if",
    "predicate": "attempt < max_attempts and is_transient(err)",
    "side": "true",
    "test": "tests/test_retry.py::test_retries_on_transient"
  },
  {
    "id": "9f8e7d6c5b4a",
    "path": "src/billing/retry.py",
    "kind": "if",
    "predicate": "attempt < max_attempts and is_transient(err)",
    "side": "false",
    "test": "tests/test_retry.py::test_raises_after_exhaustion"
  },
  {
    "id": "112233445566",
    "path": "src/billing/retry.py",
    "kind": "except",
    "predicate": "except TimeoutError",
    "side": "handler",
    "test": "tests/test_retry.py::test_timeout_is_transient"
  }
]

Empty test fields fail the gate. A nodeid that pytest does not collect fails the gate. Leftover fingerprints from a previous patch fail the gate. The table must match the current AST diff, not last week's review notes.

4. Bind rows to collected tests

Do not grep the repo for the predicate string. Agents can add a comment that matches. Bind through pytest collection.

# tools/bind_rows.py — reference sketch
import subprocess
import sys


def collected_nodeids() -> set[str]:
    proc = subprocess.run(
        [sys.executable, "-m", "pytest", "--collect-only", "-q"],
        check=False,
        capture_output=True,
        text=True,
    )
    ids: set[str] = set()
    for line in proc.stdout.splitlines():
        line = line.strip()
        if "::" in line:
            ids.add(line.split()[0])
    return ids


def assert_rows_bound(table: list[dict], nodeids: set[str]) -> list[str]:
    errors: list[str] = []
    claimed: dict[str, str] = {}
    for row in table:
        test = row.get("test")
        label = f"{row['id']} {row['path']} {row['kind']}/{row['side']}"
        if not test:
            errors.append(f"unbound {label}")
            continue
        if test not in nodeids:
            errors.append(f"not collected: {test} for {label}")
            continue
        prior = claimed.get(test)
        if prior and prior != row["id"]:
            # one test may cover several rows; record it, do not auto-fail
            pass
        claimed[test] = row["id"]
    return errors

Optional tightening, still a proposal: require the named test to fail when that branch is inverted. Rewrite only the matching If.test to not (test), run that one nodeid, expect a non-zero exit. Skip the invert when the test is an integration job with external cost. A claimed row that stays green after invert is a weak assertion, not a lock.

A marker-based bind is an alternative if nodeids are unstable:

import pytest

pytestmark = pytest.mark.branch("a1b2c3d4e5f6")


def test_retries_on_transient():
    ...

Collection then has to export marker arguments. Nodeids are simpler until tests are parametrized heavily.

5. Fail CI on an incomplete table

Keep the control flow boring. The job should be a failed exit code, not a dashboard.

1. Compute the merge base and the changed Python files.
2. For each file, extract new outcomes against the base blob.
3. If new outcomes exist and branch_rows.json is missing, write a stub table and fail with its path.
4. If the table exists, require an exact fingerprint set: no missing ids, no leftovers.
5. Collect pytest nodeids and reject unbound rows.
6. Optionally invert each new if and run the claimed nodeid.

# .github/workflows/branch-rows.yml — illustrative
name: agent-branch-rows
on:
  pull_request:
jobs:
  rows:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install pytest
      - run: python tools/check_branch_rows.py

Keep the tool in the repo. A gate that lives only in a chat transcript will not run.

Decision matrix

New syntax in the diff	Rows required	Bind to	Fail if
if / elif	true and false for each test expression	collected nodeid	one side unbound
try / except	one row per handler	collected nodeid	handler never claimed
match / case	one row per case	collected nodeid	case added, no row
for/while else	else row only	collected nodeid	else reachable on empty iterable, unbound
boolean and/or as a value	skip	—	do not explode the table

Short-circuit operators inside ordinary expressions produce too many rows. Limit extraction to statements that change control flow. That is a precision choice, not a completeness proof.

What the table does not replace

Numeric bounds, round-trips, and protocol shapes still need their own checks. Golden payloads still need fixture locks. Order-dependent tests still need an explicit flake policy. This gate answers a narrower question: did the patch add control flow that no test claims.

A claimed row can still be assert True. The invert step is the cheap detector for that class of miss. If invert-and-run stays green, the test did not kill the branch.

Where a free model loop fits

Generating many candidate patches is cheap compared with reviewing them. A local AST gate lets you discard diffs that add branches without rows before a human reads the patch.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode's free model access is enough to draft candidate patches against this gate. The free server option is enough to run the extractor, the pytest collection pass, and the optional invert step without attaching that loop to a billed runner. Neither replaces the table. If the gate is not in the repo, more patches only increase review noise.

The useful loop is mechanical: propose a diff, extract new outcomes, refuse the diff when test is null, keep iterating only when the table is complete. That is a filter. It is not a proof of correctness.

Limitations

The sketch is Python-only. Other languages need their own walkers. Macro-heavy or generated code will emit unstable predicates, and source-text fingerprints will churn.

Agents can satisfy the gate by adding a test that executes the branch and asserts a constant. Invert catches some of that. It will not catch a test that asserts an incidental log line.

Refactor-only patches that rewrite an if into an equivalent match look like new outcomes. Reviewers should accept a table rewrite in that case, not a silent skip. Do not auto-approve fingerprint churn.

Do not use this gate as the only merge check. It does not freeze clocks, identity maps, or fixture bytes. It does not measure production behavior. It does not stop an agent from editing tests if the repo allows that.

Who should not use it: repositories without a collected test runner; monorepos where parsing every changed file is too slow and no path filter exists; data-only patches; and trees where tests are generated at collection time with unstable nodeids. If the agent may rewrite tests/, bind the table first. Otherwise the agent will invent nodeids that match the JSON and delete the real assertions.

Close

Start from the AST diff. Count new outcomes. Demand a row per outcome. Refuse the merge when a row has no collected test. The suite can stay green the entire time and still be incomplete.

If you already generate patches in a loop, point that loop at this gate first. Rejects should die in CI, not in review.