CI Rebuilds the Test Matrix. The Agent Does Not Get a Vote.

You should not merge an agent patch because the agent said the tests passed. The merge queue has to rebuild the fixture set from the paths that actually changed. Local green is a hint. It is not a ticket.

This matters more now that generated diffs show up in ordinary review queues. A model can emit a plausible Tested-with: trailer, a tidy fixture list, and a green local run. None of that is evidence CI can reuse. Treat the agent's test list as untrusted input, the same way you treat a user-supplied filename.

What goes wrong if you import the list

An agent optimizes for the run it just did. It will drop fixtures that look expensive. It will keep the one test that already passed. It will retry a flaky file until the LED turns green, then hand you that retry count as if it were policy.

You cannot see that from a check mark. You only see a matrix that was chosen by the patch author. When the author is a model, that matrix is part of the patch. It is not a review of the patch.

So split the jobs. Local hooks record paths. Humans own the path-to-fixture map. CI recomputes the matrix at the merge SHA. Flake retries live in a ledger CI writes, not in the PR body.

The selector split

Use two sources of truth, and keep them apart.

1. Path record (untrusted author, trusted git). git diff --name-only against the merge base. This is the only agent-adjacent signal you keep.
2. Fixture map (human-owned). A YAML file on the default branch. Agents may read it. They may not edit it in the same commit that they change product code.
3. Recomputed matrix (CI-owned). A script CI runs on the candidate SHA. Its stdout is the only matrix the merge job is allowed to consume.
4. Flake ledger (CI-owned). Retry counts keyed by fixture id and SHA. Local retries do not transfer.

If a job reads fixtures from a git trailer, a PR label, or a comment, that job is misconfigured. Fail the workflow for that reason alone.

Artifact: recompute from paths, never from trailers

The following is a proposed workflow you can copy. It is not a production benchmark. Label it as a contract you run on a throwaway branch first.

Create ci/fixture-map.yml. Keep it small and boring.

# ci/fixture-map.yml — humans edit this; agents do not in product commits
version: 1
rules:
  - paths: ["src/billing/**"]
    fixtures: ["billing-unit", "billing-contract"]
  - paths: ["src/checkout/**"]
    fixtures: ["checkout-unit", "checkout-http"]
  - paths: ["src/shared/**", "pkg/**"]
    fixtures: ["shared-unit"]
  - paths: [".github/workflows/**", "ci/**"]
    fixtures: ["workflow-lint", "fixture-map-lint"]
require_map_hit: true
max_fixtures: 12

Then add ci/recompute_fixtures.py. It accepts a changed-file list. It refuses to read claimed fixtures.

#!/usr/bin/env python3
"""Recompute CI fixtures from changed paths. Proposed helper, not a product."""
from __future__ import annotations

import argparse
import fnmatch
import json
import sys
from pathlib import Path

try:
    import yaml
except ImportError:
    print("install pyyaml before running this helper", file=sys.stderr)
    sys.exit  = 2  # type: ignore
    raise

FORBIDDEN_FLAGS = ("--claimed", "--agent-list", "--pr-body")


def load_map(path: Path) -> dict:
    data = yaml.safe_load(path.read_text())
    if not isinstance(data, dict) or "rules" not in data:
        raise SystemExit("fixture map missing rules")
    return data


def match_fixtures(changed: list[str], rules: list[dict]) -> tuple[list[str], list[str]]:
    selected: list[str] = []
    unmatched: list[str] = []
    for rel in changed:
        hits = []
        for rule in rules:
            if any(fnmatch.fnmatch(rel, pat) for pat in rule.get("paths", [])):
                hits.extend(rule.get("fixtures", []))
        if hits:
            selected.extend(hits)
        else:
            unmatched.append(rel)
    # stable unique
    seen = set()
    ordered = []
    for name in selected:
        if name not in seen:
            seen.add(name)
            ordered.append(name)
    return ordered, unmatched


def main() -> int:
    for flag in sys.argv:
        if flag.startswith(FORBIDDEN_FLAGS):
            print(f"refusing untrusted fixture source: {flag}", file=sys.stderr)
            return 2

    parser = argparse.ArgumentParser()
    parser.add_argument("--map", required=True)
    parser.add_argument("--changed-file", required=True)
    parser.add_argument("--out", required=True)
    args = parser.parse_args()

    changed = [line.strip() for line in Path(args.changed_file).read_text().splitlines() if line.strip()]
    data = load_map(Path(args.map))
    fixtures, unmatched = match_fixtures(changed, data["rules"])

    if data.get("require_map_hit") and unmatched:
        print("unmapped paths (informational; do not import an agent list to cover them):", file=sys.stderr)
        for rel in unmatched:
            print(f"  {rel}", file=sys.stderr)
        # Fail because the map is incomplete, not because the agent omitted a test.
        return 1

    limit = int(data.get("max_fixtures", 99))
    if len(fixtures) > limit:
        print(f"recomputed matrix too large: {len(fixtures)} > {limit}", file=sys.stderr)
        return 1

    payload = {
        "selector": "ci-recompute",
        "fixtures": fixtures,
        "changed": changed,
        "unmatched": unmatched,
    }
    Path(args.out).write_text(json.dumps(payload, indent=2) + "\n")
    print(json.dumps(payload["fixtures"]))
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Fix the accidental sys.exit = 2 line if you paste carelessly. The import failure should be sys.exit(2) followed by raise. The point of the script is the refusal list: there is no flag that accepts the agent's fixtures.

Numbered path from clone to merge

Run this as a drill on a feature branch. Do not point it at main until the map covers your tree.

1. Classify the patch. Require a trailer Patch-Class: agent or Patch-Class: human on every commit that touches src/. Missing class fails the hook. The class does not choose fixtures. It only chooses how loud the logs are.
2. Record paths only. In a pre-push hook, write git diff --name-only origin/main...HEAD to .ci/changed.txt. Strip any Claimed-Fixtures, Tested-with, or Local-Green trailers before the push proceeds.
3. Block map edits beside product edits. If ci/fixture-map.yml and src/ both appear in the same diff, fail. Map changes need their own review, because they change what CI is allowed to skip.
4. CI recomputes. The first job checks out the candidate SHA, runs recompute_fixtures.py, and uploads matrix.json as an artifact. Later jobs consume that artifact. They do not parse the PR body.
5. CI owns flake. Each fixture may retry at most N times, recorded as fixture_id + sha + attempt. A local retry count in the commit message is ignored. If the ledger would exceed N, the fixture fails. It does not get a silent extra attempt because a model already burned retries on a laptop.
6. Promote only the recomputed green. Merge is allowed when the recomputed matrix is green at that SHA. A green local run with a different matrix is a comment, not a gate.

A minimal hook sketch:

#!/usr/bin/env bash
set -euo pipefail
# proposed pre-push: paths in, claimed fixtures out
range="origin/main...HEAD"
git diff --name-only "$range" > .ci/changed.txt

if git log --format=%B "$range" | grep -Eiq '^(Claimed-Fixtures|Tested-with|Local-Green):'; then
  echo "strip claimed fixture trailers; CI will recompute" >&2
  exit 1
fi

if git diff --name-only "$range" | grep -q '^src/' ; then
  if ! git log --format=%B "$range" | grep -q '^Patch-Class: '; then
    echo "add Patch-Class: agent  or  Patch-Class: human" >&2
    exit 1
  fi
fi

map_changed=$(git diff --name-only "$range" | grep -c '^ci/fixture-map.yml$' || true)
src_changed=$(git diff --name-only "$range" | grep -c '^src/' || true)
if [[ "$map_changed" -gt 0 && "$src_changed" -gt 0 ]]; then
  echo "do not edit the fixture map in the same commit as product code" >&2
  exit 1
fi

Proposed CI job (GitHub Actions). Notice the matrix file is an artifact, not a parameter from the PR.

name: recompute-matrix
on:
  pull_request:
jobs:
  select:
    runs-on: ubuntu-latest
    outputs:
      fixtures: ${{ steps.pack.outputs.fixtures }}
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: list changed paths from git, not from the PR body
        run: |
          mkdir -p .ci
          git diff --name-only origin/${{ github.base_ref }}...HEAD > .ci/changed.txt
      - name: refuse imported lists
        run: |
          if git log --format=%B origin/${{ github.base_ref }}...HEAD | grep -Eiq '^(Claimed-Fixtures|Tested-with):'; then
            echo "untrusted fixture list present; strip it and rerun" >&2
            exit 1
          fi
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install pyyaml
      - id: pack
        run: |
          python ci/recompute_fixtures.py \
            --map ci/fixture-map.yml \
            --changed-file .ci/changed.txt \
            --out .ci/matrix.json
          echo "fixtures=$(python -c 'import json; print(json.dumps(json.load(open(".ci/matrix.json"))["fixtures"]))')" >> "$GITHUB_OUTPUT"
      - uses: actions/upload-artifact@v4
        with:
          name: recomputed-matrix
          path: .ci/matrix.json

  run-fixtures:
    needs: select
    if: needs.select.outputs.fixtures != '[]'
    strategy:
      fail-fast: false
      matrix:
        fixture: ${{ fromJSON(needs.select.outputs.fixtures) }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: run one CI-selected fixture with a CI-owned retry ceiling
        env:
          FLAKE_MAX: "1"
        run: ./ci/run-fixture.sh "${{ matrix.fixture }}"

./ci/run-fixture.sh is yours. Keep FLAKE_MAX in CI. Do not read it from the commit.

Decision table: what may gate merge

Source	Show on the PR?	Gate merge?	Why
git diff paths at candidate SHA	Yes	Indirectly, via recompute	Paths are observed, not claimed
ci/fixture-map.yml at that SHA	Yes	Yes	Human-owned selector
Agent / local fixture list	At most as a comment	No	Author-controlled
PR body, labels, review comments	No	No	Trivial to spoof
Local flake retries	No	No	Burns budget off-ledger
Prior pipeline on another SHA	No	No	Wrong tree
Recomputed matrix job on this SHA	Yes	Yes	Only selector the queue trusts

Print this table in the workflow summary if you want reviewers to stop arguing with the agent log.

Where a free coding environment fits — and where it does not

You can generate the patch on a laptop, in a container, or on a hosted box. The merge rule does not change. The environment that wrote the diff does not select fixtures.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option, which is enough to draft a candidate patch and run a local smoke command. That is a writing environment. It is not your merge queue. Do not point the recompute job at that server. Do not import its logs as Claimed-Fixtures. If you try it, keep the output on a branch and let CI rebuild the matrix from ci/fixture-map.yml at the SHA you intend to merge.

No model names, quotas, or hardware claims are required for this workflow. The workflow only needs git, a map, and a job that refuses extra flags.

Flake control without inheriting the agent's budget

Agents retry. That is their loop. If you copy their retry count into CI, you launder flake into policy.

Cap CI at a small integer. One extra attempt is a reasonable default for known-noisy fixtures. Zero extra attempts is the default for everything else. Write each attempt as JSONL:

{"sha":"abc123","fixture":"checkout-http","attempt":1,"result":"fail","reason":"timeout"}
{"sha":"abc123","fixture":"checkout-http","attempt":2,"result":"pass","reason":"replay"}

If attempt 2 would exceed FLAKE_MAX, stop. Do not look at the commit message to see whether the agent already "proved" it was flaky. Proof of flake is a ticket against the fixture, opened by a human, not a trailer on the patch that needs the fixture to go green.

Limitations

Path maps lie when behavior changes without a path change. A shared library tweak can break billing without touching src/billing/**. If that is your architecture, the map must send pkg/** and src/shared/** through the heavier fixtures, or you run a scheduled full suite that is not part of the merge gate.

Dynamic test discovery (collect whole trees at runtime, then shard) does not need this script. Recompute becomes a no-op if your only matrix is "run everything." In that case the untrusted-list problem shrinks, and you should spend the effort on SHA pinning and flake ledgers instead.

The hook that forbids map edits beside product edits will annoy legitimate refactors. Split those into two PRs. If your team will not do that, the map will rot and CI will fail closed on unmatched paths.

This article does not claim the recompute step catches semantic bugs the fixtures miss. It claims something narrower: the agent does not get to shrink the gate.

Who should not use this

Do not install this if you have no agent traffic and every PR already runs the full suite. You will add YAML without changing risk.

Do not use it as a substitute for code review of .github/workflows/**. A workflow that still reads the PR body can bypass the script.

Do not use it on a repo where ci/fixture-map.yml is generated by the same model that writes src/. That collapses the selector split.

Do not treat a free remote coding box as a self-hosted runner for the merge queue. Different trust domain. Different secrets. Different failure mode.

What to do on the next agent PR

Take one open patch. Write .ci/changed.txt from git diff --name-only. Run the recompute script. Compare the printed fixtures with whatever the agent listed. If the lists differ, keep CI's list. If they match, you still run CI's list, because matching is not the same as importing.

That is the whole rule. The agent can write code. It does not choose the exam.