Unmapped Paths Fail Closed: A Diff-to-Fixture Contract for the Merge Queue

A green check that never executed tests for the files in your diff is not a merge signal. It is a hole with a badge. Fail closed when git diff contains a path that no fixture pack owns.

That rule is the article. The rest is a contract you can enforce locally, in CI, and at queue admission.

Whole-suite green hides the wrong skip

Most pipelines still run a default job list. You change internal/billing/, and CI spends its budget on docs snapshots and a flaky browser pack. The billing fixtures never ran. The check is still green.

Agent-authored diffs make the hole larger. They often add a helper, a config stub, and a file you did not ask for. Your suite maps none of those extras. You merge a path that no named test owns.

You do not need a new coverage percentage. You need a path contract.

The contract: every changed path names a pack

Keep a matrix in the repo. Each row is a glob, a fixture directory, a timeout, and a flake policy. CI loads the matrix, intersects it with the merge-base diff, and runs only the matching packs. An unmatched path is a failed job, not a skip.

Here is a starting file. Treat it as a template, not as measured production data.

# ci/path-matrix.yml
version: 1
unmapped: fail
packs:
  - id: billing-unit
    glob: ["internal/billing/**", "pkg/money/**"]
    fixtures: tests/fixtures/billing
    command: go test ./internal/billing ./pkg/money -count=1
    timeout_sec: 180
    flake:
      max_retries: 0
      quarantine_file: ci/quarantine.yml
  - id: http-contract
    glob: ["internal/api/**", "openapi/*.yaml"]
    fixtures: tests/fixtures/http
    command: go test ./internal/api -count=1
    timeout_sec: 240
    flake:
      max_retries: 1
      quarantine_file: ci/quarantine.yml
  - id: ci-self
    glob: [".github/workflows/**", "ci/**", "scripts/ci/**"]
    fixtures: tests/fixtures/ci
    command: python scripts/ci/check_matrix.py --self-test
    timeout_sec: 60
    flake:
      max_retries: 0
      quarantine_file: ci/quarantine.yml

Keep unmapped: fail as the default. If docs should be allowed to skip unit packs, add an explicit docs/** pack. Do not weaken the default so noisy diffs can pass.

Step 1 — Classify the diff from the merge base

Run this from the merge base, not from HEAD~1. Stacked commits lie.

BASE="${GITHUB_BASE_REF:-main}"
git fetch origin "$BASE" --depth=50
git diff --name-only "origin/${BASE}...HEAD" | sort -u > /tmp/changed.txt
cat /tmp/changed.txt

That list is the only input the contract should see. Job names, previous greens, and chat summaries are noise.

Step 2 — Resolve packs, then refuse leftovers

The checker should be boring. Boring is reviewable.

# scripts/ci/check_matrix.py
from __future__ import annotations

import argparse
import fnmatch
import json
import pathlib
import sys

import yaml


def load_matrix(path: pathlib.Path) -> dict:
    data = yaml.safe_load(path.read_text())
    if data.get("version") != 1:
        raise SystemExit("unsupported matrix version")
    return data


def resolve(changed: list[str], matrix: dict) -> tuple[set[str], list[str]]:
    claimed: dict[str, str] = {}
    unknown: list[str] = []
    for path in changed:
        hits = [
            pack["id"]
            for pack in matrix["packs"]
            if any(fnmatch.fnmatch(path, glob) for glob in pack["glob"])
        ]
        if not hits:
            unknown.append(path)
            continue
        claimed[path] = hits[-1]
    return set(claimed.values()), unknown


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--matrix", default="ci/path-matrix.yml")
    parser.add_argument("--changed", default="/tmp/changed.txt")
    parser.add_argument("--out", default="/tmp/promotion.json")
    parser.add_argument("--self-test", action="store_true")
    args = parser.parse_args()

    matrix = load_matrix(pathlib.Path(args.matrix))
    if args.self_test:
        packs, unknown = resolve(["ci/path-matrix.yml"], matrix)
        if "ci-self" not in packs or unknown:
            raise SystemExit("self-test failed")
        print("self-test ok")
        return 0

    changed = [
        line.strip()
        for line in pathlib.Path(args.changed).read_text().splitlines()
        if line.strip()
    ]
    packs, unknown = resolve(changed, matrix)
    record = {
        "changed": changed,
        "packs": sorted(packs),
        "unknown": unknown,
        "unmapped_policy": matrix.get("unmapped", "fail"),
        "matrix_path": args.matrix,
    }
    pathlib.Path(args.out).write_text(json.dumps(record, indent=2) + "\n")
    print(json.dumps(record, indent=2))
    if unknown and record["unmapped_policy"] == "fail":
        print("unmapped paths:", *unknown, sep="\n  ", file=sys.stderr)
        return 2
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

This is a worked example. Port the language later if you need to. The behavior you want is exit code 2 on leftovers, plus a JSON record the queue can fetch.

Step 3 — Use the same script in a pre-push hook

CI-only enforcement is how unmapped files reach the queue. Put the same check in pre-push.

# .githooks/pre-push
#!/usr/bin/env bash
set -euo pipefail
BASE="${UPSTREAM_BASE:-origin/main}"
git diff --name-only "${BASE}...HEAD" | sort -u > /tmp/changed.txt
python scripts/ci/check_matrix.py --changed /tmp/changed.txt --out /tmp/promotion.json

Point Git at that directory once:

git config core.hooksPath .githooks
chmod +x .githooks/pre-push

You now fail before the push. That is cheaper than occupying a red queue slot.

Step 4 — Run only the resolved packs

Do not keep a second, hand-maintained job list. Read promotion.json and execute those commands.

# scripts/ci/run_packs.py
from __future__ import annotations

import argparse
import json
import pathlib
import subprocess
import sys

import yaml


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--record", required=True)
    parser.add_argument("--matrix", default="ci/path-matrix.yml")
    args = parser.parse_args()

    record = json.loads(pathlib.Path(args.record).read_text())
    matrix = yaml.safe_load(pathlib.Path(args.matrix).read_text())
    wanted = set(record["packs"])
    by_id = {pack["id"]: pack for pack in matrix["packs"]}
    missing = sorted(wanted - set(by_id))
    if missing:
        print("unknown pack ids:", missing, file=sys.stderr)
        return 2

    for pack_id in sorted(wanted):
        pack = by_id[pack_id]
        retries = int(pack.get("flake", {}).get("max_retries", 0))
        timeout = int(pack["timeout_sec"])
        attempt = 0
        while True:
            attempt += 1
            print(f"==> {pack_id} attempt {attempt}")
            try:
                subprocess.run(
                    pack["command"],
                    shell=True,
                    check=True,
                    timeout=timeout,
                )
                break
            except (subprocess.CalledProcessError, subprocess.TimeoutExpired) as exc:
                if attempt > retries:
                    print(f"pack {pack_id} failed: {exc}", file=sys.stderr)
                    return 1
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Keep flake retries pack-scoped. A billing unit test that needs a retry is a different incident from a browser pack that always needs one. A repo-wide retry counter hides the pack that is rotting.

Wire both scripts in one workflow. Example GitHub Actions job:

# .github/workflows/path-contract.yml
name: path-contract
on:
  pull_request:
jobs:
  admit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: List changed paths
        run: |
          git diff --name-only "origin/${{ github.base_ref }}...HEAD" | sort -u > /tmp/changed.txt
      - name: Enforce path matrix
        run: python scripts/ci/check_matrix.py --changed /tmp/changed.txt --out /tmp/promotion.json
      - name: Upload promotion record
        uses: actions/upload-artifact@v4
        with:
          name: promotion-record
          path: /tmp/promotion.json
      - name: Run resolved packs
        run: python scripts/ci/run_packs.py --record /tmp/promotion.json --matrix ci/path-matrix.yml

If the matrix job cannot produce the artifact, later admission must not treat a skipped upload as success.

Step 5 — Quarantine by pack, and date the skip

A global flake list becomes folklore. Quarantine rows should name a pack id, a test id, and an expiry.

# ci/quarantine.yml
- pack: http-contract
  test: TestRateLimitWindow
  reason: "clock skew on shared runner; tracked in issue 1841"
  expires: "2026-10-01"

Reject expired rows in run_packs.py before you honor max_retries. An open-ended skip is a merge-policy hole. Date it or delete it.

Step 6 — Admit the queue with the record

A required check named path-contract is still a boolean at the branch-protection layer. Download promotion.json and assert the fields you actually care about.

test "$(jq '.unknown | length' /tmp/promotion.json)" -eq 0
test "$(jq -r '.unmapped_policy' /tmp/promotion.json)" = "fail"
test "$(jq '.packs | length' /tmp/promotion.json)" -gt 0

If those asserts pass, you may enqueue. If they fail, you do not. A green job that cannot produce the record does not count.

Drafting new rows without trusting the draft

When the checker prints an unmapped path, you need a glob and a fixture directory. That edit is mechanical. It is also easy to over-match.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

If you already use MonkeyCode, its free model access and free server option are enough to draft a candidate row from git diff --name-only plus a listing of tests/fixtures/. You still read the glob. You still type the commit. The assistant does not own unmapped: fail.

Keep the prompt small:

Given these unmapped paths:
internal/ledger/entry.go
internal/ledger/entry_test.go
Propose one path-matrix.yml pack row.
Constraints: glob must not include vendor/** or .github/**
fixtures dir must already exist or be named tests/fixtures/ledger
max_retries must be 0
Return YAML only.

Review the glob before you merge it. internal/** is not a pack. It is a surrender.

Who should not use this

This contract does not prove fixtures are meaningful. A pack can run and still assert nothing about the lines you changed. Pair it with tests you already trust.

It does not replace review. An unmapped-path check will not catch a mapped file with a logic error.

Skip it for generated trees, vendored blobs, or monorepos that cannot list globs without a platform team. If a typical diff contains thousands of paths, you need package ownership first.

Do not use this if CI cannot fetch the merge base. HEAD~1 classifies the wrong snapshot.

Do not switch unmapped to warn so agent PRs stop bouncing. Fix the matrix. That bounce is the signal.

Admission checklist

Copy this into the PR template if you want the queue to stay honest:

• [ ] python scripts/ci/check_matrix.py exits 0 on this branch
• [ ] promotion.json lists every changed path under a pack id
• [ ] New globs do not include vendor/**, dist/**, or secret paths
• [ ] Quarantine rows have an expires date
• [ ] core.hooksPath is .githooks and pre-push is executable

You are done when an extra file blocks merge. You are not done when the default suite is green.