Your CI Check Is Boolean. Your Merge Decision Should Not Be.

A green check is one input, not a merge signal. You still need a fixture proof, a flake budget, and a failure taxonomy before the merge button is honest. Treat CI as a state machine with three terminal states—product bug, fixture drift, flake—and refuse merge while the record is unknown.

Boolean gates hide the difference between a real regression and a snapshot that drifted. They also hide infra noise. You retry, you get green, you ship the wrong lesson.

This article is a copyable green-to-merge path. It is a proposal you can run locally; it is not a production study and it does not claim vendor benchmarks.

What the path actually requires

You collect three signals on every change:

1. Fixture proof from a local hook: the contract files you already own still parse and still match the code paths you touched.
2. Flake budget from CI: a named test may fail-and-pass only N times in a window before you must quarantine it.
3. Taxonomy record from the job that just ran: every red JUnit case lands in one bucket, or the required check stays red.

If any signal is missing, you do not merge. Green without a taxonomy file is incomplete, the same way green with a burned flake budget is incomplete.

The artifact: one record per CI run

Keep a machine-readable record in the repo, not a chat comment. Comments rot. Files can fail the required check.

# .ci/taxonomy-record.example.yml
schema: ci-taxonomy/v1
sha: REPLACE_WITH_HEAD_SHA
run_id: "0"
bucket: unknown   # product_bug | fixture_drift | flake | infra | unknown
failed_tests:
  - name: TestInvoiceSnapshot/aligns_line_items
    file: invoice_test.go
    attempts: 1
flake_budget:
  window: 20
  max_flakes: 2
  used: 0
fixture_proof:
  hook: pre-push
  passed: false
notes: "Fill this only after you classify. Do not guess."

unknown is not a soft landing. It is a blocked merge. You either classify with evidence or you keep the branch closed.

Step 1 — Local hook: prove fixtures before the push

Run cheap proofs on your laptop. Do not wait for Actions to tell you a golden file moved.

#!/usr/bin/env bash
# scripts/pre-push-fixtures.sh
# Proposal: install as .git/hooks/pre-push
set -euo pipefail

root="$(git rev-parse --show-toplevel)"
cd "$root"

if [[ ! -d testdata/fixtures ]]; then
  echo "no testdata/fixtures directory; skip proof" >&2
  exit 0
fi

# Fail closed if a fixture is empty or not valid JSON/YAML.
find testdata/fixtures -type f \( -name '*.json' -o -name '*.yml' -o -name '*.yaml' \) |
while read -r f; do
  if [[ ! -s "$f" ]]; then
    echo "empty fixture: $f" >&2
    exit 1
  fi
  case "$f" in
    *.json) python -m json.tool "$f" >/dev/null ;;
    *.yml|*.yaml) python -c "import sys,yaml; yaml.safe_load(open(sys.argv[1]))" "$f" ;;
  esac
done

# Optional: only the fixtures touched in this push.
changed="$(git diff --name-only origin/HEAD...HEAD -- testdata/fixtures || true)"
if [[ -n "$changed" ]]; then
  echo "fixture files in this push:"
  echo "$changed"
fi

echo "fixture proof ok"

Make it executable and wire it once:

chmod +x scripts/pre-push-fixtures.sh
ln -sf ../../scripts/pre-push-fixtures.sh .git/hooks/pre-push

This hook does not replace CI. It only stops the cheapest class of red jobs: broken snapshots you could have seen in two seconds.

Step 2 — CI: keep JUnit and count retries

Your suite must emit JUnit XML. If you swallow flakes with || true, the taxonomy gate has nothing to read.

# .github/workflows/taxonomy-gate.yml
name: taxonomy-gate
on:
  pull_request:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run tests with JUnit
        run: |
          set +e
          mkdir -p artifacts
          # Replace with your runner. Keep XML even on failure.
          pytest -q --junitxml=artifacts/junit.xml
          echo "TEST_EXIT=$?" >> "$GITHUB_ENV"
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: junit
          path: artifacts/junit.xml
      - name: Classify failures
        if: always()
        env:
          HEAD_SHA: ${{ github.sha }}
          RUN_ID: ${{ github.run_id }}
          FLAKE_WINDOW: "20"
          FLAKE_MAX: "2"
        run: |
          python scripts/classify_junit.py \
            --junit artifacts/junit.xml \
            --out artifacts/taxonomy-record.yml
      - name: Required check reads the record
        if: always()
        run: python scripts/enforce_taxonomy.py artifacts/taxonomy-record.yml

The last step is the merge signal. Tests can be red. The check is red until the record is in an allowed terminal state or the suite is fully green with fixture proof and leftover flake budget.

Step 3 — Classify with rules first

Do not start with a model. Start with filenames, messages, and retry counts. Most fixture drift already says snapshot, golden, or testdata in the path.

# scripts/classify_junit.py
# Proposal / worked example. Label buckets from JUnit only.
from __future__ import annotations

import argparse
import os
import xml.etree.ElementTree as ET
from pathlib import Path

FIXTURE_HINTS = ("snapshot", "golden", "fixture", "testdata")
INFRA_HINTS = ("connection reset", "timed out", "503", "runner lost")


def load_failures(path: Path) -> list[dict]:
    if not path.exists():
        return []
    root = ET.parse(path).getroot()
    failed = []
    for case in root.iter("testcase"):
        bad = case.find("failure") if case.find("failure") is not None else case.find("error")
        if bad is None:
            continue
        msg = (bad.get("message") or "") + " " + (bad.text or "")
        failed.append(
            {
                "name": case.get("name") or "",
                "file": case.get("file") or case.get("classname") or "",
                "message": msg.lower(),
            }
        )
    return failed


def bucket_for(item: dict) -> str:
    blob = f"{item['file']} {item['name']} {item['message']}"
    if any(h in blob for h in FIXTURE_HINTS):
        return "fixture_drift"
    if any(h in blob for h in INFRA_HINTS):
        return "infra"
    return "unknown"


def main() -> None:
    p = argparse.ArgumentParser()
    p.add_argument("--junit", type=Path, required=True)
    p.add_argument("--out", type=Path, required=True)
    args = p.parse_args()
    failed = load_failures(args.junit)
    buckets = {bucket_for(i) for i in failed} if failed else set()
    if not failed:
        bucket = "green"
    elif buckets == {"fixture_drift"}:
        bucket = "fixture_drift"
    elif buckets == {"infra"}:
        bucket = "infra"
    elif "unknown" in buckets or len(buckets) > 1:
        bucket = "unknown"
    else:
        bucket = next(iter(buckets))
    args.out.parent.mkdir(parents=True, exist_ok=True)
    args.out.write_text(
        "\n".join(
            [
                "schema: ci-taxonomy/v1",
                f"sha: {os.environ.get('HEAD_SHA', 'unknown')}",
                f"run_id: {os.environ.get('RUN_ID', '0')}",
                f"bucket: {bucket}",
                "failed_tests:",
                *(
                    f"  - name: {i['name']}\n    file: {i['file']}\n    attempts: 1"
                    for i in failed
                )
                or ["  []"],
                "flake_budget:",
                f"  window: {os.environ.get('FLAKE_WINDOW', '20')}",
                f"  max_flakes: {os.environ.get('FLAKE_MAX', '2')}",
                "  used: 0",
                "fixture_proof:",
                "  hook: pre-push",
                "  passed: true",
            ]
        )
        + "\n"
    )
    print(f"wrote {args.out} bucket={bucket}")


if __name__ == "__main__":
    main()

green means the XML had no failures. It still does not mean merge. The enforcer below also checks the flake budget file you keep on main.

Step 4 — Enforce the record as a required check

# scripts/enforce_taxonomy.py
from __future__ import annotations

import sys
from pathlib import Path

ALLOWED_MERGE = {"green", "flake"}  # flake still needs budget; see below
BLOCK = {"unknown", "product_bug", "fixture_drift", "infra"}


def parse_simple_yaml(text: str) -> dict:
    data = {}
    for line in text.splitlines():
        if ":" in line and not line.startswith(" ") and not line.startswith("-"):
            k, v = line.split(":", 1)
            data[k.strip()] = v.strip()
    return data


def main() -> None:
    path = Path(sys.argv[1])
    rec = parse_simple_yaml(path.read_text())
    bucket = rec.get("bucket", "unknown")
    if bucket in BLOCK or bucket not in ALLOWED_MERGE | BLOCK | {"green", "flake"}:
        raise SystemExit(f"taxonomy-gate blocked: bucket={bucket}")
    if bucket == "flake":
        # Replace with your real counter. Fail closed if missing.
        used = int(rec.get("used", rec.get("flake_budget", "0") and 0) or 0)
        raise SystemExit(
            "taxonomy-gate: flake bucket requires an explicit budget file; "
            "refusing closed until you wire scripts/flake_budget.py"
        )
    print(f"taxonomy-gate ok: bucket={bucket}")


if __name__ == "__main__":
    main()

Yes, the flake branch fails closed on purpose. A retry that went green is not a classification. You either quarantine the test or you spend budget from a file you review in the PR.

Step 5 — Optional model pass, only for unknown

Rules will miss messy logs. That is the only place a model belongs in this path. You send the failing test name, the message, and a short slice of the log. You do not send the whole repo. You do not let the model approve the merge.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

If you already use MonkeyCode, free model access and a free server option can sit behind this step: the Actions job uploads JUnit, a small classifier on the free server returns {bucket, rationale}, and the required check still reads a file. Keep secrets in Actions secrets or in the server environment. Do not put tokens in the workflow YAML.

# scripts/classify_unknown.py
# Proposal only. Unexecuted against any vendor. No model name claimed.
import json
import os
import urllib.request

PROMPT = """Classify this CI failure into one bucket:
product_bug, fixture_drift, flake, infra, unknown.
Return JSON only: {{"bucket": "...", "rationale": "..."}}.
Do not invent stack traces. If evidence is thin, use unknown.
TEST: {name}
FILE: {file}
MESSAGE: {message}
"""


def classify_unknown(name: str, file: str, message: str) -> dict:
    base = os.environ.get("MODEL_BASE_URL", "").rstrip("/")
    token = os.environ.get("MODEL_TOKEN", "")
    if not base or not token:
        return {"bucket": "unknown", "rationale": "no model configured"}
    body = json.dumps(
        {
            "messages": [
                {
                    "role": "user",
                    "content": PROMPT.format(name=name, file=file, message=message[:2000]),
                }
            ]
        }
    ).encode()
    req = urllib.request.Request(
        f"{base}/v1/chat/completions",
        data=body,
        headers={
            "Authorization": f"Bearer {token}",
            "Content-Type": "application/json",
        },
        method="POST",
    )
    with urllib.request.urlopen(req, timeout=30) as resp:
        raw = json.loads(resp.read().decode())
    # Parse conservatively. If JSON is missing, stay unknown.
    return {"bucket": "unknown", "rationale": str(raw)[:500]}

Point MODEL_BASE_URL at whatever chat API you already run. If that happens to be MonkeyCode's free model access, the job still must fail closed on unknown. The model is a labeler. The enforcer is the gate.

A free server helps when GitHub retries would otherwise re-spend inference on the same XML. You hash the JUnit file, cache the bucket, and let Actions fetch the cached record. That is an operational split, not a performance claim.

Decision table

Evidence	Bucket	Merge?	Next action
No JUnit failures, hook passed, budget intact	green	Yes	Ship
Failure path contains testdata / snapshot	fixture_drift	No	Update fixture in this PR or revert the code
Same test failed then passed inside the window, under max	flake	Only with quarantine ticket	Spend budget; do not hide XML
Timeout, 503, lost runner	infra	No	Rerun once; if repeat, stop the queue
Assertion on product code with stable fixtures	product_bug	No	Fix the code
Thin log, mixed hints, model unsure	unknown	No	Add a rule or a better log; do not merge

Print this table in the PR template if your team skips YAML. The required check still wins.

Limitations

Heuristic hints collide. A product bug can mention timeout. A fixture test can mention connection. When two buckets match, stay on unknown.

The YAML parser in enforce_taxonomy.py is intentionally tiny. Nested flake counters need a real parser before you trust them.

A model can sound confident and still be wrong. That is why it never writes green and never bypasses the required check.

This path assumes you own the test XML and the hook. If your CI vendor only shows a web UI, you cannot classify what you cannot export.

Who should not use this

Skip it if you have one job and no fixtures. A boolean check is enough.

Skip it if you cannot make a required status check on the default branch. A taxonomy file nobody enforces is documentation, not a gate.

Skip it if your policy forbids sending failure logs to any external model. Keep Step 5 unplugged. Steps 1–4 still work.

Do not use this to paper over a suite that is already 20% flaky. Quarantine first. A taxonomy gate on a burning flake budget just blocks every PR.

Close the loop

Install the hook. Emit JUnit. Fail closed on unknown. Add a model only when the rules stall. If you already have free model access on a free server, wire it to the unknown bucket and leave every other bucket on boring code. That is the whole green-to-merge path: three signals, one file, no merge on a lonely green check.