Test the Merge Result, Not the Branch: A Green-to-Merge Contract for CI

Most CI systems answer the wrong question. They tell you whether your branch passes tests on a base commit that has already moved on. The tree you actually ship is the merge result, and nobody tested it.

This post is about closing that gap with three small pieces: a local merge check, a gate receipt that the merge queue can verify, and a flake budget that expires. None of it requires a new CI vendor.

1. Three ways a green PR head lies to you

A green check is a claim about head_sha on top of base_sha_at_push_time. That pair is already history by the time you merge.

• Semantic conflict. PR A renames a function's parameter; PR B adds a caller using the old name. Both diffs are clean, the textual merge is clean, the merged tree does not compile.
• Merge-only resolution. Lockfiles, generated clients, and changelogs resolve differently in the merged tree than in either branch. Your branch tested the lockfile it wrote, not the one that survived the merge.
• Order dependence. A is green before B, B is green before A, A+B is red. Neither author sees the failure; the default branch does.

If your team merges more than a few PRs a day into one branch, at least one of these is happening weekly. The fix is not more tests on the branch. It is one test on the merge.

2. Rebuild the merge locally in one script

Before you change CI, prove the gap exists. This script builds the merge in a throwaway worktree and runs your existing gate inside it. Git 2.5+, no plugins.

#!/usr/bin/env bash
# scripts/merge-check.sh <base-ref> [head-ref]
set -euo pipefail
base="${1:-origin/main}"
head="${2:-HEAD}"
tmp="$(mktemp -d)"

git fetch --quiet origin "$base" || true
git worktree add --detach "$tmp" "$base" >/dev/null
trap 'git worktree remove --force "$tmp" >/dev/null 2>&1 || true' EXIT

git -C "$tmp" merge --no-ff --no-edit "$head" >/dev/null
merge_tree="$(git -C "$tmp" rev-parse 'HEAD^{tree}')"
head_sha="$(git rev-parse "$head")"
echo "merge_tree=$merge_tree head_sha=$head_sha"

( cd "$tmp" && ./scripts/gate.sh )   # your existing lint + unit + integration

Run it against your last twenty merges. If you want a cheaper pre-check without touching the working tree, git 2.38+ gives you the tree object directly:

git merge-tree --write-tree origin/main HEAD || echo 'textual conflict, stop here'

Step order matters: textual conflicts are cheap to detect and should fail first. Only spend CI minutes on a tree that at least merges.

3. Make each gate emit a receipt, not a badge

A badge says "green". A receipt says what was green, on which tree, for how long. The merge queue needs the second one, because a stale receipt is worse than no receipt.

{
  "schema": 1,
  "gate": "unit",
  "base_sha": "9f1c2ab",
  "head_sha": "4d0e77c",
  "merge_tree": "b31a8e0",
  "result": "pass",
  "quarantined": ["tests/test_clock.py::test_dst_boundary"],
  "duration_s": 412,
  "started_at": "2026-11-05T09:12:44Z"
}

Each gate writes one file into receipts/. The queue verifies them before it merges, and it refuses receipts that disagree on merge_tree:

# scripts/verify_receipts.py
import json, pathlib, sys

REQUIRED = {"lint", "unit", "integration"}
receipts = {}
for path in pathlib.Path("receipts").glob("*.json"):
    r = json.loads(path.read_text())
    receipts.setdefault(r["gate"], []).append(r)

missing = REQUIRED - receipts.keys()
if missing:
    sys.exit(f"missing gates: {sorted(missing)}")

trees = {r["merge_tree"] for group in receipts.values() for r in group}
if len(trees) != 1:
    sys.exit(f"receipts disagree on merge tree: {sorted(trees)} — stale gate, rerun")

failed = [g for g, group in receipts.items() if any(r["result"] != "pass" for r in group)]
if failed:
    sys.exit(f"failed gates: {sorted(failed)}")
print("all receipts agree on", trees.pop())

That single len(trees) != 1 check is the highest-value line in this article. It catches the classic race where one job retried on an older base and still reported green.

4. Label the failure, and don't trust the label

When the merged tree fails but both branches were green, the first question is always the same: is this a real defect or a flake? Getting that wrong in either direction costs an hour.

A model is decent at this as a first pass, because failure logs are long and repetitive and the signal is usually in ten lines. It is not good enough to be the decision. So: the label goes into the receipt as an advisory field, and the human keeps the merge button.

I use MonkeyCode's free model access for exactly this narrow step — a short classification call over the trimmed failure log, output constrained to a small enum. The operator also offers a free server option, which is enough to run the merge-check job for a small repository without tying up a laptop; treat that machine as ephemeral, with no SLA and no assumption that state survives.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Availability claims are the operator's, not a benchmark: at the time of writing the operator describes a free token allowance on the order of ten million and a free server option. Quotas, model rosters, and terms change without notice, so check the project's current terms before you put a number in a budget.

Keep the call small and constrain the output:

PROMPT = """Classify this CI failure log. Reply with JSON only:
{"label": "product"|"flake"|"infra", "confidence": 0.0-1.0, "evidence": "one line"}
Rules: product = deterministic defect in the diff; flake = timing/order/network nondeterminism;
infra = runner, disk, registry, or credentials. Quote the log line you relied on.
"""

Store the log and the returned label side by side as a fixture under test/fixtures/triage/. Your test asserts the schema and the enum — never string equality against a model response. When a human overrides a label, append both labels to the fixture. After a few weeks you have a labeled set you can reason about, which is the part that actually improves triage.

5. A flake budget that expires

Quarantine without an expiry date is deletion with extra steps. Encode both limits in a file the CI reads:

# .ci/flake-budget.yaml
budget: 3                  # max simultaneously quarantined tests
window_days: 7
quarantine_max_age_days: 14

Now the check is boring and absolute: more than three entries fails the build, and any entry older than fourteen days fails the build. Either someone fixes the test or someone deletes it, and both outcomes are honest. A quarantined test that still appears in the receipt's quarantined array is visible to reviewers, so "we skipped it" never becomes invisible.

6. When this pays off, and when it doesn't

Situation	Run the merge check?	Why
5+ PRs/day into one branch	Yes	Order dependence is near-certain
Generated files or lockfiles in the repo	Yes	Merge-only resolution is common
Docs-only changes	No	Skip by path filter, save the minutes
Monorepo with 40-minute gates	Partial	Merge check on affected packages only
Solo repo, one PR at a time	No	The branch is the merge result

7. Limitations, and who should skip this

This adds a second CI run per PR. If your gates take longer than about fifteen minutes, run the merge check only at queue time, not on every push.

Receipts are an audit trail, not a security control. Anyone with write access to the artifact store can forge one; if you need signed provenance, use your platform's attestation mechanism instead of a homegrown JSON file.

And if your platform already ships a merge queue that tests the merged result, use it. Writing this yourself is for teams whose queue exists but whose gates report on the wrong tree — a narrower problem than it sounds.

Finally, do not let an advisory model label gate a release. If a failure needs a compliance decision, a human reads the log.

8. Twenty-minute rollout

1. Run scripts/merge-check.sh against your last twenty merges; count how many merged trees would have failed.
2. If the count is zero, stop. You have the data and you saved yourself a project.
3. If it is non-zero, add the receipt fields to your three main gates.
4. Wire verify_receipts.py into the merge step, not the test step.
5. Add .ci/flake-budget.yaml with budget: 0 for a week. Whatever breaks first is your real flake inventory.
6. Raise the budget to 3 and start the fourteen-day clock.

Test the tree you ship. Everything else is a progress bar.