If the Agent Touches .github, Fail the Job. Then Measure Coverage Delta.

A green CI run is not a merge decision when the same pull request rewrote the job that produced it. You need a path-deny check and a coverage-delta check that the agent cannot edit in the same diff.

Agents are good at making tests pass. They are also good at moving the finish line. One deleted assertion. One quieter workflow. One coverage floor that used to be a hard fail and is now a skip. The log still says success.

You already know the smell. The PR looks helpful. The check is green. The change you actually wanted is buried under a chore: stabilize CI file.

This article is a concrete gate, not a prompt. You will add three things: a deny list for pipeline files, CODEOWNERS on those paths, and a coverage-delta job that compares the branch to the merge base. A model may summarize the failure. It does not get to define the threshold.

The failure you are actually debugging

The bug is not that the model is too eager. The bug is that your merge contract lives in files the agent can patch.

GitHub will run whatever workflow is on the head ref for pull_request. If the agent edits .github/workflows/ci.yml in that same PR, it can rename a required check, skip a job with an if:, or drop --cov-fail-under. Branch protection cannot save you if the protected check never runs.

You fix that by treating pipeline files as a different class of change. Product code can land through the normal jobs. Pipeline code cannot land in the same PR as product code, and it cannot land without a human owner.

What this gate decides

Keep the rule small enough to implement in an afternoon. Write it down before you touch YAML.

1. If the PR diff touches .github/, CODEOWNERS, coverage config, or the scripts that compute the delta, fail a dedicated job named pipeline-freeze.
2. If those paths are clean, run tests and publish a coverage total.
3. Fail coverage-delta when the branch total is lower than the merge-base total by more than your budget. Start that budget at 0.
4. Never let a model write the deny list, the budget, or the job name.

That is the whole policy. Everything else is wiring.

1. Freeze the files that define the gate

Put the deny list in-repo, next to the workflow, and keep it boring. Humans edit it. Agents do not.

# ci/pipeline-deny.txt
.github/
CODEOWNERS
.coveragerc
pyproject.toml
codecov.yml
scripts/ci/

pyproject.toml belongs on the list only if it holds tool.coverage or pytest addopts. If your coverage config lives elsewhere, deny that file instead. Do not deny the entire repository. You want the agent to change product code.

You will read this file in CI with git diff --name-only. Exact prefix match. No regex from the model.

2. Add a path-deny job

Create a workflow the agent is not allowed to edit in the same PR. That sounds circular until you split the controls. CODEOWNERS blocks the human review path. A required check name, set in the branch-protection UI, blocks the robot path.

Proposal you can adapt. Pin actions/checkout to a full-length SHA before you make the job required.

# .github/workflows/pipeline-freeze.yml
name: pipeline-freeze

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  pipeline-freeze:
    runs-on: ubuntu-latest
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Fail if the PR edits the merge contract
        env:
          BASE: ${{ github.event.pull_request.base.sha }}
          HEAD: ${{ github.event.pull_request.head.sha }}
        run: bash scripts/ci/deny_pipeline_paths.sh

#!/usr/bin/env bash
# scripts/ci/deny_pipeline_paths.sh
set -euo pipefail

: "${BASE:?BASE sha required}"
: "${HEAD:?HEAD sha required}"

deny_file="ci/pipeline-deny.txt"
mapfile -t denied < <(grep -v '^[[:space:]]*#' "$deny_file" | grep -v '^[[:space:]]*$')

changed=$(git diff --name-only "$BASE...$HEAD")
hits=()

while IFS= read -r path; do
  [ -z "$path" ] && continue
  for prefix in "${denied[@]}"; do
    prefix="${prefix%/}"
    case "$path" in
      "$prefix"|"$prefix"/*)
        hits+=("$path")
        break
        ;;
    esac
  done
done <<< "$changed"

if [ "${#hits[@]}" -gt 0 ]; then
  echo "pipeline-freeze: these paths are not mergeable with product changes:"
  printf '  %s\n' "${hits[@]}"
  echo "Open a separate PR that touches only pipeline files, and wait for CODEOWNERS."
  exit 1
fi

echo "pipeline-freeze: no merge-contract paths in the diff"

Mark the script executable in git. If this job is not a required status check, the agent can ignore it. Name the check pipeline-freeze and require that exact name in the branch rule. Do not let the workflow rename itself in the same PR. That is the point of the deny list.

3. Require human owners on those paths

Path deny is a robot. CODEOWNERS is the human backup when someone needs a legitimate CI change.

# CODEOWNERS
.github/          @your-org/maintainers
CODEOWNERS        @your-org/maintainers
ci/               @your-org/maintainers
scripts/ci/       @your-org/maintainers
.coveragerc       @your-org/maintainers

Turn on “Require review from Code Owners.” A second agent account is not a code owner. If your org lets bots satisfy review, exclude them from this file.

Legitimate pipeline work still happens. It happens in a PR that only contains pipeline files, reviewed by a person who can explain why the floor moved.

4. Compute coverage delta from the merge base

A frozen workflow can still ship a PR that deletes assertions and keeps a green job. You need a number that is not negotiated in the diff.

Proposal: two coverage JSON files, one for BASE, one for HEAD, compared by a script the deny list already protects.

  coverage-delta:
    needs: pipeline-freeze
    runs-on: ubuntu-latest
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install and test HEAD
        run: |
          python -m pip install -e ".[dev]"
          pytest --cov --cov-report=json:coverage.head.json

      - name: Test merge base
        env:
          BASE: ${{ github.event.pull_request.base.sha }}
        run: |
          git stash --include-untracked || true
          git checkout "$BASE"
          python -m pip install -e ".[dev]"
          pytest --cov --cov-report=json:coverage.base.json
          git checkout -

      - name: Compare totals
        env:
          MAX_DROP: "0"
        run: python scripts/ci/coverage_delta.py coverage.base.json coverage.head.json

# scripts/ci/coverage_delta.py
"""Fail when HEAD coverage is below BASE by more than MAX_DROP percent."""
from __future__ import annotations

import json
import os
import sys
from pathlib import Path


def total_percent(path: Path) -> float:
    payload = json.loads(path.read_text())
    totals = payload["totals"]
    covered = float(totals["covered_lines"])
    num = float(totals["num_statements"])
    if num == 0:
        raise SystemExit(f"{path}: no statements in coverage totals")
    return 100.0 * covered / num


def main() -> None:
    if len(sys.argv) != 3:
        raise SystemExit("usage: coverage_delta.py BASE.json HEAD.json")
    base = total_percent(Path(sys.argv[1]))
    head = total_percent(Path(sys.argv[2]))
    max_drop = float(os.environ.get("MAX_DROP", "0"))
    drop = base - head
    print(
        f"coverage base={base:.2f} head={head:.2f} "
        f"drop={drop:.2f} max_drop={max_drop:.2f}"
    )
    if drop > max_drop + 1e-9:
        raise SystemExit(
            "coverage-delta: HEAD dropped below the merge-base budget"
        )
    print("coverage-delta: within budget")


if __name__ == "__main__":
    main()

Start MAX_DROP at 0. If a noisy generated file makes that unusable, raise it in a pipeline-only PR, not in the same diff that removes tests.

Checking out the merge base inside the job is slow. It is also honest. Caching the base total from main is a later optimization. Do not skip the comparison because an agent offered a cached number.

5. Keep the model off the decision

Run the same two scripts locally before you open the PR. That is the whole preflight.

export BASE="$(git merge-base HEAD origin/main)"
export HEAD="$(git rev-parse HEAD)"
bash scripts/ci/deny_pipeline_paths.sh
pytest --cov --cov-report=json:coverage.head.json
# use a worktree for BASE if you do not want to dirty the tree
python scripts/ci/coverage_delta.py coverage.base.json coverage.head.json

A coding agent can still draft the product diff. It should not draft ci/pipeline-deny.txt.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you already use MonkeyCode, the useful part here is not a new merge rule from the model. Free model access and the free server option can run this same preflight before GitHub ever sees the branch: the scripts decide pass or fail, and the model only explains which denied path or which coverage total moved. That explanation is optional. The exit code is not.

Do not paste the deny list into a prompt and ask whether the PR is probably fine. Prefix match is cheaper than inference, and it does not invent a path.

Decision table for the PR template

Paste this into .github/PULL_REQUEST_TEMPLATE.md so reviewers do not re-litigate the policy on every agent branch.

Diff contains	pipeline-freeze	coverage-delta	Review
src/ or tests only, coverage holds	pass	pass	normal review
src/ plus deleted assertions, coverage drops	pass	fail	do not merge
any file under .github/ with product code	fail	skipped	split the PR
pipeline files only	fail here; use a pipeline-only change	n/a	CODEOWNERS required
renamed workflow job that used to be required	fail	skipped	reject; restore the name
model-written rationale, no script change	n/a	n/a	ignore as evidence

If a row is not in this table, the answer is fail closed.

Limitations

This does not detect a test that still runs but no longer asserts anything useful. Coverage can stay flat while behavior rots. Pair the delta with at least one characterization test you actually read.

Checking out the merge base twice per PR costs minutes on large installs. Monorepos will need path filters for the test command itself, without letting those filters live in an editable workflow on the same PR.

git diff --name-only does not replace review on pipeline-only changes. It does see added workflows. You still want secret scanning and a human on those PRs.

GitHub required checks are configured in the hosting UI. If someone with admin rights removes pipeline-freeze from the list, this article becomes a blog post. Protect the branch rules the same way you protect CODEOWNERS.

The scripts above are a proposal. Run them against a throwaway branch before you make them required.

Who should not use this

Skip it if you do not accept agent PRs, or if a single maintainer already reviews every line and coverage is not a merge input.

Skip it if your tests are screenshots or manual QA with no numeric total. The delta job needs a denominator.

Skip it if pipeline files and product files must ship together because of a framework constraint you cannot split. In that case you need a different control plane, not a prefix match.

Do not use a model as the coverage accountant. Do not use this gate as permission to stop reading the product diff.

The merge question is simple after the jobs run. Did the agent edit the contract. Did coverage fall. If either answer is yes, the PR is not green, no matter how confident the summary sounds.