Revert Production. Agent Tests Must Fail.

A parser patch lands in review at 02:11. Twelve tests sit green beside the diff. Coverage rises nine points before dawn. Two days later a trailing delimiter returns None in production.

The agent shipped code and tests together. Reviewers saw one color only. Green never proved the new behavior. It only proved the suite agreed with itself.

A lock is not proven by a key you just cut. You prove it by pulling the bolt. The door must swing open. Agent tests need the same motion. Keep the new tests. Restore production files from the merge base. Run the suite again. Something must fail.

Call that event a revert witness. No failure means the tests do not pin the patch. They exercise old paths, mirrors, or nothing. Review queues now fill with overnight agent diffs. The pattern is stable across languages. Tests arrive in the same commit as the implementation. Continuous integration reports green. Nobody saw the assertion fail against yesterday's tree.

Think of a weigh station on a bridge. Trucks roll across and the gauge never moves. Either the cargo is air or the gauge is painted. Many agent suites paint the gauge. The revert witness is a boot on the scale.

Consider a tiny tag parser under review. The old function splits on commas and keeps blanks.

# tags.py at the merge base
def parse_tags(raw: str) -> list[str]:
    if not raw:
        return []
    return raw.split(",")

The agent rewrites the function to strip tokens. It also drops empty pieces. That change is reasonable on its face. The tests decide whether the change is locked.

# tags.py after the agent patch
def parse_tags(raw: str) -> list[str]:
    if raw is None:
        raise TypeError("raw must be a string")
    parts = []
    for piece in raw.split(","):
        token = piece.strip()
        if token:
            parts.append(token)
    return parts

A busy agent suite can still stay green after a revert. It checks "a,b" and an empty string. Both results already match the merge base.

# test_tags.py — weak suite, no witness
from tags import parse_tags


def test_two_tokens():
    assert parse_tags("a,b") == ["a", "b"]


def test_empty_string():
    assert parse_tags("") == []

Those tests describe yesterday. They never go red when tags.py returns to the base. Coverage still looks healthy. The strip rule is unpaid labor. A witness-capable suite names the new contract. Spaces disappear. Empty slots disappear. None is rejected.

# test_tags.py — witness-capable suite
import pytest
from tags import parse_tags


def test_strips_spaces_around_tokens():
    assert parse_tags("a, b, c") == ["a", "b", "c"]


def test_drops_empty_slots():
    assert parse_tags("a,,b,") == ["a", "b"]


def test_rejects_none():
    with pytest.raises(TypeError):
        parse_tags(None)

Restore tags.py from the merge base and rerun. test_strips_spaces_around_tokens fails. test_drops_empty_slots fails. test_rejects_none fails. The suite has a revert witness. The patch may stay.

The check belongs in a worktree, not a slide. Local staging and CI need different bases. Locally the base is HEAD against the index. On a pull request the base is the merge-base commit. The script below reads WITNESS_BASE and restores production paths only.

#!/usr/bin/env bash
# scripts/revert_witness.sh
set -euo pipefail

ROOT="$(git rev-parse --show-toplevel)"
cd "$ROOT"

BASE="${WITNESS_BASE:-HEAD}"
if [[ "$BASE" == "HEAD" ]]; then
  mapfile -t PROD_FILES < <(
    git diff --name-only --cached --diff-filter=ACMR HEAD -- \
      "*.py" ":!test_*.py" ":!tests/*" ":!*_test.py"
  )
else
  mapfile -t PROD_FILES < <(
    git diff --name-only --diff-filter=ACMR "$BASE"...HEAD -- \
      "*.py" ":!test_*.py" ":!tests/*" ":!*_test.py"
  )
fi

if [[ ${#PROD_FILES[@]} -eq 0 ]]; then
  echo "witness: no production files in scope"
  exit 2
fi

git stash push --keep-index --include-untracked -m "witness-wip" >/dev/null || true
cleanup() {
  git checkout HEAD -- "${PROD_FILES[@]}" 2>/dev/null || true
  git stash pop >/dev/null 2>&1 || true
}
trap cleanup EXIT

git checkout "$BASE" -- "${PROD_FILES[@]}"
set +e
pytest -q --tb=line
status=$?
set -e

if [[ $status -eq 0 ]]; then
  echo "witness: tests stayed green after production revert"
  exit 1
fi

echo "witness: at least one test failed; patch is pinned"
exit 0

Stage the agent diff, then run the local path.

git add tags.py test_tags.py
bash scripts/revert_witness.sh

A weak suite prints a hard reject. A pinned suite prints a single confirmed red. Sample output from the weak pair looks like this.

.. 
witness: tests stayed green after production revert

The witness-capable pair looks like this instead.

F F F
FAILED test_tags.py::test_strips_spaces_around_tokens
FAILED test_tags.py::test_drops_empty_slots
FAILED test_tags.py::test_rejects_none
witness: at least one test failed; patch is pinned

CI should target the pull request base, not HEAD. HEAD on the runner already contains the patch. Set WITNESS_BASE to the merge-base and fetch enough history to check it out.

# .github/workflows/revert-witness.yml
name: revert-witness
on: pull_request
jobs:
  witness:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - run: pip install pytest
      - run: |
          export WITNESS_BASE="${{ github.event.pull_request.base.sha }}"
          bash scripts/revert_witness.sh

Watch the failure class with care. An ImportError after deleting a brand new module is a cheap fail. It does not specify behavior. The suite crashed on a missing file, not on a named example. Add one public case that the merge base cannot satisfy. Then the witness names the contract.

Tautological tests also fool coverage tools. They paste the function body into the assertion. A revert may still pass if the base used the same expression. Compare results to literals and examples. Do not compare results to a second copy of the code.

# tautology: tracks any comma split, including the old one
def test_matches_local_split():
    raw = "a, b"
    assert parse_tags(raw) == [p for p in raw.split(",")]

That test is a shadow, not a spec. The witness can miss it. Delete it during review even if CI stayed red for other reasons. Another miss lives in extracted helpers. The agent pulls out _split and tests _split only. Public parse_tags can still drift. Point the witness at exported functions. Private helpers remain free to change.

When the production change is a deletion, revert restores the old file. Tests that assert absence must fail on restore. That red is correct. Tests that never imported the deleted symbol stay green. Reject those. They never needed the deletion.

Some teams draft the candidate patch on a spare box before the witness runs. MonkeyCode is one option for that draft step. Disclosure: This article was prepared as part of MonkeyCode's product outreach. It offers free model access and a free server option. Use those to produce the diff in a throwaway tree. Run revert_witness.sh on that same tree before a human reads it. The server does not certify the tests. The failed revert does.

Limit the gate. The witness says nothing about product intent. Tests can fail a revert and still encode the wrong rule. A human still reads the examples. The script also assumes git, pytest, and a split between tests and production paths. Monorepos with mixed folders need an explicit path file. Path filters that hide .py production modules will skip the gate in silence. Fail closed when the file list is empty and the diff is not docs-only.

Skip this approach for comment-only edits. Skip it for generated clients whose oracle is a schema file. Skip it while the suite is order-dependent or network-bound. Those failures are noise, not witnesses. Fix isolation first. Then bring the gate back. Random clocks and shared fixtures belong to other checks. This check asks one narrower question. Would these tests pass on yesterday's tree?

Treat the witness as a cheap filter. It does not replace review, types, or staging. It only answers whether the new tests ever needed the new code. If they did not, the green build is a mirror. Mirrors do not hold production. Put the script on the branch and fail the draft there, including drafts made with free model access on a free server, before the review queue sees another all-green parser at 02:11.