Fail Closed: A Production-Readiness Checklist for AI Code

For weeks I've fed build logs and broken JSON to free model endpoints. The models were fine. My review process was the bottleneck.

There's a hot take I keep seeing this week: AI turned every developer into a reviewer. Nobody tested the reviewer. I agree. But I want a fix, not a lament.

So I built a fail-closed merge gate. It blocks merges when evidence is missing. No evidence, no merge. Here's the whole thing, plus how I run it on a free server with a free model tier.

Green checks lie

Green checks say "tests passed." They don't say "we know what this code does."

For agent-generated code, I need four answers before merge:

1. Where did this code come from?
2. Did it run anywhere safe?
3. Did tests produce real evidence?
4. Can I undo this in one command?

No answer? Block. That's fail-closed.

The gate script

Copy this into pr_gate.sh:

#!/usr/bin/env bash
# pr_gate.sh — fail-closed production readiness gate
set -euo pipefail

GATES=(provenance sandbox tests rollback)
FAILED=0

run_gate() {
  local name="$1"
  if [[ -x "gates/$name.sh" ]]; then
    echo "==> $name"
    bash "gates/$name.sh" || FAILED=1
  else
    echo "==> $name: MISSING GATE (fail closed)"
    FAILED=1
  fi
}

for g in "${GATES[@]}"; do
  run_gate "$g"
done

if [[ $FAILED -ne 0 ]]; then
  echo "BLOCKED: missing evidence. Fix gates, then re-run."
  exit 1
fi

echo "PASS: every gate produced evidence."

The script is boring on purpose. It runs four files. If a file is missing, it fails. That's the whole trick.

Gate 1: provenance

gates/provenance.sh:

#!/usr/bin/env bash
# Evidence: a label that says who wrote this
MSG=$(git log -1 --format=%B)
if ! echo "$MSG" | grep -qE "^(ai|agent|generated):"; then
  echo "FAIL: no provenance label in commit message"
  exit 1
fi
echo "PASS: provenance label present"

Why? Because I can't review code I can't classify. A label is cheap. It also feeds the provenance log I wrote about last month.

Gate 2: sandbox

gates/sandbox.sh:

#!/usr/bin/env bash
# Evidence: a clean run in a disposable environment
if [[ -z "${SANDBOX_ID:-}" ]]; then
  echo "FAIL: SANDBOX_ID not set"
  exit 1
fi
echo "PASS: ran in sandbox $SANDBOX_ID"

I run this gate on a free server instance. The server is disposable. If the agent code does something nasty, I destroy the box. Nothing touches my laptop.

Gate 3: tests

gates/tests.sh:

#!/usr/bin/env bash
# Evidence: a test log with a real summary
pytest --tb=short -q > /tmp/test.log 2>&1 || true
if ! grep -qE "[0-9]+ (passed|failed)" /tmp/test.log; then
  echo "FAIL: no test summary in log"
  exit 1
fi
if grep -qE "[1-9][0-9]* failed" /tmp/test.log; then
  echo "FAIL: failing tests"
  exit 1
fi
echo "PASS: tests ran and passed"

Exit codes are not evidence. A log line with a count is evidence. That distinction matters.

Gate 4: rollback

gates/rollback.sh:

#!/usr/bin/env bash
# Evidence: a rollback tag exists
TAG="rollback-$(git rev-parse --short HEAD)"
if git rev-parse "$TAG" >/dev/null 2>&1; then
  echo "PASS: rollback tag $TAG exists"
else
  echo "FAIL: run: git tag $TAG"
  exit 1
fi

One command to undo. If I can't tag it, I can't ship it.

Where the free model fits

MonkeyCode is an open-source project, and its free tier currently includes 10 million tokens plus a free server option. I use the free server for the sandbox gate. I use the free tokens for one job: summarizing gate logs.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The model reads the four gate outputs. It drafts a one-paragraph merge summary. It flags anything that looks like a weak pass.

But the model never decides. The gate decides.

That's the rule: AI summarizes, humans decide, scripts block.

Wire it into CI

Add one step to your pipeline:

- name: production readiness gate
  run: bash pr_gate.sh

If the gate exits 1, the merge is blocked. No human has to remember to run it. The pipeline enforces the checklist.

Decision table

Gate	Required evidence	Fail-closed criteria
provenance	commit label	no label = block
sandbox	SANDBOX_ID	no ID = block
tests	summary line with counts	no summary = block
rollback	git tag	no tag = block

Copy it. Adjust it. Keep the fail-closed default.

What I learned

The checklist caught more bad merges than the model did. The model wrote good summaries. The gates caught missing evidence.

Biggest surprise: the rollback gate failed most often. Teams tag releases, not pre-merge commits. That's a one-line habit change.

Who should not use this

This checklist is for small teams and solo builders. If you need formal compliance sign-off, this is not your gate. If your CI can't run shell scripts, start there instead.

The checklist is opinionated. It assumes git, pytest, and a disposable server. Swap the tools. Keep the principle.

Try it today

Copy pr_gate.sh into a repo. Add the four gate files. Run it on your next AI-assisted PR. The free tier is enough to test all of this.

Then ask yourself: which gate produced no evidence? That's the gap you fix next.

What does your merge pipeline do when evidence is missing — block, or shrug?