Make the Checklist Exit Nonzero: Six Gates, One TOML File, No Skipped Rows

Last Friday I opened our launch checklist. Fourteen rows. Eleven checkmarks.

Every checkmark was mine. I had ticked them from memory at 6pm.

Then the free-tier canary returned HTTP 200 with error: quota exceeded inside the body. The checklist said green. The product said no.

A checklist you read is a wish you wrote

Why do we keep writing launch checklists as prose? Because prose is cheap and nobody has to run it.

A row that can't fail is decoration. A row that can't name an owner is folklore. A row with a "skip" state is a hole with a checkbox drawn on it.

So I rewrote ours as six commands. The file exits nonzero or it doesn't ship.

Step 1: Write the gates as commands, not sentences

One gate per deploy-blocking risk. Each gate gets a command, an expected exit code, an owner, and one line of why.

# gates.toml — every row must be runnable by a machine
[meta]
receipt_dir = ".gates"

[[gate]]
id = "key-present"
kind = "blocking"
owner = "sam"
cmd = 'test -n "$MONKEYCODE_API_KEY"'
expect_exit = 0
why = "No key, no call. Stop before we waste an hour."

[[gate]]
id = "quota-budget"
kind = "blocking"
owner = "sam"
cmd = "python3 scripts/quota_guard.py --budget 200000"
expect_exit = 0
why = "Stop at my budget, not at the provider's 429."

[[gate]]
id = "canary-one-task"
kind = "blocking"
owner = "sam"
cmd = "python3 scripts/canary.py --task summarize --timeout 60"
expect_exit = 0
why = "One task, one call. Cheap proof the wire is alive."

[[gate]]
id = "silent-error-scan"
kind = "blocking"
owner = "sam"
cmd = "! grep -qi 'error' .gates/canary.log"
expect_exit = 0
why = "Exit 0 with 'error' in the log is a lie. Catch the lie."

[[gate]]
id = "rollback-dry-run"
kind = "blocking"
owner = "sam"
cmd = "python3 scripts/rollback.py --dry-run"
expect_exit = 0
why = "An untested rollback is a hope, not a plan."

[[gate]]
id = "latency-note"
kind = "advisory"
owner = "sam"
cmd = "python3 scripts/latency.py --p95-max 8000"
expect_exit = 0
why = "Slow is not broken. Record it. Do not block on it."

Notice gate four. It exists because our worst incident was a process that exited 0 while printing an error. Exit codes lie when nobody reads the log.

Step 2: Run them with a runner that refuses to skip

Forty lines of Python. No framework, no dashboard, no YAML indentation debates.

#!/usr/bin/env python3
"""run_gates.py — fail-closed checklist runner (Python 3.11+)."""
import hashlib, json, subprocess, sys, time, tomllib
from pathlib import Path

ROOT = Path(__file__).resolve().parent
SPEC = tomllib.loads((ROOT / "gates.toml").read_text())
OUT = ROOT / SPEC["meta"]["receipt_dir"]
OUT.mkdir(exist_ok=True)

def run(gate):
    t0 = time.time()
    p = subprocess.run(gate["cmd"], shell=True, capture_output=True, text=True)
    log = OUT / f"{gate['id']}.log"
    log.write_text(p.stdout + p.stderr)
    return {
        "id": gate["id"], "kind": gate["kind"], "owner": gate["owner"],
        "cmd": gate["cmd"], "exit": p.returncode,
        "expected": gate.get("expect_exit", 0),
        "seconds": round(time.time() - t0, 2),
        "log": str(log.relative_to(ROOT)),
        "sha256": hashlib.sha256(log.read_bytes()).hexdigest()[:16],
        "why": gate.get("why", ""),
    }

results = [run(g) for g in SPEC["gate"]]
for r in results:
    ok = r["exit"] == r["expected"]
    r["status"] = "PASS" if ok else "FAIL"

blocked = [r for r in results if r["kind"] == "blocking" and r["status"] == "FAIL"]
receipt = {
    "when": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
    "verdict": "BLOCK" if blocked else "GO",
    "results": results,
}
(OUT / "receipt.json").write_text(json.dumps(receipt, indent=2))

for r in results:
    print(f'{r["status"]:5} {r["id"]:18} exit={r["exit"]:<4} owner={r["owner"]}')
print(f'\nverdict: {receipt["verdict"]}')
sys.exit(1 if blocked else 0)

There is no SKIP. A missing script returns 127, which never equals 0, so a deleted gate fails loudly. That single decision removed half the arguments we used to have at 6pm.

Run it:

python3 run_gates.py; echo "exit=$?"
head -20 .gates/receipt.json

Step 3: Break a gate on purpose before you trust it

A checklist you have never seen fail is a checklist you have never tested.

# failure fixture: remove the credential, expect BLOCK
export MONKEYCODE_API_KEY=""
python3 run_gates.py; echo "exit=$?"   # exit=1, verdict: BLOCK

# second fixture: make the canary exit 0 but log an error
printf 'error 429 quota_exceeded\n' > .gates/canary.log
python3 run_gates.py; echo "exit=$?"   # gate 4 catches it

Two minutes of deliberate breakage. That is what turned my fourteen checkmarks into six gates I actually believe.

Step 4: Rehearse on free capacity, then pay for the real thing

Here is where the free tier earns its keep. MonkeyCode is an open-source project that, per its operator, offers free model access and a free server option, along with a free token allowance (the operator states 10M tokens; quotas change, so read the current terms before you plan around it).

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The point is not the number. The point is having a disposable target for the runner. My sequence:

1. Point MONKEYCODE_API_KEY and the base URL at the free server.
2. Run all six gates. Save the receipt as receipt-free.json.
3. Fix whatever fails — usually the budget guard.
4. Swap the base URL to the paid endpoint and run the same file again.
5. Diff the two receipts before anyone touches production.

python3 run_gates.py && cp .gates/receipt.json receipts/receipt-free.json

Same gates, two environments, two hashed receipts. If the paid run is green and the free run was red, you just found a config drift for free instead of at 2am.

I would not call the free tier a production recommendation. I call it a rehearsal room. You cannot break anything that matters there.

The table I keep next to the file

Gate kind	Evidence produced	Owner signs	If it fails
Credential	exit code + stderr	sam	Stop. Nothing else runs.
Budget	spend counter, call count	sam	Cap the feature, not the user
Canary	one task, one log	sam	Revert the last config change
Silent error	grepped log	sam	Treat as a failed deploy
Rollback	dry-run transcript	sam	Do not ship until it passes
Latency (advisory)	p95 number	sam	Record, ship, watch

Every row has a name. If a row has no name, delete the row. It was never a gate, it was a feeling.

Where this breaks, and who should skip it

Six gates is a solo-builder shape. It does not cover auth, data residency, or audit trails — do not take this to a compliance review and call it done.

Skip this approach if your deploy already runs a real policy engine. Skip it if you cannot name an owner per row, because the file will quietly become another document nobody runs. Skip it if rollback is genuinely impossible — fix that first, then come back.

Also, pin nothing about free tiers in a contract. Allowances and server terms move. Your gates should read the current number from config, not from a blog post. Mine does:

python3 run_gates.py --budget "${MONTHLY_TOKEN_BUDGET:-200000}"

One flag today. A different number next month. No rewrite.

The question I actually want answered

Which row on your launch checklist has no owner's name on it? Paste that single row and what evidence it would produce. I want to see the one you would delete first — that tells me which gate to design next.