The Gate Goes in the Repo: A Fail-Closed Promotion Checker for Hosted AI Prototypes

A free model allowance is a starting condition, not a capacity plan. The prototypes that hurt teams are rarely the ones that failed in the demo — they're the ones that passed the demo and then quietly became a dependency nobody had budgeted for.

So put the promotion decision in a file that a script reads. Not a wiki page. Not a slide. A manifest with gates, artifacts, and fail-closed criteria, checked by a program that exits non-zero when evidence is missing.

Below is the checker I use, the gate manifest it reads, and the six gates I have found actually catch problems. It runs on the Python standard library, so there is nothing to install and nothing to keep updated.

The failure mode this is built for

Demo-grade evidence looks like proof and isn't. Someone runs a prototype against a hosted runtime, it answers correctly, and the team concludes the system works.

Three things are missing from that conclusion:

1. A pinned runtime. Nobody wrote down which model identifier or version answered. When it changes, no one can tell whether behavior changed or the model did.
2. A usage ledger. There is a bill (often zero) but no record of tokens per request, so there is no way to project cost at 10x or 100x the demo volume.
3. A degradation path. Every hosted runtime has a bad day. If the prototype has only ever seen happy responses, its failure behavior is undefined.

A gate manifest forces those three answers to exist as files before anyone says the word "production."

Six gates, each with an artifact and a fail-closed rule

Gates are only useful if they are decidable. Each row below answers one question and points at one artifact. If the artifact is missing or empty, the gate fails. No judgement call, no "we'll add it later."

Gate	Question	Evidence artifact	Fail-closed when
G1 Runtime pin	Which runtime produced this result?	runtime.json with model id, version string, and the date it was captured	Any result file predates the pin
G2 Quota ledger	What does one real workflow cost?	ledger.csv with per-request token counts from a recorded run	Ledger has fewer rows than the run had requests
G3 Degradation drill	What happens when the runtime is slow, rate-limited, or unavailable?	degradation.md with the observed behavior and the user-visible fallback	No drill was executed in the last 30 days
G4 Egress	What leaves the host, and where does it go?	egress.txt allowlist plus a test that fails on an unlisted destination	Any destination is listed as "unknown"
G5 Cost crossover	At what volume does the free tier stop being the cheaper option?	crossover.md with the arithmetic and the input assumptions	Assumptions are not stated as numbers
G6 Portability	Can the workload move to another host in a day?	export.md describing state export and config replay, plus a dry run	Export was never executed end to end

Two of these deserve a note. G3 is the gate teams skip and later regret — a drill is not a load test, it is a deliberately induced failure to watch your own error path run. G5 is the gate that protects you from an awkward conversation: if you cannot state the volume at which a free allowance stops covering you, you do not have a cost model, you have a hope.

The checker

Save this as promote_gate.py. It reads a JSON manifest, checks that every required artifact exists and is non-empty, and exits non-zero if anything is unmet.

#!/usr/bin/env python3
"""promote_gate.py - fail-closed promotion checker for hosted AI prototypes.

Usage:
    python promote_gate.py gates.json --evidence ./evidence
Exit code 0 means every gate is satisfied by an artifact on disk.
"""
from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path


def missing_for_gate(gate: dict, evidence_dir: Path) -> list[str]:
    missing = []
    for artifact in gate["artifacts"]:
        path = evidence_dir / artifact
        if not path.is_file() or path.stat().st_size == 0:
            missing.append(artifact)
    return missing


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("manifest")
    parser.add_argument("--evidence", default="evidence")
    args = parser.parse_args()

    manifest = json.loads(Path(args.manifest).read_text())
    evidence_dir = Path(args.evidence)

    failed: list[str] = []
    for gate in manifest["gates"]:
        missing = missing_for_gate(gate, evidence_dir)
        print(f"[{'PASS' if not missing else 'FAIL'}] {gate['id']}: {gate['question']}")
        for artifact in missing:
            print(f"        missing or empty: {artifact}")
        if missing:
            failed.append(gate["id"])

    if failed:
        print(f"\nfail-closed: {len(failed)} gate(s) unmet, promotion blocked")
        for gate_id in failed:
            print(f"  - {gate_id}")
        return 1

    print("\nall gates satisfied, promotion allowed")
    return 0


if __name__ == "__main__":
    sys.exit(main())

The manifest

{
  "gates": [
    {"id": "G1", "question": "Which runtime produced this result?", "artifacts": ["runtime.json"]},
    {"id": "G2", "question": "What does one real workflow cost?", "artifacts": ["ledger.csv"]},
    {"id": "G3", "question": "What happens when the runtime degrades?", "artifacts": ["degradation.md"]},
    {"id": "G4", "question": "What leaves the host?", "artifacts": ["egress.txt", "egress_test.py"]},
    {"id": "G5", "question": "Where is the cost crossover?", "artifacts": ["crossover.md"]},
    {"id": "G6", "question": "Can this move hosts in a day?", "artifacts": ["export.md"]}
  ]
}

Run it, and wire it into the same place your tests run:

python promote_gate.py gates.json --evidence ./evidence
# [PASS] G1: Which runtime produced this result?
# [FAIL] G2: What does one real workflow cost?
#        missing or empty: ledger.csv
#
# fail-closed: 1 gate(s) unmet, promotion blocked

The exit code is the whole point. A blocked promotion should be a red build, not a meeting.

Populating G2 without guessing an endpoint

Do not assume your provider exposes a usage API you can poll. Wrap your own calls and log what you sent and what came back. A dozen lines is enough:

import csv, time
from pathlib import Path

LEDGER = Path("evidence/ledger.csv")


def log_call(request_id: str, prompt_chars: int, response_chars: int) -> None:
    new_file = not LEDGER.exists()
    with LEDGER.open("a", newline="") as handle:
        writer = csv.writer(handle)
        if new_file:
            writer.writerow(["ts", "request_id", "prompt_chars", "response_chars"])
        writer.writerow([int(time.time()), request_id, prompt_chars, response_chars])

Character counts are a proxy, not tokens. That is acceptable for G2's purpose, which is to prove you measured something real. Swap in exact counts if your runtime returns them; the gate only checks that the file exists and has rows.

Where MonkeyCode fits, and where it does not

The gates above are runtime-agnostic, which matters because you will change runtimes. If you are starting from nothing and need a runtime and a host for the drill in G3 and the export in G6, MonkeyCode is one option: it offers free model access and a free server option, which is enough to run a small prototype and a degradation drill without a procurement conversation.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat the numbers as claims to re-verify rather than constants. The allowance is described as 10 million free tokens alongside a free server option, but quotas, model lists, and hosts change on their own schedule. Before you put a figure in crossover.md, read the project's current documentation yourself and date the entry. An undated quota is exactly the kind of stale data that makes G5 worthless.

What MonkeyCode does not do is satisfy a single gate for you. It gives you somewhere to run. The ledger, the drill record, and the export are still yours to produce.

Limitations and who should not use this

• Small prototypes with one user. Six gates is overhead. Write G1 only, and revisit when a second person depends on the thing.
• Regulated workloads. egress.txt is a checklist, not a compliance control. Do not use it as evidence of data-handling policy.
• Teams without a CI system. The checker still works locally, but the fail-closed property weakens sharply when a human has to remember to run it.
• Anyone treating an artifact's existence as proof of quality. The script checks presence and size. It cannot tell whether degradation.md describes a real drill or a paragraph of optimism. Review the contents; the script only enforces that there is something to review.

Adoption order

1. Add gates.json with G1 and G2 only.
2. Run the checker in CI on the branch that proposes the promotion.
3. Capture runtime.json and a real ledger row on your next prototype run.
4. Add G3, then G5, then G4 and G6 as the workload gains users.
5. Re-date every artifact on a schedule you choose, and let G3 expire on its own if the drill lapses.

The gates are cheap. The conversation you avoid is not.

If you want to try this end to end, the only file you have to write is gates.json — start with two gates, and get the runtime for your first drill wherever it is cheapest to start.