Rehearse the Failure Before You Promote: A Degradation Gate for Free-Tier AI Apps

Your release will not fail because a feature is missing. It fails because nobody rehearsed the path where the upstream stops answering. So build the failure, assert on it, and block the merge when the evidence is absent.

Below is a checklist you can copy into a repo today: three failure modes, one fault-injection harness with no dependencies, and a gate that refuses to pass unless evidence is tied to the current commit.

The demo path is the easy path

A demo proves the happy path once, on your machine, with a warm endpoint. That is the cheapest thing you will ever test. It tells you nothing about the hour when a request comes back throttled and your app decides to invent an answer.

Free tiers make this worse in a useful way: the failure is not hypothetical, it is scheduled. Your job is to decide the response before it happens.

Three failure modes you own

Every app sitting on a hosted model endpoint and a small server has the same three failure shapes. The names differ per provider; the shapes do not.

• Throttle. The request is refused because you are over a rate limit. It usually clears in seconds. Retrying is reasonable; retrying blindly is not.
• Exhausted. The allowance for the period is gone. Retrying is pointless until the window resets, and your app must say so plainly.
• Unavailable. The endpoint or the server behind it is down or cold. Duration is unknown, so the only honest answer is a refusal with an error code.

Any of these can become a silent wrong answer if your error handling returns an empty string that a template happily renders.

The degradation matrix your team can copy

Fill this in for your own product before writing code. One row per mode, one row per user-facing surface.

Mode	User must see	Evidence you keep	Fail-closed condition
Throttle	Retry notice, no partial content	Recorded status + retry count	Any generated content is returned
Exhausted	Explicit "allowance used up" state	Status code + timestamp	The app retries in a loop
Unavailable	Error with a stable code	Status code + build SHA	A cached answer appears unlabeled
Cached fallback	Answer marked source: cache + age	Sample payload	Age is missing or the label is lost

The last column is the whole point. A gate that only checks "did the response look fine" passes everything.

Fault injection you can run in thirty seconds

The harness below stubs the upstream so it needs no credentials. Route every model call in your app through one function, then swap that function's fault switch during the drill.

#!/usr/bin/env python3
"""fault_gate.py - inject upstream faults, assert the app fails closed.

    python fault_gate.py                  # run the drill, print results
    python fault_gate.py evidence/degradation   # also write evidence files
"""
import json
import os
import pathlib
import subprocess
import sys
from dataclasses import dataclass


class UpstreamError(Exception):
    def __init__(self, status: int):
        super().__init__(f"upstream {status}")
        self.status = status


@dataclass
class Gateway:
    """The single choke point for every model call in your app."""

    def ask(self, prompt: str) -> dict:
        fault = os.environ.get("FAULT", "none")
        codes = {"throttle": 429, "exhausted": 402, "unavailable": 503}
        if fault in codes:
            raise UpstreamError(codes[fault])
        return {"text": f"echo:{prompt}"}


def answer(prompt: str, gw: Gateway) -> dict:
    """The only function your UI is allowed to call."""
    try:
        out = gw.ask(prompt)
    except UpstreamError as exc:
        return {"ok": False, "text": None, "code": f"upstream_{exc.status}"}
    return {"ok": True, "text": out["text"], "code": "ok"}


EXPECTED = {
    "none": {"ok": True, "text": "echo:ping", "code": "ok"},
    "throttle": {"ok": False, "text": None, "code": "upstream_429"},
    "exhausted": {"ok": False, "text": None, "code": "upstream_402"},
    "unavailable": {"ok": False, "text": None, "code": "upstream_503"},
}


def main(argv: list) -> int:
    emit = pathlib.Path(argv[1]) if len(argv) > 1 else None
    if emit:
        emit.mkdir(parents=True, exist_ok=True)
    head = subprocess.check_output(
        ["git", "rev-parse", "HEAD"], text=True
    ).strip()

    for mode, want in EXPECTED.items():
        os.environ["FAULT"] = mode
        got = answer("ping", Gateway())
        if got != want:
            print(f"FAIL {mode}: want {want} got {got}", file=sys.stderr)
            return 1
        print(f"PASS {mode:12s} -> {got['code']}")
        if emit and mode != "none":
            record = {
                "mode": mode,
                "commit": head,
                **got,
                "repro": f"FAULT={mode} python fault_gate.py",
            }
            (emit / f"{mode}.json").write_text(json.dumps(record, indent=2))
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv))

Two properties matter here. The expected result for a fault is text: None, and the drill compares whole dictionaries, so a stray fallback string breaks the test loudly instead of quietly shipping.

The gate: evidence tied to a commit

Evidence that is not pinned to a build is a rumor. This checker refuses to pass unless each failure mode has a record from HEAD, reports ok: false, returned no content, and includes a reproduce command.

#!/usr/bin/env python3
"""promote_gate.py - exit 1 unless degradation evidence matches HEAD."""
import json
import pathlib
import subprocess
import sys

HEAD = subprocess.check_output(["git", "rev-parse", "HEAD"], text=True).strip()
MODES = ["throttle", "exhausted", "unavailable"]
EVIDENCE = pathlib.Path("evidence/degradation")


def check(rec: dict) -> list:
    problems = []
    if rec.get("commit") != HEAD:
        problems.append("evidence is not from this commit")
    if rec.get("ok") is not False:
        problems.append("failure path did not report ok=false")
    if rec.get("text") not in (None, ""):
        problems.append("failure path returned content")
    if not rec.get("repro"):
        problems.append("no reproduce command recorded")
    return problems


def main() -> int:
    blocked = []
    for mode in MODES:
        path = EVIDENCE / f"{mode}.json"
        if not path.exists():
            blocked.append(f"{mode}: missing {path}")
            continue
        try:
            rec = json.loads(path.read_text())
        except json.JSONDecodeError as exc:
            blocked.append(f"{mode}: not JSON ({exc})")
            continue
        blocked += [f"{mode}: {p}" for p in check(rec)]

    if blocked:
        print("BLOCKED", *blocked, sep="\n  ")
        return 1
    print(f"PASS: {len(MODES)} failure modes evidenced at {HEAD[:8]}")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Wire it into CI as two lines. The drill regenerates evidence; the gate decides whether the branch is promotable.

python fault_gate.py evidence/degradation && python promote_gate.py

A thirty-minute runbook

1. Find the choke point. If more than one file imports the model client directly, fix that first; a drill cannot cover paths it cannot reach.
2. Write the matrix. One row per failure mode, and name the exact user-visible string or error code you expect.
3. Implement refusals. Return a code, not a blank string. Never let a template render None as content.
4. Label fallbacks. A cache hit must carry source: cache and its age, or the gate should treat it as fabricated content.
5. Run the drill, emit evidence, run the gate. Commit both scripts with the app so the check survives the next refactor.
6. Re-run before every release. Faults you have not re-tested since the last refactor are assumptions again.

Where a free allowance actually fits

The drill above is deliberately offline, but you still need a live endpoint to confirm that real status codes map to your handlers. That is where a no-cost allowance is useful: you can rehearse throttling and exhausted states against a live endpoint without watching a bill.

MonkeyCode offers free model access and a free server option, which makes it a practical place to run this harness next to a small deployment instead of on your laptop. It is not a substitute for the drill; free capacity is exactly the kind of resource that gets throttled, so treat it as the environment you test in, not a guarantee you build on.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

One caution on numbers: allowances and server limits change, and any figure quoted in a blog post goes stale. Check the project's current terms yourself before you plan a release around them — including whatever you read here.

Who should not ship this way

Skip free-tier hosting for this workflow if your product carries a contractual uptime or latency commitment, if a delayed answer is a safety issue rather than an inconvenience, or if you cannot verify where the data is processed. Skipping the drill, however, is not an option anywhere.

What to do on Monday

Write the matrix before lunch. It takes twenty minutes and it will surface at least one place where your app guesses instead of refusing. Then keep the gate in the repo, where it keeps working after you have forgotten this article.

If you want to run the harness against something live first, the free tier is a reasonable sandbox to start in — just bring the gate with you.