Build a Production Checklist for Promoting a Tiny AI CLI

I almost promoted a folder watcher last Friday night.

The laptop lid sat half closed. The worker still needed a remote home.

Do you know that late Friday itch?

The CLI watched ./inbox for new text files. It asked a model for one line. I kept the whole worker tiny on purpose.

I still wanted it off my machine. I wanted it overnight, unattended, and cheap.

That itch is how quiet failures get a host. Does that sound like your Friday too?

The real goal

I needed a promotion checklist, not a vibe. Laptop to remote box, extra spend zero.

If any gate failed twice, I stayed local. There was no heroic third try.

I gave the whole drill forty-five minutes. Rollback meant I stopped the user unit.

What this file is not

This is not a two-host canary. I already keep that drill.

This is not an assumption ledger either. That file answers a different question.

This is the ship-or-stay moment only. One YAML file, one checker, one abort.

Would you promote a worker without that file? I would not.

Why a free box tempts a solo builder

Free model access looks like production. It is not.

A free server looks like spare capacity. It is a loaner you can lose overnight.

I needed a remote target that could disappear. That constraint shaped every gate below.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I pointed the checker at MonkeyCode's free model access and free server option as staging, then practiced the stay path there.

The checklist still works if the loaner vanishes. That is the entire point.

The artifact you copy

Keep one YAML file next to the CLI. Name it ship_or_stay.yaml and commit it.

Refuse to promote without it. The checker exit code is the only vote.

Exit 0 means ship. Exit 2 means stay on the laptop.

# ship_or_stay.yaml
version: 1
worker:
  name: inbox-summarizer
  entry: "./summarize.py"
  inbox: "./inbox"
  outbox: "./outbox"
budget:
  extra_usd: 0
  max_calls_per_hour: 20
  max_prompt_chars: 4000
remote:
  required: true
  health_url_env: "MODEL_HEALTH_URL"
  timeout_sec: 8
secrets:
  allow_env:
    - MODEL_HEALTH_URL
    - MODEL_API_KEY
  forbid_files:
    - ".env"
    - "secrets.json"
fail_closed:
  on_missing_field: stay
  on_health_fail: stay
  on_secret_file: stay
rollback:
  unit: "inbox-summarizer.service"
  command: "systemctl --user stop inbox-summarizer.service"
abandon_after_failures: 2

Copy the file, then change names. Do not invent a budget you cannot see.

The checker

Here is the whole gate. It is boring on purpose.

Treat it as a template you can run. It does not grade model quality.

#!/usr/bin/env python3
"""Fail-closed ship-or-stay checker for a small AI CLI."""
from __future__ import annotations

import os
import sys
import urllib.request
from pathlib import Path

import yaml

ROOT = Path(__file__).resolve().parent
CFG_PATH = ROOT / "ship_or_stay.yaml"
STAY = 2
SHIP = 0


def stay(msg: str) -> int:
    print(f"STAY: {msg}", file=sys.stderr)
    return STAY


def load_cfg() -> dict:
    if not CFG_PATH.exists():
        raise SystemExit(stay("missing ship_or_stay.yaml"))
    with CFG_PATH.open() as fh:
        data = yaml.safe_load(fh) or {}
    if not isinstance(data, dict):
        raise SystemExit(stay("config is not a mapping"))
    return data


def gate_files(cfg: dict) -> str | None:
    worker = cfg.get("worker") or {}
    for key in ("entry", "inbox", "outbox"):
        rel = worker.get(key)
        if not rel:
            return f"worker.{key} missing"
        path = ROOT / rel
        if key == "entry" and not path.is_file():
            return f"entry not found: {rel}"
        if key != "entry" and not path.is_dir():
            return f"{key} is not a directory: {rel}"
    return None


def gate_budget(cfg: dict) -> str | None:
    budget = cfg.get("budget") or {}
    extra = budget.get("extra_usd")
    calls = budget.get("max_calls_per_hour")
    chars = budget.get("max_prompt_chars")
    if extra != 0:
        return "extra_usd must be 0 for this weekend"
    if not isinstance(calls, int) or calls <= 0 or calls > 60:
        return "max_calls_per_hour out of range"
    if not isinstance(chars, int) or chars <= 0 or chars > 8000:
        return "max_prompt_chars out of range"
    return None


def gate_secrets(cfg: dict) -> str | None:
    secrets = cfg.get("secrets") or {}
    for name in secrets.get("allow_env") or []:
        if not os.environ.get(name):
            return f"missing env: {name}"
    for rel in secrets.get("forbid_files") or []:
        if (ROOT / rel).exists():
            return f"secret file present: {rel}"
    return None


def gate_health(cfg: dict) -> str | None:
    remote = cfg.get("remote") or {}
    if not remote.get("required"):
        return "remote.required must be true to ship"
    env_name = remote.get("health_url_env")
    url = os.environ.get(env_name or "")
    if not url:
        return "health URL env is empty"
    timeout = float(remote.get("timeout_sec") or 8)
    req = urllib.request.Request(url, method="GET")
    try:
        with urllib.request.urlopen(req, timeout=timeout) as resp:
            if resp.status != 200:
                return f"health status {resp.status}"
    except Exception as exc:
        return f"health call failed: {exc}"
    return None


def main() -> int:
    cfg = load_cfg()
    for gate in (gate_files, gate_budget, gate_secrets, gate_health):
        err = gate(cfg)
        if err:
            return stay(err)
    print("SHIP: all gates passed")
    return SHIP


if __name__ == "__main__":
    sys.exit(main())

Install the one dependency, then dry-run the checker.

python3 -m pip install pyyaml
chmod +x check_ship.py
./check_ship.py; echo $?

Exit 2 means stay. Exit 0 means ship.

Anything else is a bug in the checker. Fix the checker. Do not ship.

Decision table

Stay is the default until every gate prints clean. Shipping is earned, never assumed.

Checker result	Action	Evidence you keep
Exit 0	start the user unit	SHIP: all gates passed
Exit 2	keep the worker local	STAY: line on stderr
Crash / traceback	treat it as stay	the stack trace itself
Two stays in a row	abandon the weekend	the two stderr lines

If the table feels too strict, good. Promotion should feel slightly annoying.

Numbered gates

Run them in this order. Do not skip a row because the demo worked once.

1. Files exist on disk

Does summarize.py exist on this machine? Do inbox and outbox exist as directories?

If the entrypoint is still a wish, you stay. The remote box will not invent files for you.

2. Budget is visible and tiny

extra_usd stays at zero. That is the weekend rule, not a metaphor.

max_calls_per_hour is twenty because I can count twenty. Why twenty and not a slogan?

If you cannot name the cap, you do not have a cap. Stay on the laptop.

3. Secrets stay in the environment

The checker forbids .env and secrets.json on disk. I have copied a repo onto a loaner before.

The secret file came along for the ride. That was enough education for me.

Env vars only. If a forbidden file exists, you stay.

4. Model health is a GET, not a hope

Set MODEL_HEALTH_URL to a cheap health endpoint. I do not pin model names here.

Names change and URLs rot. The gate should not care about branding.

If the GET fails, you stay. The worker gets no consolation prize.

5. Rollback is a real command

The YAML holds a stop command. Test it once against a dummy unit.

mkdir -p ~/.config/systemd/user
cat > ~/.config/systemd/user/inbox-summarizer.service <<'EOF'
[Service]
ExecStart=/usr/bin/python3 /home/you/inbox-summarizer/summarize.py
Restart=on-failure
EOF

systemctl --user stop inbox-summarizer.service || true

If you cannot stop it, you cannot ship it. Obvious, and very easy to skip.

Failure fixture

Here is a fixture that must fail. Save it as fixtures/missing_budget.yaml.

version: 1
worker:
  name: inbox-summarizer
  entry: "./summarize.py"
  inbox: "./inbox"
  outbox: "./outbox"
remote:
  required: true
  health_url_env: "MODEL_HEALTH_URL"
  timeout_sec: 8
secrets:
  allow_env:
    - MODEL_HEALTH_URL
  forbid_files:
    - ".env"
fail_closed:
  on_missing_field: stay

Point the checker at it. Expect STAY and exit code 2.

cp fixtures/missing_budget.yaml ship_or_stay.yaml
./check_ship.py; echo $?
# expect: STAY: extra_usd must be 0 for this weekend
# expect: 2

If that fixture ships, your checker is theater. Throw the checker out.

Want a second fixture? Drop a .env beside the CLI and run it again.

Secret files should fail closed even when the health GET looks fine. That surprise is the lesson.

How I spend the forty-five minutes

1. Copy the YAML and fill names only.
2. Export the two env vars, nothing else.
3. Run the checker on the laptop first.
4. Sync the repo to the loaner with no secret files.
5. Run the checker on the box itself.
6. If it stays, I stop. The lid closes. The worker stays local.

Did I mention abandon-after-two-failures? Yes. The third retry is ego.

I do not enable the user unit until the remote checker prints SHIP. Enabling first is how you debug in production by accident.

Limitations

This checklist does not measure model quality. It cannot, and it should not pretend.

It does not prove the summarizer is right. It only proves you may leave the laptop.

It does not pin a provider. Free access can vanish without a meeting.

It does not replace logs. Add a provenance line later if you still care.

Who should not use this file? Anyone holding customer data, payments, or secrets you cannot delete tonight.

Solo folder watchers, internal drafts, throwaway summarizers: that is the audience. Teams with an on-call rotation already have better gates.

I will not quote token quotas. They go stale by Monday.

I will not quote hardware for a free box. Loaners change under you.

I will not claim a free server is forever. The checklist has to survive the marketing copy.

Which gate fails first on your cheap box: secret files, the health GET, or the budget cap?