Build a Fail-Closed Ship Folder for a Small AI CLI

Last Tuesday I had a zip file ready.
The CLI called a free AI server.
I was one send away from a teammate.

Then I froze.
What would I keep if it failed at 2am?
A README is not evidence.

That is the whole problem.
Free models help a solo side project.
They are not a production contract.

I wanted a folder I could copy.
Eight files. One checker. Zero vibes.
If a file is missing, we do not ship.

The goal and the hard stop

Goal: hand a small AI CLI to one teammate.
Budget: forty-five minutes and zero dollars.
Backend: a free model on a free server.

Missing input: proof it should leave my laptop.
Not a blog post. Not a demo GIF.
A folder a reviewer can grep.

If the checker exits nonzero, I stop.
No "just this once" merge.
That is the fail-closed rule.

Why a folder, not another runtime gate

Runtime gates belong in CI later.
This is earlier and meaner.
This is evidence you keep next to the binary.

A gate is a check that runs once.
A ship folder is a check you reread tomorrow.
Can you explain the failure without Slack archaeology?

I usually cannot.
So we write the archaeology first.
Then the teammate can reject the zip with receipts.

What goes in ship/

Create this next to your CLI.
Do it before you polish flags.

mkdir -p ship out

cat > ship/GOAL.md << 'EOF'
# GOAL
User: one teammate on macOS or Linux.
Task: turn a stack trace into a 12-line summary.
Never: write files outside ./out.
EOF

cat > ship/BOUNDARY.md << 'EOF'
# BOUNDARY
Time box: 45 minutes to fill this folder.
Money: $0. Free model. Free server.
Blast radius: local ./out only.
Network: one HTTPS endpoint.
EOF

Each file has one job.
If you cannot fill it, you are not ready.
Would you sign that sentence in git?

1. GOAL.md — one task, one user

Three lines max. Then stop writing.
Who runs it. What it returns. What it must never do.

If the goal needs a second user, stop.
This checklist is for one-task tools.
Second users hide second failure modes.

2. BOUNDARY.md — time, cost, blast radius

Declare the walls before the demo.
Did you promise overnight cron?
Then this folder is the wrong artifact.

Cron needs a different runbook.
I am not writing that runbook today.
Forty-five minutes is the wall.

3. PROMPT_FIXTURE.json — one frozen input

Do not "try a few prompts" in chat.
Freeze one fixture you can replay.
Production is not a playground.

cat > ship/PROMPT_FIXTURE.json << 'EOF'
{
  "id": "trace-001",
  "input": "TypeError: 'NoneType' object is not subscriptable\n  File app.py, line 41, in load",
  "expect_contains": ["NoneType", "line 41"],
  "expect_forbidden": ["rm -rf", "curl | sh", "API_KEY"]
}
EOF

If the model cannot hit this fixture, we do not ship.
No extra retries. No prompt shopping after handoff.
Would you debug a moving target on someone else's laptop?

4. FAIL_CLOSED.md — what silence means

Free servers fail in boring ways.
Timeouts. Empty JSON. Partial sentences.
Write the rule in English first.

cat > ship/FAIL_CLOSED.md << 'EOF'
# FAIL_CLOSED
Empty body → exit 2.
Missing JSON key `summary` → exit 2.
Fixture miss → exit 2.
Unknown HTTP status → exit 2.
Never retry more than once.
Never print a guessed summary.
EOF

Would you rather look careful or look broken?
Broken. Always broken.
A guessed summary is a lie with extra steps.

5. ROLLBACK.md — the exit you can run tired

If I cannot roll back in one minute, I do not ship.
Tired-me is the real operator.
Clever-me does not get a vote at 2am.

cat > ship/ROLLBACK.md << 'EOF'
# ROLLBACK
1. Ctrl-C the CLI.
2. rm -rf ./out
3. git switch main
4. Tell the teammate: revert to v0, no AI path.
EOF

No database. No shared cache.
If you need those, you left the small-tool lane.
Stay in the lane.

6. COST.md — even free has a budget

Free is not infinite.
I still write a kill switch I will honor.
I do not invent a vendor quota I cannot see.

cat > ship/COST.md << 'EOF'
# COST
Backend: free model access plus a free server option.
Local cap: stop after 20 fixture runs today.
If the server asks for a card, abandon.
If two calls exceed 30s, abandon.
EOF

Write only the cap you will enforce yourself.
Anything else is marketing in a runbook.
I refuse that mix.

7. HEALTH.json — a tiny canary shape

Keep a schema, not a dashboard.
Your CLI must print this on --health.
If ok is false, exit 2.

cat > ship/HEALTH.schema.json << 'EOF'
{
  "ok": true,
  "http_status": 200,
  "bytes": 1,
  "has_summary": true,
  "fixture_id": "trace-001",
  "elapsed_ms": 1
}
EOF

No pretty table. JSON only.
Humans lie. Schemas argue back.
Which one do you want at handoff?

8. ABANDON.md — when I walk away

Shipping includes quitting.
Write the walk-away rules before the first success.
Success makes people greedy.

cat > ship/ABANDON.md << 'EOF'
# ABANDON
Walk away if:
- fixture fails twice in a row
- health JSON cannot be parsed
- rollback takes more than one minute
- I start adding a second task
EOF

Second task is how side projects die.
One task. Then stop.
Can you say no to a "tiny extra flag"?

The checker: refuse to print READY

This script is the whole product.
It does not call the model.
It only certifies the folder.

#!/usr/bin/env python3
"""Fail closed unless ./ship has required evidence."""
from __future__ import annotations

import json
import sys
from pathlib import Path

REQUIRED = [
    "GOAL.md",
    "BOUNDARY.md",
    "PROMPT_FIXTURE.json",
    "FAIL_CLOSED.md",
    "ROLLBACK.md",
    "COST.md",
    "ABANDON.md",
]


def die(msg: str, code: int = 2) -> None:
    print(f"NOT READY: {msg}", file=sys.stderr)
    raise SystemExit(code)


def main(root: Path) -> None:
    if not root.is_dir():
        die(f"missing directory {root}")

    for name in REQUIRED:
        path = root / name
        if not path.is_file():
            die(f"missing {name}")
        if path.stat().st_size < 24:
            die(f"{name} is too small to be evidence")

    goal = (root / "GOAL.md").read_text(encoding="utf-8")
    if "Never:" not in goal:
        die("GOAL.md must declare a Never: line")

    fail_closed = (root / "FAIL_CLOSED.md").read_text(encoding="utf-8")
    if "exit 2" not in fail_closed:
        die("FAIL_CLOSED.md must mention exit 2")

    rollback = (root / "ROLLBACK.md").read_text(encoding="utf-8")
    if "rm -rf ./out" not in rollback:
        die("ROLLBACK.md must include rm -rf ./out")

    abandon = (root / "ABANDON.md").read_text(encoding="utf-8")
    if "Walk away" not in abandon:
        die("ABANDON.md must say Walk away")

    fixture = json.loads((root / "PROMPT_FIXTURE.json").read_text(encoding="utf-8"))
    for key in ("id", "input", "expect_contains", "expect_forbidden"):
        if key not in fixture:
            die(f"PROMPT_FIXTURE.json missing {key}")
    if not fixture["input"].strip():
        die("PROMPT_FIXTURE.json input is empty")
    if "API_KEY" not in fixture["expect_forbidden"]:
        die("fixture must forbid leaking API_KEY")

    print("READY")
    raise SystemExit(0)


if __name__ == "__main__":
    target = Path(sys.argv[1] if len(sys.argv) > 1 else "./ship")
    main(target)

Save it as check_ship.py.
Then run the happy path.

python3 check_ship.py ./ship
# expected stdout: READY

Now the failure fixture I keep in git.
Delete rollback. Watch the checker refuse.

mv ship/ROLLBACK.md /tmp/ROLLBACK.md
python3 check_ship.py ./ship
echo "exit=$?"
# expected stderr: NOT READY: missing ROLLBACK.md
# expected exit=2
mv /tmp/ROLLBACK.md ship/ROLLBACK.md

If that does not fail, the checker is theater.
Theater is how I almost mailed the zip.
Do you still trust a green demo after this?

A tiny CLI stub that honors the folder

Label this as a template, not a benchmark.
I am not publishing latency numbers I did not record today.
The stub shows fail-closed shape only.

#!/usr/bin/env python3
"""Template CLI. Unset env vars must fail before any HTTP call."""
from __future__ import annotations

import json
import os
import sys
import time
from pathlib import Path
from urllib.error import URLError
from urllib.request import Request, urlopen


def fail(msg: str) -> None:
    print(json.dumps({"ok": False, "error": msg}))
    raise SystemExit(2)


def main() -> None:
    base = os.environ.get("AI_BASE_URL")
    key = os.environ.get("AI_API_KEY")
    if not base or not key:
        fail("missing AI_BASE_URL or AI_API_KEY")

    fixture = json.loads(Path("ship/PROMPT_FIXTURE.json").read_text())
    payload = json.dumps({"input": fixture["input"]}).encode("utf-8")
    req = Request(
        base.rstrip("/") + "/summarize",
        data=payload,
        headers={
            "content-type": "application/json",
            "authorization": f"Bearer {key}",
        },
        method="POST",
    )

    started = time.monotonic()
    try:
        with urlopen(req, timeout=30) as resp:
            raw = resp.read()
            status = resp.status
    except URLError as exc:
        fail(f"network: {exc}")

    elapsed_ms = int((time.monotonic() - started) * 1000)
    if status != 200 or not raw:
        fail(f"bad response status={status} bytes={len(raw)}")

    try:
        body = json.loads(raw.decode("utf-8"))
    except json.JSONDecodeError:
        fail("body is not JSON")

    summary = body.get("summary")
    if not isinstance(summary, str) or not summary.strip():
        fail("missing summary")

    low = summary.lower()
    for needle in fixture["expect_contains"]:
        if needle.lower() not in low:
            fail(f"fixture miss: {needle}")
    for banned in fixture["expect_forbidden"]:
        if banned.lower() in low:
            fail(f"forbidden content: {banned}")

    Path("out").mkdir(exist_ok=True)
    Path("out/summary.txt").write_text(summary + "\n", encoding="utf-8")
    print(json.dumps({
        "ok": True,
        "http_status": status,
        "bytes": len(raw),
        "has_summary": True,
        "fixture_id": fixture["id"],
        "elapsed_ms": elapsed_ms,
    }))


if __name__ == "__main__":
    main()

The first fail-closed gate is the shell, not Python.
Unset means we do not start.

export AI_BASE_URL="${AI_BASE_URL:?missing}"
export AI_API_KEY="${AI_API_KEY:?missing}"
python3 check_ship.py ./ship
python3 summarize.py

Where does a free backend enter this story?
I point AI_BASE_URL at a server that offers free model access.
I keep the key out of git.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

For this template I aim those env vars at MonkeyCode's free model access and free server option.
The folder does not depend on that name.
Remove the name and the checker still works.

If you need a concrete place to point the env vars, that is the only product note.
Try the folder first. Then pick a backend.

Numbered ship procedure

1. Fill all eight files in ship/.
2. Run python3 check_ship.py ./ship.
3. Delete ROLLBACK.md once, confirm exit 2, restore it.
4. Run the CLI against the frozen fixture.
5. Confirm health JSON on stdout with ok: true.
6. Run the rollback commands once, for real.
7. Only then zip the CLI for a teammate.

Skip any step and you are demoing, not shipping.
Which step would you skip under time pressure?
That is the step that needs the checker.

Who should not use this

Do not use this for multi-tenant products.
Do not use this for medical, legal, or payroll text.
Do not use this if you need an SLA.

This is a solo-builder folder.
It protects a teammate from a half-built CLI.
It does not replace monitoring.

Also skip it if you cannot name one fixture.
If every prompt is "whatever the user pastes," stop.
You do not have a tool yet.

Limitations I will not hide

The checker does not prove the model is good.
It proves you wrote evidence.
Those are different claims.

Free servers change behavior without a changelog.
A passing fixture today can fail Thursday.
Re-run the checker after every prompt edit.

I did not publish latency numbers here.
I do not have a stable public benchmark for this backend.
If a number is not in your own health JSON, it is rumor.

The 45-minute box is a discipline tool.
It is not a performance claim.
If you cannot fill ABANDON.md in that box, the tool is too big.

What I do after READY

I send the zip plus the ship/ folder.
The teammate can reject it with evidence.
That is the point.

No dashboards. No weekly review ritual.
One folder. One checker. One rollback.

If the first run after handoff fails, we abandon.
We do not tune the prompt on their machine.
Tuning happens on mine, then I re-check.

Which evidence file did you fail to fill first?
GOAL, COST, or ABANDON?
That answer tells me the next template to write.