DEV Community

Sam Rivera
Sam Rivera

Posted on

Build a Schema Receipt for a Free-Model CLI

I needed proof before a free remote touched files. Not a vibe check. A receipt I could grep.

The CLI was tiny on purpose. The blast radius was not. A free model can change shape overnight. Your parser can still print ok. Then what do you roll back?

I asked one rude question before writes. What hash marks the last good body? If that file is missing, you do not have production. You have hope with a progress bar.

This is a production-readiness card. Gates, evidence, fail-closed rules. Copy it today. Throw it away if it lies.

What this card refuses to be

This is not a launch checklist. Shipping copy is a different job.

This is not a two-host canary. One endpoint. One frozen contract.

This is not an agent think piece. Most loops are still if-statements. The loop still needs a receipt.

Hot threads still argue about model skill. I care about schema drift. Skill debates do not restore a dirty repo.

Build goal, budget, stop rule

Goal: freeze the remote response shape first. No real files until the shape holds.

Time box: forty-five minutes. Cash box: zero dollars. That boundary is part of the card.

Abandon if three canary calls disagree. Rollback means delete receipts. Then un-chmod the CLI binary.

Why zero dollars? This path is free-tier work. If you need a contract, stop here.

The three files

Keep the surface tiny. Three files only. No framework.

  1. schema_receipt.json — the contract you trust.
  2. check_receipt.py — the fail-closed checker.
  3. fixtures/stale_schema.json — the lie you must catch.

If a fourth helper appears, you are decorating. Delete it. Forty-five minutes does not include a platform.

Step 1: write the stop rules first

Do not start with the prompt. Start with the brakes.

{
  "tool": "tiny-ai-cli",
  "budget_minutes": 45,
  "cash_usd": 0,
  "max_latency_ms": 8000,
  "max_bytes": 4096,
  "writes_allowed": false,
  "abandon_after_mismatches": 3
}
Enter fullscreen mode Exit fullscreen mode

writes_allowed stays false on purpose. The card is a gate. It is not the worker.

Would you let a canary mkdir in your repo? I would not. A canary that writes is already production.

Step 2: freeze one boring canary

Pick one prompt. Make it dull. Dull is testable.

Return JSON only. Keys: ok, echo, schema_version.
ok must be true. echo must equal "canary-alpha".
schema_version must be "1".
Enter fullscreen mode Exit fullscreen mode

Then freeze success in schema_receipt.json.

{
  "schema_version": "1",
  "required_keys": ["ok", "echo", "schema_version"],
  "echo": "canary-alpha",
  "ok": true
}
Enter fullscreen mode Exit fullscreen mode

A free model is not a coworker. It is a function with drift. Functions get receipts. Coworkers get standups.

Would you ship a HTTP client with no content-type check? Then do not skip this freeze.

Step 3: define the receipt fields

A receipt is not the prose. It is the evidence pack.

Record these six fields every run:

  1. UTC timestamp
  2. SHA-256 of the raw body
  3. Byte length
  4. Latency in milliseconds
  5. Checker exit status
  6. Whether writes stayed blocked

No hash means no run. Do not negotiate that line.

Missing latency also fails closed. Silent slowness is still a production bug. Free remotes stall in boring ways.

Step 4: list the fail-closed trips

Drift is not a feeling. Drift is a missed key.

Fail closed when any trip hits:

  1. A required key is missing
  2. schema_version does not equal 1
  3. echo does not equal canary-alpha
  4. Latency exceeds 8000 ms
  5. Body exceeds 4096 bytes
  6. Three live mismatches land in a row

No fluency score. No "it seemed helpful." Helpful is how bad JSON sneaks in.

Use this table when someone asks for a softer gate.

Signal Gate Action
missing required key contract exit 1, do not exec
schema_version != 1 contract exit 1, do not exec
latency > 8000 ms budget exit 1, do not exec
body > 4096 bytes budget exit 1, do not exec
three live mismatches abandon chmod -x the CLI
receipt file missing evidence treat as failure

Notice the missing row. There is no "retry with a nicer prompt." That retry is how you burn the time box.

Step 5: drop the checker

Label this as a local utility. I am not claiming live latency numbers. Run it on your machine.

Save check_receipt.py.

#!/usr/bin/env python3
"""Fail-closed schema receipt checker for a tiny AI CLI."""
from __future__ import annotations

import argparse
import hashlib
import json
import sys
import time
import urllib.request
from datetime import datetime, timezone
from pathlib import Path

CONTRACT = {
    "schema_version": "1",
    "required_keys": ["ok", "echo", "schema_version"],
    "echo": "canary-alpha",
    "ok": True,
}
BOUNDARIES = {"max_latency_ms": 8000, "max_bytes": 4096, "writes_allowed": False}


def load_payload(path: Path) -> tuple[dict, bytes, int]:
    raw = path.read_bytes()
    return json.loads(raw.decode("utf-8")), raw, 0


def live_payload(endpoint: str) -> tuple[dict, bytes, int]:
    # Placeholder POST shape. Replace this adapter for your host.
    body = json.dumps({
        "prompt": (
            "Return JSON only. Keys: ok, echo, schema_version. "
            "ok must be true. echo must equal canary-alpha. "
            "schema_version must be 1."
        )
    }).encode("utf-8")
    req = urllib.request.Request(
        endpoint,
        data=body,
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    started = time.perf_counter()
    with urllib.request.urlopen(req, timeout=10) as resp:
        raw = resp.read()
    latency_ms = int((time.perf_counter() - started) * 1000)
    return json.loads(raw.decode("utf-8")), raw, latency_ms


def evaluate(payload: dict, raw: bytes, latency_ms: int) -> list[str]:
    failures: list[str] = []
    if latency_ms > BOUNDARIES["max_latency_ms"]:
        failures.append(f"latency {latency_ms}ms")
    if len(raw) > BOUNDARIES["max_bytes"]:
        failures.append(f"bytes {len(raw)}")
    for key in CONTRACT["required_keys"]:
        if key not in payload:
            failures.append(f"missing {key}")
    if payload.get("schema_version") != CONTRACT["schema_version"]:
        failures.append(
            f"schema_version: got {payload.get('schema_version')!r} want 1"
        )
    if payload.get("echo") != CONTRACT["echo"]:
        failures.append(f"echo: got {payload.get('echo')!r}")
    if payload.get("ok") is not True:
        failures.append("ok was not true")
    return failures


def write_receipt(raw: bytes, latency_ms: int, failures: list[str], out: Path) -> None:
    out.mkdir(exist_ok=True)
    stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
    receipt = {
        "ts": datetime.now(timezone.utc).isoformat(),
        "sha256": hashlib.sha256(raw).hexdigest(),
        "bytes": len(raw),
        "latency_ms": latency_ms,
        "writes_allowed": BOUNDARIES["writes_allowed"],
        "ok": not failures,
        "failures": failures,
    }
    path = out / f"receipt-{stamp}.json"
    path.write_text(json.dumps(receipt, indent=2) + "\n", encoding="utf-8")
    print(path)


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--fixture", type=Path)
    parser.add_argument("--live", action="store_true")
    parser.add_argument("--endpoint", default="")
    parser.add_argument("--receipts", type=Path, default=Path("receipts"))
    args = parser.parse_args()
    if args.fixture:
        payload, raw, latency_ms = load_payload(args.fixture)
    elif args.live:
        if not args.endpoint:
            print("live path needs --endpoint", file=sys.stderr)
            return 2
        payload, raw, latency_ms = live_payload(args.endpoint)
    else:
        print("pass --fixture or --live", file=sys.stderr)
        return 2
    failures = evaluate(payload, raw, latency_ms)
    write_receipt(raw, latency_ms, failures, args.receipts)
    if failures:
        for item in failures:
            print(f"FAIL {item}", file=sys.stderr)
        return 1
    print("PASS schema receipt")
    return 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

The live POST body is a sketch. Swap live_payload() for your host client. evaluate() is the real artifact. Do not treat the POST as an SDK.

Step 6: break the fixture on purpose

This fixture is the lie. Keep it committed. Do not pretty it up.

fixtures/stale_schema.json:

{
  "ok": true,
  "echo": "canary-alpha",
  "schema_version": "2",
  "extra_field": "i-look-helpful"
}
Enter fullscreen mode Exit fullscreen mode

See the trap? ok is true. echo matches. A lazy parser would continue. Your card must not.

schema_version moved to 2. That is enough. The extra field is just perfume.

Run the dry path first. Always. Live comes last.

python3 check_receipt.py --fixture fixtures/stale_schema.json
echo $?
Enter fullscreen mode Exit fullscreen mode

You want a non-zero exit. You want FAIL schema_version on stderr. You also want a receipt file that records the failure.

ls -l receipts/
python3 -m json.tool receipts/$(ls receipts | tail -n 1)
Enter fullscreen mode Exit fullscreen mode

If the fixture command exits zero, stop. The gate is inverted. You built a welcome mat.

Step 7: wrap the CLI so writes cannot skip

The wrapper is the production control. The Python file only writes evidence.

#!/usr/bin/env bash
set -euo pipefail
: "${MODEL_ENDPOINT:?set MODEL_ENDPOINT}"
python3 check_receipt.py --live --endpoint "$MODEL_ENDPOINT"
exec ./tiny-ai-cli "$@"
Enter fullscreen mode Exit fullscreen mode

set -e matters. A failed receipt must kill exec. Did your last wrapper swallow stderr? Fix that before new flags.

Make the wrapper the only entrypoint.

chmod +x wrap-tiny-ai-cli.sh
chmod -x ./tiny-ai-cli
# later, only the wrapper may exec the binary
Enter fullscreen mode Exit fullscreen mode

No receipts/ directory after a claimed pass? Treat that as failure. Evidence you cannot find is not evidence.

Where a free model host belongs

I needed a zero-cash box for the live path. That is the only reason a host shows up here.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source project. It offers free model access and a free server option. I point the live checker at that option when I want a cheap canary box. The card does not grade the vendor. It grades the body.

Use your own endpoint if you already have one. The gates stay identical. If the host cannot return stable JSON, do not aim the CLI at a real tree.

If you want a zero-cash canary box, that free server is one candidate. Run the stale fixture first. Same abandon rule. No special casing for the logo.

Who should not use this

Skip this if the payload has secrets. A free remote is not a vault.

Skip this if you need an SLA. Free includes no uptime promise.

Skip this if the first call must write files. You need a sandbox, not a receipt.

Skip this if a committee owns prompts. This card is a solo-builder brake.

Skip this if you will bump schema_version to silence failures. That bump is how drift becomes policy.

Rollback and abandon

Rollback is deletion. Keep the ritual short. Do not hold a meeting.

rm -rf receipts/
chmod -x ./tiny-ai-cli
unset MODEL_ENDPOINT
Enter fullscreen mode Exit fullscreen mode

If the card and the CLI disagree, trust the card. Then disable the binary. Do not debug in place on a dirty tree.

Abandon after three live mismatches. Do not retry with a friendlier prompt. The canary is supposed to be boring.

Forty-five minutes is the ceiling. If you are still tuning JSON, the remote is not ready. Walk away while the repo is still clean.

Limits I will not dress up

This card does not prove the model is smart. It proves the shape held.

It does not pin weights. A free server can swap backends. Your hash will move. That is a feature.

It does not replace review. A valid receipt can wrap bad advice. Someone still reads the diff.

It is not a load test. One canary is not traffic. Do not quote it as capacity.

It will not save you from prompt injection. Different gate. Different day. Do not stretch this file.

Trend threads this week argued about AI replacing developers. Fine. My smaller question still stands. Can your free-model CLI show a schema hash from the last trusted body?

If the answer is no, do not add another flag. Add the card. Then keep writes locked until it passes.

One check for tomorrow

Copy the three files. Fail the fixture first. Then stop for the night.

Which gate would have fired first on your worst free-model run? Missing key, schema bump, or a write that skipped evidence?

Top comments (0)