Build a Fail-Closed Assumption Ledger for a Small AI Sidecar

A sanitized failure fixture still sits in my notes. The sidecar returned HTTP 200 and billed the wrong tenant. Why did a green model call still wreck the invoice path?

The model never received a tenant id. It guessed one from a nearby filename. Have you watched a success hide a missing field?

This post is a copyable gate file, not a platform tour. You get a ledger, a checker, and fail-closed exits. Skip it if CI already proves every assumption.

The build goal and the hard stop

I needed a sidecar that talks to a free model. I did not want another quiet default. The budget was one evening and zero new paid hosts.

MonkeyCode is an open-source project with free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am not naming models, hardware, or promised uptime here.

Ten million free tokens is a declared ceiling, not a benchmark. I treat that number as a budget boundary in the ledger. I do not treat it as measured burn.

Rollback stays boring on purpose. Delete the unit, delete the ledger, stop the sidecar. If a gate cannot fail closed, that gate does not ship.

What the ledger must prove

Every assumption becomes a row with evidence. No evidence means exit code 2. Does your current start script do that?

I keep six gates. Each gate has a proof artifact. Each gate has a fail-closed rule.

1. Callee identity is pinned, not guessed.
2. Required inputs exist before the first token.
3. Response schema is pinned to a file.
4. Budget boundary is declared in the ledger.
5. Abort path returns non-zero on purpose.
6. Evidence file is written before traffic.

Who should skip this list? Anyone running a multi-region control plane. Anyone who needs formal SLA math. Anyone without a named rollback owner.

The one-file ledger

Create assumption_ledger.yaml next to the sidecar. Keep it short enough to read aloud. If a row needs a paragraph, the assumption is too big.

version: 1
service: invoice-sidecar
callee:
  kind: free-model-http
  base_url_env: SIDECAR_MODEL_BASE_URL
  identity_sha256_env: SIDECAR_CALLEE_SHA256
required_inputs:
  - TENANT_ID
  - INVOICE_SCHEMA_PATH
  - ABORT_TOKEN
schema:
  path: ./fixtures/invoice.schema.json
  pin: required
budget:
  token_ceiling_declared: 10000000
  source: operator-declared-free-grant
  not_a_measurement: true
abort:
  command: ["python", "abort_probe.py"]
  expect_exit: 2
evidence:
  path: ./evidence/last-start.json

Notice the budget row refuses to look like telemetry. It is a ceiling you typed by hand. Can your start script still lie about remaining tokens?

Yes. The ledger only records the lie class. It will not scrape a dashboard for you.

The checker you actually run

Save this as check_ledger.py. It is a gate, not a framework. Run it before the sidecar process starts.

#!/usr/bin/env python3
"""Fail closed if any assumption lacks evidence."""
from __future__ import annotations

import hashlib
import json
import os
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path

import yaml

LEDGER = Path("assumption_ledger.yaml")
EXIT_MISSING = 2


def fail(msg: str) -> None:
    print(f"GATE FAIL: {msg}", file=sys.stderr)
    raise SystemExit(EXIT_MISSING)


def load_ledger() -> dict:
    if not LEDGER.exists():
        fail("assumption_ledger.yaml is missing")
    data = yaml.safe_load(LEDGER.read_text())
    if not isinstance(data, dict):
        fail("ledger is not a mapping")
    return data


def require_env(name: str) -> str:
    value = os.environ.get(name, "").strip()
    if not value:
        fail(f"missing required env {name}")
    return value


def gate_identity(ledger: dict) -> dict:
    callee = ledger.get("callee") or {}
    base = require_env(callee["base_url_env"])
    expected = require_env(callee["identity_sha256_env"])
    digest = hashlib.sha256(base.encode("utf-8")).hexdigest()
    if digest != expected:
        fail("callee identity does not match pin")
    return {"base_url_present": True, "identity_ok": True}


def gate_inputs(ledger: dict) -> dict:
    names = ledger.get("required_inputs") or []
    found = {name: require_env(name) for name in names}
    schema_path = Path(found["INVOICE_SCHEMA_PATH"])
    if not schema_path.is_file():
        fail("invoice schema file is missing")
    return {"inputs": sorted(found)}


def gate_schema(ledger: dict) -> dict:
    path = Path(ledger["schema"]["path"])
    if not path.is_file():
        fail("pinned schema path is missing")
    text = path.read_text(encoding="utf-8")
    json.loads(text)  # fail closed on invalid JSON
    digest = hashlib.sha256(text.encode("utf-8")).hexdigest()
    return {"schema_sha256": digest}


def gate_budget(ledger: dict) -> dict:
    budget = ledger.get("budget") or {}
    ceiling = budget.get("token_ceiling_declared")
    if not isinstance(ceiling, int) or ceiling <= 0:
        fail("token ceiling is not a positive int")
    if not budget.get("not_a_measurement"):
        fail("budget row must declare it is not telemetry")
    return {"token_ceiling_declared": ceiling}


def gate_abort(ledger: dict) -> dict:
    abort = ledger.get("abort") or {}
    cmd = abort.get("command")
    expect = abort.get("expect_exit")
    if not cmd:
        fail("abort command is missing")
    result = subprocess.run(cmd, check=False)
    if result.returncode != expect:
        fail(f"abort probe exited {result.returncode}, expected {expect}")
    return {"abort_exit": result.returncode}


def write_evidence(payload: dict, path: Path) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(json.dumps(payload, indent=2) + "\n")


def main() -> int:
    ledger = load_ledger()
    evidence = {
        "service": ledger.get("service"),
        "checked_at": datetime.now(timezone.utc).isoformat(),
        "identity": gate_identity(ledger),
        "inputs": gate_inputs(ledger),
        "schema": gate_schema(ledger),
        "budget": gate_budget(ledger),
        "abort": gate_abort(ledger),
    }
    write_evidence(evidence, Path(ledger["evidence"]["path"]))
    print("GATE PASS: evidence written")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

The abort gate is the one people skip. Why prove a failure path before success? Because a sidecar that cannot die will guess.

Numbered start sequence

Follow this order on a clean shell. Do not invert steps four and five.

1. Write the ledger with an empty evidence path ready.
2. Pin SIDECAR_MODEL_BASE_URL to the free server you use.
3. Export SIDECAR_CALLEE_SHA256 from that URL, not memory.
4. Export TENANT_ID from a real tenant record.
5. Drop fixtures/invoice.schema.json and refuse extra keys.
6. Run the abort probe and confirm exit 2.
7. Run python check_ledger.py and keep the evidence file.
8. Only then start the sidecar process.

Commands I keep in the README look like this.

python - <<'PY'
import hashlib, os
url = os.environ["SIDECAR_MODEL_BASE_URL"]
print(hashlib.sha256(url.encode()).hexdigest())
PY

export SIDECAR_CALLEE_SHA256="paste-the-digest"
export TENANT_ID="tenant_042"
export INVOICE_SCHEMA_PATH="./fixtures/invoice.schema.json"
export ABORT_TOKEN="do-not-send"

python abort_probe.py; echo $?   # must print 2
python check_ledger.py           # must print GATE PASS

Need an abort probe tonight? Keep it stupid. Do not add retries.

#!/usr/bin/env python3
import os
import sys

if os.environ.get("ABORT_TOKEN") != "SEND-NOW":
    print("abort probe refusing live traffic", file=sys.stderr)
    raise SystemExit(2)
raise SystemExit(0)

If that probe ever returns 0 during a normal start, the ledger is wrong. Stop. Do not say "just this once."

Fail-closed criteria you can copy

I use this table in pull requests. A reviewer can scan it in one minute. If a row has no evidence path, the row is theater.

Gate	Evidence	Fail-closed exit
Callee identity	SHA-256 of SIDECAR_MODEL_BASE_URL matches env pin	2
Required inputs	TENANT_ID, schema path, abort token are non-empty	2
Schema pin	fixtures/invoice.schema.json parses as JSON	2
Budget boundary	Positive int ceiling plus not_a_measurement: true	2
Abort path	abort_probe.py returns 2 on a normal start	2
Evidence file	evidence/last-start.json written before boot	2

No row says "warn and continue." Warnings are how the wrong tenant got billed. Are you still shipping a warn-only wrapper?

Failure fixture you should copy

I keep this as fixtures/missing-tenant.env. It is the opening scene, encoded. Run it on purpose before you trust the checker.

env -u TENANT_ID \
  SIDECAR_MODEL_BASE_URL="http://127.0.0.1:8080" \
  SIDECAR_CALLEE_SHA256="$(python -c 'import hashlib; print(hashlib.sha256(b"http://127.0.0.1:8080").hexdigest())')" \
  INVOICE_SCHEMA_PATH="./fixtures/invoice.schema.json" \
  ABORT_TOKEN="do-not-send" \
  python check_ledger.py
echo "exit=$?"

Expected stderr line: GATE FAIL: missing required env TENANT_ID. Expected exit: 2. If you get 0, the checker is not fail-closed.

A second fixture flips the identity pin. Same shape, wrong digest, exit 2, no sidecar. Would your current wrapper still start the process?

Wire both fixtures into CI as plain shell. Do not hide them behind a plugin. A solo repo can afford two extra jobs.

# ci-assumption-gates.sh
set -euo pipefail
python check_ledger.py && exit 1  # missing-tenant fixture must not reach here

That last line looks backwards on purpose. The missing-tenant job should die inside the checker. If the script continues, your gate leaked.

Time box, cost box, abandon box

Time box: one evening, then ship the gates or drop the sidecar. Cost box: stay on the declared free token ceiling and the free server option. No extra GPU invoice to explain later.

Abandon if any gate needs a human to "just know." Abandon if the evidence file is optional. Abandon if the abort probe cannot run in CI.

I do not log remaining tokens in this checker. I do not scrape a usage page. Those numbers go stale while you blink. The ledger only stores a ceiling you typed.

A tiny sidecar stub that respects evidence

The sidecar should refuse to boot without evidence. That keeps the checker honest. Otherwise someone will start the process by hand.

#!/usr/bin/env python3
import json
import sys
from pathlib import Path

EVIDENCE = Path("evidence/last-start.json")


def main() -> int:
    if not EVIDENCE.is_file():
        print("sidecar refuse: no evidence file", file=sys.stderr)
        return 2
    payload = json.loads(EVIDENCE.read_text())
    if "identity" not in payload or "budget" not in payload:
        print("sidecar refuse: evidence incomplete", file=sys.stderr)
        return 2
    print("sidecar start: evidence accepted")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Yes, this stub does no inference. That is deliberate. Get the gates green before you spend free tokens. Why burn a grant on a boot that cannot name its tenant?

What this does not do

This is not load testing. This is not eval quality. This is not an authz system.

The identity gate hashes a URL string. It does not prove TLS. It does not prove the process on that host. If you need mTLS, add it somewhere else.

The budget gate does not know what ten million tokens buy in practice. Here the number is only a declared stop sign. Do not paste it into a marketing slide.

The schema gate does not generate invoices. It only refuses to start without a pin. Agents still invent fields after start if you let them. Pair this with output validation inside the sidecar itself.

Do not use this checklist for enterprise change-management theater. It will look too small. Do not use it as a beginner tour of agents. It assumes you already have a sidecar.

Shipping notes I actually keep

Put check_ledger.py in pre-start, not in a wiki. Put the failure fixtures in CI. Put the evidence file in .gitignore if it can leak tenant ids.

If the free server moves, rotate SIDECAR_CALLEE_SHA256 in the same commit. If you cannot rotate in one commit, you do not have a pin. You have a comment.

A ledger that only lives on a laptop will rot. Copy the YAML into the repo. Copy the fixtures next. Then the next late-night start has something to fail against.

Want a box to run the same gates without standing up hosts? MonkeyCode's free server option is one place I would point the ledger. No ranking. No countdown. Just a callee the identity gate can hash.

Which required input does your sidecar still treat as obvious? If you drop the field name and the fail step, I can tighten the ledger rows next.