DEV Community

Taylor Zhu
Taylor Zhu

Posted on

A Zero Bill Is Not Evidence: Production Gates for a Free Model Allowance

A zero invoice does not make an agent job production-ready. If you cannot show a token ledger, a pinned host, and a floor that fails the run when the allowance thins out, you do not have a release. You have a demo that has not been billed yet.

That gap shows up whenever model quality moves faster than the machinery around it. People retry, swap hosts, and call the result shipped. You should not. Treat a free allowance as untrusted input, and promote only when the gates below all close on the same run record.

What a zero bill hides

A free model allowance removes the invoice. It does not remove spend, drift, or a bad retry loop. Three failures stay cheap until they are not.

  • The job keeps calling the model after the allowance is gone, then fails in a way your pager does not name.
  • The free server you tested is not the free server you promoted, because nothing pinned the host identity.
  • A retry helper multiplies calls when the model hedges, and the ledger never notices.

A passing local suite never sees the host, the receipt, or the cap. You need evidence from the run itself. Store that evidence before you argue about prompt quality.

Four gates, all fail closed

Copy the gates into the review checklist. Do not soften a missing field into a warning. A missing field is a failed promote.

Gate 1: Stamp the allowance source today

Write down the allowance you are willing to burn, the source you read, and the date you read it. If you cannot point at a current primary doc, stop.

The operator brief for this piece states that MonkeyCode is an open-source project with free model access, a free server option, and an allowance figure of 10 million tokens. Disclosure: This article was prepared as part of MonkeyCode's product outreach. That figure is a claim for you to re-check, not a number this article certifies. Open the project docs on the day you promote. Stamp what they actually say. If the docs are silent, or the number moved, the gate fails. Do not backfill from memory, from this post, or from a screenshot older than the promote date.

Read the license in the repository you clone. If the repo, the license, or the server terms are unclear, you are not ready to depend on them. Confirm those facts in the project itself. This page is not a primary source.

Gate 2: One ledger line per run

Every candidate run emits one JSON record. No record, no promote. The record is the evidence. A dashboard screenshot is not.

Require these fields:

  • run_id
  • allowance_tokens and tokens_used
  • floor_tokens, the remainder you refuse to cross
  • host_fingerprint and pinned_host_fingerprint
  • retry_count and retry_cap
  • docs_checked_on and promote_date
  • allowance_source, a doc path or URL you actually opened

Provider token counts can lag. If your runner only has an estimate, store the estimate and mark it as such. Still fail closed when the field is missing. A stored estimate you can diff beats a blank you hoped was small.

Gate 3: Pin the host, free server included

A free server is still a host. Pin the identity the control plane returns. That might be a hostname, an image digest, or an account-scoped id. If the API does not return a stable identity, do not invent one. Fail the gate and keep the job where it is.

Do not stamp a region, a machine size, or an uptime promise the docs do not state. Empty adjectives are not a fingerprint. Unknown hardware is a blank field, and a blank field fails.

Gate 4: Tie the retry cap to the floor

Retries are a budget decision, not a courtesy. Cap them in the same record as the tokens. If retry_count exceeds retry_cap, fail. If another attempt would push tokens_used past allowance_tokens - floor_tokens, fail before the call.

That is the practical rule when someone asks whether to retry. Retry only inside a cap you already wrote down. Outside that cap, stop and surface the ledger. Another attempt is not free just because the invoice is zero.

Run this checker before you discuss the model

The script is a copy-ready gate. It is an unexecuted example, not a benchmark, and it does not call a model. It only judges a record you already collected. Point it at fixtures before you point it at a real job.

#!/usr/bin/env python3
"""Fail-closed promote gate. Example only — wire your runner's field names first."""
import json
import sys
from pathlib import Path

REQUIRED = [
    "run_id",
    "allowance_tokens",
    "tokens_used",
    "floor_tokens",
    "host_fingerprint",
    "pinned_host_fingerprint",
    "retry_count",
    "retry_cap",
    "docs_checked_on",
    "promote_date",
    "allowance_source",
]

def fail(msg: str) -> None:
    print(f"FAIL: {msg}")
    raise SystemExit(2)

def main(path: str) -> None:
    data = json.loads(Path(path).read_text())
    missing = [k for k in REQUIRED if data.get(k) in (None, "")]
    if missing:
        fail("missing evidence: " + ", ".join(missing))
    if data["host_fingerprint"] != data["pinned_host_fingerprint"]:
        fail("host drift")
    if data["docs_checked_on"] != data["promote_date"]:
        fail("allowance source was not re-checked on promote date")
    remaining = int(data["allowance_tokens"]) - int(data["tokens_used"])
    if remaining < int(data["floor_tokens"]):
        fail(f"remaining {remaining} is below floor {data['floor_tokens']}")
    if int(data["retry_count"]) > int(data["retry_cap"]):
        fail("retry cap exceeded")
    print("PASS")

if __name__ == "__main__":
    if len(sys.argv) != 2:
        fail("usage: promote_gate.py run.json")
    main(sys.argv[1])
Enter fullscreen mode Exit fullscreen mode

The passing fixture below is synthetic. It is not a usage report from any host, and the allowance number is only valid if your docs stamp matches it on promote day.

{
  "run_id": "dry-run-014",
  "allowance_tokens": 10000000,
  "tokens_used": 180000,
  "floor_tokens": 500000,
  "host_fingerprint": "host-pin-example",
  "pinned_host_fingerprint": "host-pin-example",
  "retry_count": 1,
  "retry_cap": 2,
  "docs_checked_on": "2026-10-13",
  "promote_date": "2026-10-13",
  "allowance_source": "project-docs-opened-this-date"
}
Enter fullscreen mode Exit fullscreen mode
python3 promote_gate.py pass.json
# expected for this example: PASS

python3 promote_gate.py drift.json
# expected for this example: FAIL: host drift
Enter fullscreen mode Exit fullscreen mode

Build drift.json by changing one fingerprint. Build a floor failure by raising tokens_used until the remainder drops under floor_tokens. If either broken fixture passes, the checker is wrong. Fix the checker before you talk about model choice.

How to read the result

Evidence you have Gate result What you do
Full record, pin match, remainder above floor, retries inside cap, docs dated today Pass this run only Eligible to promote this artifact
Any required field blank Fail closed Keep the job off the shared path
Fingerprints differ Fail closed Re-pin from the control plane, or discard the host
Docs date is not the promote date Fail closed Re-read the allowance source and stamp it again
Remainder under the floor, or retries over cap Fail closed Stop calls. Do not sneak in one more try
Local tests passed, record absent Fail closed A suite is not a ledger

A pass covers this run only. It does not travel to tomorrow, a larger prompt, or a different host. Change any of those, and you collect a new record.

Rehearse the red path on a free host

You do not need a paid fleet to practice failure. A stated free server and a stated free model allowance are enough to produce one real record: a short prompt, a saved usage line, and a host identity if the API returns one. Then you break the fixtures on purpose.

  1. Read the FAIL line. It names one condition. Fix that condition in the run, not in the gate.
  2. If evidence is missing, emit the field from the runner. Do not default it inside the checker.
  3. If the fingerprints differ, print both and re-pin from the control plane. Do not edit the pin so a surprise host looks familiar.
  4. If the remainder is under the floor, stop the job. Raising the floor after the fact is how quiet overruns start.
  5. Re-run the same broken fixtures. A checker that passes drift.json is a broken checker.

If the free path does not expose usage or a host identity, that is a result. You learned you cannot promote from that path. Keep the job local, or move to a plan that emits the fields. Do not paper over the gap with a hardcoded 10000000.

Put that 10 million token figure in the record only after you see it in current docs. If the docs show a different cap, stamp the docs. If they show no cap, Gate 1 fails. A blog post, including this one, is not the source you stamp.

Who should not use this

This checklist does not score answer quality. A clean ledger can still be a confident wrong answer. Pair it with your own eval set when a bad reply can mislead a user.

It does not prove isolation. A free server may share fate with other tenants. If your data cannot sit on a shared host, do not use this path.

It does not freeze the product. Allowances, server terms, and model routing can change without a version pin you control. The date check exists for that reason. Do not promise a customer a quota you do not operate.

Skip the approach when you need a contractual uptime target, a named model build, a private network boundary, or an audit trail the free control plane does not emit. Also skip it if your team will override a FAIL to save a demo. The gate is useless once people treat it as optional advice.

Close on evidence, not on the invoice

Write the record. Run the checker. Keep the failing fixtures beside the script so the next person can see what closed means. Use the free model access and free server option for that rehearsal only after the docs still match the stamp in your record. Let a failed fixture decide, not a bill that happens to be zero.

Top comments (0)