The Stop List for Free Inference and Free Hosts

Most teams can name the day they adopted a free inference endpoint or a free server. Almost none can name the sentence that would make them leave it. That unwritten sentence is the actual risk, and it has nothing to do with price.

A stop list fixes this. It is a short document, dated, that pairs each free dependency with an observable signal and a pre-agreed action. Written before adoption, it turns a diffuse unease into a decision any on-call engineer can execute at 3 a.m. without a committee.

The artifacts below are three small ones: a capability ledger you can lint, a detach drill that proves your job survives losing the endpoint, and an exit-criteria table. MonkeyCode appears once, exactly where it is useful — as a disposable target for the drill. Its free model access and free server option are described only as the project states them. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why benchmarks answer the wrong question

A benchmark tells you how good a tier is while it works. Your incident review will never be about that. It will be about what the code did in the ninety seconds after the endpoint stopped answering, and whether that behavior was designed or accidental.

Dependency shape matters more than quality. A summarizer that posts into a review queue and can be re-run by hand is shallow. A nightly job that mutates a durable table and has no second writer is deep, and depth is what makes removal expensive. Deep dependencies are not automatically wrong; they are wrong when nobody wrote down the cost of cutting them.

The useful classification is therefore about blast radius, not model quality. That is what the ledger encodes.

Artifact 1: a capability ledger you can lint

One JSON file per job, checked in next to the code. It records what the job actually asks of the endpoint, in vocabulary a reviewer can argue about.

{
  "job": "nightly-summarizer",
  "endpoint": "free-inference",
  "capabilities": ["mutates_durable_state"],
  "hard_latency_ms": null,
  "fallback": "enqueue to primary model",
  "owner": "platform"
}

The lint is deliberately blunt. Five capability flags are treated as red lines, because each one converts a vendor-side change into an incident you cannot cancel.

# stoplist.py - unexecuted template; wire into CI as a pre-merge check.
import json
import sys

RED_LINES = {
    "mutates_durable_state": "a free tier must never be the only writer",
    "grants_authority": "issued credentials outlive the endpoint",
    "carries_identity": "session trust cannot depend on a vendor's uptime",
    "bills_money": "spend needs a ceiling you control, not one they set",
    "hard_latency_ms": "no service level means no latency floor",
}


def audit(ledger: dict) -> list[tuple[str, str]]:
    hits = [(c, RED_LINES[c]) for c in ledger.get("capabilities", []) if c in RED_LINES]
    if ledger.get("hard_latency_ms") is not None and not ledger.get("fallback"):
        hits.append(("hard_latency_ms", RED_LINES["hard_latency_ms"]))
    return hits


ledger = json.loads(open(sys.argv[1]).read())
hits = audit(ledger)
for cap, why in hits:
    print(f"RED LINE  {ledger['job']}: {cap} -> {why}")
sys.exit(1 if hits else 0)

Running it against the sample above exits 1 with one line naming the durable write. The team then either adds a second writer or marks the job as an accepted, owned exception. Both outcomes are fine. Silence is the only failure mode the lint exists to prevent.

Artifact 2: the detach drill

Exit criteria are worthless if nobody can measure the exit. The drill runs one command twice: once normally, once with every provider base URL pointed at a port that refuses connections. Nothing else changes. The job's behavior in the second run is your true portability story.

#!/usr/bin/env python3
# detach_drill.py - unexecuted template. Usage:
#   python detach_drill.py ./data/jobs.db python -m app.worker --once
import hashlib
import os
import subprocess
import sys
import time

DETACHED = {
    "OPENAI_BASE_URL": "http://127.0.0.1:9/v1",
    "ANTHROPIC_BASE_URL": "http://127.0.0.1:9/v1",
}  # port 9 refuses immediately; no server is started on purpose


def digest(path: str) -> str:
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(65536), b""):
            h.update(chunk)
    return h.hexdigest()[:12]


def run(cmd: list[str], extra_env: dict, timeout: int = 180) -> dict:
    env = {**os.environ, **extra_env}
    t0 = time.monotonic()
    proc = subprocess.run(cmd, env=env, capture_output=True, text=True, timeout=timeout)
    stderr_tail = proc.stderr.strip().splitlines() or [""]
    return {
        "exit": proc.returncode,
        "secs": round(time.monotonic() - t0, 1),
        "tail": stderr_tail[-1][:160],
    }


def drill(cmd: list[str], state_file: str) -> bool:
    before = digest(state_file)
    wired = run(cmd, {})
    detached = run(cmd, DETACHED)
    after = digest(state_file)

    checks = {
        "wired_succeeds": wired["exit"] == 0,
        "detached_fails_loud": detached["exit"] != 0,
        "detached_fails_fast": detached["secs"] < 15,
        "state_untouched": before == after,
    }
    for name, ok in checks.items():
        print(f"{'PASS' if ok else 'FAIL'}  {name}")
    print(f"  wired    exit={wired['exit']} {wired['secs']}s {wired['tail']}")
    print(f"  detached exit={detached['exit']} {detached['secs']}s {detached['tail']}")
    return all(checks.values())


if __name__ == "__main__":
    sys.exit(0 if drill(sys.argv[2:], sys.argv[1]) else 1)

Four checks, each with a reason. detached_fails_loud catches the quietly-degrading job that swallows the error and writes perfect-looking zeros. detached_fails_fast catches the retry stack whose default backoff turns a refused connection into a six-minute hang; fifteen seconds is a starting threshold, tune it to your job's own budget. state_untouched catches partial writes, which are the only failure here that a rerun cannot repair.

Run the drill on a schedule, not once. A dependency that fails cleanly in October can fail messily in December after someone adds a single try/except.

Artifact 3: exit criteria with thresholds

A criterion without a number is a feeling. Each row below names something you can observe in logs or a dashboard, a threshold committed to in advance, and the action that follows automatically.

Observable signal	Committed threshold	Action when crossed
Free path p95 vs paid path p95	Free path above 2x for 3 consecutive days	Route the job to the paid path
Detach drill result	Two consecutive failures	Remove the free endpoint from that job
Pages attributable to the endpoint	More than one per month	Move to a paid or self-hosted deployment
Vendor-stated allowance remaining	Consumed before day 20 of the month	Add budgeting in code, not throttling in comments
Fallback path usage	Zero invocations for 90 days	The fallback is untested; drill it or delete it

The last row is the one teams skip. An untested fallback is a comment.

Where a free endpoint legitimately belongs

MonkeyCode is an open-source project offering free model access and a free server option; the operator states a free allowance on the order of ten million tokens, which should be read as a vendor-stated figure and verified on the project's own pages rather than treated as measured capacity. The honest use in this article is narrow and real: the drill needs a target you are willing to break.

Pointing detach_drill.py at a free endpoint costs nothing and teaches you the shape of your own code. The same free server option is a reasonable place to host the throwaway harness, the ledger lint, and a staging copy of the job under test — work whose worst outcome is a wasted afternoon.

That is also the boundary. Regulated data, identity issuance, anything that writes a system of record, and any latency promise you have put in a contract all sit outside it. Those need a paid tier with terms, or your own hardware, and the stop list should say so in the first paragraph rather than the last.

Limitations of this approach

The drill measures your code's shape, not the endpoint's reliability. It will not warn you that a free allowance is being reduced, and it cannot distinguish a graceful refusal from a graceful refusal that happens to return the wrong answer — for that you need assertion-level tests on output content, which is a different artifact.

All three snippets above are unexecuted templates. Port numbers, timeouts, file paths, and environment variable names must be adapted to your stack before anyone trusts a pass.

Finally, a stop list ages. Put a review date on the file, and treat a stale ledger as a red line of its own, because the risk being managed is not the free tier — it is the assumption that nothing about it will change.