Opinion: Free Model Access Is Worthless Without a Kill Switch

Free capacity is not the hard part anymore. Throwing it away safely is.

My position: free model access and a free server only matter if your workload is disposable. If it is not, the free tier just moves the failure later. You get the same outage with worse logs.

MonkeyCode is an open-source coding agent project with free model access and a free server option. Both availability claims come from the operator, not from a benchmark I ran. I have not measured their throughput, limits, or uptime.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The thing everyone skips

Cheap inference is easy to demo. It is hard to operate.

Nobody posts about the run that stalled at task 37. Nobody shows the ledger where token spend silently tripled. That failure mode is boring, and boring is where the money goes.

So the real question is not "how many free tokens do I get". It is "what stops the run".

Three properties of a disposable workload

Before I put anything on free capacity, I check three things.

1. Stateless. Kill the box mid-run and nothing is lost but time.
2. Bounded. The run has a hard call cap and a wall-clock cap.
3. Replayable. Inputs are in version control, so a rerun is one command.

Fail any one and free capacity becomes expensive. You pay in debugging hours instead of dollars.

The artifact: a budget-guarded runner

Here is the smallest kill switch I trust. It works against any OpenAI-style chat completions endpoint.

# budget_runner.py — hard caps first, retries never
import json, os, sys, time, urllib.request

BASE = os.environ["LLM_BASE_URL"].rstrip("/")
KEY = os.environ["LLM_API_KEY"]
MODEL = os.environ.get("LLM_MODEL", "default")
MAX_CALLS = int(os.environ.get("MAX_CALLS", "20"))
MAX_SECONDS = int(os.environ.get("MAX_SECONDS", "300"))
LEDGER = os.environ.get("LEDGER", "ledger.jsonl")


def call(prompt):
    body = json.dumps({
        "model": MODEL,
        "messages": [{"role": "user", "content": prompt}],
    }).encode()
    req = urllib.request.Request(
        f"{BASE}/chat/completions",
        data=body,
        headers={
            "Authorization": f"Bearer {KEY}",
            "Content-Type": "application/json",
        },
    )
    with urllib.request.urlopen(req, timeout=60) as r:
        return json.loads(r.read())


def main():
    prompts = [line.strip() for line in sys.stdin if line.strip()]
    started = time.time()
    with open(LEDGER, "a") as log:
        for i, prompt in enumerate(prompts):
            if i >= MAX_CALLS or time.time() - started > MAX_SECONDS:
                print(f"STOP: budget hit at task {i}", file=sys.stderr)
                sys.exit(2)
            t0 = time.time()
            try:
                resp = call(prompt)
                rec = {"i": i, "ok": True, "secs": round(time.time() - t0, 2),
                       "tokens": resp.get("usage", {}).get("total_tokens")}
            except Exception as exc:
                rec = {"i": i, "ok": False, "secs": round(time.time() - t0, 2),
                       "err": type(exc).__name__}
            log.write(json.dumps(rec) + "\n")
            log.flush()
            print(json.dumps(rec))


if __name__ == "__main__":
    main()

Run it like this. The caps are the point.

printf '%s\n' "summarise src/a.py" "summarise src/b.py" \
  | MAX_CALLS=5 MAX_SECONDS=120 LLM_MODEL=default \
    LLM_BASE_URL=$BASE LLM_API_KEY=$KEY \
    python budget_runner.py

Three behaviours matter here.

• Exit code 2 means the budget stopped you, not the network.
• No retries. A failed call is data, not a trigger.
• The ledger is append-only. One JSONL file per run, diffable across runs.

I have not executed this against the free server. Treat it as a template you adapt to the documented base URL and auth shape.

Why no retries is a feature

Free capacity blips. That is expected. Silent retries hide it.

A retry loop turns one bad minute into a spend spike and a flaky test suite. Your ledger stops meaning anything. You can no longer answer "did the run fail, or did we paper over it".

Count failures instead. If the failure rate is above your tolerance, the workload is not ready for shared capacity. That is a design answer, not an infrastructure answer.

Decision table

Workload	Free capacity OK?	Reason	Guard you need
Refactors on a scratch clone	Yes	Throwaway branch, no secrets	Call cap + clone deletion
Test generation for review	Yes	Human reviews every diff	Diff size cap
Docstring and changelog drafts	Yes	Low blast radius	Read-only repo access
Long agent loops with tool use	Carefully	One bad loop burns hours	Wall-clock cap, no auto-merge
Anything touching customer data	No	Shared capacity, unclear retention	None exists — use your own
Latency-sensitive endpoints	No	No SLA on free anything	Not a guard problem

Read the table as a filter, not a ranking. Most teams should start at row one.

Debugging workflow when the box dies

It will die. Here is the order I check things in.

1. Read the ledger tail. Confirm the last task index and elapsed seconds.
2. Check the exit path. Code 2 means your cap fired. Anything else means the environment.
3. Rerun with MAX_CALLS=1. If that fails, the problem is config, not load.
4. Diff two ledgers. Same prompt count, different timing means shared capacity noise.
5. Only then widen the cap. One variable at a time, always.

Step 4 is the one people skip. Two ledgers from the same inputs tell you more than any dashboard.

Where this breaks

The approach has real limits, and I would rather name them.

• Shared capacity is shared. Timing will vary and you cannot schedule around it.
• The free server is disposable by design. Do not treat local disk as storage.
• No SLA means no on-call story. Route alerts to a human who can switch providers.
• You still need a paid path. Free capacity is a ramp, not a floor.
• Compliance does not care about free. Data retention rules apply either way.

Who should not use this

Skip this workflow if you run production traffic, handle regulated data, or promise latency numbers. Skip it if your team cannot review generated diffs. Free capacity will not fix an unreviewed merge queue. It just makes the queue longer and cheaper to fill.

If you are prototyping, generation-testing, or teaching an agent a new repo, the economics flip. You want cheap, disposable, and boring.

The position, restated

Stop evaluating free capacity by how much you get. Evaluate it by how fast you can walk away.

A kill switch is ten lines of Python. It is also the difference between a free experiment and an unowned incident.

If you want to try the setup, MonkeyCode's free model access and free server option are the cheapest way I know to run this harness for an afternoon and read your own ledger.