Red Flags Before Routing a Workload Through a Free Tier

Free model access and a free server are probe instruments. They answer one question well — does this idea survive contact with real input — and they answer it cheaply, which is exactly why they get promoted into roles they were never shaped for. The usual failure is not a crash. It is a quiet dependency on capacity nobody promised, wrapped around a code path that a team now treats as production.

This field guide is the inverse of an adoption post. It lists the conditions under which a workload should stay off a free tier, the measurements that decide it, and the criteria that force a migration before the decision gets made for you.

Why free capacity leaks into critical paths

The leak happens for a boring reason. A prototype on free capacity behaves like the real thing during a demo: it returns tokens, it streams, it looks fine. Nothing in the response tells the caller that the model version may move underneath it, that capacity is shared with strangers, or that consumption is drawn from a pool with no contract attached. Only failure surfaces separate the two, and failure surfaces appear after the prototype already has callers.

The distinction worth keeping in the head is accountability. A component is load-bearing when someone can be held to its behavior — a pager, a stored record, an invoice, an audit line. Anything load-bearing belongs somewhere with an owner, a version, and a stated limit.

Four red flags, ordered by how often they bite

The output outlives the request. If a response lands in a database, an object store, a customer inbox, or a compliance archive, the tier that produced it has joined a provenance chain. Free tiers generally do not commit to traceability, and the same prompt replayed next month may not produce the same text.

The caller retries on its own. Retries turn a transient refusal into multiplied consumption, and multiplied consumption into an argument with a ceiling you did not set. Free capacity is most fragile precisely when traffic is spiky, which is when automatic retries fire hardest.

The workload carries secrets. Prompts assembled from internal documents, stack traces, or customer records push sensitive data across a boundary the operator does not control. That is a legal and architectural problem, not a throughput problem.

A human latency promise sits on top. Shared capacity gives a fine median and a jittery tail. The median is what the demo shows; the tail is what the user experiences at 4pm.

Any single flag is survivable with eyes open. Two flags on the same call path is the point where the migration is already late.

A probe harness you can run before committing

The decision should come from numbers, so record them from the first call. The harness below uses only the standard library, runs without a network key in --dry-run mode, and prints a verdict against a policy file. Treat it as a template: swap the synthetic generator for a thin client that wraps your provider call. No performance figures for any provider are claimed here; the dry-run data is deliberately synthetic.

#!/usr/bin/env python3
"""tier_probe.py - decide whether a workload belongs on a free tier.

  python tier_probe.py --dry-run --calls 40 --policy policy.json
  echo $?   # 1 means at least one exit criterion was breached
"""
import argparse, json, random, statistics
from dataclasses import dataclass


@dataclass
class Attempt:
    call_id: int
    attempt: int      # 1 = first try, >1 = retry
    status: str       # ok | rate_limited | timeout | error
    tokens_in: int
    tokens_out: int
    latency_ms: int


def dry_run(calls: int, seed: int = 7):
    rng = random.Random(seed)
    rows = []
    for cid in range(calls):
        attempt = 1
        while True:
            roll = rng.random()
            status = "ok" if roll > 0.18 else rng.choice(
                ["rate_limited", "timeout", "error"])
            rows.append(Attempt(cid, attempt, status,
                                rng.randint(300, 1500), rng.randint(50, 600),
                                int(rng.gauss(900, 250))))
            if status == "ok" or attempt >= 3:
                break
            attempt += 1
    return rows


def measure(rows):
    per_call = {}
    for r in rows:
        per_call.setdefault(r.call_id, []).append(r)
    total = sum(r.tokens_in + r.tokens_out for r in rows)
    wasted = sum(r.tokens_in + r.tokens_out for r in rows if r.attempt > 1)
    ok = [r.latency_ms for r in rows if r.status == "ok"]
    failures = sum(1 for r in rows if r.status != "ok")
    return {
        "calls": len(per_call),
        "attempts_per_call": round(len(rows) / max(len(per_call), 1), 2),
        "failure_rate": round(failures / max(len(rows), 1), 3),
        "retry_token_share": round(wasted / max(total, 1), 3),
        "calls_never_succeeded": sum(
            1 for rs in per_call.values() if all(r.status != "ok" for r in rs)),
        "median_ok_latency_ms": int(statistics.median(ok)) if ok else 0,
    }


def breaches(m, policy):
    return {k: (m[k], limit) for k, limit in policy.items() if m[k] > limit}


if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("--calls", type=int, default=40)
    ap.add_argument("--policy", default="policy.json")
    ap.add_argument("--dry-run", action="store_true")
    a = ap.parse_args()
    if not a.dry_run:
        raise SystemExit("wire your own client, then drop --dry-run")
    m = measure(dry_run(a.calls))
    with open(a.policy) as fh:
        policy = json.load(fh)
    hit = breaches(m, policy)
    print(json.dumps(m, indent=2))
    print("breaches:", hit or "none")
    raise SystemExit(1 if hit else 0)

A policy file turns engineering taste into a gate. The defaults below are proposed starting points for a backlogged team, not measured properties of any service, and they should be tightened for user-facing paths and loosened for overnight batch work.

{
  "failure_rate": 0.10,
  "retry_token_share": 0.20,
  "attempts_per_call": 2.00,
  "calls_never_succeeded": 0
}

Run it on day one, save the JSON next to the prototype, and re-run it whenever the workload changes shape. retry_token_share is the metric most teams never look at, and it is the one that pre-announces a wall: above roughly a fifth of consumption, the workload is not being served, it is gambling.

Routing by workload shape

Once the numbers are in hand, routing becomes a lookup rather than a debate.

Workload shape	Route	Reason
Scratch classification, output read once	free tier	the loss on failure is a discarded row
Demo, workshop, throwaway preview	free tier plus free server	reset is the feature, not a gap
Batch enrichment into a durable store	paid or self-hosted	provenance and version pinning
Anything inside a CI gate	deterministic check, or pinned paid model	a flaky gate teaches people to ignore it
Long-lived agent with memory	self-hosted with checkpointing	state must survive a capacity change
Regulated or secret-bearing input	self-hosted or contracted vendor	boundary control

Where MonkeyCode's free access fits

The operator of MonkeyCode states that the project provides free model access, described as up to ten million tokens, along with a free server option. Those figures are operator-supplied; I have not independently verified them, and this article claims no benchmark, uptime, quota-per-window, hardware detail, or permanence for either offer.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What that shape of offer is genuinely good for is the first two rows of the table above: probe work, throwaway demos, teaching material, and experiments whose output nobody will be held to. The free server option is a reasonable home for a preview environment you are happy to rebuild from scratch, since the interesting property of a sandbox is that deleting it costs nothing. The same reasoning makes it a poor home for the bottom four rows: the moment output persists, feeds a gate, or carries data with a boundary around it, an owner, a version, and a contract stop being optional.

If the experiment in mind is discardable by design, the free access and free server are worth an afternoon — but run the probe first, so the decision rests on a number rather than a feeling.

Exit criteria and who should not use this

The exit plan should exist before the first call. Keep exactly one seam in the code:

import os

ROUTE = os.environ.get("INFERENCE_ROUTE", "free")
ENDPOINTS = {
    "free":  (FREE_BASE_URL, FREE_MODEL_ALIAS),
    "paid":  (PAID_BASE_URL, PINNED_MODEL_VERSION),
    "local": ("http://127.0.0.1:8080/v1", LOCAL_ALIAS),
}
base_url, model = ENDPOINTS[ROUTE]

With that seam in place, migration is a config change rather than a refactor, and a reviewer can see which environment is hitting which endpoint. The trigger to flip it is the policy file above, not a hunch: two breached criteria on a user-facing path, one breach plus a red flag, or any change that makes the output auditable.

Three groups should skip the free tier entirely for their main path. Teams handling regulated data, because the boundary question has one acceptable answer. Teams that owe an uptime commitment to anyone, because shared capacity cannot be paged. And anyone whose artifact must be reproducible in ninety days, because a moving alias makes that promise unkeepable.

Nobody has to be talked out of using free capacity. They have to be given the moment to leave it, and that moment is cheaper to define with a probe script than to discover during an incident.