Your Free Tier Has a Concurrency Knee. Go Find It.

The free endpoint didn't fail me with a wrong answer. It failed me with a right answer, eleven seconds late, and that is a much harder bug to notice.

So this is an evaluation of MonkeyCode's free model access and its free server option — both as the operator describes them, not as a benchmark I can hand you. Disclosure: This article was prepared as part of MonkeyCode's product outreach. As of this writing the offer is advertised as a large free token allowance plus a free server; I'm treating every number on that page as a claim to re-check on the day you read this, because free-tier terms move and stale numbers are worse than no numbers. What I can give you is a measurement you run yourself in about an hour, and a rule for reading it.

The mistake is counting tokens

Everyone audits a free tier by watching the quota drain. That's the visible meter, so it gets the attention. But the meter rarely decides whether your job finishes — the queue does. Two requests with identical prompt sizes can land a second apart or thirty seconds apart depending on how many other people hit the same free capacity at that moment. Which means the question isn't "how many tokens do I get?" It's "at what concurrency does the shape of the response stop being mine?"

That threshold has a name in capacity work: the knee. Below it, latency grows gently and errors are rare. Above it, p95 detaches from p50, and the kind of failure changes — first slow, then 429, then truncated bodies, then connection timeouts. Your retry logic behaves completely differently on each side of that line, and if you never located the line you tuned your retries against a phantom.

A harness that produces one number

The experiment is deliberately boring: hold the request byte-identical, step concurrency 1 → 2 → 4 → 8 → 16 → 32, and record latency percentiles plus a failure shape for every call. Failure shape is the part people skip. A 429 is not a timeout is not a 200 with an empty choices array, and lumping them into "error" throws away the only signal that tells you which fix to apply.

# knee.py — find the concurrency where latency stops being yours
import asyncio, json, os, statistics, time, httpx

BASE  = os.environ["FREE_BASE_URL"].rstrip("/")
KEY   = os.environ["FREE_API_KEY"]
MODEL = os.environ["FREE_MODEL"]
PROMPT = "Return the SHA-256 of the string 'knee' plus one sentence explaining it."

def body():
    return {"model": MODEL, "temperature": 0,
            "messages": [{"role": "user", "content": PROMPT}]}

async def one(client, sem, out):
    async with sem:
        t0, shape = time.perf_counter(), "ok"
        try:
            r = await client.post(f"{BASE}/chat/completions", json=body())
            dt = time.perf_counter() - t0
            if r.status_code == 429:            shape = "429"
            elif r.status_code >= 500:          shape = f"5xx_{r.status_code}"
            elif r.status_code != 200:          shape = f"http_{r.status_code}"
            else:
                try: j = r.json()
                except json.JSONDecodeError:    shape = "bad_json"
                else:
                    if not j.get("choices"):    shape = "empty_choices"
            out.append((dt, shape))
        except httpx.ReadTimeout:      out.append((time.perf_counter()-t0, "read_timeout"))
        except httpx.ConnectTimeout:   out.append((time.perf_counter()-t0, "connect_timeout"))
        except Exception as e:         out.append((time.perf_counter()-t0, type(e).__name__))

async def step(c, n):
    out, sem = [], asyncio.Semaphore(c)
    timeout = httpx.Timeout(60.0, connect=10.0)
    async with httpx.AsyncClient(timeout=timeout) as client:
        t0 = time.perf_counter()
        await asyncio.gather(*(one(client, sem, out) for _ in range(n)))
        wall = time.perf_counter() - t0
    lat = sorted(d for d, _ in out)
    shapes = {}
    for _, s in out: shapes[s] = shapes.get(s, 0) + 1
    return statistics.median(lat), lat[int(0.95*(len(lat)-1))], wall, shapes

async def main():
    for c in (1, 2, 4, 8, 16, 32):
        p50, p95, wall, shapes = await step(c, max(8, c * 4))
        print(json.dumps({"c": c, "p50": round(p50, 2), "p95": round(p95, 2),
                          "wall": round(wall, 2), "shapes": shapes}), flush=True)
        bad = sum(v for k, v in shapes.items() if k != "ok")
        if p95 > 4 * p50 or bad > 0.05 * sum(shapes.values()):
            print(f"KNEE at concurrency {c}")
            break

asyncio.run(main())

Run it under tee so each day's numbers are a file, not a memory:

FREE_BASE_URL=... FREE_API_KEY=... FREE_MODEL=... \
  python knee.py | tee knee-$(date +%F).jsonl
jq -s 'map({c, ratio: (.p95 / .p50)})' knee-*.jsonl

Two definitions make the output actionable instead of decorative. The latency knee is the first rung where p95 exceeds four times p50 — that's where a shared queue starts deciding your user experience. The error knee is the first rung where more than five percent of calls aren't plain ok. They are almost never the same rung, and the gap between them is your entire safety margin. If you ship at the latency knee because the errors haven't started yet, you've shipped into the gap.

Then run it twice, from two places

Here's where the free server earns its keep as an instrument, not just a host. Run the identical script from your laptop, then from the free server. Same prompt, same code, different client. If p50 barely moves but p95 collapses, your problem was the path from your network to the endpoint, not the endpoint. If both move together, you're looking at shared capacity and no amount of moving hosts will help. That's a fifteen-minute experiment that replaces a week of guessing, and it's the reason I'd rather have a free box next to the endpoint than a bigger quota far away from it.

What the number tells you to build

Workload	Verdict after the knee test
Single-user CLI or dev loop	Fine if c=1 p95 stays under your patience
Latency-blind batch backfill	Fine one rung below the error knee, with a retry budget
Chat UI with a human staring at it	Only if p95 at your real peak stays under budget
Anything with a contract attached	Measure, then don't promise it

Set your production concurrency to the error knee minus one, not to the biggest number that didn't explode. Set client timeouts slightly above the p95 at that rung, because a timeout below your own tail is a self-inflicted error generator. And log the shape, always — "it failed" cost me hours; 429 costs me minutes.

Where this approach breaks

It breaks when your real traffic isn't this uniform. One long-context request in a batch of short ones drags the tail in ways a fixed prompt never shows, so re-run with your heaviest real payload before you trust the rung. It breaks when the free capacity changes with the crowd, which is why the tee files matter more than any single run. And it breaks completely if you need a number you can put in a contract — a shared free queue cannot give you a p95 guarantee, and asking it to is a category error, not a tuning problem.

Who shouldn't bother? Anyone whose workload is already one request at a time, anyone who can't spend the hour, and anyone who needs an SLA on the response. For the rest of us — people prototyping agents on free capacity and quietly hoping the tail behaves — the knee is the only number worth knowing, and it takes one script to find. If you want to run it against the free models and free server, the project page has the setup; just check today's terms before you budget against them.