Build a 45-Minute Demand Probe Before You Trust a 30M-Token Free Server

Why this is worth reading: a free tier with 30 million tokens and a free server sounds like a default yes, but free infrastructure becomes risky when you never measure it. You do not need another watchdog; you need a 45-minute demand probe that tells you whether the endpoint tolerates your concurrency, what each request costs you, and when to walk away before your side project quietly depends on a changing free tier.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat the free server claim as a hypothesis

MonkeyCode's operator describes free model access and a free server option. That is a starting claim, not a permanent capability guarantee. Free tiers change, shared servers get cold, and an open source repository does not automatically equal a hosted endpoint with stable behavior. The practical move is to test the shape of traffic you will actually send.

The probe below does three things:

1. Sends a fixed payload in a controlled concurrency loop.
2. Records status codes, latencies, failures, and whatever token usage the endpoint returns.
3. Exits with a nonzero code when a small set of pass/fail thresholds is broken.

You can run it against any free endpoint before you wire it into a CLI, sidecar, or batch job.

Start with a single curl smoke test

Create a deliberately small JSON payload first. A small payload means a timeout cannot hide behind long generation.

payload.json:

{"messages":[{"role":"user","content":"Reply with exactly: ok"}],"temperature":0}

Then run one request before you start the loop:

export PROBE_ENDPOINT="https://your-endpoint.example/v1/chat/completions"
export PROBE_API_KEY="your-key"
curl -sS -X POST "$PROBE_ENDPOINT" \
  -H "Authorization: Bearer $PROBE_API_KEY" \
  -H "Content-Type: application/json" \
  -d @payload.json | jq .

If the first request does not return a normal JSON body, stop. The probe cannot interpret an undocumented failure format.

Install the probe

The script needs Python 3.10 or newer.

python -m venv .venv && source .venv/bin/activate
pip install httpx

The probe script

Save this as probe_free_tier.py. It is intentionally single-file so you can read it in one sitting and edit it when the API response shape changes.

import asyncio
import json
import os
import sys
import time
from pathlib import Path

import httpx

ENDPOINT = os.environ['PROBE_ENDPOINT']
API_KEY = os.environ['PROBE_API_KEY']
PAYLOAD = json.loads(Path(os.environ.get('PROBE_PAYLOAD_FILE', 'payload.json')).read_text())
REQUESTS = int(os.environ.get('PROBE_REQUESTS', '300'))
CONCURRENCY = int(os.environ.get('PROBE_CONCURRENCY', '4'))
TIMEOUT = float(os.environ.get('PROBE_TIMEOUT', '15'))

sem = asyncio.Semaphore(CONCURRENCY)
status_counts: dict[int, int] = {}
latencies: list[float] = []
tokens: list[int] = []
errors: list[str] = []

def pct(xs: list[float], p: float) -> float | None:
    if not xs:
        return None
    s = sorted(xs)
    i = min(len(s) - 1, int((p / 100) * len(s) - 1))
    return round(s[max(0, i)], 3)

async def one(client: httpx.AsyncClient, i: int) -> None:
    async with sem:
        start = time.perf_counter()
        try:
            r = await client.post(
                ENDPOINT,
                headers={'Authorization': f'Bearer {API_KEY}'},
                json=PAYLOAD,
            )
            status_counts[r.status_code] = status_counts.get(r.status_code, 0) + 1
            if r.status_code == 200:
                try:
                    data = r.json()
                    usage = data.get('usage', {})
                    if isinstance(usage.get('total_tokens'), int):
                        tokens.append(usage['total_tokens'])
                except Exception:
                    pass
            else:
                errors.append(f'{i}: {r.status_code} {r.text[:120]}')
        except Exception as exc:
            errors.append(f'{i}: {type(exc).__name__} {exc}')
        finally:
            latencies.append(time.perf_counter() - start)

async def main() -> None:
    limits = httpx.Limits(max_connections=CONCURRENCY * 2)
    async with httpx.AsyncClient(timeout=TIMEOUT, limits=limits) as client:
        await asyncio.gather(*(one(client, i) for i in range(REQUESTS)))
    ok = status_counts.get(200, 0)
    total = REQUESTS
    p50 = pct(latencies, 50)
    p95 = pct(latencies, 95)
    print(f'ok={ok}/{total} status={status_counts}')
    print(f'p50={p50}s p95={p95}s max={max(latencies):.3f}s')
    if tokens:
        avg = sum(tokens) / len(tokens)
        print(f'token_observations={len(tokens)} mean={avg:.1f} estimated_total_burn={avg * total:.0f}')
    else:
        print('no token usage found in responses')
    if errors:
        print(f'errors={len(errors)}')
        for e in errors[:10]:
            print(f'  {e}')
    if ok / total < 0.98 or p95 is None or p95 > 8:
        sys.exit(2)
    print('probe passed: success rate >= 98% and p95 < 8s')

if __name__ == '__main__':
    asyncio.run(main())

Run the probe

Use a small run first to avoid burning a meaningful chunk of a 30 million token allowance on a broken loop.

export PROBE_REQUESTS=300
export PROBE_CONCURRENCY=4
python probe_free_tier.py

Then, if the first run passes, scale to a five-minute run:

export PROBE_REQUESTS=1000
export PROBE_CONCURRENCY=8
python probe_free_tier.py

Read the pass/fail table

Metric	Pass	Abandon or lower concurrency
HTTP 200 rate	>= 98%	< 98% or any non-transient 5xx
p95 latency	< 8s	missing p95 or p95 > 8s
Token usage	response includes usage.total_tokens	no token count means you cannot budget
429 responses	0 in the final run	any 429 means the free server is throttling you

The exact thresholds are arbitrary, but they are explicit. You can change them after the first run, but write the new numbers down before you retry.

What the probe does and does not prove

The probe proves whether the endpoint survived your payload and concurrency for the minutes you tested. It does not prove that 30 million tokens will last all month, that the free server has no cold start, that the model is the same later, or that the node you hit is the one you will hit tomorrow.

Because MonkeyCode is an open source project, you can inspect the server code and run a local copy for baseline tests. That is a useful escape hatch: if the hosted free server changes, you still have the source to evaluate, not a black-box dependency.

Who should not use this approach

Do not use a free server probe as your only test for payment processing, regulated personal data, long-context batch jobs, or any path with a hard latency objective. The probe is a preflight for a side project or a local tool, not a substitute for an SLA.

If you are evaluating MonkeyCode's current free tier and free server for a small build, start with this probe, read the official limits, and keep the script in your repo. A free server is useful when you know its failure shape before your code does.