Before You Automate a Free Model Endpoint, Measure Rate, Shape, Drift, and Budget

A free model endpoint is not ready for automation until you measure its rate limit, response shape, and drift—not its uptime.

The failure mode is common. A free endpoint returns 200 once, so you wire it into a job. Later, under concurrency, it returns 429, a truncated JSON body, or a sudden latency spike. Your pipeline does not know the difference and acts on a bad output.

Previous posts on this account covered gates after the model responds: diffing the filesystem, a permission matrix, buffering streaming JSON, and read-only SQL. This piece moves earlier. It is a preflight acceptance probe you run before any of those gates matter.

One relevant setup is MonkeyCode's free model access and free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Those two availability claims are operator-supplied, not a guarantee about a particular model, quota, uptime, or latency. Do not assume any of those from a marketing page.

What the probe measures

A 200 status code is not acceptance. The probe checks four signals:

1. Rate shape - how many requests succeed before a 429, and whether Retry-After is present.
2. Response shape - whether the endpoint returns the promised JSON fields across repeated calls.
3. Drift - whether the same prompt changes structure or latency over a short window.
4. Budget - token counts in and out, so you can estimate the cost of a real job.

These are not quality checks. They do not tell you whether the model is good at coding or reasoning. They tell you whether the endpoint is safe to automate at all.

The probe

The script below uses only the Python standard library. It assumes a non-streaming JSON endpoint with a chat-style messages field:

import json, os, time, urllib.error, urllib.request
from concurrent.futures import ThreadPoolExecutor

ENDPOINT = os.environ.get("FREE_ENDPOINT", "")
API_KEY = os.environ.get("FREE_API_KEY", "")

PROMPTS = [
    [{"role": "user", "content": 'Return JSON with keys "ok" and "summary".'}],
    [{"role": "user", "content": 'Return JSON with keys "ok" and "summary" for another topic.'}],
    [{"role": "user", "content": 'Return JSON with keys "ok" and "summary".'}],
]

def call(prompt, timeout=20):
    payload = {
        "messages": prompt,
        "temperature": 0,
    }
    # Only include response_format if the endpoint documents it.
    # payload["response_format"] = {"type": "json_object"}

    req = urllib.request.Request(
        ENDPOINT,
        data=json.dumps(payload).encode(),
        headers={
            "Authorization": f"Bearer {API_KEY}",
            "Content-Type": "application/json",
        },
        method="POST",
    )
    start = time.monotonic()
    result = {"status": None, "data": None, "latency": None, "retry_after": None}

    try:
        with urllib.request.urlopen(req, timeout=timeout) as r:
            result["status"] = r.status
            result["data"] = json.loads(r.read().decode())
            result["latency"] = time.monotonic() - start
    except urllib.error.HTTPError as e:
        result["status"] = e.code
        result["data"] = {"error": e.read().decode()[:200]}
        result["latency"] = time.monotonic() - start
        result["retry_after"] = e.headers.get("Retry-After")
    except Exception as e:
        result["data"] = {"exception": type(e).__name__}
        result["latency"] = time.monotonic() - start

    return result


def shape_ok(result):
    try:
        content = result["data"]["choices"][0]["message"]["content"]
        return result["status"] == 200 and isinstance(content, str) and bool(content)
    except Exception:
        return False


def shape_check():
    return [call(p) for p in PROMPTS]


def rate_check(concurrency=3, rounds=2):
    def worker(_):
        return call(PROMPTS[0])

    with ThreadPoolExecutor(max_workers=concurrency) as ex:
        return [f.result() for f in [ex.submit(worker, i) for i in range(concurrency * rounds)]]


def budget_check(results):
    total_in = 0
    total_out = 0
    for r in results:
        usage = r.get("data", {}).get("usage", {})
        total_in += usage.get("prompt_tokens", 0)
        total_out += usage.get("completion_tokens", 0)
    return {"input_tokens": total_in, "output_tokens": total_out}


if __name__ == "__main__":
    results = shape_check()
    print("shape_ok:", [shape_ok(r) for r in results])

    rate_results = rate_check()
    print("status:", [r["status"] for r in rate_results])
    print("latency_ms:", [round((r["latency"] or 0) * 1000) for r in rate_results])
    print("budget_estimate:", budget_check(results + rate_results))

To use it:

• Set ENDPOINT and API_KEY from your environment, not in the file.
• If the endpoint rejects response_format, delete that field and validate the returned string with a JSON parser instead.
• Run shape_check once, then rate_check with a concurrency close to your real workload.
• budget_check only works when the response includes a usage object. Many free endpoints do not expose it; treat absence as an unknown, not zero.

This is a template, not a benchmark result. It has not been run against a current quota.

Go/no-go table

Set thresholds before running. They depend on your job.

Signal	Accept	Investigate	Reject
Non-200 rate	None in 10 calls	One 429 in 10 calls	Repeated 429 or no Retry-After
Missing choices[0].message.content	None	One malformed body	Repeated malformed bodies
Latency	Every call under your max	Occasional spikes	Consistent timeout or >2x baseline
Rate-limit ceiling	Over your expected concurrency	Equal to expected concurrency	Below expected concurrency

Do not copy these numbers. Your free tier may allow one concurrent request or twenty. Measure it.

Why this is not a trust signal

Passing the probe does not mean the model is safe. A free endpoint can:

• Return correct JSON with wrong content.
• Pass the shape check and hallucinate a command.
• Have a short cooling-off period after the probe.
• Change schema tomorrow.

Run this probe before the gates, not instead of them. The earlier gates still decide what the output may touch.

Also measure drift over time rather than once. A one-off acceptance probe can false-positive if the endpoint is in a quiet window. Run it at two times of day, or compare the current pass with the previous pass saved to a small JSON file.

Example drift record:

{
  "checked_at": "2026-08-15T10:00:00Z",
  "requests": 10,
  "non_200": 0,
  "median_latency_ms": 842,
  "shape_ok": 10
}

If the next record jumps to 8 non-200s or 2400 ms median latency, stop the pipeline before it starts.

Who should not use this

• You need a hard SLA or production secrets.
• You are in a compliance or safety-critical domain.
• You expect free tiers to be stable and versioned.
• You want to evaluate model quality. This probe measures transport, not reasoning.

For those cases, a paid or dedicated endpoint with a contract is the right tool.

Conclusion

A free model endpoint earns its place in automation by surviving a preflight probe, not by answering one curl. Measure rate shape, response shape, drift, and token budget first. Then hand the output to the existing gates.

If you are using MonkeyCode's free model or server option, run this probe before you connect it to anything that can write or spend.