200 OK Proves Nothing About Tool Calls: A Free-Model Capability FAQ With a Probe

I wired up a free model endpoint last month. I sent one request. It returned 200 OK. I shipped the feature. Then the tool calls never happened. The endpoint accepted the request. It just ignored half of it. Sound familiar?

Here is my question. What does a 200 actually prove? My answer: almost nothing about capabilities. It proves the transport worked. That is it.

So I stopped trusting status codes and started probing the contract. This post is the FAQ I wish I had read first. It ends with a script you can run in about two minutes.

Myth 1: A 200 means the model supports my feature

The server can accept a tools array and silently ignore it. The model then answers in prose. No error. No warning. Your code sees a normal response and skips the branch. Is that a bug in the model? No. It is a bug in your assumption.

The corrected model: support is a property of the endpoint, not of the model name. Probe it. Record it. Re-probe when anything changes.

Myth 2: finish_reason is optional metadata

It is the most important field in the response. length means the model ran out of output budget. Your answer looks fine and is probably cut in half. I have watched a parser happily eat a truncated JSON object.

{"choices":[{"finish_reason":"length","message":{"content":"{\"items\": [1, 2, 3, 4"}}]}

If you do not branch on finish_reason, you are parsing whatever bytes arrived. That is not a feature. That is a coin flip.

Myth 3: Streaming and non-streaming are interchangeable

They are two different code paths. In streaming, finish_reason arrives in the final chunk, not the first. Some servers only report token usage when you ask for it explicitly. Some omit it entirely.

Ask yourself: does my client read the last event? Mine did not, for a week. I was logging None as the finish reason every single time.

Myth 4: JSON mode means my schema

It means valid JSON. It does not mean your keys. It does not mean your types. Validate every response against a schema before it touches business logic. Treat model output like user input, because that is what it is.

Myth 5: Retries are always safe

Retry a failed read, fine. Retry a tool call that already wrote a row, and now you have two rows. I learned this from a duplicate charge, not a blog post.

Add an idempotency key your tool honors. Make the retry decision depend on the operation, not on the exception class.

The artifact: a five-minute capability probe

This script uses only the Python standard library. It assumes a chat-completions-shaped endpoint. Adjust the paths if yours differs.

# probe_capabilities.py — smoke test only, not a benchmark
import json, os, time, urllib.request, urllib.error, concurrent.futures as cf

BASE  = os.environ["BASE_URL"].rstrip("/")
KEY   = os.environ["API_KEY"]
MODEL = os.environ["MODEL"]

def call(payload, stream=False, timeout=60):
    req = urllib.request.Request(
        BASE + "/chat/completions",
        data=json.dumps(payload).encode(),
        headers={"Content-Type": "application/json",
                 "Authorization": f"Bearer {KEY}",
                 "Accept": "text/event-stream" if stream else "application/json"},
        method="POST")
    t0 = time.time()
    try:
        with urllib.request.urlopen(req, timeout=timeout) as r:
            return {"status": r.status, "ms": int((time.time()-t0)*1000),
                    "raw": r.read().decode("utf-8", "replace")}
    except urllib.error.HTTPError as e:
        return {"status": e.code, "ms": int((time.time()-t0)*1000),
                "raw": e.read().decode("utf-8", "replace")}

def probe_truncation():
    p = {"model": MODEL, "max_tokens": 8,
         "messages": [{"role": "user", "content": "List ten colors."}]}
    r = call(p)
    if r["status"] != 200:
        return {"probe": "truncation", "ok": False, "why": r["status"]}
    ch = json.loads(r["raw"])["choices"][0]
    return {"probe": "truncation", "ok": True,
            "finish_reason": ch.get("finish_reason"),
            "reports_usage": "usage" in json.loads(r["raw"])}

def probe_tools():
    p = {"model": MODEL, "max_tokens": 128,
         "messages": [{"role": "user", "content": "Read ./README.md and report its first line."}],
         "tools": [{"type": "function", "function": {
             "name": "read_file",
             "description": "Read a file from disk.",
             "parameters": {"type": "object",
                            "properties": {"path": {"type": "string"}},
                            "required": ["path"]}}}]}
    r = call(p)
    if r["status"] != 200:
        return {"probe": "tools", "ok": False, "why": r["status"]}
    msg = json.loads(r["raw"])["choices"][0]["message"]
    return {"probe": "tools", "ok": True,
            "returned_tool_calls": bool(msg.get("tool_calls"))}

def probe_json():
    p = {"model": MODEL, "max_tokens": 128,
         "messages": [{"role": "user",
                       "content": 'Reply with only JSON: {"status":"ok","count":3}'}]}
    r = call(p)
    if r["status"] != 200:
        return {"probe": "json", "ok": False, "why": r["status"]}
    text = json.loads(r["raw"])["choices"][0]["message"]["content"]
    try:
        obj = json.loads(text)
        return {"probe": "json", "ok": True,
                "keys_match": sorted(obj) == ["count", "status"]}
    except json.JSONDecodeError:
        return {"probe": "json", "ok": True, "keys_match": False,
                "note": "not parseable as JSON"}

def probe_concurrency(n=2):
    p = {"model": MODEL, "max_tokens": 16,
         "messages": [{"role": "user", "content": "Say ok."}]}
    with cf.ThreadPoolExecutor(max_workers=n) as ex:
        codes = list(ex.map(lambda _: call(p)["status"], range(n)))
    return {"probe": "concurrency", "ok": True, "statuses": codes,
            "saw_429": 429 in codes}

if __name__ == "__main__":
    for fn in (probe_truncation, probe_tools, probe_json, probe_concurrency):
        print(json.dumps(fn(), sort_keys=True))

Run it like this. Keep the key in the environment, never in the file.

export BASE_URL="$YOUR_ENDPOINT"
export API_KEY="$YOUR_KEY"
export MODEL="$YOUR_MODEL"
python3 probe_capabilities.py

Reading the output: a decision table

Probe result	What it means	What I do next
returned_tool_calls: false	Tools accepted, not acted on	Drop the agent branch, or change endpoint
finish_reason: "length" on an 8-token cap	Normal, expected	Keep the check; raise budget in real calls
finish_reason: null in streaming	Client is not reading the final event	Fix the parser before anything else
reports_usage: false	No token accounting from server	Count tokens client-side, or accept blind cost
keys_match: false	Valid JSON, wrong shape	Add schema validation at the boundary
saw_429: true at n=2	Very low concurrency ceiling	Queue work; never fan out blindly

Nothing here measures quality. Nothing here measures speed. Those are different experiments with different controls.

Where I run this

I run probes from a throwaway box, not my laptop. Same reason I do not test migrations on my main database. The probe should not touch anything I care about.

MonkeyCode's free model access and free server option are the two pieces I use for that. The free model access gives the probe something to talk to. The free server option gives it a disposable place to run. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I am not claiming the free tier is fast, generous, or permanent. I do not know its limits, and I am not going to guess at them. Run the concurrency probe and read your own numbers. That is the whole point of this post.

Limitations and who should skip this

• This is a smoke test. It is not a benchmark and not a load test.
• Results describe one endpoint, one model, one moment. They expire.
• Paid tiers can behave differently from free ones. Re-probe after switching.
• Only probe endpoints you own or are authorized to test.
• Do not use this to grade answer quality. It checks contract shape only.

Skip this approach if you need contractual latency SLOs, deterministic output, or guaranteed tool-call behavior. Buy a plan with a written contract. A probe will not save you from a missing SLA.

The corrected mental model

Treat every free endpoint as a set of separate promises. Transport, truncation, tool calls, JSON shape, and concurrency are five different promises. A 200 only confirms the first one.

Which promise broke on your last integration? I keep a one-page probe log per endpoint now. It has saved me two weekends so far.