200 OK Is Not a Model Answer: Triage Silent Empty Responses from Free Endpoints

#ai #api #debugging #serverless

200 OK is a transport result, not a model output. A free model endpoint can answer the connection and still hand you an empty, stale, or truncated payload; run five checks before you let that response into your workflow.

The failure mode nobody logs

A 200 status only says the HTTP request reached a server and got a response. It does not prove any of these:

The body has usable bytes.
The model returned a non-empty string.
The JSON is complete and parseable.
The answer is fresh, not cached from a previous request.
finish_reason matches your expectation.

The most dangerous case is the silent one: choices[0].message.content is empty, but the request looks successful in logs, so a downstream parser fails later with no clear cause.

Start with a deterministic probe

Send the smallest request that should always produce text. Use a single-word response prompt, a low token budget, and a hard timeout. This probe assumes an OpenAI-compatible response shape; adapt the field path if your provider uses a different contract.

#!/usr/bin/env bash
set -euo pipefail

ENDPOINT="${1:?usage: probe.sh <endpoint>}"
PAYLOAD='{"model":"test-model","messages":[{"role":"user","content":"Reply with the single word OK."}],"max_tokens":64,"stream":false}'

tmp=$(mktemp)
code=$(curl -sS -o "$tmp" -w '%{http_code}' \
  -H 'content-type: application/json' \
  --max-time 30 \
  -d "$PAYLOAD" \
  "$ENDPOINT")

bytes=$(wc -c < "$tmp")
content=$(jq -r '.choices[0].message.content? // empty' "$tmp" 2>/dev/null || true)
finish=$(jq -r '.choices[0].finish_reason? // empty' "$tmp" 2>/dev/null || true)

if [ "$code" != "200" ]; then
  echo "TRANSPORT_FAIL code=$code"
elif [ "$bytes" -eq 0 ] || [ -z "$content" ]; then
  echo "EMPTY code=$code bytes=$bytes finish=$finish"
elif [ "$finish" = "length" ]; then
  echo "TRUNCATED_OR_STOP code=$code bytes=$bytes finish=$finish"
else
  echo "OK code=$code bytes=$bytes finish=$finish content=${content:0:40}"
fi

rm -f "$tmp"

The script returns four states: TRANSPORT_FAIL, EMPTY, TRUNCATED_OR_STOP, or OK. The point is not to fix the endpoint; it is to make the silent failure visible before it reaches your pipeline.

Decision table

Signal	Likely cause	Next action
`code != 200`	auth, quota, route	read the error body; retry once with the same nonce
`code=200`, `bytes=0`	proxy dropped the read	retry; compare total time
`content` empty	model returned stop without text	simplify the prompt or raise `max_tokens`
content exists but JSON invalid	truncation or mixed content type	set `stream=false`, inspect headers, record byte count
`finish_reason=length`	token budget cut the output	raise `max_tokens` or shorten the input
same content across nonce runs	cached result	add a nonce to the prompt; override cache if available

A free endpoint is a test surface, not a contract

MonkeyCode's free model access and free server option make this probe cheap to run. The free server is useful for smoke tests and disposable CI jobs, but it should be treated as an unmanaged test surface: expect cold starts, shared capacity, and occasional silent responses.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use the same rules when you point the probe at a free model or free server:

Keep the payload deterministic and secret-free.
Set a hard timeout, then stop; do not loop around a rate limit.
Save the raw response as a CI artifact before parsing.
Retry once with a nonce, not with the same exact prompt.
Compare content across warm and cold runs.

What the probe does not tell you

The check is about transport and response shape. It does not score output quality, detect hallucination, measure long-term stability, or guarantee privacy. Treat a passing probe as a precondition, not a result.

Repeatable smoke sequence

Use this order when you first touch a new free endpoint:

Send one-word request with a unique nonce.
Record code, bytes, finish_reason, and total time.
Wait 60 seconds and send the same prompt with a new nonce.
Run once with stream=false and once with stream=true.
Diff the returned content for each nonce.

If any step returns EMPTY or TRUNCATED_OR_STOP, do not feed the response into a larger workflow. Fix the request shape first.

Who should skip this

Do not use a free model or free server when you need strict availability, stable latency, batch throughput, private data handling, or a guaranteed response body. Free tiers are useful for evaluation, smoke tests, and debugging, not for production guarantees.

A 200 is the beginning of the check, not the end. Five small signals—status, bytes, content, finish reason, and freshness—turn an invisible failure into a repeatable triage step.

DEV Community