The API Said 200 OK. The Model Returned Nothing. My Code Saved It Anyway.

Last week my nightly summarizer started writing empty strings into the database, and the worst part was that nothing in the logs looked wrong. Every request returned HTTP 200, every response parsed cleanly, and no exception ever fired. The model simply answered with an empty completion, and my code happily saved that silence as a finished summary. I spent a full evening blaming the server before I thought to inspect the payload I was actually sending.

The symptom that looked like success

The job pulled open GitHub issues from a queue, asked a model to summarize each one, and stored the result in a Postgres table. One morning the table contained rows where summary was an empty string, and the timestamps proved the job had run without a single failure. My first instinct was to check the usual suspects: network timeouts, rate limits, and malformed prompts. None of them had left a trace, because the API had answered every single call with a clean 200.

Here is the shape of the code that was quietly succeeding:

resp = httpx.post(
    API_URL,
    headers=HEADERS,
    json={
        "model": MODEL,
        "messages": [{"role": "user", "content": prompt}],
        "max_tokens": max_tokens,
    },
)

if resp.status_code == 200:
    data = resp.json()
    summary = data["choices"][0]["message"]["content"]
    save_summary(issue_id, summary)

The status check passed, the JSON parsed, and the empty string was written to the database. Nothing in that flow was designed to notice that content contained zero characters. From the application's perspective, the job had succeeded, which is exactly why the bug survived for so long.

The hypotheses I burned an hour on

• Maybe the free server was overloaded and silently dropped the work. The response times were stable, so that theory died quickly.
• Maybe the prompt exceeded the context window and got truncated into nothing. I logged prompt lengths, and they were all well within normal range.
• Maybe the model was having a bad night. I retried the same request and got the same empty result, which ruled out randomness.

Retrying actually made things worse, because every retry returned the same 200 with the same empty content, and my retry logic treated that as a successful completion. The response was not an error; it was a perfectly valid answer that happened to contain zero tokens. That is the moment I stopped guessing and started reproducing.

Reproducing with the smallest possible script

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I used MonkeyCode's free model access and free server option to build a minimal reproduction, because I did not want to burn paid credits on a bug that probably lived in my own code. The free server gave me a throwaway endpoint where I could hit the same prompt repeatedly until the behavior became obvious. The script that finally revealed the truth was almost embarrassingly small:

import httpx

def build_payload(prompt, overrides=None):
    defaults = {"model": MODEL, "max_tokens": 512, "temperature": 0.2}
    return {**defaults, **(overrides or {})}

payload = build_payload("Summarize this issue in one sentence.")
print("SENT PAYLOAD:", payload)

resp = httpx.post(API_URL, json=payload, headers=HEADERS)
print("STATUS:", resp.status_code)
print("BODY:", resp.json())

The output stopped me cold:

SENT PAYLOAD: {'model': '...', 'max_tokens': 0, 'temperature': 0.2}
STATUS: 200
BODY: {'choices': [{'message': {'content': ''}}]}

The server did exactly what I asked. I asked for zero tokens, and it delivered zero tokens with a smile.

The root cause was a merge I trusted too much

The payload builder merged defaults with user overrides, and one code path passed {"max_tokens": 0} because an environment variable had never been set. The merge did exactly what I told it to do: it replaced the sane default of 512 with a literal zero, and the API happily honored the instruction. Most providers treat max_tokens=0 as "return nothing," which is technically a valid completion, so the server had no reason to complain.

def build_payload(prompt, overrides=None):
    defaults = {"model": MODEL, "max_tokens": 512}
    return {**defaults, **(overrides or {})}

# somewhere else, the override was built like this:
overrides = {"max_tokens": int(os.getenv("MAX_TOKENS", "0"))}  # bug: default is 0

The environment variable was missing in production, the fallback was zero, and the merge happily accepted it. No validation layer existed between that number and the API call.

The fix, in three layers

I added a guard before the request, a check after the response, and a canary test to catch the class of bug early. Each layer catches the failure at a different stage, so a regression cannot sneak through silently again. Here is what each one looks like.

Validate before sending:

if payload["max_tokens"] <= 0:
    raise ValueError("max_tokens must be a positive integer")

Treat empty content as a failure instead of a result:

content = data["choices"][0]["message"]["content"]
if not content:
    raise EmptyCompletionError(f"empty completion for issue {issue_id}")

Add a canary prompt to the test suite:

def test_canary_prompt_returns_text():
    content = call_model("Say the word 'ok'.")
    assert content.strip(), "model returned an empty completion"

Reusable debugging techniques

1. Log the exact request payload, not just the status code. The status code told me the server was happy; the payload told me why it had nothing to say.
2. Treat "200 with empty content" as its own failure class. If your code only checks status codes, you are blind to a whole family of silent bugs.
3. Reproduce with a minimal script before you blame infrastructure. The smallest possible client stripped away every layer of my application and exposed the payload.
4. Add a canary prompt to your test suite. A known prompt with a known non-empty answer will catch this class of bug on the very first run.

Limitations and who should skip this

This approach assumes your provider returns 200 with an empty completion instead of a 400, and that assumption does not hold everywhere. Some APIs reject max_tokens=0 outright, in which case you would see a loud error and this whole story would be much shorter. Also, if your product legitimately allows empty model output, do not blindly raise on it; make the empty case explicit in your data model instead.

Debugging is mostly about asking the right question, and the right question here was not "why is the server failing" but "what exactly did I send." If you have a free tier or a free server option available, use it to reproduce silent failures before they reach production, because it costs nothing and it will show you the truth faster than any dashboard. What silent 200 have you chased lately?