The 200 Came Back Empty and the Job Still Ran

An empty string is a valid answer. That is why nobody noticed.

A batch job returned 200 OK for nine nights. It also returned zero summaries. Every check I had measured the transport, not the result.

Why did green mean nothing? Because green only ever meant "the socket opened."

The failure, rebuilt

I am not going to pretend this is a clean production postmortem. It is a harness. I rebuilt the pattern in about forty lines so you can watch it fail on your own laptop.

The pattern is boring and common. An upstream hits a throttle window and answers 200 with an empty choices array. Your client returns "". Downstream code treats "" as data.

Is that the API's fault? Partly. Mostly it is mine. I asserted on the wrong layer.

Step 1 — Reproduce it without spending a token

Save this as repro_empty_200.py. Run python repro_empty_200.py. It fakes a throttle window on hits two and three.

# repro_empty_200.py
import json
import threading
import time
from http.server import BaseHTTPRequestHandler, HTTPServer


class Flaky(BaseHTTPRequestHandler):
    hits = 0

    def do_POST(self):
        Flaky.hits += 1
        if Flaky.hits in (2, 3):
            body = {"choices": []}          # 200, but no content
        else:
            body = {"choices": [{"message": {"content": "ok"}}]}
        raw = json.dumps(body).encode("utf-8")
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(raw)))
        self.end_headers()
        self.wfile.write(raw)

    def log_message(self, *args):
        pass


def serve():
    HTTPServer(("127.0.0.1", 8787), Flaky).serve_forever()


if __name__ == "__main__":
    threading.Thread(target=serve, daemon=True).start()
    time.sleep(0.2)
    print("fake endpoint on http://127.0.0.1:8787/v1/chat")
    time.sleep(600)

No key. No quota. No network. That matters, and I will come back to it.

Step 2 — Write the client that fails

Here is the version I shipped. Read the return line twice.

# naive_client.py
import json
import urllib.request

ENDPOINT = "http://127.0.0.1:8787/v1/chat"


def summarize(text: str) -> str:
    req = urllib.request.Request(
        ENDPOINT,
        data=json.dumps({"input": text}).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    with urllib.request.urlopen(req, timeout=30) as resp:
        assert resp.status == 200        # the whole bug, in one line
        payload = json.load(resp)
    choices = payload.get("choices") or []
    return choices[0]["message"]["content"] if choices else ""

That else "" is the crime. It turns a protocol failure into a value.

Step 3 — Watch it pass

Run the fake server, then call summarize five times. You get ok, '', '', ok, ok. Exit code zero. No traceback.

Is that a crash? No. That is worse. A crash gets paged. An empty string gets stored.

Step 4 — Name the contract before you fix it

Write the promise down. Mine has three clauses.

1. A completion must be non-empty after .strip().
2. Transport errors and empty payloads are the same failure class to the caller.
3. Every retry needs backoff plus jitter, and a hard attempt cap.

If you cannot state those three sentences, you cannot test them.

Step 5 — Enforce the contract in the client

Same shape, different honesty.

# safe_client.py
import json
import random
import time
import urllib.error
import urllib.request

ENDPOINT = "http://127.0.0.1:8787/v1/chat"   # swap for a real endpoint


class EmptyCompletion(RuntimeError):
    """Upstream answered, but not with content."""


def _post(text: str):
    req = urllib.request.Request(
        ENDPOINT,
        data=json.dumps({"input": text}).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    try:
        with urllib.request.urlopen(req, timeout=30) as resp:
            return resp.status, json.load(resp)
    except urllib.error.URLError as exc:
        return 0, {"error": str(exc)}


def summarize(text: str, max_attempts: int = 4) -> str:
    for attempt in range(max_attempts):
        status, payload = _post(text)
        if status == 200:
            choices = payload.get("choices") or []
            content = ""
            if choices:
                content = (choices[0].get("message", {}).get("content") or "").strip()
            if content:
                return content
            reason = "empty_completion"
        else:
            reason = f"http_{status}"

        if attempt == max_attempts - 1:
            raise EmptyCompletion(f"{reason} after {max_attempts} attempts")

        time.sleep(2 ** attempt + random.random())

    raise EmptyCompletion("unreachable")

The jitter is not decoration. Without it, four workers retry in lockstep and rebuild the throttle window you just left.

Step 6 — Enforce it downstream too

Do not trust the client. Test the pipeline output as a contract.

# test_pipeline_contract.py — run: python -m pytest -q
from pipeline import run_pipeline


def test_every_row_has_text():
    out = run_pipeline(["row-a", "row-b", "row-c"])
    assert len(out) == 3, "a row went missing"
    empty = [k for k, v in out.items() if not isinstance(v, str) or not v.strip()]
    assert not empty, f"empty summaries for {empty}"

Then emit one counter, empty_completion_total, and alert when it is greater than zero. A number that should never be positive is the cheapest alarm you will ever install.

Why free access changes the design

Throttling and soft failures are normal states on any shared endpoint. On free access they are the default weather, not the exception. That does not make free tiers bad. It makes contract checks mandatory.

This is where MonkeyCode actually showed up in my workflow. The project offers free model access and a free server option, so I could point the harness at a real endpoint instead of my fake one and exercise the retry path without watching a bill.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Check the project page for the current model list and limits. Quota and capacity numbers move, and stale numbers make bad decision tables.

Decision table

Situation	Free model access + free server	Why
Learning the API, building a harness	Good fit	Failed calls cost you time, not money
Prototype with tolerant latency	Good fit	Retries and backoff absorb the jitter
Nightly batch that must not skip rows	Use with Step 6 checks	Contract tests catch silent empties
Hard p99 latency target	Poor fit	Shared capacity is not a guarantee
Regulated data with retention rules	Verify first	Read the current data policy yourself
Anything where an empty string ships to a customer	Poor fit	Fix the contract before the tier

Who should not use this approach

Skip the free-server route if you need a written SLA. Skip it if your data cannot leave your network. Skip it if you cannot spare an hour to write the contract test.

And do not adopt any tier, free or paid, based on a blog post. Read the project's own current numbers.

The reusable part

Here is what I keep from this failure. Symptom first, reproduce offline second, fix third. Never the other order.

Ask three questions every time a job goes green on empty output.

1. What did my assertion actually measure?
2. Can I rebuild this failure with zero dependencies?
3. Which counter would have caught it on night one?

My answer to the third question is now a single line. empty_completion_total > 0 pages me. If you want to run the same harness against a real endpoint instead of my forty-line fake, the project's free model access and free server option are a reasonable place to start. Bring the checklist.