200 OK, content: null — what actually breaks when you build on AI APIs

Timur — Wed, 29 Jul 2026 07:07:08 +0000

One morning our SEO copy generator started failing with AttributeError: 'NoneType' object has no attribute 'strip'. The API call had returned 200 OK with finish_reason: "stop". A completely healthy-looking response, except message.content was null.

Some context before we chase that null. We build Hitou, a small web product that writes and produces a personalized song about someone you love: an LLM writes lyrics from the story you tell, a music model sings them, and a few minutes later there's a track with a gift page. The happy path took a few weeks. Everything since has been learning, in production, how AI APIs fail.

This post is the field notes: the failure modes that actually reached (or almost reached) paying users, and the guards that stopped them from happening twice.

The pipeline in one paragraph

A wizard collects the story (who the song is about, the occasion, the inside jokes). An LLM turns that into structured lyrics — verses, chorus, style notes — as JSON. The lyrics go to Suno v5 through a provider API, which returns two rendered takes plus word-level timestamps. We cut a preview, build a karaoke-style highlight track from the timestamps, and serve the whole thing from a FastAPI app. Two music providers sit behind a switch: a primary and a fallback, so one vendor having a bad day doesn't stop orders.

Sounds simple. The interesting part is that every arrow in that pipeline is a third-party API that can fail in ways the status code will never tell you.

War story #1: 200 OK, `content: null`

Back to that null. Here's the thing about LLM routers (we use OpenRouter): one model slug is served by many independent hosts, and the router picks one per request. We eventually ran the same prompt across every host serving that model. Exactly one of them was broken. It returned the entire completion in the reasoning field and null in content, wrapped in a clean 200 with a normal finish_reason.

Three lessons that now live in our code:

Your exception guard is probably shaped wrong. We had except (KeyError, IndexError, TypeError) around the response parsing. None of those fire here: the key exists, the value is null. The failure only surfaced two calls later, as an AttributeError in unrelated-looking code.
Log the host, not just the model. The response body (and every streaming chunk) carries a provider field. Without logging it, "the model is flaky" and "one host out of 33 is broken" are indistinguishable — and the second one has a config-level fix.
Validate emptiness after stripping decoration. The same class of host bug can return a bare fenced json code block with nothing inside it. If you check if not content before stripping the fence, the garbage passes.

The fix was boring, which is the point: a per-request provider: {"ignore": [...]} list, driven by an env var, plus an account-level ignore list in the router's dashboard as the no-deploy emergency lever.

War story #2: the song titled "None"

A week later, the same bug shape came back one level deeper. This one cost real money.

The LLM response was now validated: non-null, non-empty after stripping code fences, parsed JSON, all the required sections in place. What our checks didn't cover was leaf types: a key can be present, the structure can look right, and the value can still be null. And our code did this:

lyrics = str(payload.get("lyrics", ""))

Looks defensive, right? It isn't. The default in .get() only applies when the key is missing. When the payload is {"lyrics": null}, the key exists, .get() returns None, and str(None) is the string "None" — eight characters, non-empty, sails straight past if not lyrics and past every or "fallback" downstream.

So we paid a music API to professionally sing the word "None". A related null in a word-timestamps payload put the literal word "None" into a paying customer's karaoke highlight. Nothing crashed. No exception handler in the codebase had anything to catch.

The correct idiom is one token different:

lyrics = str(payload.get("lyrics") or "")

After it bit us twice in one week, we stopped trusting review to catch it and wrote a test that walks the AST of the whole codebase:

def _offenders(tree: ast.AST, where: str) -> list[str]:
    """Flag every str(<x>.get(<key>, "<literal>")) call."""
    found = []
    for node in ast.walk(tree):
        if (isinstance(node, ast.Call)
                and isinstance(node.func, ast.Name)
                and node.func.id == "str"
                and len(node.args) == 1):
            inner = node.args[0]
            if (isinstance(inner, ast.Call)
                    and isinstance(inner.func, ast.Attribute)
                    and inner.func.attr == "get"
                    and len(inner.args) == 2
                    and isinstance(inner.args[1], ast.Constant)
                    and isinstance(inner.args[1].value, str)):
                found.append(f"{where}:{node.lineno}")
    return found

Why AST and not grep? Because grep missed a real offender: a multi-line call with indexing in the middle (str(messages[-1].get("content", ""))). The AST doesn't care how the call is formatted.

We deliberately gate only str(...). float(None) raises a loud TypeError that any error path catches; str(None) degrades silently and ships to a user. Gate the quiet failures — the loud ones catch themselves.

"Just put pydantic on the boundary" is a fair response, and strict response models would catch this too. In practice we have several third-party boundaries (an LLM router, two music providers, a transcription API), they evolve at different speeds, and not all of them have earned a full typed model yet. The AST gate is one 30-line test that covers every str(.get()) in the codebase — including the boundaries nobody has gotten around to modelling.

War story #3: word timestamps lie (in three specific ways)

Suno v5 returns word-level timestamps with the audio. That's what our karaoke highlight is built on. The contract looks clean: {word, startS, endS} per token, and the data mostly matches it. "Mostly" is the whole job:

Phantom first lines. The aligner sometimes emits the opening line twice: one copy with all word starts compressed into under 2 seconds (raced over the intro instrumental), then the real sung line right after. Detection: two adjacent identical lines where the first spans <2s. Drop the compressed one.
Squeezed lines. A sung line packed into 0.85 seconds — six words — with the stolen time dumped onto the next word, stretched to ~3s. The trick is detecting it relative to the track's own tempo: we compare each line's per-word pacing against the median inter-word step, and redistribute the line evenly across its real window.
Ends absorb silence. A word's endS can swallow an instrumental gap; we've measured 11 seconds. If your highlight follows word ends, it drifts badly. Follow the starts.

The umbrella policy above all three: no karaoke beats broken karaoke. The normalizer returns None on anything it can't trust — too few words, character-error-rate above threshold (healthy sung tracks measure ~0.20-0.25 on the vendor's CER metric; we reject above 0.4), non-monotonic timestamps, timestamps past the track duration. The page quietly renders without the feature. Graceful degradation is a product decision, not just an ops one.

War story #4: generation takes 3-9 minutes. Sometimes it takes 30.

Music generation is slow and occasionally very slow: same API, same payload, 25-30 minutes instead of the usual 3-9. Early on we treated the long tail as failure — time out, refund, re-create. That's exactly wrong, because re-creating a paid render doubles the cost while the original job is often still cooking.

What we run now:

Resume, never re-create. Every generation stores its provider task id. Anything that looks stuck is resumed — polled on the same task — not re-submitted. The provider switch resolves adapters by name, so an in-flight job always resumes on the provider that created it, even if we've flipped the primary since.
A reconciler as the safety net. A background loop sweeps orders that stopped moving (process restarts mid-poll, crashed workers, the lot) and re-attaches to their provider tasks. The deploy script waits on a health check that includes the reconciler's heartbeat, so "the app is up but the sweeper is dead" fails the deploy instead of failing a customer.
Asymmetric fallback. Primary healthy → new orders go primary, fall back on error. Primary manually switched off → the chain is the fallback only. A kill switch that still quietly routes to the "disabled" provider isn't a kill switch. Resumes bypass the chain entirely (see above).

What we'd tell our past selves

Status codes are decoration. Every incident above arrived wrapped in a 200. Validate the shape and content of what you got, at the boundary, before it enters your system.
str(x.get(k, "")) is a bug. Write str(x.get(k) or ""). If the codebase is bigger than one file, write the AST gate — it's 30 lines and it never gets tired.
Identify the host behind the model. Routed APIs mean your "flaky model" may be one broken machine with a config-level fix.
Prefer degradation over garbage. A missing feature block is fine. The word "None" sung in a birthday song is not.
Resume beats retry for anything long-running and paid. Store the task id; treat "stuck" as "re-attach", not "re-buy".

None of this is exotic engineering. It's the unglamorous layer between "the demo works" and "strangers pay for it" — which, if you're gluing LLMs and generative audio together in 2026, is where most of the actual work turns out to live.

If you're curious what all this plumbing produces: hitou.site — it turns a story about someone into a finished song in a few minutes.

DEV Community: Timur