The Schema That Got Too Big To Compile

Episodes as data, not as prose

I generate short animated episodes for a children's channel, and the model does not write a screenplay. It emits a structured object: a list of beats, each with a narrator line, an optional character line, staging cues, camera framing, props, and a symbol on screen. A renderer interprets that object into video.

The reason for structured output here is not tidiness. A model asked to invent an action will invent an action that does not exist, and the failure is invisible until someone watches the finished file. If every action and prop is a Literal in the schema, an invented one cannot leave the parser. The schema is the vocabulary of the world.

Which means the schema grows every time the world does. That is by design, and it is exactly what walked me into the ceiling.

One field, and nothing generates

I drew ten new props, which widened one enum, and separately added a backdrop field so a beat could say where it happens. The next call came back:

400 invalid_request_error: The compiled grammar is too large, which
would cause performance issues. Simplify your tool schemas or reduce
the number of strict tools.

The important part is what "the next call" means. It was not the long-format episode that broke, the one with fifty beats. It was every call. A thirty-second short with nine beats failed identically, because the limit is on the compiled schema, not on the output. Nothing in the system could produce an episode at all.

This was the second time in three days. The first was a one-line field I had added to record which structural template an episode used, and the symptom then was exactly as total. Both times the overnight run failed into a container log and nobody found out until morning.

Where the space actually goes

There is no published number for the ceiling, so I measured against my own schema by serialising it and printing the size of each definition:

Beat      1551 chars, 13 fields
    backdrop   164
    tone       163
    drift      132
    props      121
PropCue   1212 chars, 10 fields
    kind       296
    at         121
CharCue    654 chars,  4 fields
    act        246

Working: about 4,680 characters of JSON schema. Failing: about 4,850. Those are my numbers on one provider and should not be treated as a universal constant, but the shape of the cost generalises.

Two things dominate, and only one of them is obvious.

Enums cost what they list. kind is 296 characters because it enumerates twenty-seven props. Drawing ten new ones was not free; it widened that enum in every prop, in every beat.

Optional enums cost roughly double. This is the part I had not thought about. In Pydantic, x: Literal["a","b"] | None = None produces an anyOf with two branches: the enum and a null type. The same field written as x: Literal["a","b"] = "a" produces just the enum. Same expressiveness for the model, meaningfully less grammar.

Defaults instead of nullables

I converted three optional enums to non-nullable fields with defaults: camera framing, camera drift, and the mood tag handed to the speech synthesiser.

- shot:  Literal["wide", "mid", "close"] | None = None
+ shot:  Literal["wide", "mid", "close"] = "wide"
- drift: Drift | None = None
+ drift: Drift = "hold"
- tone:  Tone | None = None
+ tone:  Tone = "calm"

That took the schema back under the ceiling and generation resumed, with the new field and the ten new props still in place. No capability was lost. The model simply always states the framing, the drift and the mood rather than sometimes leaving them out, which if anything makes the output easier to read.

Worth saying plainly: a field the model must not fill has no business in the schema at all. My first ceiling breach was a bookkeeping field recording which template an episode used, something the code stamps after validation. It became a private attribute and left the grammar entirely.

Assert the budget where you can see it

Twice in three days, with the same total symptom, is a pattern rather than an accident. Vocabulary grows; that is the whole point of this design. So the ceiling needs to announce itself at build time rather than at one in the morning:

SCHEMA_BUDGET_CHARS = 4820

_size = len(json.dumps(EpisodeDraft.model_json_schema()))
if _size > SCHEMA_BUDGET_CHARS:
    log.error(
        "Episode schema is %d chars against a budget of %d. The provider "
        "will refuse to compile the grammar and NO episode will be "
        "generated. Drop a field or turn an optional Literal into one "
        "with a default.",
        _size, SCHEMA_BUDGET_CHARS,
    )

A log line, not an exception, because crashing the API process over a schema measurement trades one outage for another. The message names the failure and the two fixes, so whoever hits it next does not have to rediscover the arithmetic.

If you use structured outputs

• The limit is on the schema, not the response. Crossing it fails your smallest call as surely as your largest, which makes it look like an outage rather than a size problem.
• Prefer a default over a nullable enum. It is the cheapest headroom available and it changes nothing about what the model can express.
• Keep bookkeeping fields out. Anything your code fills in after validation should not be in the grammar the model is compiled against.
• Measure before you widen a vocabulary. Adding ten items to an enum is a schema change in every place that enum appears, which in a nested list is more places than you think.
• Put the budget in the code. Discovering the ceiling from a 400 during an unattended run is the expensive way to learn it.

And the diagnostic habit that mattered most: after the long-format call failed, I tested a short one before assuming the long format was special. It failed too, which turned "the complicated case is broken" into "everything is broken" in one cheap call. Verifying the blast radius is usually faster than reasoning about it.

---

Originally published at neuragrowth.co. I run a one-person digital-products studio and write up what breaks in production.

If you write CLAUDE.md files, I keep a set of working templates here: neuragrowth.co/free/claude-md-templates.