Why JSON Schema Field Order Breaks Structured Output Accuracy

Someone on your team reorders a Pydantic model so the API response reads better — label first, rationale last. No prompt change, no model change, no temperature change. The classification eval drops. The rationales still look great; they just now describe a decision the model had already made.

That is not a coincidence and it is not prompt superstition. With strict structured outputs, JSON Schema field order is a decoding constraint. The grammar compiler turns your property order into a finite-state machine, and that machine assigns probability zero to any token that would start the wrong key. The model cannot reorder. It cannot think first and answer second unless your schema lets it.

TL;DR

• JSON Schema field order determines generation order under constrained decoding (OpenAI strict structured outputs, vLLM guided_json, Outlines/XGrammar/llguidance). Objects are emitted in declaration order because tracking arbitrary property order needs 2^k FSM states for k required keys.
• Autoregressive models compute in the token stream. A field declared before the answer is scratchpad; a field declared after it is post-hoc rationalization that cannot influence the answer at all.
• Put evidence → rationale → label → confidence. Never label first. The only cost is time-to-first-useful-token, not accuracy.
• Forced key tokens are free prompt real estate: step_by_step_reasoning injects that phrase into the KV cache right before the model writes the value. Their logprobs are meaningless — don't compute confidence over forced spans.
• Design enum values to differ at the first token (urgent/normal/low, not p0/p1/p2) so the masked, renormalized logprobs at that position give you a usable class posterior.

Why does JSON Schema field order change structured output accuracy?

Because a decoder-only model has exactly one place to do intermediate computation: the tokens it has already emitted. There is no hidden scratch buffer that persists across steps. Everything the model "works out" has to exist as tokens in the prefix, or it does not exist.

So when the schema forces this:

{"label": "p0", "rationale": "The stack trace shows ..."}

the token for p0 is sampled from a context that contains the ticket, the system prompt, and the seven characters {"label":. That is it. Whatever the rationale says afterwards was conditioned on p0, not the other way around. You did not get reasoning. You got a language model doing what it is extremely good at — writing a persuasive defense of a position already on the page.

Flip the order and the same tokens become causal:

{"evidence": ["OOM at 03:12 on the payments pod"], "rationale": "...", "label": "p0"}

Now every token of evidence and rationale is in the prefix that conditions label. This is chain-of-thought, just wearing a JSON hat.

The part people miss: with an unconstrained model you might get away with a bad order, because a strong model will sometimes ignore your schema and emit keys in the order it prefers. Strict mode removes that escape hatch. The constraint is enforced at the logit level.

How does constrained decoding actually enforce the order?

The schema is compiled into a grammar, the grammar into a state machine over the token vocabulary. At each step the engine computes the set of tokens that keep the output on a valid path, masks everything else to -inf, and renormalizes:

# What every guided-decoding backend does, stripped down.
mask = fsm.allowed_token_mask(state)     # bool[vocab_size]
logits[~mask] = float("-inf")
probs = softmax(logits)                  # renormalized over the legal set only
tok = sample(probs)
state = fsm.advance(state, tok)

Consider the step right after {. The model's unconstrained top token might be "reason with high probability — it wants to think first. If your schema declared label first, that token is masked. The mass gets redistributed over legal tokens, and the model emits "label. You never see the preference; you only see the degraded downstream answer.

Why does declaration order win? Because permitting properties in any order is expensive. To allow k required properties in arbitrary order, the automaton must remember which subset has already been emitted — that is 2^k states, before you nest anything. Some engines support unordered objects; the hosted strict modes generally pin declaration order instead, and it is the sane default. Which means your schema file is a program, and property order is control flow.

Two more strict-mode consequences worth knowing:

• OpenAI's strict mode requires every property in required and additionalProperties: false. If a field has no evidence in the input, the model must still emit something — that is where placeholder hallucinations come from. Model genuine optionality as a nullable union ("type": ["string", "null"]), not by omitting the key.
• The grammar is compiled and cached per schema. The first request with a brand-new schema pays extra latency. Generating schemas dynamically per request throws that cache away every time.

What does the fix look like in code?

Same fields, same descriptions, same model. Only the order changes.

from typing import Literal
from pydantic import BaseModel, Field
from openai import OpenAI

client = OpenAI()

# WRONG — the decoder commits to `label` before writing one token of analysis.
class TriageBad(BaseModel):
    label: Literal["urgent", "normal", "low"]
    confidence: float
    rationale: str

# RIGHT — evidence and rationale are in the prefix that conditions `label`.
class TriageGood(BaseModel):
    evidence: list[str] = Field(
        ..., description="Verbatim quotes from the ticket. No paraphrase."
    )
    ruled_out: str = Field(
        ..., description="Which severity you considered and rejected, and why."
    )
    label: Literal["urgent", "normal", "low"]
    confidence: float

resp = client.responses.parse(
    model="gpt-5.1",
    input=[{"role": "user", "content": ticket_text}],
    text_format=TriageGood,
)
out = resp.output_parsed

Note ruled_out. Under strict mode the model must fill it, so you have effectively forced a contrastive step into the decode path. You can attach compute to a task by declaring a field, and remove it by deleting one. That is a stronger lever than most prompt edits.

The self-hosted equivalent is the same idea through vLLM:

llm.generate(
    prompt,
    sampling_params=SamplingParams(
        max_tokens=512,
        guided_decoding=GuidedDecodingParams(json=TriageGood.model_json_schema()),
    ),
)

model_json_schema() preserves Pydantic field order, and XGrammar walks the properties map in that order. Reorder the class, reorder the decode.

Does the same thing happen with Claude tool use?

Yes, with a softer mechanism and the same fix. Claude Sonnet 4.5 and Opus 4.x tool use is schema-guided rather than compiled to a hard token-level grammar the way OpenAI strict mode is, so Claude has more freedom to deviate. But your input_schema is serialized into the request in property order, and the model overwhelmingly writes keys in the order it read them. Answer-first schemas produce answer-first generation, which produces post-hoc rationales.

tools = [{
    "name": "triage_ticket",
    "description": "Record a triage decision for one support ticket.",
    "input_schema": {
        "type": "object",
        # Order is the plan. Cheap observations first, commitments last.
        "properties": {
            "evidence":   {"type": "array", "items": {"type": "string"}},
            "ruled_out":  {"type": "string"},
            "label":      {"type": "string", "enum": ["urgent", "normal", "low"]},
            "confidence": {"type": "number"},
        },
        "required": ["evidence", "ruled_out", "label", "confidence"],
    },
}]

With Claude there is a better option available: turn on extended thinking and let the reasoning happen in thinking blocks before the tool call, then keep the tool input lean. That gives you a real scratchpad with no schema contortions, and the tool result stays clean for downstream consumers. Use in-schema reasoning fields when you are on a non-thinking configuration or when you need the reasoning persisted as structured data.

Why are forced key tokens worth designing?

Because they are prompt tokens you get for free, positioned exactly where they matter. The model does not choose "rationale" — the FSM does — but those tokens still enter the KV cache and still condition everything after them. Renaming a field from notes to contradicting_evidence_from_the_ticket inserts that phrase directly before the value is generated. It is the highest-leverage prompt edit that costs nothing at the call site.

The flip side: never treat logprobs over forced spans as signal. The key tokens, the colons, the braces, the closing quotes are all masked down to a single legal option, so their logprob is ~0 by construction. Averaging token logprobs across a structured output to get "confidence" mostly measures how much punctuation your schema has.

Where logprobs are meaningful is the first token that distinguishes one legal continuation from another. For a Literal["urgent","normal","low"] enum, the FSM has already forced the opening quote, so the next token's renormalized distribution is a genuine posterior over your classes. That only works if the values diverge at the first token. p0/p1/p2 all start with the same p token, so the informative step is buried one position deeper and may split unevenly across the vocabulary. Pick lexically distinct enum values and you get a calibratable classifier out of a generative call.

Also drop the self-reported confidence: float. A model writing 0.9 after its own label is producing a plausible-looking number, not a probability. The enum logprobs are the real thing.

How do you measure this in your own stack?

Change one variable. Duplicate the schema, permute only the property order, hold everything else — same model snapshot, same prompt, same temperature, same seed if available — and run both over the same N examples.

Then analyze it as a paired experiment, not two independent accuracy numbers. Look only at the examples where the two orders disagree, and test whether the disagreements are lopsided (McNemar's test on the discordant pairs). Paired analysis needs far fewer examples to reach significance than comparing two marginal accuracies, which matters because per-call eval noise on a few hundred items is large enough to swallow a real effect.

The cost of reasoning-first is real but narrow: you cannot stream the answer early, and you pay for the extra tokens. Bound the scratchpad fields (maxItems on evidence, an explicit "one sentence" in the description) and the tax stays small. Latency to the field you care about is the trade — accuracy is not.

The short answer

JSON Schema field order breaks structured output accuracy because constrained decoding compiles your property order into a state machine that emits keys in declaration order and masks every other token to zero probability. A decoder-only model can only compute in tokens it has already written, so any field declared before the answer becomes usable scratchpad and any field declared after it is a post-hoc justification with zero causal effect on the answer. Declare evidence and rationale first, the label and any derived numbers last, model optional fields as nullable unions rather than omissions, name fields so the forced key tokens act as instructions, and read confidence from the enum's first divergent token instead of asking the model for a number. Reordering four lines of a Pydantic model is the cheapest accuracy fix in a structured-output pipeline.