Originally published at heycc.cn. This is a mirrored copy — the canonical version is kept up to date at the source.

LLM API Structured Outputs in 2026: JSON Schema, Strict Mode, and the Constrained-Decoding Divide

For years the standard way to get JSON out of a language model was to ask nicely, hope, and wrap a try/except around json.loads. In 2026 that era is over: every major provider ships a native structured-output mode that takes a JSON Schema and binds the response to it. But they do not bind it equally hard, and the field names, schema dialects, and guarantee semantics differ enough that "it works on OpenAI" tells you almost nothing about how it behaves on Gemini.

One rule survives all three platforms: parse and validate, never trust the string. Even the strictest mode can return a refusal instead of your object, and the weakest mode can hand you schema-shaped JSON whose values are wrong. If you are choosing a provider more broadly, see the companion piece comparing Claude, GPT, and Gemini APIs in 2026; what follows zooms into the one feature that quietly determines how much glue code you end up writing.

The core distinction: prompt-and-pray vs. constrained decoding

There are two fundamentally different ways a provider can "support" a schema.

A soft guarantee means the model is steered toward producing JSON that matches your schema — via fine-tuning and prompt injection — but the token sampler is still free to emit anything. You usually get valid JSON; occasionally you get a value that violates your intent. Output is syntactically correct, semantically your problem.

A hard guarantee means the schema is compiled into a grammar (or finite-state machine) that restricts the token sampler itself. At each step, tokens that would make the output diverge from the schema are masked out before sampling. The model literally cannot emit a token that breaks the structure — no missing required key, no invalid enum value, no trailing comma. This is constrained decoding, and it is the difference between "almost always valid" and "structurally cannot be invalid."

The catch worth internalizing: a hard structural guarantee is not a hard semantic guarantee. Constrained decoding forces a field typed integer to be an integer; it does not force that integer to be the right answer. Validation of business rules — ranges, cross-field consistency, referential integrity — is always yours.

Provider comparison

This table is the heart of the article — the field names you actually type, and whether each path hard-guarantees the schema.

Provider / mode	How you enable it (exact fields)	Hard-guarantees the schema?	Mechanism	Key constraint to remember
OpenAI Structured Outputs	response_format: { type: "json_schema", json_schema: { name, schema, strict: true } }	Yes	Constrained decoding	Every property must be in required; additionalProperties: false; optional = union with null
Anthropic JSON outputs	output_config: { format: { type: "json_schema", schema } }	Yes	Schema compiled to grammar, cached 24h	No recursive schemas, no minimum/maxLength; unsupported keywords stripped into descriptions
Anthropic strict tool use	"strict": true on a tool's input_schema	Yes (validates the tool call's input)	Same grammar/constrained sampling	Max 20 strict tools, 24 optional params, 16 union-typed params per request
Google Gemini	generationConfig: { responseMimeType: "application/json", responseSchema }	No — valid JSON shape, validate values	Decoder steered to schema	Output is "syntactically correct JSON"; docs explicitly say validate values app-side

What that table comes down to: OpenAI strict mode and Anthropic structured outputs both bind the schema at the token level. Gemini's responseSchema gives you reliable JSON of the right shape but stops short of a sampler-level guarantee — Google's own documentation tells you to validate values in your application.

OpenAI Structured Outputs

You enable it by passing a response_format whose type is json_schema, with a nested json_schema object carrying a name, your schema, and strict: true:

{
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "extraction",
      "strict": true,
      "schema": { "...": "..." }
    }
  }
}

When strict is true, the response is guaranteed to adhere to the supplied schema — no omitted required keys, no out-of-enum values. The price is a rigid schema dialect:

• additionalProperties: false is mandatory on every object.
• Every property must be listed in required. There is no "optional by omission."
• To express an optional field, give it a union with null (e.g. "type": ["string", "null"]) rather than leaving it out.
• Supported types are string, number, boolean, integer, object, array, null, and enum.

Two ergonomics that matter in production. First, refusals are a first-class field. A safety-based refusal lands in message.refusal, not in content, so you detect it with a field check instead of discovering a malformed object at parse time. Second, the SDKs do the schema plumbing for you: the Python SDK accepts a Pydantic BaseModel and the Node SDK accepts a Zod schema, auto-converting to JSON Schema, deserializing the response, and surfacing refusals as parsed objects.

Availability starts at gpt-4o-2024-08-06 and gpt-4o-mini, and extends through the GPT-5 family; OpenAI's current docs recommend gpt-5.6 for new projects.

Anthropic structured outputs

Anthropic offers two distinct patterns, both backed by the same grammar engine.

JSON outputs shape the model's final answer. You set output_config.format to { "type": "json_schema", "schema": ... }. Strict tool use instead guarantees that a tool call's input conforms: you add "strict": true to a tool definition, and the call's arguments are validated against that tool's input_schema. Use the first when you want a structured answer; use the second when you want a reliably callable function.

Under the hood, Anthropic compiles your JSON Schema into a grammar and constrains token generation, so the model cannot emit schema-violating tokens. The compiled grammar is cached for 24 hours, which means the first call pays a compilation latency cost and subsequent calls with the same schema are fast. The cache is keyed on schema structure and the tool set — changing a name or description does not invalidate it.

The supported subset is broad but has sharp edges: basic types, enum, const, anyOf/allOf, $ref/definitions (no external refs), a rich set of string formats (date-time, date, time, email, uri, uuid, ipv4, ipv6), and additionalProperties: false. Not supported: recursive schemas, numeric minimum/maximum, and string minLength/maxLength. The SDKs handle this gracefully — they strip unsupported keywords and move them into field descriptions so the model still sees the intent even though the grammar cannot enforce it.

A timeline note for anyone reading older guides: Anthropic launched structured outputs in public beta on 2025-11-14, originally gated behind the header anthropic-beta: structured-outputs-2025-11-13. As of 2026 the feature is generally available, the parameter lives at output_config.format, and the beta header is no longer required (it keeps working during a transition period).

What "the SDK converts your Pydantic model" actually does

All three vendors say some version of "pass a Pydantic model, we handle the rest," but the conversion path underneath differs — and it explains why a schema that validates fine under Pydantic can still get silently rewritten before it reaches the model:

# OpenAI (openai-python): client.responses.parse(text_format=MyModel)
# -> type_to_response_format_param() -> to_strict_json_schema()
#    (src/openai/lib/_pydantic.py): calls MyModel.model_json_schema(),
#    then mutates it via _ensure_strict_json_schema() into OpenAI's
#    strict subset (forces additionalProperties: false, required: [...]).

# Anthropic (anthropic-sdk-python): client.messages.parse(output_format=MyModel)
# -> TypeAdapter(MyModel).json_schema() -> transform_schema()
#    (src/anthropic/lib/_parse/_transform.py): a hand-written normalizer,
#    not Pydantic-generated output — it strips the keywords Anthropic's
#    grammar engine doesn't support (see the unsupported list above) and
#    relocates them into field descriptions.
#    Note: the plain client.messages.create() call has no native Pydantic
#    support — you build output_config yourself and pass it through
#    transform_schema() if you want the same normalization.

# Google (google-genai): response_schema=MyModel inside GenerateContentConfig
# -> t_schema() (src/google/genai/_transformers.py): calls
#    MyModel.model_json_schema() then its own process_schema() pass.
#    Known rough edge: nested BaseModels have had schema-generation bugs
#    (see google-genai issue #60) — flatten nested models if you hit
#    unexpected schema output.

The common pattern: every SDK calls Pydantic's own model_json_schema() first, then runs a provider-specific normalization pass. That second pass is where your minLength, recursive $ref, or bare-optional field either survives, gets silently relocated into a description, or gets rejected outright — which is why testing the actual emitted schema, not just your Pydantic model, catches problems the type checker won't.

Google Gemini

Gemini takes the lightest-touch approach. You set two fields inside generationConfig: responseMimeType to "application/json" and responseSchema to your schema. The Python SDK's config accepts response_mime_type and response_schema, and like the others it will take a Pydantic model or Zod schema directly.

Gemini's supported subset covers string, number, integer, boolean, object, array, and null, plus enum, format, properties, required, additionalProperties, and array constraints. On 2025-11-05 Google substantially expanded its dialect, adding anyOf, $ref (so recursive schemas work), minimum/maximum, additionalProperties, type: null, and prefixItems, and made Pydantic and Zod work out of the box. That same update made the API preserve the order of keys as written in the schema for Gemini 2.5 models and later.

Ordering is an easy way to get burned on Gemini. Because the model generates fields in sequence and earlier fields condition later ones, property order affects quality. Gemini exposes a propertyOrdering field inside responseSchema to force generation order; properties named there are generated first, in that order. The practical tip from the docs: make your prompt's few-shot examples match the schema's property ordering, or you can get malformed output.

The defining caveat: Gemini guarantees the JSON is syntactically valid and matches your schema's shape, but the docs are explicit that outputs can be schema-compliant yet semantically incorrect — so always validate values in your application.

Choosing a mode by what your schema needs

• You need the structure to be impossible to break (downstream code indexes fields without guards): use OpenAI strict: true or Anthropic structured outputs. Both constrain the sampler.
• You are building reliable function/tool calls: prefer Anthropic strict tool use or OpenAI strict mode over hand-validated tool arguments.
• Your schema is recursive or needs numeric/length bounds: Gemini's post-2025-11 dialect supports $ref recursion and minimum/maximum that Anthropic's grammar does not — but you trade away the hard sampler guarantee, so validate.
• You want minimal SDK ceremony: all three accept Pydantic/Zod; pick the one whose schema dialect already covers your constraints so the SDK does not silently strip them.
• Always, regardless of provider: deserialize into a typed model and run your business-rule validation. The schema controls structure; only your code controls correctness.

Tracing the field names back to the docs

Each provider's enabling fields and guarantee semantics come from that provider's own pages in Sources. The OpenAI response_format/strict: true behavior, the rigid dialect rules, and the gpt-5.5 recommendation are from OpenAI's Structured Outputs guide and launch post. Anthropic's output_config.format parameter, the strict-tool-use limits, the grammar-compilation-and-24h-cache mechanics, the supported subset and its exclusions (recursion, numeric/length bounds), and the beta-to-GA timeline (public beta 2025-11-14 under header structured-outputs-2025-11-13, now GA with no header required) are from Anthropic's structured-outputs docs and its advanced-tool-use writeup. Gemini's responseMimeType/responseSchema fields, the propertyOrdering behavior, the explicit "validate values app-side" guidance, and the 2025-11-05 dialect expansion are from Google's structured-output docs and its expanded-schema announcement. One genuinely moving target: the recommended model defaults — gpt-5.6 here — change faster than the field names do, so confirm the version before pinning it in production.

Sources

• OpenAI Structured Outputs guide
• OpenAI: Introducing Structured Outputs in the API
• Anthropic Structured Outputs documentation
• Anthropic Engineering: Advanced tool use
• Google Gemini API: Structured output
• Google: Expanded JSON Schema support in the Gemini API