Originally published on AI Tech Connect.
What you need to know There are four levels of reliability. Plain text plus a parser, JSON mode, function/tool calling, and native structured output with constrained decoding. Each is more reliable and slightly more constrained than the last. Constrained decoding makes the structure a guarantee, not a hope. Your schema is compiled to a grammar or finite-state machine, and at every step any token that would break the grammar is masked before sampling. The output cannot be structurally invalid. Every major provider now supports it. OpenAI Structured Outputs (strict json_schema), Google Gemini (responseSchema), Anthropic (Structured Outputs beta plus strict tool use), and the open-source stack — vLLM guided decoding, Hugging Face TGI, llama.cpp GBNF, plus the Outlines and Instructor…
Top comments (0)