Structured Outputs in GPT-4o: Strict JSON Schema End to End

#ai #machinelearning #llm #programming

Strict mode is not “JSON mode but better”. It constrains decoding so that only tokens permitted by your schema can be emitted, which is why the output validates without a retry loop — and why the schema itself has to fit inside a restricted subset of JSON Schema before OpenAI will accept it.

Where the schema goes

The schema is passed in response_format, wrapped in an object that also carries a name and the strict flag:

{
  "model": "gpt-4o-2024-08-06",
  "messages": [
    {"role": "system", "content": "Extract the event details."},
    {"role": "user", "content": "Standup moved to Thursday 09:30 in room B2, Ana and Kwame."}
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "event",
      "strict": true,
      "schema": { /* see below */ }
    }
  }
}

strict: true is the whole feature. With it absent or false, the schema is a hint the model tries to follow. With it present, OpenAI compiles the schema into a constraint on decoding, and the returned string is guaranteed to parse and to conform. Support arrived with the gpt-4o-2024-08-06 snapshot, which is a concrete reason to pin a snapshot — an earlier GPT-4o snapshot behind the same alias does not have it.

The schema you would normally write

Here is an ordinary, valid JSON Schema. It is the shape a developer writes without thinking about it, and OpenAI will reject every one of the things marked below:

{
  "type": "object",
  "properties": {
    "title":     { "type": "string", "minLength": 1 },
    "day":       { "type": "string", "enum": ["Mon","Tue","Wed","Thu","Fri"] },
    "start":     { "type": "string", "format": "time" },
    "room":      { "type": "string" },
    "attendees": { "type": "array", "items": { "type": "string" }, "minItems": 1 }
  },
  "required": ["title", "day", "start"]
}

Three fields are required and two are optional; two constraints use keywords (minLength, minItems) that express bounds; one uses format; and additionalProperties is unspecified, which in JSON Schema means “anything else is allowed”.

The same schema, strict

{
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "title":     { "type": "string" },
    "day":       { "type": "string", "enum": ["Mon","Tue","Wed","Thu","Fri"] },
    "start":     { "type": "string", "description": "24-hour HH:MM" },
    "room":      { "type": ["string", "null"] },
    "attendees": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["title", "day", "start", "room", "attendees"]
}

Every key is now in required, additionalProperties is explicitly false, optionality has moved into the type of room, and the bounds keywords are gone — one of them replaced by a description, which is the correct home for a constraint the grammar cannot express.

The five transformations

Every object needs additionalProperties: false. Not just the root — every nested object, including objects inside array items and inside $defs. Omitting it on one nested object is the single most common rejection, and the error names the path, so read the path rather than re-reading the root.
Every property must appear in required. All of them, at every level. This exists because the constraint is compiled into a decoding grammar, and a grammar that may or may not emit a key is a far harder object than one that always does.
Optional becomes nullable. Since you cannot omit a key, express “no value” as a permitted null: "type": ["string", "null"]. The consequence reaches your application code — room is now always present and sometimes null, so if (obj.room) is the check, not if (“room” in obj). If you generate types from the schema, they will be nullable rather than optional and your compiler will make you handle it, which is the outcome you want.
Unsupported keywords must go. The supported subset covers the structural keywords — types, enum, anyOf, $ref and $defs, arrays and nested objects. Value-range keywords such as minLength, maxLength, pattern, minItems, maxItems, minimum and maximum have historically not been part of it. Move each one into a description, which the model reads, and validate it yourself after parsing. A constraint the grammar cannot enforce is a constraint you still own.
Respect the structural limits. OpenAI documents caps on the number of object properties, on nesting depth, and on the total size of enum values in one schema. A schema generated from a large database model will breach these. The fix is to model the output you need rather than mirroring your domain: an extraction schema with 200 fields is usually asking one call to do the work of five.

The exact supported-keyword list and the structural caps are the parts of this feature most likely to have expanded since this page was written; OpenAI has added to the subset over time. Check OpenAI’s Structured Outputs guide before removing a keyword you would rather keep. The five transformations themselves follow from how the feature works and are stable.

Two things that are supported and are worth knowing about, because they save the awkward workarounds people reach for. anyOf works anywhere except at the root, which is how you model a discriminated union of result types. And $defs with $ref works, including recursive references, so a tree or a nested comment thread is expressible directly.

Generating the schema instead of writing it

Hand-maintaining both a strict schema and the type your code parses it into guarantees they drift. The OpenAI SDKs take the type as the source of truth and emit a compliant schema from it — Pydantic models in Python, Zod schemas in TypeScript — which also gives you a parsed, typed object rather than a string:

import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";

const Event = z.object({
  title: z.string(),
  day: z.enum(["Mon", "Tue", "Wed", "Thu", "Fri"]),
  start: z.string().describe("24-hour HH:MM"),
  room: z.string().nullable(),
  attendees: z.array(z.string()),
});

const res = await client.beta.chat.completions.parse({
  model: "gpt-4o-2024-08-06",
  messages,
  response_format: zodResponseFormat(Event, "event"),
});

const event = res.choices[0].message.parsed;   // typed, or null on refusal

Note what the helper did with the schema: nullable() became the string-or-null union, additionalProperties: false was added everywhere, every key went into required, and .describe() became the description that carries the constraint the grammar cannot. It is the five transformations, automated. The same restrictions still apply underneath — a .min(1) on a string is not enforceable and will either be dropped or rejected, so the helper removes a class of mistakes rather than the restrictions themselves.

The same subset governs strict function calling, where the schema describes a tool’s arguments rather than the reply. If you have done this work for one, it transfers directly — strict mode on function definitions is the same compiler applied to a different field.

The two failures strict mode does not prevent

The guarantee is precise: valid JSON conforming to your schema, if the model produces a completion at all. Two ways it does not.

Truncation. If generation hits the token budget mid-object, you get finish_reason: “length” and a string that is a prefix of valid JSON, which parses as nothing. This is the failure that surprises people most, because the feature is sold as a guarantee. Check the finish reason before you parse, always — see what each finish_reason value means. Deep schemas with long string fields are the ones that hit it.

Refusal. The model can decline, and when it does the message carries a refusal string instead of content:

{
  "message": {
    "role": "assistant",
    "content": null,
    "refusal": "I can't help with that."
  },
  "finish_reason": "stop"
}

This field exists precisely because strict mode makes a refusal otherwise unrepresentable: a schema-conforming object has no slot for “no”. Handle it explicitly rather than treating a null content as an error, and note that the refusal comes from the model’s own training rather than from any moderation call you made — those are two separate systems.

const msg = res.choices[0].message;
if (res.choices[0].finish_reason === "length") throw new Error("truncated");
if (msg.refusal) return { refused: msg.refusal };
const event = JSON.parse(msg.content);   // safe: strict mode guarantees the shape

Those three lines in that order are the whole contract. Check the finish reason, check for a refusal, then parse without a try/catch and without a repair loop — because a repair loop that never fires is dead code that will one day mask a real problem, and strict mode is the reason it never fires. If you are migrating from JSON mode, deleting the retry-on-invalid-JSON path is the point at which the feature actually pays for the schema work.

A last note on scope. Strict mode constrains the shape of the output and nothing else. A field typed as a string will be a string; it will not necessarily be a correct string. An enum guarantees the value is one of your five options and says nothing about whether it is the right one. Schema conformance removes an entire class of parsing failures and moves the remaining problem, unchanged, to where it belongs: evaluating whether the extracted content is accurate.