- Book: AI That Answers
- The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
Three things get used as if they were the same feature. They are not.
They sit at different points in the request, they fail in different
ways, and exactly one of them produces a type your compiler knows
about.
Sorting them out takes one question per mechanism: what does this
guarantee, and to whom?
JSON mode: syntax, nothing else
JSON mode tells the provider to constrain generation so the output
parses as JSON. That is the entire promise.
// conceptually, across providers
const res = await call({ responseFormat: { type: "json_object" } });
const data = JSON.parse(res.text); // will not throw
JSON.parse will not throw. That is worth something — it removes the
failure where the model wraps its answer in prose or a fenced code
block and your parser dies on the backtick.
What it does not promise is your shape. All of these are valid JSON:
{"total": "1,240.00"}
{"Total": 1240}
{"result": {"total": 1240}}
{}
Every one parses. Every one breaks a consumer expecting
{ total: number }. JSON mode moved you from "might not be JSON" to
"is JSON of unknown shape," which is progress and is not a
guarantee about content.
In TypeScript terms the return type is still any. You have changed
the runtime failure mode without changing the type story at all.
Structured output: shape, at generation time
Provider structured-output modes go further. You supply a JSON Schema
and generation is constrained so the tokens produced conform to it.
Field present, type correct, enum value from the list.
const res = await call({
responseFormat: {
type: "json_schema",
schema: {
type: "object",
properties: {
total: { type: "number" },
status: { type: "string", enum: ["draft", "sent", "paid"] },
},
required: ["total", "status"],
additionalProperties: false,
},
},
});
This is a real guarantee and a stronger one than a prompt asking
nicely. The model cannot emit "PAID" if the enum says otherwise,
because that token path is not available to it.
Three limits worth holding onto.
It constrains shape, not truth. {"total": 0, "status": "paid"}
conforms perfectly and may be entirely wrong about the invoice.
Schema conformance is not correctness, and the confidence a
guaranteed shape produces is the main way this feature misleads
people.
Schema support is a subset. Providers implement part of JSON
Schema. Regex patterns, numeric bounds, tuple types, conditional
subschemas — support varies and changes. A constraint your schema
declares may be silently ignored rather than enforced. Check what
your provider currently supports rather than assuming your whole
schema is live.
It gives you nothing at compile time. The schema is a JSON object
you handed to an API. TypeScript does not know it exists. The
response is still any, and every property access downstream is
still unchecked.
Zod: shape, after the fact, with a type
Zod runs in your process, after the bytes arrive. It cannot influence
generation. What it can do is refuse to let unverified data into your
domain — and produce a static type while doing it.
const Invoice = z.object({
total: z.number().positive(),
status: z.enum(["draft", "sent", "paid"]),
});
type Invoice = z.infer<typeof Invoice>; // the part the others lack
z.infer is the difference in kind. The schema and the type are one
declaration, so they cannot drift. Add a field and every consumer
updates in the compiler. With a provider schema you maintain the JSON
Schema and a hand-written interface, and nothing tells you when they
diverge.
Zod also expresses constraints no provider schema will enforce for
you:
const Booking = z.object({
start: z.coerce.date(),
end: z.coerce.date(),
seats: z.number().int().min(1).max(8),
}).refine((b) => b.end > b.start, {
message: "end must be after start",
path: ["end"],
});
A cross-field rule like end > start is a domain invariant. No
generation-time constraint is going to enforce it, and it is exactly
the kind of thing a model gets wrong on an ambiguous input.
The cost is that failure happens after you have paid for the tokens.
Side by side
| JSON mode | Structured output | Zod | |
|---|---|---|---|
| Runs | at generation | at generation | in your process |
| Guarantees parseable JSON | yes | yes | no |
| Guarantees your field shape | no | mostly, subject to support | yes, or it throws |
| Cross-field rules | no | no | yes |
Coercion ("2026-08-06" → Date) |
no | no | yes |
| Gives a TypeScript type | no | no | yes |
| Failure costs tokens | — | — | yes, already spent |
The row that decides most architectures is the last one against the
second-to-last. Generation-time constraints are cheaper because they
prevent the bad output. Zod is stronger because it knows your domain
and your compiler.
Use both, in that order
They are not alternatives. Constrain generation to reduce how often
you get bad output, then parse at the boundary because the constraint
is partial and does not produce a type.
export async function extractInvoice(doc: string): Promise<Invoice> {
const res = await client.messages.create({
model: "claude-opus-5",
max_tokens: 1024,
tools: [{
name: "record_invoice",
description: "Record the extracted invoice.",
input_schema: zodToJsonSchema(Invoice),
}],
tool_choice: { type: "tool", name: "record_invoice" },
messages: [{ role: "user", content: prompt(doc) }],
});
const block = res.content.find((b) => b.type === "tool_use");
if (!block) throw new NoStructuredOutput(res.stop_reason);
return Invoice.parse(block.input);
}
Two details. Forcing a tool call with tool_choice is the structured
output mechanism on providers that expose it that way — the tool's
input_schema is the constraint, and the model must call it.
And zodToJsonSchema(Invoice) keeps one source of truth. The
generation constraint and the runtime check are generated from the
same Zod object, so they cannot disagree. Maintaining two schemas by
hand is how you end up constraining one shape and validating another.
Invoice.parse at the end is not redundant. It applies the
constraints the provider does not enforce, performs coercion, and —
the part that matters most — is what makes the return type
Promise<Invoice> rather than Promise<any>.
When one is enough
Only JSON mode, no schema: throwaway scripts and exploration where a
wrong shape costs you a re-run.
Only Zod, no generation constraint: when your provider or model does
not support constrained output, or when you are on a path where a
retry with the validation error is cheap and acceptable.
Only structured output, no Zod: hard to justify in TypeScript. You
gave up the type, which is most of why you are writing TypeScript.
The sentence to keep
JSON mode promises it will parse. Structured output promises the
shape, within the subset your provider implements. Zod promises your
domain rules and hands your compiler a type.
None of them promises the answer is right. That is a different tool
at a different layer, and confusing a guaranteed shape for a correct
value is the most expensive mistake of the three.
If this was useful
AI That Answers works through
structured output properly — generating provider schemas from Zod,
what each mode enforces, retry strategies when the parse fails, and
where shape guarantees stop being useful.
Checking that the answer is right — evals — is book five. The full
series is at
xgabriel.com/ai-in-typescript.
Top comments (0)