You ask for JSON, a constrained decoder gives you JSON, and a validator says it is valid. That is the pitch: structured output turns a model into something you can plug into code.
Run it: https://dev48.infy.uk/prompt/day78-structured-output.html
Nothing here simulates language
There is no model and nothing is sampled. What is real: a JSON Schema validator written out rather than imported — so the page is not testing a library's opinion of "valid" — a coercion layer, and a corpus of 40 outputs that is data, each with the correct answer beside it.
The corpus is invented, hand-written, and the page says so above the first number.
Valid and correct are different sets
| schema-valid | 26 of 40 — 65.00% |
| actually true | 18 of 40 — 45.00% |
| valid and wrong | 8 |
| precision of "valid" | 69.23% |
Every one of those eight is a value a schema has no opinion about: a plausible email that belongs to somebody else, an invented resolution date, a ticket id in exactly the right format for a ticket that does not exist, a refund of £49.99 where the truth is £4.99.
A schema constrains shape. It cannot say this ticket was urgent.
The gentle repair layer is genuinely good news
Eight outputs fail for reasons a few lines of coercion handle — a number sent as a string, "High" instead of "high", slashes in a date, a currency symbol, one extra key.
All eight become valid, and all eight become correct. Not one is coerced into a valid-but-wrong answer. A coercion step applied only where it applies is nearly free, and that deserves saying before the next part.
The repair layer that never fails is a trade in both directions
Once a dashboard is watching the validity rate, the coercion grows: an unparseable email becomes unknown@example.com, a priority outside the enum becomes medium, a negative refund becomes its absolute value. Each rule is defensible alone.
| valid | true | valid and wrong | |
|---|---|---|---|
| as returned | 26 | 18 | 8 |
| + never-fail repair | 40 (100%) | 29 | 11 |
Validity hits 100%. Correctness genuinely improves, 18 → 29 — those rules are not useless. And the silently-wrong count goes up, from 8 to 11, because six loud failures became six quiet answers and only three happened to be right.
Both things are true at once. Reporting only the first is what makes it dangerous — and "100% schema-valid" is exactly the number that gets reported.
202 verifier asserts, 17 in-page checks, 0 failures.
Top comments (0)