Why JSON Schema Isn't Enough for Production AI

Your LLM Can Return Perfect JSON and Still Be Completely Wrong

Most developers (including me) assumed one thing when Structured Outputs became available:

If my JSON validates against the schema, my extraction is correct.

Unfortunately, that's not true.

After spending the last few months building ShapeCraft, a structured output library for Node.js that works across OpenAI, Claude, Ollama, and Groq, I realized something important:

JSON Schema guarantees structure—not truth.

Once you understand that distinction, you'll design much more reliable AI systems.

---

Let's look at an example

Suppose you're extracting data from an invoice.

The invoice says:

Invoice Total: $900

Your LLM returns:

{
  "invoiceTotal": 1200
}

Now let's validate it.

• Valid JSON? Yes
• Matches JSON Schema? Yes
• Correct data type? Yes
• Required field present? Yes
• Correct answer? No

The response is structurally perfect but factually wrong.

That's the difference between structural correctness and semantic correctness.

---

JSON Schema only solves one problem

JSON Schema is excellent at validating structure.

It ensures things like:

• Valid JSON
• Required fields
• Correct data types
• Arrays and objects
• Enum values
• Consistent response shape

That's incredibly useful.

But JSON Schema cannot answer questions like:

• Did the model extract the correct value?
• Is this value actually present in the source document?
• Did the model hallucinate?
• Can I trust this extraction?

Those are semantic questions.

---

Structural Validation vs Semantic Validation

Structural Validation	Semantic Validation
Valid JSON	Correct value
Required fields	Grounded in source
Correct data types	No hallucination
Schema compliance	Evidence exists
Safe to deserialize	Safe to trust

Most AI SDKs solve the first column.

Production AI requires both.

---

Not all structured output works the same way

While building ShapeCraft, I discovered something else that surprised me.

Every provider advertises structured output, but they don't enforce it the same way.

Provider	Implementation	Guarantee
OpenAI	Server-side schema enforcement	Native
Groq	Server-side schema enforcement	Native
Ollama	Grammar-constrained decoding (GBNF)	Constrained
Anthropic	Prompt + parse + retry	Best-effort

They all produce structured output.

But they provide different guarantees.

That's an important distinction that's often hidden behind a single API.

---

Structure ≠ Truth

Even with the strongest schema enforcement available today, a model can still return:

{
  "customerName": "John Smith",
  "invoiceTotal": 1200,
  "invoiceDate": "2026-07-01"
}

The JSON is valid.

The schema passes.

The data types are correct.

The answer can still be wrong.

No JSON Schema validator can detect that.

---

What production AI actually needs

I now think of AI validation as two independent layers.

Layer 1 - Structural Validation

Questions like:

• Is this valid JSON?
• Does it match my schema?
• Can my application deserialize it safely?

This is where JSON Schema shines.

Layer 2 - Semantic Validation

Questions like:

• Is the extracted value correct?
• Can this answer be traced back to the original source?
• Does evidence exist?
• Should I trust this response?

This layer is significantly harder.

And it's where I believe the next generation of AI tooling needs to evolve.

---

Building ShapeCraft

While exploring these differences, I built ShapeCraft.

ShapeCraft is an open-source TypeScript library that provides a unified structured output API across multiple providers while exposing the actual guarantee level instead of pretending every backend behaves the same.

import { generate, openai } from "@aviasole/shapecraft";

const result = await generate(
  openai({ model: "gpt-4o-mini" }),
  schema,
  prompt
);

console.log(result.guaranteeLevel);

// "native"
// "constrained"
// "best-effort"

Current providers:

• OpenAI
• Groq
• Ollama (GBNF constrained decoding)
• Anthropic

Supported schema types:

• Zod
• JSON Schema
• XML
• Regular Expressions
• Custom Validators

Streaming, retries, and multi-turn structured conversations are also supported through the same API.

GitHub:

https://github.com/aviasoletechnologies/shapecraft

npm:

npm install @aviasole/shapecraft

---

Where I think AI tooling is heading

Structured output solved one of the biggest problems in LLM development.

Before it, we spent countless hours:

• Fixing malformed JSON
• Cleaning invalid responses
• Writing regex parsers
• Handling missing fields

That problem is largely solved.

The next challenge isn't better JSON.

It's better confidence.

Imagine every response also including:

• Evidence for every extracted field
• Field-level confidence scores
• Source attribution
• Semantic verification
• Reliability metrics

That's where I believe production AI is heading.

---

Final thoughts

Structured output is one of the biggest improvements in modern LLM APIs.

But it's important to understand what it actually guarantees.

It guarantees structure.

It does not guarantee correctness.

Once I separated those two ideas, it completely changed how I think about designing reliable AI systems.

How are you validating AI output in production today?

• Schema validation only?
• Human review?
• Semantic verification?
• Something else?

I'd love to hear how others are solving this problem.