Your LLM's JSON passed the schema. It was still wrong.

Deterministic contract tests for structured LLM output on the JVM, and what 120 real documents taught me about where schemas stop.

If you put an LLM behind a feature that returns structured data, you have probably written this code:

var result = mapper.readValue(modelOutput, MyRecord.class);
schemaValidator.validate(result);
// ship it

The JSON parses. It matches the schema. The types bind. And the answer can still be wrong in ways none of that can see: a line dropped from a list while its total stayed the same, a number the model computed instead of copying, a value quoted from the wrong table, an apology tucked inside a text field.

I built outputgate, an MIT-licensed Java 21 library, to catch those. It treats a model's structured output the way you would treat any untrusted input: as something that has to satisfy a contract before your system acts on it. No gate calls a model. Every check is plain, deterministic code, so the same output always gets the same verdict, and you can run it in JUnit, in CI, or at runtime.

This post covers the idea, the API, and the benchmark that showed where a JSON Schema alone falls short.

The problem: valid shape, wrong content

A JSON Schema answers one question: is this the right shape? For structured LLM output that is necessary, but it is the easy half. The hard half is whether the content is consistent with itself and with the source it came from.

Take any record whose parts must add up and whose text must come from a document. Here the model dropped a line, kept the old subtotal, and apologised inside a field:

{"vendor": "Acme", "subtotal": 100.00,
 "line_items": [{"description": "Widgets", "amount": 40.00},
                {"description": "Sorry, I could not read this line", "amount": 34.50}]}

A schema that requires vendor, subtotal and line_items is perfectly happy with that. A contract is not:

Contract contract = Contract.builder()
        .gate(SchemaGate.fromString("""
                {"type": "object", "required": ["vendor", "subtotal", "line_items"]}"""))
        .gate(MathGate.builder()
                .sumEquals("line_items[*].amount", "subtotal")
                .tolerance(0.01)
                .build())
        .gate(RegexGate.builder()
                .forbid("line_items[*].description", "(?i)\\b(sorry|as an ai)\\b")
                .build())
        .build();

Verdict verdict = contract.evaluate(output);

REJECTED
[math] subtotal: sum of line_items[*].amount is 74.50 but subtotal is 100.00 (tolerance 0.01)
[regex] line_items[1].description: forbidden pattern /(?i)\b(sorry|as an ai)\b/ matched "Sorry"

Two things to notice. Every gate runs, so the verdict lists every violation, not just the first. And the output is either accepted or rejected with reasons, which is exactly what you need to decide whether to accept the answer, retry it, or send it to a person.

That README example is itself run as a test on every build, so the docs can't drift from the code.

The gates

A contract composes gates. Each one checks something you can write down as an invariant:

Gate	What it checks
SchemaGate	JSON Schema 2020-12. The schema itself is validated against the meta-schema when the gate is built.
TypeGate<T>	Strict binding to a Java type: unknown properties, null primitives, "2" for 2, floats for ints all fail.
MathGate	Cross-field arithmetic with an explicit tolerance: sums, products, ranges, custom rules, and rounding-aware sums for printed figures.
ProvenanceGate	Each required value carries evidence, {value, source_span: {page, text}}, and the quote must literally occur on the page it names.
RegexGate	Text rules per field: required patterns, forbidden patterns, ISO dates, ISO 4217 currency codes.
BudgetGate	Ceilings on cost, latency and output tokens for the call.

Before any gate runs, output that is not exactly one JSON value is rejected: prose around a code fence, trailing notes, duplicate keys and empty output included.

The gate that does the most work is ProvenanceGate. Asking the model to return a quote alongside each value turns "I hope this number is right" into "show me where you read it", and the check is just a string search over the source text. No embeddings, no second model.

Fail closed, everywhere

The most dangerous guardrail failure is a silent pass, because a wrong output then looks validated. So outputgate treats everything ambiguous as a rejection: a missing or null value, a gate that throws, a gate configured with no rules, grounding with no source document, unknown token usage when a cost ceiling is set.

That rule came from a real bug. Mutation testing (PIT) on a core with 95% line coverage showed that 23% of injected bugs went unnoticed. One of the real ones: a schema typo like "required": "vendor" (a string, not an array) was silently ignored by the validator, so SchemaGate passed every output. The fix was to validate each schema against the bundled 2020-12 meta-schema when the gate is built. The mutation score went from 77.7% to 95.4%. Coverage told me which lines ran; mutation testing told me whether the tests would notice a bug.

The benchmark: 120 real documents, 4 models

Claims about guardrails are cheap, so I measured. I needed a dataset that is public, messy and checkable, where totals must add up and the right answer is independently known. SEC 10-Q balance sheets fit: every filing ships official XBRL figures to compare against.

Four models (three hosted Gemini models and qwen2.5:14b running locally on Ollama) extracted 60 company balance sheets. The contract was then frozen and run on 60 different companies, so the second set is a genuine held-out test. Each accepted answer was compared with the filing's XBRL values.

Then each run got 50 corrupted copies of accepted answers, one seeded defect each: a dropped line, swapped digits, an invented quote, a wrong page, a bare value, a non-ISO date, an amount as text, a code fence, a trailing note and so on. Same answers scored three ways:

	Naive parse	JSON Schema only	Full contract
Seeded defects caught per run	4/50	20/50	50/50 in all 8 runs
Real answers accepted but wrong against the filing (of 452 with ground truth)	3	3	0 in-sample, 1 held out

To be clear about what that means: the defect kinds were written knowing what the gates check, so the first row shows that each gate fires on the error it targets, not how often models make that error. The second row is the one about real model behaviour.

The balance sheets are only the eval set. The same contract shape applies to anything with a structure and rules: extracted form fields, lab results, contract clauses, tool-call arguments, classification output with a justification.

Finding 1: the retry trap

This was the result that surprised me most.

A common pattern for structured output is validate, send the errors back, ask again. In the benchmark, one model's line items added up to 78,902 while the document printed a total of 70,402. The retry was told the sum didn't match.

It left the lines alone and changed the total to 78,902.

Schema: pass. Arithmetic: pass. The answer now agrees with itself and is wrong. outputgate rejected it because every value carries a quote, and 78,902 is not the number its own quote prints (numbersMatchQuotes()). That happened in 3 of the 12 retries, and the quote check caught all 3.

The takeaway generalises well beyond finance: a retry loop will game any validator that only checks the output against itself. Retrying is only safe when at least one check is tied to the source.

Finding 2: every quote was real, and the answer was still wrong

The one wrong answer that got through the frozen contract was Corteva's held-out filing. The page prints two near-identical balance sheets, the parent company's and then its subsidiary's. Same labels, similar numbers. The local model took one block from each.

Every quote was on the page. Every sum added up. The contract couldn't tell.

Proving a value is in the document is not the same as proving it came from the right part of the document. So I added regions:

static final Region FIRST_STATEMENT =
        Region.fromDocumentStart().through("(?i)^liabilities and equity\\b");

ProvenanceGate.builder()
        .require("period_end", "total_assets", "total_liabilities_and_equity" /* ... */)
        .groundedIgnoringWhitespace()
        .numbersMatchQuotes()
        .within(FIRST_STATEMENT, "period_end", "total_assets", "total_liabilities_and_equity")
        .build();

Re-scoring every recorded run offline, that check rejects Corteva and changes no other verdict, and the seeded defects are still 50 of 50. But I wrote it after seeing the failure, so it is a fix, not a held-out result, and the published held-out table still shows the miss. If you build guardrails, keep that distinction honest; it is the easiest number to inflate.

Any document that repeats its labels sets the same trap: a lab report with prior results next to current ones, a contract with an amendment, a form with a summary page.

Finding 3: most early failures were mine, not the model's

Before the balance sheets, I tried scanned public invoices. The first local run passed 1 of 20. It was tempting to blame the model. Classifying every rejection told a different story:

• 4 real model errors caught. It computed a 15% commission and invented a quote for the result, though the document only prints "15%".
• 1 document problem. A garbled PDF text layer.
• 14 problems on my side. The prompt never said an ID field was text, so it came back as a number. Quotes were stitched across table rows. Spacing differed from the PDF's extracted text.

And one infrastructure bug worth knowing about: Ollama's default context in that setup was 4,096 tokens. A ~12k-token prompt was silently cut to about 2,050 tokens, dropping the start, which is where the instructions were. No error. A model "ignoring instructions" may never have seen them. Set limits explicitly and log what was applied, not what was requested.

After fixing my side (and constraining generation with Ollama's structured-output format, while still judging every answer), the same model went from 1 to 13 of 20 without loosening a single gate.

Lesson: a low pass rate needs a root cause per violation (model, document, or contract) before it means anything.

PDF and OCR text without fuzzy matching

Grounding compares a quote with extracted text, and PDF or OCR text differs from what a model types in ways that carry no meaning: line breaks, ligatures (fi), thousands separators, curly quotes, a currency sign present or absent. The obvious fix, fuzzy matching, is also how a misread digit slips through.

So each tolerance is a named, opt-in fold that applies to the quote and the document alike: groundedIgnoringWhitespace(), ignoringCurrencySigns(), ignoringTypography(), joiningHyphenatedLines(), ignoringDigitGrouping(). None of them ever lets one letter or digit match another. 0 and O are still different characters. When OCR is too poor for faithful quotes to pass, fix the extraction rather than loosen the contract: a rejection costs a retry, an accepted wrong value costs more.

Using it

Add the core module and, optionally, the JUnit module and an adapter:

dependencies {
    implementation("io.github.ravismash:outputgate-core:0.1.0")
    testImplementation("io.github.ravismash:outputgate-junit:0.1.0")
    // implementation("io.github.ravismash:outputgate-spring-ai:0.1.0")
    // implementation("io.github.ravismash:outputgate-langchain4j:0.1.0")
    // implementation("io.github.ravismash:outputgate-ollama:0.1.0")
}

In tests, assert on recorded outputs and run golden sets (JSONL, one recorded answer per line):

LlmAssert.assertThat(output).passes(contract);
LlmAssert.assertThat(output).isRejectedBy(contract).byGate("math");

GoldenRunner.of(contract).withContext(context)
        .run("golden/extractions.jsonl")   // also writes a self-contained HTML report
        .assertPassRateAtLeast(1.0);

At runtime, the same contract decides what to do with a live answer:

LlmOutput output = client.complete(prompt);
Verdict verdict = contract.evaluate(output, EvalContext.empty().withSource(document));

A few design choices that matter in real builds:

• Core depends only on Jackson and slf4j-api. The JSON Schema validator is shaded and relocated, because Spring AI 1.x and 2.x pull newer, incompatible versions of it, and a guardrail that throws NoClassDefFoundError in your build is no guardrail.
• Adapters are tiny. The Spring AI and LangChain4j clients are one class each, under 60 lines, and CI replays a real recorded model answer through each framework's own client on several framework versions.
• Tokens aren't money, and unknown isn't zero. Cost comes from explicit per-million-token pricing. A provider that reports no usage means unknown cost, which fails a cost ceiling rather than passing as free.
• Contracts are thread-safe. Build one, share it across parallel tests or a server.

There's also a Python/Pydantic v2 port of the grounding and arithmetic checks in the repo; it gives the same verdict as the Java library on all 480 recorded benchmark answers.

What it doesn't do

outputgate checks what can be written as an invariant: shape, arithmetic, provenance, patterns, budgets. It is not an LLM-as-judge and doesn't score tone, relevance or toxicity. It's meant to sit next to a judge, handling the part that should never depend on another model's opinion. It also doesn't manage prompts or run retries for you; it tells you, deterministically, whether an answer can be trusted.

Takeaways

1. A schema checks shape, not truth. Add cross-field rules for anything that must add up.
2. Ask for evidence and verify it in code. A value plus a quote, checked literally against the source, catches invented and rewritten numbers.
3. Tie at least one check to the source before you retry. Otherwise the retry loop learns to satisfy the validator instead of the document.
4. "In the document" isn't "from the right place". Scope quotes when a document repeats its labels.
5. Fail closed and measure your tests with mutation testing. Silent passes are the worst failure a guardrail can have.
6. Classify every rejection before you blame the model. Mine were mostly prompt and pipeline bugs.

The code, every benchmark run, and a write-up of each failure are public:

• Repo: https://github.com/ravismash/outputgate
• Every failure, case by case: https://ravismash.github.io/outputgate/sec-10q-llm-extraction-failures.html
• Maven Central: io.github.ravismash:outputgate-core:0.1.0

If your model quoted the right label from the wrong table, would your checks notice? I'd like to hear how you validate structured output in production, and what broke.