Schema-Valid Isn't Correct: 4 Checks Before You Trust LLM Extraction

Finance posts an invoice. The JSON matched the schema. The PDF even shows the same total. Subtotal plus VAT still does not add up — the vendor transposed two digits, the model copied them faithfully, and nothing asked whether the numbers made sense.

The opposite failure is just as common: a perfectly typed dueDate that appears nowhere in the document, "helpfully" inferred from payment terms.

I treat structured output as the starting line, not the finish. The full pipeline (Microsoft.Extensions.AI GetResponseAsync<T>, grounding, bounded repair, review queue, tests, offline mode) is in Tech Skill Builder. This post is the short checklist: four checks that separate schema-valid JSON from data you can trust.

Check 1: Forgiving wire contract ≠ your domain model

Your domain wants DateOnly, non-null totals, and validated currency codes. The model needs room to say "this is not printed." Put the schema on a draft type; build the domain type only when validation is clean.

// Wire: nullable fields + Evidence quotes; dates as strings
public sealed record InvoiceDraft
{
    public DocumentKind DocumentKind { get; init; }
    public TextField? InvoiceNumber { get; init; }
    public TextField? IssueDate { get; init; }
    public AmountField? Total { get; init; }
    // … VendorName, DueDate, Currency, lines, subtotal, tax
}

public sealed record AmountField
{
    public decimal? Value { get; init; }
    public string? Evidence { get; init; } // verbatim quote from the document
}
// Domain Invoice is separate and only built from a draft with zero errors
// full implementation in the complete project

Ask the model to copy, never calculate. Strict JSON schema still does not enforce [Range] / regex the way you hope — with some adapters those keywords become description hints. Validate in code.

Call shape:

var response = await chatClient.GetResponseAsync<InvoiceDraft>(
    messages, serializerOptions, chatOptions, useJsonSchemaResponseFormat: true, ct);
// Prefer TryGetResult — Result throws on bad JSON
// full implementation in the complete project

Check 2: Handle every outcome that is not data

Schema adherence does not mean you got an invoice. Before business rules:

Outcome	Status
Refusal / content filter	Rejected
Truncated (FinishReason.Length)	Failed (do not blind-retry the same token budget)
Unparseable	one parse-repair turn, then Failed
NotAnInvoice	Rejected

Nothing throws for bad documents. Nothing bad posts silently.

Check 3: Ground every value, then classify repairable vs not

For each field: is it valid, and does it actually appear in the source?

var evidence = context.CheckEvidence(name, field.Evidence);
if (evidence == EvidenceState.Found
    && !EvidenceVerifier.AmountMatchesEvidence(value, field.Evidence!))
{
    context.Error(name, "value_not_in_evidence",
        $"{name} {value} does not appear in its evidence.", repairable: true);
}
// Dates: year + day + month name/number must show up in the quote
// full implementation in the complete project

Then business rules: required fields, ISO dates, currency allow list, qty × unit = line, lines → subtotal, subtotal + tax = total.

The important flag: if every number is verified as printed and the math still fails, the vendor is wrong. Mark that error not repairable. Asking the model to "fix" it only tempts it to invent a balancing total. Send it to a human.

Policy issues (PO required above a threshold, currency not allowed) are warnings → still NeedsReview, even when the extraction itself is clean.

Check 4: Bound repair, then route to review

When errors are repairable, keep the conversation and list the exact field failures. Cap attempts (for example 2). If targeted repairs do not fix it, stop paying for the same answer.

outcome = validator.Validate(draft, documentText);
if (!outcome.HasErrors)
    return Accepted(outcome);

if (!outcome.HasRepairableErrors || attempt == maxAttempts)
    return NeedsReview("validation_failed", outcome);

messages.AddRange(response.Messages);
messages.Add(CreateRepairMessage(outcome.RepairableErrors));
// End repair prompt with: if it is not printed, return null — do not guess
// full implementation in the complete project

Four explicit statuses cover the product:

Status	Meaning
Accepted	clean, grounded, postable
NeedsReview	repair exhausted, non-repairable math, or policy warning
Rejected	refusal, filter, not an invoice
Failed	truncation / unrecoverable parse

NeedsReview is success of the pipeline, not a crash. Reviewers can approve with corrections — and those corrections should go through the same validator (evidence checks off when a human is looking at the original).

Checklist before you trust an extraction

• [ ] Wire draft ≠ domain model; every money/date field carries Evidence
• [ ] TryGetResult + explicit paths for refusal / length / bad JSON
• [ ] Evidence substring + amount/date match against the document
• [ ] Repairable vs not (especially grounded math that still fails)
• [ ] Hard MaxRepairAttempts; sum token usage across turns
• [ ] Statuses: Accepted / NeedsReview / Rejected / Failed
• [ ] Fence the document; injection screen → review, not silent approve
• [ ] Schema snapshot / version in the cache key so contract changes invalidate old results

What this post skips

Schema strict-mode transforms, the review API with optimistic concurrency, document-hash caching, and the full test suite live in the complete project. Use this as the decision guide.

If you are extracting invoices, claims, or any record that hits a ledger, the ready-to-run package is in Tech Skill Builder: validate → ground → repair → review, with offline demos and tests. Limited-time membership pricing is on the product page. Grab it once, then spend your time on the domain rules — not on rediscovering that valid JSON can still be wrong.

Structured output fixed parsing. Trust still needs ordinary engineering: evidence, bounded repair, and a human for what a model should not decide.