The first four invoice formats fit the schema. The fifth is a credit note with negative lines, two tax rates, a freight charge that is not a line item, and a legal statement that changes who owes the tax. Nothing about that document is unusual. It was simply not in the sample you designed from.
The schema that broke on the fifth format
A first-pass invoice schema almost always looks like this: an invoice number, an invoice date, a supplier name, a currency, a total, and an array of line items each with a description, a quantity and an amount. It is a reasonable model of the invoices you looked at. Here is what the next few documents do to it, all of them ordinary.
- A credit note. The total is negative, the document references the invoice it credits, and the number in the “invoice number” position is a credit note number in a different series. A schema with one identifier field cannot say which it holds, and a validation rule that rejects negative totals now rejects a valid document.
- Two tax rates on one document. A single
tax_amountfield forces the extractor to sum them, which destroys the only information a tax reconciliation needs. - Charges that are not line items. Freight, insurance, a settlement discount, a rounding adjustment. Forcing them into the line-item array makes the line total disagree with the sum of quantities times prices; leaving them out makes the document not foot.
- Two currencies. Goods priced in one currency and freight in another, or a document that states an amount and its converted equivalent. A currency field at document level is now a lie.
- A reverse-charge or self-billed invoice. The tax is zero for a reason that is stated in prose, and the party in the “supplier” position may be the recipient. A schema with no field for the basis of the treatment loses the only thing that explains the zero.
Each of those is fixable in isolation. The point is that they were all knowable in advance, from the document type’s own rules, without having seen an example — which is what makes the standard advice to “iterate on real documents” insufficient rather than wrong.
Design from the rare variant
Invert the usual method. Instead of writing the schema that fits your sample and widening it when something breaks, enumerate the variants the document type is known to have and design for the widest one, then check that the common case is still expressible. The enumeration comes from the domain, not from the corpus: the standard, the form’s own instructions, the regulation that mandates the fields, the list of document subtypes the issuing system can produce.
Concretely, before writing a schema for a document type, answer four questions in writing. Which subtypes exist under this name and what does each add or remove? Which fields are mandated by an authority, and which are the issuer’s choice? Which fields can legitimately appear more than once? And which fields exist only in a particular jurisdiction or a particular revision of the form? A page written about a specific document type is exactly the place to answer those, which is why the document-specific pages in this wave are worth reading before the schema is fixed.
The economics favour this strongly. A schema change after launch is not a code change: it is a code change plus a decision about every document already extracted under the old shape, which is the subject of schema evolution. Adding a field you never populate costs almost nothing.
Assume everything repeats
The single highest-value default is to make anything that could plausibly occur twice an array from the beginning. Not because it usually does, but because widening a scalar to an array later changes the type of a field that consumers have already been reading, and widening an array from one element to two changes nothing.
The candidates are predictable: tax lines, addresses, purchase order references, bank accounts, contact people, dates that share a kind, and any party role that a second person can occupy. On a lease that means tenants; on a shipment it means containers; on a claim it means treatment lines. Where the repetition is about parties or roles rather than quantities, the naming problem it creates is handled on designing a schema for a multi-entity document, and the question of how deep the resulting structure should be is on nested versus flat extraction schemas.
The escape hatch has to be in the schema
This is the part that surprises people who have used a model in free JSON mode and then switched on a constrained decoding mode. In an unconstrained mode, a model that meets a field you did not anticipate will often just add it, and you can notice it in the output. Under a strict schema-constrained mode, it cannot: the decoder will only emit tokens that keep the output valid against the schema, and the usual requirement that objects declare no additional properties means the unanticipated field has nowhere to go. It is silently dropped, and the output looks perfectly clean.
So the ability to tell you about something unexpected has to be a declared field. Two of them, doing different jobs.
{
"type": "object",
"properties": {
"invoice_number": { "type": ["string", "null"] },
"other_charges": {
"type": "array",
"items": {
"type": "object",
"properties": {
"label": { "type": "string" },
"amount_minor":{ "type": "integer" },
"currency": { "type": "string" }
},
"required": ["label", "amount_minor", "currency"],
"additionalProperties": false
}
},
"unmapped_fields": {
"type": "array",
"items": {
"type": "object",
"properties": {
"label_verbatim": { "type": "string" },
"value_verbatim": { "type": "string" },
"page": { "type": "integer" }
},
"required": ["label_verbatim", "value_verbatim", "page"],
"additionalProperties": false
}
}
},
"required": ["invoice_number", "other_charges", "unmapped_fields"],
"additionalProperties": false
}
other_charges is a typed escape hatch for a known category with an unknown membership: you know charges exist, you do not know which ones. unmapped_fields is untyped and exists purely so the extraction can say “this document had a labelled value I was not asked about”. Mine it periodically; the labels that keep appearing are your next schema version, discovered from production rather than from a fresh sampling exercise.
Which JSON Schema keywords a provider’s strict mode accepts, and whether unspecified fields are dropped or rejected, is provider behaviour that has changed more than once. Check the current supported subset in the provider’s own documentation before assuming a keyword works, and see structured output support and JSON mode versus structured outputs.
Three kinds of absent
A strict mode typically requires every declared property to be present in the output, which means optionality is expressed as a nullable type rather than by omission. That constraint is a gift, because it forces you to confront something a loose schema lets you ignore: null is three different facts and a downstream consumer cannot act on them identically.
- Not present. The document does not contain this field. On an invoice with no purchase order reference, that is correct and final.
- Present but unreadable. The field is there and the extraction could not read it — a torn scan, a stamp over the digits. This is a review item, not a value, and it is handled on handling an illegible field.
- Not applicable. The field cannot exist for this subtype. A credit note has no due date; asking for one and getting null is not the same as an invoice missing its due date, which is a missing required field.
Model this as a small status enum alongside the value rather than as three sentinel values, and let the model populate it. A field that can say why it is empty removes an entire class of review work.
Changing it later without a full re-run
The schema will change anyway. What decides whether that is cheap is whether the change is additive. Adding a nullable field, adding an element to an enum, adding an array: cheap, because existing records remain valid and only new extractions populate it. Renaming a field, narrowing a type, splitting one field into two: expensive, because every stored record now has a shape that no longer exists.
Two practices make the expensive case survivable. Version the schema and store the version on every extracted record, so that a consumer can tell which shape it is holding — the mechanics are on schema versioning. And keep the extraction inputs, so a re-run is possible at all; a pipeline that discards the page images after extraction has made every future schema change a data-loss event. When you do re-run, re-run only the fields the change touches, which requires the field-level provenance described on the field audit trail.
Schema-constrained decoding is the part of an extraction stack that differs most between providers: the accepted JSON Schema subset, the strictness guarantee and the failure behaviour when a schema is rejected are all provider-specific, so a schema tuned for one provider is not portable by default. If you route or fall back across providers — which document work pushes you towards, because a model that declines on one class of document needs somewhere to go — you need one request shape that works across them. That normalisation is what Multigrid’s single API and key is for, alongside the per-request cost tracking a long extraction run needs.
Top comments (0)