5 Undocumented Rules for Gemini Structured Output, Measured in Production

We run a document extraction pipeline on Gemini with a native responseSchema attached, not a "please reply with JSON" instruction in the prompt text. Over two months, three separate production problems traced back to behaviours of that schema that are not in Google's documentation.

These are the rules we ship with now, and the measurement behind each one. The domain is anonymized (no client, no industry, role codes renamed). Every number, date, model name and error string is real.

TL;DR

1. Order the required array identity, then evidence, then derived. It controls emission order, and emission order controls correctness.
2. Count your total enum values before shipping. There is an undocumented ceiling and it is lower than you think.
3. Only real enum arrays are enforced. Values listed in a description are not constrained at all.
4. Author schemas in Gemini's subset, not in JSON Schema.
5. PROVIDER_EXHAUSTED right after a prompt change means your schema is broken, not that Google is busy.

Rule 1: order required identity, evidence, derived

The law

Gemini emits every required property first, in exactly the order the required array lists them, then the optional ones. The declaration order inside properties is ignored for the required set.

This is empirical. It is not in Google's docs. We measured it on gemini-3-flash-preview twice, with opposite orders, reading the raw response text:

schema	position of role_code in required	position in emitted JSON
v18	3rd	3rd
v19	14th (last)	14th

In v18 that field was 12th in properties and 3rd in required. It came out 3rd. properties is not the lever.

propertyOrdering is the documented knob, but if you do not set it (we do not, anywhere), the required order is what governs.

Why this is correctness and not cosmetics

A model writing JSON does one forward pass. Whatever it has already emitted is in context. Whatever it has not is not. And an emitted token cannot be revised when a later field contradicts it.

So a field emitted early is decided with almost no self-generated evidence, and a field emitted late is decided with everything above it visible.

The failure this came from

v18 put role_code third in required, after only first_name and last_name. Its instruction was a priority ladder:

1. the position stated in the application (in the input, available)
2. the title of the most recent work_history entry (emitted 4 fields later, unavailable)
3. the CV header (in the input, available)

Nine CVs, all advertising the same role. Four came back with L3-OPS, a role from a different department. All four were internally self-contradictory:

{
  "role_code": "L3-OPS",
  "department": "Technical",
  "work_history": [{ "title": "L3-TECH", "...": "..." }]
}

Both codes are valid members of the 143-value enum, so nothing rejected the output. department, which agreed with the correct reading in all four cases, was emitted 11th, long after the wrong token was committed. The consistency check that would have caught the error was generated downstream of the error.

Ruled out first: environment drift (schemas byte-identical), downstream mapping (the wrong code was already in the raw provider response), a missing enum value (the correct code was present, and used correctly elsewhere in the same responses), and ambiguous source documents (zero matches for any operations wording, 11 to 20 matches for technical wording per document).

The fix

Reorder required. Nothing else. No type change, no enum change, no shape change.

v18:  first_name, last_name, role_code, contacts, nationalities, date_of_birth,
      work_history, certifications, documents, education, languages, address,
      home_airport, department

v19:  first_name, last_name, date_of_birth, nationalities, contacts,
      work_history, certifications, education, documents, languages, address,
      home_airport, department, role_code

Result: 9 of 9 correct, up from 5 of 9. Emitted position of role_code moved 3 to 14, exactly as predicted.

Classify every field

class	meaning	position
identity	copied off the document, no reasoning (first_name, date_of_birth)	first
evidence	the substantive extracted content (work_history, certifications)	middle
derived	a judgement about the evidence (role_code, department, any score, total or summary)	last

Three things that come with the reorder:

Descriptions must not forward-reference. Once department moved ahead of role_code, its old text ("classify from the stated role_code") became the same bug in miniature. After any reorder, re-read every description for references to fields that now come later.

Tell the model the evidence is already there. Reordering alone is silent. v19's priority 2 became: "You have ALREADY emitted the work_history array above. Read the title of its first entry and use it."

Check for over-anchoring. The goal is grounding, not echoing. Two candidates whose most recent entry was one level below the applied-for role still correctly emitted the applied-for level, because priority 1 legitimately outranks priority 2. If every derived value suddenly equals evidence[0], you have over-corrected.

And the trap: required is a set to a JSON Schema validator. Reordering it is semantically inert, so a formatter that sorts the array, or a tool that round trips the JSON, silently reverts the behaviour with a diff that looks like whitespace and passes every test. Say so in the file.

Do not respond to a wrong derived field by adding more prose first. v18 already carried four bullets of correct guidance for that field and was still wrong about 44% of the time. The instruction was not being disobeyed. It was being evaluated at a token position where its input did not exist.

Rule 2: count your enum values before you ship

Gemini rejects a schema above an undocumented ceiling on the total enum-value count across the whole schema. Google publishes no number, only that "very large or deeply nested schemas may be rejected".

total enum values	result	when
467	accepted	v9, production
610	accepted, months of clean runs	v10-revised through v14
740	rejected	v15, 2026-08-17
754	rejected, reverted	v10-initial, 2026-07-13

The boundary is in (610, 740]. We never bisected it.

Three checkpoints were tried with the 740 schema, one preview and two GA releases:

model	outcome
gemini-3-flash-preview	400 invalid argument
gemini-3.5-flash	400 invalid argument
gemini-3.6-flash	400 invalid argument

Identical rejection across releases spanning months. This is a property of the constrained-decoding compiler, not of a checkpoint, so waiting for a newer model is not a mitigation.

You cannot deduplicate your way under the limit. Gemini's subset has no $ref and no $defs (see Rule 4), so every repeated list is paid for in full. A 145-value list used in three places costs 435, not 145.

Practical consequences:

• Recount the total before adding any enum to a large schema.
• For low-value fields, put the code list in a description instead. Descriptions cost nothing against the budget. Just know they are not enforced either (Rule 3).
• If a vocabulary genuinely needs enforcement and does not fit, split the extraction into two calls, and split it by data dependency, not by document section. Fields that derive from each other must stay in the same call.

A rough counter is worth having in CI:

// Sums every enum array in a schema, nulls included.
function countEnums(node) {
  if (Array.isArray(node)) return node.reduce((n, v) => n + countEnums(v), 0);
  if (node && typeof node === 'object') {
    return Object.entries(node).reduce(
      (n, [k, v]) => n + (k === 'enum' && Array.isArray(v) ? v.length : countEnums(v)),
      0
    );
  }
  return 0;
}

Rule 3: only real enums are enforced

responseSchema guarantees JSON shape and types. It does not guarantee values, with one exception.

how you express it	enforced?
"enum": ["L3-TECH", "L3-OPS"]	yes, by constrained decoding
allowed values listed in description	no, purely advisory
maxLength	no
array uniqueness	no

In July, a schema change meant constrained decoding stopped being applied for five days. Nothing failed, nothing turned red, and 75 values that do not exist in the vocabulary reached production in a field the rest of the system indexes on.

So:

• Validate and normalise bounded fields in code after extraction, even with a schema attached.
• Add a canary: any value outside its enum proves constrained decoding was not applied to that call.

Rule 4: write schemas in the Gemini subset

If the same prompt may run on more than one provider, store the schema in the more restrictive format. Gemini's subset is the floor.

feature	OpenAI	Gemini
$ref / $defs	supported	not supported, inline everything
$schema, $id	supported	not supported, strip
oneOf	supported	not supported, single type + nullable
["string", "null"]	supported	not supported, use nullable
exclusiveMinimum	supported	not supported, use minimum
pattern	supported	stripped
format: "uri"	supported	stripped
nullable	not used	required for nullable fields
max nesting	no limit	5 levels
property ordering	not enforced	required order drives emission

Rejected:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "role_code":  { "$ref": "#/$defs/RoleCode" },
    "start_year": { "type": "integer", "exclusiveMinimum": 1900 },
    "email":      { "type": ["string", "null"] },
    "website":    { "type": "string", "format": "uri" },
    "ref":        { "type": "string", "pattern": "^[A-Z]{2}-\\d{4}$" }
  }
}

Accepted:

{
  "type": "object",
  "properties": {
    "start_year": { "type": "integer", "minimum": 1901 },
    "email":      { "type": "string", "nullable": true },
    "website":    { "type": "string" },
    "ref":        { "type": "string", "description": "Two uppercase letters, hyphen, four digits" },
    "role_code":  { "type": "string", "enum": ["L3-TECH", "L3-OPS"] }
  },
  "required": ["start_year", "email", "website", "ref", "role_code"]
}

Note role_code is last in required, per Rule 1.

A malformed schema does not degrade politely. On 2026-08-27 a single "type": ["string", "null"] union in one prompt failed every execution of it until the union was removed.

Rule 5: the error names the wrong culprit

This is the one that costs the most hours, because the label sends you to the wrong system.

What the provider actually returns:

400 . Request contains an invalid argument.

No field pointer, no property name, no mention of size or enums. Deterministic on every retry, every key and every service tier.

What the operator sees by the time it surfaces:

processing_error   PROVIDER_EXHAUSTED: Provider capacity unavailable
error_reason_code  PROVIDER_EXHAUSTED
model              (empty)

The chain is 400, then an API error, then the key circuit opens, then the retry ladder exhausts, then the last event gets reported instead of the first. It reads as a transient capacity shed. It is a permanent schema defect.

Triage table:

symptom	actual meaning
PROVIDER_EXHAUSTED with an empty model field, starting right after a prompt change	schema defect, not capacity
the same failure on both STANDARD and FLEX tiers	not a tier or quota problem
identical failure across model checkpoints	constrained-decoding compiler, not the model
valid JSON with out-of-vocabulary values	constrained decoding was not applied at all

The real error survives only in a WARN line, on whichever replica ran the worker, which is usually not the replica that logged the submission. Grep all of them:

for P in $(kubectl -n <ns> get pods -o name | grep -E "^pod/ai-" | grep -v db); do
  kubectl -n <ns> logs $P --since=2h \
    | grep -E "invalid argument|Circuit OPEN|keys exhausted"
done

Time-to-failure tells you nothing. We saw 30s and 165s for the same rejection and briefly read the slow one as "this model accepted the schema". It had not. The difference was retry parking.

Verifying emission order

If you store parsed responses in a jsonb column, that column loses key order. Read the raw response text instead:

const raw = require('fs').readFileSync('raw.json', 'utf8').trim();
Object.keys(JSON.parse(raw)).forEach((k, i) => console.log(`${i + 1}. ${k}`));

Compare that against your required array. If they diverge, the ordering law has changed for your model family and needs re-measuring.

Checklist

• [ ] Every field classified identity, evidence or derived.
• [ ] required ordered identity, then evidence, then derived.
• [ ] No description references a field emitted later.
• [ ] Each derived field's instruction names the already-emitted field to read back.
• [ ] Derived fields spot-checked for over-anchoring.
• [ ] A comment states that the required order is deliberate and must not be sorted.
• [ ] Total enum count recounted, well under the last known-good number.
• [ ] Bounded fields validated in code after extraction.
• [ ] Schema written in the Gemini subset (no $ref, no oneOf, no type arrays, nullable used, 5 levels max).
• [ ] Runbook says PROVIDER_EXHAUSTED after a prompt change means schema first, capacity second.

Two of these five rules describe limits Google does not document, and both were established by breaking production. If you are running structured output at any scale, measure them for your own model family and write your own numbers down. The alternative is rediscovering them next quarter at the same price.

Comments

[Vinh Nguyen]

Demoting low-value lists from enum to description to get under the ceiling quietly costs you a row in your own triage table: once some vocabularies are unenforced, valid JSON with out-of-vocabulary values no longer separates "constrained decoding was not applied" from "that field was never constrained to begin with". Keeping one small enum that stays enforced, ideally over values the model would not produce unprompted, preserves the signal, because an out-of-vocab value there can only mean decoding did not run. Worth pinning deliberately, since that check is the cheap one for silent decoding failure and the demoted fields are the ones nobody reads closely.

[Kane Lim]

This is the kind of production detail that’s actually useful. The point about required order affecting emission order is especially interesting because it turns something that looks purely schema-related into a decoding/grounding issue.

I also like the distinction between “schema accepted” and “value actually valid.” Even with constrained output, I’d still keep application-level validation as the final boundary, especially for fields that become indexes or drive business logic.

The PROVIDER_EXHAUSTED case is probably the most dangerous operationally. If the original 400 gets buried behind retries and circuit-breaker errors, the system can make a deterministic schema bug look like an infrastructure incident.

I’d definitely keep the enum counter and schema-subset checks in CI. Catching those before deployment is much cheaper than discovering a provider constraint from production traffic.

Really good write-up especially because you measured the behavior instead of assuming the JSON Schema semantics would map directly to Gemini.