Here is the bank-transfer block from a Japanese invoice, rendered at 300 dpi. The fine print is 7.5 pt and every character is crisp. The bank is みずなら銀行 — a fictional institution I invented for a benchmark. It exists nowhere except in this document.
gemini-3.5-flash@high read this block five times. It answered みずほ銀行 — Japan's largest megabank — five times out of five.
It did not fail to read the document. It read it, and overruled it.
Where this came from
This is the strangest cell in the legibility map I published recently. That project degraded one Japanese invoice through seven simulated scan resolutions and ran 27 vision model variants down the ladder, to find where each one stops reading — and what it does after. An earlier article established the pattern everyone now expects: when a model can't read, some models invent.
This is different. This happened at the very top of the ladder, on a fully legible original. The trigger wasn't degradation.
An accidental controlled experiment
The benchmark's fine print held two fictional financial institutions, and — more by instinct than by plan — they differed in exactly one way.
The first, みずなら銀行, sits one character away from みずほ銀行, a real megabank. The second, ほしかげ信用金庫, is a small credit union with no real-world neighbor: nothing in the space of Japanese financial institutions sounds like it.
Same font size, same resolution, same prompt. The results:
- At 300 dpi,
@highread ほしかげ信用金庫 correctly in every run — while turning みずなら into みずほ in every run. - Across the three Gemini variants and the full ladder, the substitution happened 48 times: 25 on
@low, 18 on@high, 5 on@medium. - When ほしかげ finally did break, deep in the blur, it drifted to はしかぜ信用金庫 and はしかわ信用金庫 — plausible-sounding institutions that do not exist. Across the entire run it was never once pulled to a real one.
The trigger is not legibility. It is the existence of a nearby real entity. Where a real neighbor exists, there is an attraction. Where none exists, the model reads what is on the page — or, at worst, invents something exactly as fictional as the truth.
Vision proposes, language disposes
I can't see inside these pipelines, so take this as the simplest explanation rather than a claim about architecture. These systems read with their language model as much as with their eyes. The visual evidence says みずなら; the language prior says みずほ is overwhelmingly more probable; and somewhere in decoding, probability wins — silently, with no flag, at settings you don't control. One character of visual evidence is not enough to outvote a name the model has seen millions of times.
Which is also why the credit union survived. There was no gravity well next to it.
It's a trait, not a law
claude-fable-5 read the same field on the same ladder and never substituted the real bank — not once, at any resolution, while reading the 7.5 pt tier correctly down to 50 dpi. Same input, same prompt, opposite disposition.
Prior capture is not a law of vision models. It is a measurable individual trait — which means you can select against it.
Why this is the scary one
The earlier fabrication article showed invented values that reconcile: totals that add up around a counterparty that was never there. This failure is worse in one specific way: the fabricated value is more plausible than the truth.
Run the human review in your head. A reviewer checking "does this bank look real?" passes みずほ銀行 without blinking — and would actually flag the true value as a typo. Every plausibility check you have, human or automated, is aligned with the error.
And notice what made it visible at all: the ground truth was fictional. Real invoices carry real banks, so in production this substitution produces output indistinguishable from a correct read. A benchmark built on real documents is structurally blind to prior capture. Unguessable, fictional ground truth isn't a convenience for scoring — it is the only instrument that detects this failure mode.
Concretely: never validate payment fields by plausibility. Validate against your counterparty master — the extracted bank either matches the registered account or a human looks at the page. And if fine print matters in your pipeline, benchmark for disposition, not just accuracy: a model's willingness to overrule the page matters as much as its eyesight.
Receipts
Everything is published in the benchmark repo — github.com/ldxhub-io/examples → analyzedoc/legibility-benchmark/: the deterministic material generator (the crops above are the actual benchmark materials), the raw outputs of all 4,158 jobs, and the scorer. The free-tier reproduction subset described in the map article includes @high, so you can watch the correction happen on your own API key without paying anything.
Disclosure, as before: I run LDX hub, the harness used here. It builds no models, and nothing in this post gets better or worse for me depending on which model wins.
The errors to fear are not the implausible ones — those get caught. The ones to fear are the corrections: answers improved in the direction of the world's expectations, away from what the page says. A model that corrects reality will pass every review that checks for plausibility. The only defense is knowing what the document really said — which is exactly the thing you were asking the model to tell you.
Top comments (0)