Here is the issuer name from a Japanese invoice, rendered at 300 dpi. A red company seal sits directly on top of it — the kind stamped on nearly every invoice in Japan. On the left, the seal is a normal vermilion impression: translucent, the way real 朱肉 ink sits on paper. On the right, the same seal at full opacity, covering the same characters.
Under the translucent seal, gpt-5.6-sol@high read the company name perfectly — the black ink survives underneath the red, and the model reads straight through it. Under the opaque seal, it read nothing. Same model, same characters, same position. The only variable is whether the ink underneath still exists.
That gap is the first of two findings. The second one is stranger, and it runs the other way.
What this measures
This is a companion to a benchmark I published recently, which degraded one Japanese invoice through seven scan resolutions to find where each vision model stops reading. That axis was blur. This one holds the document perfectly sharp and introduces the second stress factor Japanese paperwork routinely carries and no Western benchmark tests: occlusion by a seal.
One variable: how much of a target field's ink a stamp covers, from 0% (a control, the seal sitting just below the text) to 100% (fully buried), in six steps. Two things vary in a 2×2:
- Opacity. A translucent seal (multiply blend, the black ink readable underneath) versus an opaque pad (the ink physically gone — the generator asserts, pixel by pixel, that nothing dark survives inside the target box).
- Target. The issuer name, which has no arithmetic relationship to anything else on the page. And the subtotal, which by construction equals total − tax, with both of those left visible at every step.
The two splits ask two different questions. Opacity tests whether a model can see through red ink to the black underneath. Target tests whether it can rebuild a buried field from the ones around it. Finding 1 comes from the first axis; Finding 2, from the second.
The extraction prompt never mentions the stamp. Whether a model reads through it, leaves the field empty, or fills it some other way is the thing being measured — so instructing any of those behaviors would destroy the measurement. Everything runs against 27 model variants from the same catalog as the blur benchmark, five repetitions per cell, 3,240 jobs.
Finding 1: the chromatic channel
Human OCR pipelines have handled seals for decades with a red-channel drop — strip the vermilion, the black text underneath comes back. The translucent-versus-opaque split measures whether a vision model does the same thing implicitly.
The top-resolution variants do, almost universally. Under the translucent seal at full 100% coverage of the issuer name, @high variants read it correctly 60 out of 60 times; under the opaque seal at the same coverage, the same variants scored 0 out of 60. The red is not what stops them — the destruction of the ink underneath is. The effect holds across nearly every strong reader, with a chromatic advantage of 0.56 to 0.76.
One model inverts it. nova-2-lite posts a negative score: the translucent seal hurts it more than the opaque one — layered color confuses it worse than a field that is simply gone.
Finding 2: the readers go silent, the non-readers answer
Now the subtotal, under the opaque seal, deep in the ladder — 80% and 100% coverage, where the digits are physically destroyed. Any correct answer here was not read off the page. There is nothing left to read.
The Anthropic models score 0 out of 30 — three models, two coverage levels, five runs each. When the number disappears, they leave it blank.
The OpenAI and Azure @low variants score 80 out of 100. On a field whose pixels are gone, they return the correct value four times in five.
This is not a small effect at the margins. It is a clean inversion, and the ladder makes it visible: on the destroyed-subtotal panel, the models that can read the number go red — blank — at the bottom, while the models that can't stay white. The scariest OCR errors in this whole series aren't misreads. This one isn't even a read.
What's happening is arithmetic. The subtotal equals total − tax, and both of those are visible on the page at every step by construction. A model that has given up reading the buried cell can still reconstruct it from its neighbors. This is a cousin of a pattern I've written about before: when a model can't read a field, some models invent a value rather than leave it blank. There the invented number was pulled from a plausible prior; here it's computed from the visible neighbors. Either way the model answers with something other than what the document says. And here is the part that matters for anyone wiring this into an accounts-payable flow: a derived value is internally consistent by definition. It reconciles. Every validation check that asks "do subtotal, tax, and total add up?" passes — because the number was built to add up, not read. The one instrument that would catch it is the thing you were trying to avoid buying: a human who looks at the actual document.
Which arithmetic
Two routes could produce it — total − tax, or total ÷ 1.1 — and since the tax is exactly 10%, they give the same answer, so the value can't distinguish them. But the @low variants answered the tax field correctly 49 times out of 50 at zero occlusion: with tax visible and answered, subtraction is available directly and division is redundant. The materials for total − tax were in hand. That is as far as the evidence goes — I can say the materials were present, not that I watched them used.
What doesn't generalize, and one honest gap
Two cautions, because this series lives on stating them.
The route conclusion above holds only because the tax here is a round 10%, which collapses two arithmetic paths into one answer. A future version with a mixed tax rate would make total − tax and total ÷ 1.1 produce different numbers, fingerprinting the route in the output itself. That's the clean way to settle it, and this run can't.
And the seal text. The stamp reads 検収済印 — an inspection stamp applied by the receiving company, chosen precisely because it carries zero issuer information. claude-fable-5 reads it correctly 94% of the time, including the traditional right-to-left column order, while most models ignore it as decoration. But 検収済印 is a common, real phrase, so a model reading it can't be separated from a model recognizing it from training. Whether that number reflects reading or memory is genuinely unresolved. A future version needs a fictional seal phrase — the same lesson the blur benchmark taught about fictional ground truth, applied to the stamp.
The picture across all four panels
Put the two findings together. A translucent seal is not much of an obstacle to a strong reader — the black ink is right there under the red. An opaque seal destroys the field, and what a model does next sorts it into a type. The strongest readers in the benchmark fall silent. A specific class of weak readers quietly computes the answer from the fields around it, and hands it back with no indication that it was never seen.
The blur benchmark had readers who invent and readers who go blank. This one adds a third: the deriver, who answers a question you didn't ask — what should this number be, given the others — in place of the one you did: what does the document say. And because the derived answer reconciles, it is the hardest of the three to catch.
The companion to this finding is The model corrected reality — a case where a model read a field perfectly and overruled it. Both are about answers that survive every plausibility check.
Method notes: 3,240 analysis jobs against a synthetic invoice (all names fictional), five repetitions per cell, run July 2026 via my document-processing gateway. The stamp compositing, the pixel-level occlusion checks, the scorer, and the recorded runs are public: ldxhub-io/examples › analyzedoc/hanko-benchmark. Every figure in this article is reproducible from the scored output.
Top comments (0)