A lot of document pipelines share one shortcut: call extract_text(), and only fall back to OCR when it comes back empty. Archive scans rarely come back empty: Internet Archive runs tesseract over them and stores the result as an invisible text layer, so the shortcut treats a book printed in 1734 like a born-digital file. To see what that costs, I took two public-domain books from archive.org, transcribed one printed page of each by hand, and compared. The two books landed on opposite sides, which turned out to be the useful part.
Where the invisible text comes from
PDF text has a rendering mode, set with the Tr operator. Mode 3 means neither fill nor stroke: the glyphs have positions, you can select and copy them, and nothing gets painted. An OCR'd scan is a page image with the recognised words drawn on top in mode 3. The viewer shows you the image, extract_text() hands you the words, and nothing in the file checks that the two agree. To spot such pages I walk the content stream with pypdf, track the current Tr value, and count text-showing operators (Tj, TJ, ', ") under mode 3 versus everything else. Tr is part of the graphics state, so it gets saved by q and restored by Q; without a small stack for that, a mode change inside a nested block leaks out and the counts drift. While I'm on the page, I also list each image XObject's /Filter and the filter of its /SMask, if it has one.
On the two pages I cut from each book, the counts came out as 461 and 470 invisible text operators for 1921, 296 and 302 for 1734, and zero visible ones anywhere. Every text operator on all four pages is invisible. Each page also carries two images: a JPEG 2000 layer (/JPXDecode), and a second JPEG 2000 layer with a JBIG2 soft mask (/JBIG2Decode). That's a mixed raster setup, a smooth background plus a 1-bit mask that keeps glyph edges sharp. A born-digital PDF usually looks the other way round, with no mode-3 text at all, so the ratio separates the two kinds of file cleanly. I spent three years writing codecs at an SDK company, and I still admire how small these pages get.
Two books, opposite results
Book one is Psycho-Analysis and the War Neuroses (London, 1921), text layer from tesseract 5.3.0. On printed page 2 (460 words, 2,706 characters, transcription checked line by line against a 3489-pixel-high render), the text layer is off by 4 character edits, 0.15%, with errors like whichis and a stray | at a line end. I also rendered the page and ran OCR on the pixels again: 7 edits, 0.26%, including aro for "are" and renderedl. Re-recognising did not beat the layer that was already there.
Book two is Round About Our Coal Fire (4th edition, 1734), tesseract 4.1.1: long s, italics, a ct ligature, catchwords. On page (3), 301 words and 1,577 characters, the text layer comes to 2.2% if I don't count ſ read as f, 4.1% if I do. It gives you Loudon for London, Bithop ot Wincheffer's for Biſhop of Wincheſter's and Ele@tiion for Election. A rendered re-OCR brings the page to 1.0% (3.2% strict). Neither pass knows the long s; both read it as f.
So the result is not "never trust a text layer". A good tesseract layer on a clean 20th-century book was the best text I had. The old typeface is what broke it, and you only find that out by looking at a page.
What a browser extractor did with them
I also ran both files through Extract PDF content in ImgIng, a tool I help build. My part is codecs and model loading; the PDF engine is someone else's work. With either book, the source preview was a blank page, and the TXT export with "Recognize text inside images" switched on was byte-identical to the export with it switched off. No Image OCR block, no warning. That fits the page images never being decoded: Chromium ships no JPEG 2000 or JBIG2 decoder, so a web page has to bring its own, and when that step produces no pixels, OCR has nothing to read and only the embedded layer is left. That's my inference from the codec side; I haven't traced it through the engine.
For the 1921 book the fallback is harmless. For the 1734 book it did real damage. The output lost 12 words that pypdf finds in the layer (Countenances;, the honeft, at Chelfea; and others, mostly at the start of a line) and moved fragments to the wrong line, which put page (3) at 19.5% against the layer's own 2.2%. I haven't found the cause yet. It also cut justified lines with wide word gaps into two-cell tables: 48 of the 109 lines on the two 1921 pages carry a Tab in the TXT.
Render, re-OCR, compare one page
The workaround is to render the pages with something that decodes both formats, save them as an image-only PDF, and OCR that. My measured run used macOS sips for the render. On Linux, PyMuPDF decodes JPEG 2000 and JBIG2, and in the version I checked, the only line that needs any thought is the resolution (excerpt):
widest = max(doc.extract_image(x[0])["width"] for x in page.get_images())
dpi = round(widest / (page.rect.width / 72))
That dpi goes straight into get_pixmap, each rendered page then goes to PIL and the list is saved as one image-only PDF. For the 1734 file this comes out at 72 dpi, 1483×2746 for the first page and 1522×2839 for the second. The "dpi" just maps one scan pixel to one output pixel (the 1921 file comes out at 400). These renders decode fine and carry no text layer, but the recognition numbers above came from the sips render. Fed to the same extractor, an image-only PDF gets a normal preview and an Image OCR block:
You don't need a hand transcription to decide. Comparing the embedded layer with a re-OCR of the same page gets you most of the way. I split both texts into words, after folding every quote variant into one and joining words hyphenated across a line break, then let difflib find the matching blocks (excerpt):
def words(text):
text = re.sub(r"[“”‘’'`\"]", "'", text)
text = re.sub(r"-\s*\n\s*", "", text) # join end-of-line hyphenation
return re.findall(r"[A-Za-z']+|\d+", text)
sm = SequenceMatcher(None, layer, ocr, autojunk=False)
same = sum(b.size for b in sm.get_matching_blocks())
The agreement is same divided by the longer of the two word lists, and the non-equal opcodes give you the disagreements side by side. On the 1921 page the layer has 463 words, the re-OCR 462, and they agree on 99.1%; the differences are a lone 2 the re-OCR dropped, a quote mark, are against aro and rendered against renderedl. On the 1734 page it's 302 words against 299 and 88.4%, with pairs like weicome / welcome, Hull / Hall and greafy / grcafy running down the list. The gap between the two books is wide, and the 1734 list reads like an index of the layer's mistakes. The check has a blind spot: whichis is wrong in both versions, so it never appears. Two readers agreeing is not the same as both being right.
What I do now is sample one page per source. If the layer and the re-OCR mostly agree, keep the layer; it's cheaper, and on the 1921 book it was the better text. If they disagree word by word, re-render and re-OCR the whole book. Two books aren't enough to put a number on "mostly"; 99 against 88 was a wide enough gap that I didn't need one. ImgIng is at https://imging.ai/
Top comments (0)