I tilted a scanned page 1 , 3 and 5 : Tesseract shrugged, a browser OCR scrambled the lines

#webdev #javascript

![ ](https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/wShould a deskew step come before OCR at all? I wanted to know what a small tilt really costs, so I rotated two clean public-domain scans by 1°, 3° and 5° in code and ran three engines over them. Short answer: on the English page Tesseract went from 98.9% to 96.2% at 5°, the built-in text recognition in macOS barely reacted at all, and a browser-based PP-OCR kept almost every letter but lost the line order from 3° on. On the Chinese page that last one dropped about half the text at 5°.

How I counted the errors

The English page is page 8 of H. G. Wells' The Time Machine (Heinemann, 1895), 1,002 characters, checked against the validated Wikisource transcription. The Chinese page is the 1986 explanation note of the Simplified Characters table, 557 characters. Both are clean one-column prints, a friendly case, and the tilt is added by code, not by a real scanner. One run each, on an Apple M4 Mac. Tesseract 5.5.3 had only its English pack, so it skipped the Chinese page. macOS ran through the Vision framework in accurate mode. The browser engine was the PDF content extraction in ImgIng https://imging.ai/ in a desktop browser, which runs PP-OCR inside the page.

"Errors" here means character edit distance: how many single characters you have to insert, delete or replace to turn the OCR output into the true text. I didn't see its side effect until the results came in: if two lines swap places, almost every character counts as wrong though nothing was misread. So I also counted "lost" characters, which ignores order and only asks which characters from the truth never showed up. Below is a simplified illustration of both, written from the definitions; the numbers in the rest of this post come from the full scoring script, which also normalizes quotes, dashes and whitespace first:

import collections as cl

def edits(ref, hyp):
    row = list(range(len(hyp) + 1))
    for i, r in enumerate(ref, 1):
        nxt = [i]
        for j, h in enumerate(hyp, 1):
            nxt.append(min(row[j] + 1, nxt[j - 1] + 1, row[j - 1] + (r != h)))
        row = nxt
    return row[-1]

def lost(ref, hyp):
    return sum((cl.Counter(ref) - cl.Counter(hyp)).values())

A made-up toy case, not one of the sample pages: swap the two lines "the time traveller" and "was expounding" (33 characters with the line break), and edits returns 30 while lost returns 0.

What to look at: in the 3° and 5° rows the two browser-OCR columns jump into the hundreds, while both macOS columns stay at 8 or below. Samples are the two public-domain pages credited at the end; the rotated and noisy versions are self-made.

Tesseract lost a little at 5 degrees

The percentages here come from a separate Tesseract grid on PNG copies of the English page; the figure used JPEG copies, so I don't mix the two. With no noise, accuracy was 98.9 straight, 99.5 at 1°, 98.1 at 3° and 96.2 at 5°. So 1° cost nothing, 3° cost 0.8 points and 5° cost 2.7. Why 1° beat the straight page I honestly don't know. One run, probably noise. Deskewing alone pulled the 5° page back to 99.0. With Gaussian noise (σ 25) it went 97.9 to 92.9 at 5°, still far from what 5% salt-and-pepper noise did to the straight page: 75.5.

macOS Vision did not react to skew

The English page had 0 errors straight, 1 at 3° and 0 at 5°. The Chinese page stayed at 4 to 8 errors from 0° to 5°; on the straight page, 3 of its 5 errors were pinyin tone marks. Its one bad number came from the noisy 3° English page: 50 errors with only 2 characters lost, because one line got moved below the next one. Same lesson as the toy pair, the letters were fine and the order wasn't.

The browser OCR kept the letters and lost the order

This one surprised me, because on the straight Chinese page it had the fewest errors of the engines I ran there, 2 against 5 for macOS, and only 1 on the English page (it skipped the page number). At 1° the Chinese page was still fine with 1 error. At 3° the English output had 400 errors and only 1 lost character. In the noisy 3° run, which scored the same 400, the first line of each paragraph came out after the line beneath it. The Chinese page at 3° had 175 errors, 31 lost, with two short lines gone. At 5° it was 446 errors and 115 lost for English, 362 and 275 lost for Chinese, about half the page missing. The page still said the job was done, with no warning; only comparing line counts would catch it.

Denoise and deskew before OCR

On the noisy 3° page I ran non-local means denoising, a projection-profile deskew, then Otsu binarization. ImgIng came back to 2 errors on both pages. Tesseract went from 25 to 13 on the English page, and macOS Vision from 50 to 1 there, 9 to 6 on Chinese. For the browser engine that was the difference between lines in reading order and a scrambled page.

This is narrow: two clean pages. I didn't test real scanner skew, phone photos, two-column layouts or low-resolution scans, and ImgIng can't currently read scanned pages stored as JPEG 2000 or JBIG2. Still, whether skew matters depended almost entirely on the engine. Before adding a deskew stage, rotate one of your own pages by 3°, run your engine, and compare both the edit distance and the lost characters. If the lost count stays low while edits explode, it's an ordering problem, and on these two pages deskewing was what fixed it.

Samples: H. G. Wells, The Time Machine, William Heinemann 1895, p. 8, Wikimedia Commons, public domain (truth text from Wikisource). Chinese page: State Language Commission, 1986 edition of the Simplified Characters table, explanation page, Internet Archive, Public Domain Mark 1.0.
1jizpbbljad1a08fkzp.png)

DEV Community