DEV Community

Cover image for Calibre couldn't see my OCR: building a Mac app that gets scanned PDFs onto a Kindle
Indiorlei de Oliveira
Indiorlei de Oliveira

Posted on Edited on

Calibre couldn't see my OCR: building a Mac app that gets scanned PDFs onto a Kindle

I read a lot on a Kindle, and a lot of what I want to read only exists as a PDF. Some are clean exports with real text. Many are scanned books: 400 photos of pages, 180 MB, slightly crooked.

You can send a PDF straight to a Kindle, but the result is bad. A PDF has fixed pages, so on a 6" screen the text is tiny and you can't make it bigger. What you want is an EPUB, which reflows to fit the screen and lets you change the font and text size, like a book bought from the store.

The usual recipe takes four tools: one for OCR, Calibre to convert, something to shrink the images, then email or Amazon's app to send. I wanted one window where I drop the PDF and the book shows up on my Kindle.

The Lumink queue: each book shows the stage it's in

The pipeline

Every PDF goes into a queue and through the same five stages, one book at a time:

PDF ─▶ Analyze ─▶ OCR ─────────────▶ Convert ─▶ Compress ─▶ Send
       PDFKit     ocrmypdf or Vision  Calibre    ImageIO      Send to Kindle
                  (only if scanned)   or image mode           or SMTP
Enter fullscreen mode Exit fullscreen mode

I'll go through each one, but the interesting part is the third.

1. Analyze: is this PDF text, scanned, or both?

Running OCR on a PDF that already has text wastes minutes and can make things worse, so the first step is to find out what we have.

Lumink uses PDFKit to pull the text from up to 30 pages spread across the book. A page "has text" if it has 40 or more non-whitespace characters:

let ratio = Double(withText) / Double(indices.count)
let kind: PDFKind =
    ratio >= 0.9 ? .text :     // text PDF: skip OCR
    ratio < 0.5  ? .scanned :  // scanned
                   .mixed      // mixed
Enter fullscreen mode Exit fullscreen mode

Because it only reads a sample, this step takes under a second, even on a 600-page book. When OCR does run, Lumink checks every page and only processes the ones with fewer than 40 characters.

2. OCR, with a fallback

For scanned and mixed PDFs, Lumink runs ocrmypdf (Tesseract) with --skip-text, so pages that already have text are left alone.

ocrmypdf is optional. If it isn't installed or it fails, Lumink falls back to Apple's Vision framework (VNRecognizeTextRequest in accurate mode), which ships with macOS. It's slower, but it means OCR works on a fresh Mac with nothing installed.

If a scanned PDF yields fewer than 100 characters per page on average after OCR, it's probably a comic or a photo book. In that case Lumink switches to image mode on its own (more on that below).

3. The bug: Calibre ignores invisible text

This is the part that cost me an evening.

OCR doesn't replace the scanned image. It keeps the image and adds an invisible text layer on top, positioned over the words. In PDF terms, that's text drawn with render mode 3, a paint mode that draws nothing. That's how you can select and search text in an OCR'd PDF while still seeing the original scan.

I fed that PDF to Calibre's ebook-convert, and the EPUB came out... with no text at all. Just images, one per page.

I tried Calibre's other PDF engine and got the same result. Both drop invisible text. As far as Calibre is concerned, an OCR'd page is a picture with nothing written on it.

The fix: build the PDF that goes to Calibre myself (TextLayerPDFBuilder in the code). It takes two steps.

First, read back the text the OCR wrote, line by line, with each line's position. PDFKit reads it straight from the invisible layer:

let box = page.bounds(for: .mediaBox)
let lines = page.selection(for: box)?.selectionsByLine() ?? []
// each line: text + rectangle on the page
Enter fullscreen mode Exit fullscreen mode

Then, write a new PDF. Pages that already had text are copied as they are. On scanned pages, every line is drawn as visible text in the same spot, and the scan image is left out:

// The line box includes ascenders and descenders; ~75% of it is the font size.
let fontSize = max(4, rect.height * 0.75)
// ...
// Stretch or squeeze horizontally to fill the line's original width.
let sx = min(max(rect.width / width, 0.5), 2)
ctx.textMatrix = CGAffineTransform(scaleX: sx, y: 1)
ctx.textPosition = CGPoint(x: rect.minX, y: rect.minY + descent)
CTLineDraw(ctLine, ctx)
Enter fullscreen mode Exit fullscreen mode

The horizontal stretch looks like a detail, but it matters. Calibre decides whether two lines belong to the same paragraph by looking at where each line ends. If a redrawn line came out shorter than the original, Calibre would break the paragraph in the middle.

Now Calibre sees real text, and reflow works. As a bonus, the EPUB doesn't carry a photo of every page, which makes it far smaller.

The trade-off is that figures inside scanned pages don't make it into the EPUB. For books that are mostly illustrations, image mode is the better choice.

4. Convert with Calibre

For text, Lumink calls ebook-convert with options tuned for books rather than documents. Here are the ones that matter:

ebook-convert input.pdf output.epub \
  --output-profile kindle_pw3 \
  --pdf-engine calibre \
  --pdf-header-skip -1 --pdf-footer-skip -1 \
  --pdf-footer-regex '^\s*[-–—]?\s*\d{1,4}\s*[-–—]?\s*$' \
  --unwrap-factor 0.45 \
  --enable-heuristics \
  --filter-css font-family,color,background-color \
  --epub-version 3
Enter fullscreen mode Exit fullscreen mode
  • --pdf-header-skip/--pdf-footer-skip -1 detect and remove the running titles at the top and bottom of every page.
  • The regex removes stray page numbers, including ones like — 42 —.
  • --enable-heuristics joins broken lines and undoes hyphenation.
  • --filter-css strips the PDF's fonts and colors, so the Kindle applies the ones the reader picked.

Without these options, the EPUB reads like a photocopy, with a page number in the middle of a sentence every few paragraphs.

A note on language: Lumink detects the book's language with NLLanguageRecognizer (from the PDF's text or from the OCR output) and writes it to dc:language in the EPUB metadata. It looks cosmetic, but the Kindle uses that field to pick the dictionary, hyphenation and the text-to-speech voice. The same language also goes to the front of the OCR language list, because both Vision and Tesseract give more weight to the first entry.

Image mode, for comics and manga

Comics, manga and photo books don't have text worth reflowing. Image mode skips OCR and Calibre and builds a fixed-layout EPUB 3 (rendition:layout set to pre-paginated) directly, one image per page. It's the only mode that doesn't need Calibre at all.

5. Compress

Scanned books are heavy, and both delivery paths have size limits. So Lumink unzips the EPUB and recompresses the images in parallel with ImageIO:

Level Max height JPEG quality
High 1600 px 85%
Medium 1600 px 75%
Maximum 1200 px 55%

On top of that, opaque PNGs of 40 KB or more become JPEGs, after a pixel-by-pixel transparency check. A new image is only kept if it's actually smaller than the original.

When a PNG becomes a JPEG, the references in the HTML, CSS and OPF, and the media-type, are updated too. Then the app zips the book back up, with mimetype as the first, uncompressed entry, as the EPUB spec requires (zip -X -0). Some readers reject the file if that rule isn't followed.

If the book is still over the limit, it tries the next level. If even maximum compression doesn't fit, it saves the EPUB and tells you, instead of sending a file that would bounce.

6. Send, without asking for your password

The first version sent books by email, which meant asking for an SMTP password. Even stored in the Keychain, I didn't like asking anyone for a password.

The default now is Amazon's own Send to Kindle app for Mac. Lumink opens the EPUB in it, and you pick your Kindle and confirm. The login stays with Amazon: no password in Lumink, no server in between, and a 200 MB limit.

Email is still there as an option for fully automatic delivery. Two details I liked solving there:

The size math. Send to Kindle by email accepts 50 MB, but Gmail rejects messages over 25 MB, and base64 makes the attachment about 33% bigger. That leaves roughly 18 MB of attachment. When the server is Gmail, Lumink uses 18 MB as the limit automatically.

Why curl. I could have used a Swift SMTP package, but most are barely maintained, and each would bring its own TLS and auth code for me to audit. Instead, Lumink builds the MIME message itself and hands it to the curl that ships with macOS. The password goes to curl through stdin (--config -), so it never shows up in ps or in a file on disk.

What I learned

  • Test the tool with the real input, not the one you picture. I tested Calibre with text PDFs, and it worked. The problem only showed up with PDFs that came out of OCR, because "has text" and "has visible text" are different things.
  • A native fallback is worth it. Having Vision as a fallback for ocrmypdf made the app work on a fresh Mac, with no Homebrew and no Python.
  • Sometimes the safest feature is the one you remove. Taking the password out of the default path did more than any amount of Keychain care.

Things it doesn't do (yet)

  • No right-to-left reading for Japanese manga: pages stay in PDF order.
  • Figures inside scanned pages are dropped (see the Calibre section).
  • Vision OCR is much slower than ocrmypdf, especially the first time, when it loads its models.
  • It isn't notarized by Apple yet, so on first launch you need to go to System Settings → Privacy & Security → Open Anyway, or run one xattr command.

Try it

Lumink is free and open source, for macOS 14 or later, on Apple Silicon and Intel. The interface is in English and Portuguese. To convert text books, you need Calibre installed. ocrmypdf is optional.

If you read PDFs on a Kindle, I'd love to hear what breaks. Issues and pull requests are welcome, and the code is small enough to read in an afternoon.

And a question for you: have you ever run into a tool that silently ignored an "invisible" part of a file? Tell me in the comments.

Why the name? **Lume* is an old Portuguese word for fire and light (from Latin* lumen*). Ink is for the e-ink screen. And in English, to* kindle means to set alight.


Leia este post em português: O Calibre não enxergava o meu OCR: como fiz um app para Mac que leva PDFs escaneados ao Kindle

Top comments (0)