DEV Community

Cover image for My PDF converter returned an empty Markdown for a perfectly readable scan — image-only PDFs have no text layer
InApp
InApp

Posted on Originally published at imapp.blogspot.com

My PDF converter returned an empty Markdown for a perfectly readable scan — image-only PDFs have no text layer

The bug report arrived sideways: an agent kept failing to answer questions about a "signed supplier agreement" even though the contract was supposedly in its knowledge base. Tracing the ingestion pipeline, the culprit was my PDF-to-Markdown converter — it had returned a document with page markers and headings but essentially no text.

Opening the file in a viewer showed perfectly readable pages. That was the trap. Text extraction only reads the text layer, and this PDF didn't have one. Every page was a 300dpi scan wrapped in a PDF shell. To a human it's a contract; to an extractor it's an album of images.

Worse, the failure was invisible. Empty output looks exactly like "document with no content" — nothing crashed, nothing logged. The gap only surfaced weeks later when retrieval found nothing to retrieve.

So I built detection from three signals:

  • near-zero extractable characters across the whole document
  • zero embedded font resources (scanned pages carry none)
  • content streams consisting almost entirely of full-page image XObjects

Two of three now flips a text_layer_detected: false flag in the API response, plus a per-page image inventory so the caller can decide whether to route the file to OCR. I deliberately did not auto-OCR — contracts and invoices deserve an explicit choice, and OCR quality varies too much to promise silently.

The bigger lesson: silence is the worst failure mode an ingestion pipeline can have. "No text" and "cannot extract text" must never look identical downstream — one honest, distinguishable response beats a thousand empty strings.

That detection now ships behind my PDF-to-Markdown endpoint (https://x402.freeq.one/tools/pdf_to_markdown.html) so pipelines can branch before they quietly index nothing.

Top comments (0)