DEV Community

Jonson800
Jonson800

Posted on Originally published at toolgenix.nxtniche.com

Stop OCRing Every PDF: Route It First with pdf-inspector

#pdf #rust #ocr #opensource

OCR is often the most expensive and slowest step in a document-ingestion pipeline. The frustrating part is that many PDFs already contain usable text, yet a naive pipeline sends every document through OCR anyway.

pdf-inspector takes a better approach: classify first, extract native text when possible, and route only the pages that actually need OCR.

The routing pattern

The core decision is simple:

PDF arrives
  ↓
Classify the document and its pages
  ├─ native text available → extract locally → Markdown
  └─ text missing/broken   → route those pages to OCR
Enter fullscreen mode Exit fullscreen mode

That small decision can remove a large amount of unnecessary OCR work from RAG ingestion, invoice processing, research-paper parsing, and document search.

The library classifies PDFs as:

  • TextBased
  • Scanned
  • ImageBased
  • Mixed

It also returns a confidence score and the specific pages that need OCR. A 40-page report with one scanned appendix does not have to become a 40-page OCR job.

Quick start in Python

Install the package:

pip install pdf-inspector
Enter fullscreen mode Exit fullscreen mode

Then process a PDF:

import pdf_inspector

result = pdf_inspector.process_pdf("document.pdf")

print(result.pdf_type)
print(result.pages_needing_ocr)
print(result.markdown)
Enter fullscreen mode Exit fullscreen mode

For selective OCR, the native package also exposes an OCR-aware pipeline:

ocr_result = pdf_inspector.process_pdf_with_ocr("document.pdf")
print(ocr_result.pages_routed_to_ocr)
Enter fullscreen mode Exit fullscreen mode

The OCR runtime remains separate and is only touched when a page is routed to it. That keeps the default extraction path lightweight.

Node.js and browser support

The same idea is available for Node.js:

npm install @firecrawl/pdf-inspector
Enter fullscreen mode Exit fullscreen mode 
import { readFileSync } from "fs";
import { processPdf } from "@firecrawl/pdf-inspector";

const pdf = readFileSync("document.pdf");
const result = processPdf(pdf);

console.log(result.pdfType);
console.log(result.markdown);
Enter fullscreen mode Exit fullscreen mode

There is also a WebAssembly package for running the Rust parser locally in a browser or Web Worker:

npm install @firecrawl/pdf-inspector-wasm
Enter fullscreen mode Exit fullscreen mode

This is useful when documents should not be uploaded to a parsing service just to determine whether they contain native text.

What the extractor preserves

Classification is only half the project. For text-based PDFs, the extractor attempts to preserve structure such as:

  • headings derived from font-size tiers
  • bold and italic text
  • numbered and bulleted lists
  • code blocks detected from monospace fonts
  • tables detected from drawing rectangles and text alignment
  • multi-column reading order
  • links, page breaks, captions, and common font encodings

The output is Markdown, which makes the library convenient for search indexing and LLM/RAG pipelines.

How classification works

At a high level, the detector inspects PDF content streams for text operators such as Tj and TJ, and image operators such as Do. It can scan all pages, stop early, sample a large document, or inspect a caller-provided page set.

This is a routing signal, not a promise that every PDF will be perfectly parsed. PDFs with broken encodings, text converted to vector paths, or extremely complex layouts may still need OCR or a specialized parser. The library explicitly reports encoding problems so callers can fall back instead of silently accepting bad text.

About the benchmark numbers

The project publishes a reproducible benchmark against a 200-document corpus. Its July 2026 results report strong reading-order and table scores as well as fast local processing. Those are project-published measurements on specified hardware—not a universal latency guarantee—so benchmark your own document mix before committing to production thresholds.

The more durable takeaway is architectural: OCR should be a fallback chosen per page, not the default chosen per file.

A practical production rule

A conservative router might look like this:

result = pdf_inspector.process_pdf("document.pdf")

if result.pdf_type == "text_based" and result.confidence >= 0.95:
    store_markdown(result.markdown)
else:
    send_pages_to_ocr(result.pages_needing_ocr)
Enter fullscreen mode Exit fullscreen mode

Your threshold should depend on the cost of a false positive. A casual knowledge base can tolerate more extraction noise than a legal or financial workflow.

If your pipeline currently OCRs every incoming PDF, classification-first routing is a small change with a clear operational payoff.

The longer version and implementation notes are available on ToolGenix.

Top comments (0)