When to Self-Host LightOnOCR-3 vs. Calling a Hosted PDF-to-Markdown API

When an AI agent or RAG pipeline needs to turn a PDF into clean Markdown, teams hit a practical fork: run an open-weight OCR model on their own GPU, or pay per page for a hosted conversion API.

LightOnAI's October 8, 2026 release of LightOnOCR-3 (0.8B, 1B, and 4B Apache-2.0 checkpoints) makes the tradeoffs concrete. The 0.8B and 4B models move onto Qwen3.5 vision-language backbones and post strong numbers on clean technical layouts, while exposing the exact places where self-hosting stops being free.

What LightOnOCR-3's benchmarks show

Across olmOCR-Bench, ParseBench, and fr-bench-pdf2md (leaderboard revision 3576c52), LightOnOCR-3-0.8B scores 85.5 overall on olmOCR-Bench, hitting 90.8 on tables, 89.6 on arXiv math, 85.2 on multi-column text, and 94.1 on long tiny text. The 4B variant reaches 86.3 on olmOCR-Bench and 75.1 on ParseBench.

Degraded pages tell a different story. On olmOCR-Bench old scans, LightOnOCR-3-0.8B drops to 48.7 (52.5 for 4B). On fr-bench-pdf2md, 0.8B scores 33.3 on handwriting (46.1 for 4B) and 47.8 on forms (49.6 for 4B). Clean digital papers and tables work well at sub-1B scale; low-contrast scans and handwritten forms remain hard.

Throughput also shifts with input resolution. Measuring 512 benchmark pages on a single H100 with vLLM 0.30.0 and guidellm, LightOnAI hit its peak olmOCR-Bench accuracy at 400 DPI (5 MP cap, roughly 4,800 image tokens per page). Lowering the long side to 1540 px reduced image tokens by almost two-thirds and raised peak throughput by 44% to 4.78 pages/s on 0.8B (3.36 pages/s on 4B), while 1B reached 2.7 seconds single-page latency.

The surrounding engineering bill

Running a compact OCR model handles page-level transcription, yet production document ingestion still requires three pieces around the model weights.

The first bottleneck is postprocessing and normalization. LightOnAI reports that simple deterministic format rewrites, such as HTML <sup>1</sup> versus LaTeX $^{1}$ footnotes, move edit-distance scores noticeably across top models. Grounding mode outputs inline labels, 0-1000 bounding boxes, image captions, and HTML chart tables that your pipeline must parse into Markdown sections, cross-page paragraphs, and RAG chunks.

The second bottleneck is PDF rendering and resolution routing. The worker still needs safe PDF rasterization, rotation handling, image extraction or hosting, and rules deciding when a math-heavy page needs 400 DPI instead of 1540 px.

The third bottleneck is utilization on bursty queues. LightOnAI notes that self-hosted compute can fall below $0.01 per 1,000 pages at high utilization. That assumes steady batch volume. An agent workflow that parses a handful of research papers or filings in irregular bursts either pays for an idle GPU or eats container cold-start latency.

Offloading bursty parses to a hosted Jobs API

For irregular agent workloads, delegating rendering, cross-page merging, and queue scaling to an async API keeps the agent container thin. KolmoPDF provides a REST Jobs API alongside an agent skill (npx skills add komoai2026/kolmopdf-skill) and a local stdio MCP server (@kolmopdf/mcp-server).

Its API handles files up to 300 MB and 800 pages per job:

• POST /api/v1/jobs/parse accepts Idempotency-Key headers for retry safety and returns 202 Accepted with a job ID (up to 3 concurrent jobs per key, extras queued).
• Clients track completion via SSE (GET /api/v1/jobs/{id}/events), webhooks, or polling.
• Options cover table_mode (markdown or image), formula_format (dollar or bracket), enable_cross_page_merge, and layout-preserving translation across nine languages.
• images_as_url=true with enrichment=none returns a plain Markdown file with 30-day image URLs; otherwise (or with default outline,summary sidecars), downloads arrive as a ZIP bundle (PK header) stored for 7 days.

Free accounts include 300 welcome credits and 50 daily check-in credits, while pay-as-you-go bundles cost $3 per 1,000 credits without a subscription (Go, Plus, and Pro plans start at $4.99/month for 3,000 credits). Parsing runs 2 credits per page ($0.006/page PAYG), or 3 credits per page with translation.

Decision rule

Pick self-hosted LightOnOCR-3 when steady daily batch volume keeps a GPU busy, when you need raw 0-1000 bounding boxes for custom UI overlays, or when compliance requires on-premise weights. Pick a hosted async PDF-to-Markdown API when document parses happen in sporadic agent bursts, mix multi-column math with translation, or when maintaining PDF rasterizers and vLLM queues costs more than $0.006 per page.