I benchmarked 5 open-source PDF to Markdown tools for RAG, on real documents (2026)

If you feed PDFs to an LLM or a RAG pipeline, the PDF-to-Markdown step is where quality quietly dies. The text comes out fine, then a table collapses, a formula shatters, and a number changes to something that looks right but isn't.

So I benchmarked the open-source converters people actually reach for: MinerU, Marker, Docling, PyMuPDF4LLM, MarkItDown. I ran them on five deliberately hard documents (a research paper, a Japanese financial report, a US legal brief, a page of nothing but data tables, and a 1973 scan) and scored what an LLM actually needs. I also threw a few hosted tools (Mathpix, CloudConvert, pdf2md, and PDF·Markdown App) in for comparison, since sometimes you just want it done without a Python setup.

Two of the nine silently changed numbers. More on that below.

It's all reproducible. Every source document, all nine tools' raw outputs, the answer keys, and the scoring scripts are open:

→ Benchmark repo on GitHub · Full methodology & per-document scores

Pick one in ten seconds

TL;DR

Open source (run it yourself):

• Strongest overall: MinerU (top-end VLM). Excellent tables and formulas, but it deletes footnote text and turns contents lines into fake H1s. The VLM wants a GPU.
• Safe, balanced default: Marker. Consistent, good on formulas and figures; stumbles on multi-column layout and reference lists. (Check its commercial-use license.)

• RAG pipelines: MinerU, Marker and Docling all keep headings.

  • Skip MarkItDown: it emits zero Markdown headings, which kills heading-based chunking.

• Speed on simple PDFs: PyMuPDF4LLM, until a figure breaks the reading order.

Just want it done, no setup:

• Mathpix has the cleanest formulas, but it silently drops numbers in dense tables.
• CloudConvert is fine for a one-off, but collapses dense tables into a single cell.
• pdfmarkdown.app runs in the browser with nothing uploaded, and stayed the most consistent across every document type.

Capability profiles

Here's how the nine compare across five capabilities. A bigger, rounder shape is a better all-rounder; a dent is a weak spot. It's five hard documents, not a representative sample, so read the shapes, not fine rankings. (The full per-document scores are on the original article.)

The open-source tools

MinerU: strongest overall, two blind spots

Highest recovery of any open-source tool here.

• Strong: perfect tables (including a brutal 26-row one) and perfect formulas.
• Deletes footnote text: on the legal brief it kept every footnote marker but dropped the text, so 10 of 11 citations point at nothing.
• Fake headings: on the financial report it promoted about 13 contents lines into H1s.
• Setup: I ran the top-end VLM, a few points above the lighter pipeline on MinerU's own OmniDocBench numbers (about 91 vs 86); it needs a GPU.

MinerU nails the hard tables, including Toyota's dense metrics grid.

Formulas come through intact.

The original brief: its legal citations sit in page-bottom footnotes.

MinerU keeps the superscript markers but drops the footnote text (red box), so the references point at nothing.

Marker: the balanced default

Consistently good, rarely the winner.

• Strong: formulas and figures, and it never fell apart on any document.
• Scrambles multi-column blocks: it interleaved a two-column attorney signature block into one address.
• Breaks reference lists: a Table of Authorities came out with each case name split from its page number.
• Detaches row labels: it stripped the labels off a US-GAAP table, leaving 29,929,992 with nothing to say it's net sales.
• License: commercial-use conditions worth checking before you ship it.

Tables render cleanly, and the math survives too, which are Marker's real strengths.

The original: a two-column block, left column then right column.

Marker interleaves the two columns line by line, scrambling the reading order (red box).

The original US-GAAP table, with every row labelled: net sales, total assets, and so on.

Marker keeps the numbers but drops the row labels (red box): 29,929,992 with nothing to say it is net sales.

Docling: honest, and allergic to math

Clean text, and it fails out loud instead of faking.

• Strong: clean tables, and it marks anything it can't handle as undecoded rather than inventing it. Zero silent errors in the whole test.
• Drops every formula and image.
• Mangled a table of contents into a duplicated table.
• Good fit for text- and table-heavy RAG, poor for anything with math or figures.

Docling's tables are clean and well aligned.

It drops the formulas and images, marking them as undecoded rather than inventing something wrong (red box). Honest, but a gap.

PyMuPDF4LLM: fast, until layout carries meaning

A quick Python library, clean and very fast on simple single-column PDFs.

• Strong: speed and dead-simple integration.
• Reading order breaks at figures: on the paper it collapsed at the Figure 1 boundary and severed whole sections.
• Drops most display formulas.

On the paper, the reading order breaks around the figures and the display formulas and images fall out (red box).

MarkItDown: no headings, anywhere

Handles many formats with a simple API, but it disqualifies itself for RAG.

• Zero Markdown headings on every document I tested; section titles become plain text, so heading-based chunking loses its outline.
• Tables and formulas shatter.
• Drops images.
• No OCR: scans need an external OCR or LLM you wire in yourself.

Section titles come out as plain text: not a single Markdown heading in the whole file (red box), so the outline is gone.

Tables and formulas break down as well.

The hosted & browser tools

These need no Python setup, which is the whole appeal when you just want a file converted. The fidelity trade-offs are real, though, especially the silent ones.

Mathpix: beautiful output, with a knife in it

The single cleanest result in the test, with a perfect score on the paper. But that clean output hides two silent errors:

• Dropped dividends: on the financial report it kept only the parenthetical interim, so an AI reads 100 where the real number is 220.
• Swapped a total: on the data tables it replaced 87,416 with 100.0%.
• Both look completely normal. Cloud, paid, with a roughly 10-page free tier.

On the research paper, Mathpix is the cleanest of all nine: formulas and figures come through beautifully.

The original dividend row: the annual figure (220, 240, 260, 300, 375) with the interim dividend in parentheses.

Mathpix keeps only the parenthetical figure (red box): 100, 105, 120, and so on. An AI now reads 100 as the dividend when the real number is 220. The table looks perfectly normal.

CloudConvert: valid tables that are secretly unusable

Emits proper Markdown tables, which sounds good, until a dense one.

• Strong: easy, no install, many formats.
• Crams rows into one cell: the Markdown is valid but useless as data.
• Formulas mangled, figures dropped.

On dense tables it crams several rows into a single cell (red box): valid Markdown, but you can't tell which number belongs to which row.

Formulas come out badly mangled.

And figures are dropped entirely.

pdf2md: the one that quietly cuts your numbers off

The scariest visible failure.

• Truncates numbers at the thousands separator (458,614 becomes 458,). At least the dangling comma is greppable.
• No real tables, every formula shatters, images dropped.
• Invents headings: it promoted a copyright notice and formula fragments into 30+ headings.
• One genuine silent swap on the financial report, on top of the visible damage.

The original land-use data table.

pdf2md: no table at all, and numbers cut off at the thousands separator (red box), so 546,030 and 87,416 lose their tails.

The Attention formula shattered across headings and code blocks: the fraction is gone.

pdfmarkdown.app: mine, including where it loses

I'll be straight, since it's mine.

• Strong: top tier across all five documents with zero silent errors. It kept Toyota's minus signs, bracketed counts and dividends, got all five reading-order seams right on the legal brief, and produced the most complete figure crops of anything I tested.
• Where it loses: MinerU and Mathpix edged it on the pure paper; it kept the scan's cover as an image but didn't OCR the cursive title; and on the nastiest multi-level tables (the Attention paper's Table 4) it crammed parser rows into stacked cells, where MinerU reads the layout better.
• It's a browser tool, not a scriptable library, so for automated batch pipelines an open-source option fits better.

The Toyota table comes through with the minus signs and bracketed temp-worker counts intact, where Mathpix and pdf2md broke.

Figure 2 comes out whole, both the Scaled Dot-Product and Multi-Head Attention panels with their caption, and the equation renders as clean LaTeX.

The original: Table 4 from the Attention paper, where each parser is its own row.

pdfmarkdown.app on the same table: the four cited-parser rows get crammed into single stacked cells (red box). The values survive but the row structure blurs, and this is the kind of dense table where MinerU is sharper.

How it was scored, and the raw data

Two axes:

• Recovery score: the share of a fixed per-document checklist that came through correctly.
• Silent-error count: plausible-but-wrong values, kept separate and never averaged in.

I graded one criterion at a time across all nine tools, checked it by hand against the source PDFs, and ran an adversarial second pass. The full writeup, with the methodology, the capability-score derivation, and the reproducible test set and raw outputs, lives on the original article:

→ Best PDF to Markdown Tools in 2026: 9 Converters Tested on Real Documents

I'm Jerome, builder of pdfmarkdown.app. I included direct competitors and tried to credit each fairly. If you think I got a call wrong, tell me.