PDF to clean Markdown for RAG: chunking-ready text from PDFs, Word and scanned files

Most RAG quality problems I have debugged were not embedding or prompt problems. They were extraction problems: a table flattened into a soup of numbers, a heading glued to the paragraph above it, a scanned appendix that came back empty, so the retriever never had a chance.

This post walks through turning PDFs, Word files and scans into Markdown that is ready to chunk, with a real test run and the actual output, warts included.

Disclosure: I built the Apify actor used below (Document to Markdown). I also list the free DIY options, which are the right choice in plenty of cases.

Why Markdown and not plain text?

Three reasons that matter for retrieval:

1. Structure becomes chunk boundaries. If section titles survive extraction, you can split on them. A chunk that is "6.1 Machine Translation" plus its paragraphs embeds far better than a fixed 1,000-character window that starts mid-sentence and ends in the next section.
2. Tables stay rows and columns. An LLM can read | Transformer (big) | 28.4 | 41.8 |. It cannot reliably read Transformer (big) 28.4 41.8 2.3 1019 after the column headers were lost.
3. Page markers enable citations. If every page starts with <!-- Page 8 -->, each chunk can carry a page number, and your answer can say "see page 8".

The DIY options, honestly

• pdfplumber: Excellent low-level control, including table extraction with tunable settings. You write the logic for headings, reading order and Markdown output yourself, and there is no OCR. Best if you have one document layout and time to tune it.
• MarkItDown (Microsoft, MIT): One pip install and you get Markdown from PDF, DOCX, PPTX, XLSX, HTML and more. Office formats come out very well because they carry real structure. PDFs are mostly extracted as text, and scanned pages need extra setup. My actor uses MarkItDown internally for Office, HTML and EPUB files.
• marker: Uses layout and OCR models and produces the nicest PDF Markdown of the three, including real headings. The trade-off is heavy models, a GPU for reasonable speed on large batches, and licence terms you should read before commercial use.

If you process a few hundred PDFs once and you are happy running Python locally, start with MarkItDown or marker. A hosted API makes sense when you do not want to maintain OCR models and dependencies, when the documents arrive as URLs inside an automation (n8n, Make, an AI agent), or when you need per-page control and billing.

Step 1: Convert a document via the API

The actor takes public URLs (PDF, DOCX, PPTX, XLSX, EPUB, HTML, images). Scanned pages are detected and OCRed automatically, and normal pages are not.

With curl, using the synchronous endpoint that returns the results directly:

curl -X POST "https://api.apify.com/v2/acts/fast_data_asap~document-to-markdown/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "documentUrls": ["https://arxiv.org/pdf/1706.03762"],
    "pages": "1,8,9",
    "ocr": "auto",
    "outputChunks": 1500
  }'

Or in Python with pip install apify-client:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("fast_data_asap/document-to-markdown").call(run_input={
    "documentUrls": ["https://arxiv.org/pdf/1706.03762"],
    "pages": "1,8,9",
})
doc = next(client.dataset(run["defaultDatasetId"]).iterate_items())
markdown = doc["markdown"]

pages is worth knowing about: you only convert (and pay for) the pages you select.

Step 2: What came back (real run)

I ran exactly this input on "Attention Is All You Need" (pages 1, 8 and 9, which contain the abstract and the two results tables). The result:

{
  "status": "success",
  "fileType": "pdf",
  "title": "Attention Is All You Need",
  "pageCount": 15,
  "pagesProcessed": 3,
  "ocrPages": 0,
  "billedPages": 3,
  "wordCount": 1587
}

The conversion itself took 1.8 seconds, the whole run 12 seconds including container start. With outputChunks: 1500 it also returned 11 ready-made chunks, each with pageStart and pageEnd.

The tables come out as Markdown tables. The simple rows are clean:

<!-- Page 8 -->

Table 2: The Transformer achieves better BLEU scores than previous state-of-the-art models ...
|     | Transformer | (big)         |          |      |     | 28.4 41.8   |     | 2.3    | · 1019     |

Here is the honest part. This is a complex academic table with multi-level headers, and the output shows it: some cells are split across columns, a superscript exponent became 1019 instead of 10^19, and a few short prose lines and section titles such as 6.1 Machine Translation were detected as one-row tables. PDFs do not store "this is a heading", so PDF headings come out as plain lines, not ##. Word, PowerPoint and HTML files do keep real headings.

So a heading-based chunker for PDFs has to recognise numbered section titles too, which leads to the next step.

Step 3: Chunk by headings (and keep page numbers)

The built-in outputChunks splits by paragraphs and size. If you want one chunk per section, this snippet handles Markdown headings (DOCX/HTML) and numbered titles (PDF), and tracks the page each chunk starts on:

import re

PAGE = re.compile(r"<!-- Page (\d+) -->")
HEADING = re.compile(r"^(#{1,6} .+|\d+(\.\d+)*\s+[A-Z][^.]{2,60}|Abstract|References)$")

def chunk_by_headings(markdown, max_chars=2000):
    chunks, buf, section, page, start_page = [], [], "Front matter", 1, 1

    def flush():
        text = "\n".join(buf).strip()
        for i in range(0, len(text), max_chars):
            chunks.append({"section": section, "page": start_page, "text": text[i:i + max_chars]})
        buf.clear()

    for line in markdown.splitlines():
        if m := PAGE.match(line.strip()):
            page = int(m.group(1))
            continue
        flat = re.sub(r"[|\s]+", " ", line).strip()  # PDF headings sometimes arrive as 1-row tables
        if HEADING.match(flat):
            flush()
            section, start_page = flat.lstrip("# "), page
        if not buf:
            start_page = page
        buf.append(line)
    flush()
    return chunks

On the test output it produced 11 chunks with these sections: Front matter, Abstract, 6 Results, 6.1 Machine Translation, 6.2 Model Variations and 6.3 English Constituency Parsing. Prepend the section title to each chunk before embedding (f"{c['section']}\n\n{c['text']}") and store page as metadata for citations. In production, also merge tiny chunks (the 6 Results heading alone was a 9-character chunk) into the next one.

What it costs

Pricing is per page: $1.50 per 1,000 pages, plus $5 per 1,000 pages that need OCR. Higher Apify plans get tiered discounts of up to 25%. Failed downloads and documents without readable text are not charged.

• The test run above: 3 text pages = $0.0045.
• A 2,000-page document library where 300 pages are scans: 2,000 × $0.0015 + 300 × $0.005 = $3.00 + $1.50 = $4.50.
• Word files have no fixed pages, so every 3,000 characters of output count as one page.

Summary

• Extract to Markdown, not plain text, and keep page markers.
• Expect PDF tables with complex headers to need cleanup. Simple tables and Office tables convert well.
• Chunk by section, not by a fixed window, and attach section and page as metadata.
• Use pdfplumber, MarkItDown or marker if you want everything local, and a hosted actor if you would rather not run OCR models yourself.

The actor is here if you want to try it on your own files: Document to Markdown on Apify. If you have a PDF that breaks it, send it my way. Those are the most useful bug reports.