DEV Community

Cover image for Docling: turn messy PDFs into clean, heading-aware chunks for RAG
M.T.Ramkrushna
M.T.Ramkrushna

Posted on

Docling: turn messy PDFs into clean, heading-aware chunks for RAG

#ai #rag #llm #python

Thursday AI Tool/Workflow. This week's tool: Docling, the open-source document converter started by IBM Research Zurich, now hosted by the LF AI & Data Foundation. It turns PDFs, DOCX, PPTX, HTML and more into clean Markdown or lossless JSON, and it can chunk the result for RAG without throwing away your section headings.

If your RAG pipeline starts with pdftotext or a basic PDF text extractor, you've probably seen what comes out: two-column pages read straight across, tables flattened into a stream of numbers, page headers and footers mixed into body text, and no idea which section a sentence came from. Then you split that text every N characters and wonder why retrieval is weak.

Docling attacks the problem at the source. Here's exactly how to use it.

What Docling gives you

From the official README, the features that matter for RAG:

  • Parsing of many formats, including PDF, DOCX, PPTX, XLSX, HTML, EPUB, images and more
  • Advanced PDF understanding: page layout, reading order, table structure, code, formulas and image classification
  • OCR support for scanned PDFs and images
  • A unified DoclingDocument representation, exportable to Markdown, HTML, DocTags and lossless JSON
  • Local execution, which matters for sensitive or air-gapped data
  • Integrations with LangChain, LlamaIndex, Crew AI and Haystack
  • MIT-licensed codebase (individual models have their own licenses)

1. Install 

pip install docling
Enter fullscreen mode Exit fullscreen mode

Docling needs Python 3.10 or higher (Python 3.9 support was dropped in version 2.70.0). It runs on macOS, Linux and Windows, on both x86_64 and arm64.

2. Convert a document from the CLI

The fastest way to see what Docling does:

docling https://arxiv.org/pdf/2206.01062
Enter fullscreen mode Exit fullscreen mode

This writes a .md file into the current directory. The CLI takes local files, directories or URLs, and a few options are worth knowing:

# Markdown + lossless JSON into ./out
docling ./policies --to md --to json --output ./out

# Scanned documents: OCR is on by default; turn it off for born-digital PDFs to save time
docling report.pdf --no-ocr

# Table structure model in accurate (default) or fast mode
docling report.pdf --table-mode fast

# Ask the CLI for chunks directly (hybrid chunker by default)
docling report.pdf --to chunks --chunks-max-tokens 512
Enter fullscreen mode Exit fullscreen mode

Run docling --help or check the CLI reference for the full list. The default chunk tokenizer for --to chunks is sentence-transformers/all-MiniLM-L6-v2.

3. Convert in Python

The Python API is three lines:

from docling.document_converter import DocumentConverter

converter = DocumentConverter()
result = converter.convert("policies/leave-policy.pdf")  # local path or URL
doc = result.document

print(doc.export_to_markdown())
Enter fullscreen mode Exit fullscreen mode

doc is a DoclingDocument. Besides Markdown you can call export_to_dict(), export_to_text(), export_to_doctags(), or save lossless JSON with doc.save_as_json(path). The JSON keeps structure and provenance that Markdown drops, so it is the better format to store if you might re-chunk later.

4. Tables and layout

Tables are where naive extraction hurts most. Docling runs a table structure model and gives you each table as a real object:

for i, table in enumerate(doc.tables):
    df = table.export_to_dataframe(doc=doc)  # pandas DataFrame
    df.to_csv(f"table-{i + 1}.csv")
Enter fullscreen mode Exit fullscreen mode

In Markdown output, tables come out as Markdown tables instead of loose numbers. If you need to tune the PDF pipeline, pass pipeline options:

from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.document_converter import DocumentConverter, PdfFormatOption

opts = PdfPipelineOptions()
opts.do_ocr = False              # born-digital PDFs don't need OCR
opts.do_table_structure = True   # keep table structure

converter = DocumentConverter(
    format_options={InputFormat.PDF: PdfFormatOption(pipeline_options=opts)}
)
Enter fullscreen mode Exit fullscreen mode

5. Chunk with HybridChunker

Docling has native chunkers that work on the DoclingDocument instead of on flat text. The docs describe HybridChunker as tokenization-aware refinements on top of document-based hierarchical chunking:

  • It starts from the hierarchical chunker (one chunk per document element, with headings and captions attached).
  • It splits chunks that are oversized for your tokenizer.
  • It merges undersized neighbouring chunks that share the same headings (merge_peers, on by default).
  • For tables that span chunks, repeat_table_header (on by default) repeats the header row in each chunk.

The docs stress one thing: the chunker and your embedding model should use the same tokenizer.

from docling.chunking import HybridChunker
from docling_core.transforms.chunker.tokenizer.huggingface import HuggingFaceTokenizer
from transformers import AutoTokenizer

EMBED_MODEL_ID = "sentence-transformers/all-MiniLM-L6-v2"

tokenizer = HuggingFaceTokenizer(
    tokenizer=AutoTokenizer.from_pretrained(EMBED_MODEL_ID),
    max_tokens=256,  # pick to fit your embedding model
)
chunker = HybridChunker(tokenizer=tokenizer)

for chunk in chunker.chunk(dl_doc=doc):
    print(chunk.meta.headings)               # e.g. ["Leave Policy", "Carry-over"]
    text_to_embed = chunker.contextualize(chunk=chunk)
Enter fullscreen mode Exit fullscreen mode

Note contextualize(): it returns the chunk text enriched with metadata such as its headings. The docs say this is typically the text you want to feed the embedding model, while chunk.text is the raw content. If you prefer OpenAI tokenizers, install docling-core[chunking-openai] and use OpenAITokenizer with tiktoken.

6. Example workflow: a folder of policy PDFs into a RAG index

This is an example workflow, not a benchmark. Say HR has a folder of policy PDFs and you want a Q&A bot that cites the right section.

from pathlib import Path

from docling.chunking import HybridChunker
from docling.datamodel.base_models import ConversionStatus
from docling.document_converter import DocumentConverter

converter = DocumentConverter()
chunker = HybridChunker()  # default tokenizer; align it with your embedder in production

pdfs = sorted(Path("policies").glob("*.pdf"))
Path("parsed").mkdir(exist_ok=True)
records = []

for res in converter.convert_all(pdfs, raises_on_error=False):
    if res.status != ConversionStatus.SUCCESS:
        print("needs review:", res.input.file.name, res.status)
        continue

    doc = res.document
    doc.save_as_json(Path("parsed") / f"{res.input.file.stem}.json")  # keep lossless copy

    for i, chunk in enumerate(chunker.chunk(dl_doc=doc)):
        records.append({
            "id": f"{res.input.file.stem}-{i}",
            "text": chunker.contextualize(chunk=chunk),  # embed this
            "source": res.input.file.name,
            "section": " > ".join(chunk.meta.headings or []),
        })

# embed records[*]["text"] and upsert into your vector store
# (pgvector, Qdrant, Milvus...), storing source + section as metadata
Enter fullscreen mode Exit fullscreen mode

What this buys you:

  1. Section headings travel with every chunk. "Carry-over" under "Leave Policy" doesn't get confused with "Carry-over" under "Expense Policy".
  2. Citations are free. Store source and section, and your answer can say "Leave Policy > Carry-over" instead of "page text, somewhere".
  3. Failures are visible. convert_all(..., raises_on_error=False) keeps going, and you log the files that only partially converted instead of silently indexing garbage.
  4. You can re-chunk without re-parsing. The saved JSON is a DoclingDocument, so changing chunk size later doesn't mean running the PDF models again.

Tips before you ship

  • Measure retrieval, not vibes. Build 20 to 50 real questions with known answer sections and check recall@k before and after switching to Docling.
  • Turn OCR off for born-digital PDFs and on for scans. It is a real speed/quality trade-off.
  • Spot-check tables. Open a few Markdown exports next to the original PDF, especially for merged cells.
  • Match tokenizers. Chunk limits only mean something if they're counted with your embedding model's tokenizer.

Garbage in, garbage retrieved. Fix the parsing step and every step after it gets easier.

References

What does your RAG pipeline use to parse PDFs today, and where does it break?

Top comments (0)