From Messy PDFs to Structured JSON: An LLM-Ready Workflow

Most AI pipelines don't fail because of the model.

They fail because of the input.

If you're:

• building RAG pipelines,
• automating invoice or contract intake,
• or feeding documents into an LLM for Q&A,

the PDF is usually the weakest link in the chain.

TL;DR

• A PDF is a layout format, not a data format. Generic text extraction often loses reading order, table structure, and source context.
• Nutrient Data Extraction API turns PDFs, scans, images, and Office files into structured output: structured JSON for schema-defined extraction and downstream systems, or Markdown for AI/search workflows.
• Use Studio to test documents first, compare processing modes, generate schemas, and inspect output before integrating with the API.
• The API supports four processing modes — Text, Structure, Understand, and Agentic — so teams can balance speed, cost, and extraction quality based on document complexity.

The PDF was never built for this

A PDF remembers how a page looks. It doesn't remember what the content is.

That distinction breaks most naive extraction attempts:

• Text is stored in fragments, often out of reading order
• Tables are just aligned text — there's no actual "table" object
• Scanned pages have no text layer at all
• Two-column layouts get merged into a single unreadable stream

Developers usually discover this too late — after a regex-based parser works fine on three test invoices and then falls apart on the fourth, because the vendor used a different template.

Why text extraction alone isn't enough

Getting text out of a PDF is easy. Getting text out with the relationships preserved — which value belongs to which row, which heading owns which paragraph, which number came from which page — is the actual job.

An LLM does not just need more text. It needs structured content that preserves context, order, and source grounding.

Nutrient Data Extraction API: Studio-first, API-next

Nutrient Data Extraction API processes PDFs, scans, images, and Office files, and returns either structured JSON or clean Markdown, with source context such as confidence scores, page references, and coordinates.

Before wiring extraction into an application, you can test documents in Data Extraction API Studio. Upload a file, choose Parse or Extract, compare processing modes, inspect Markdown or JSON output, and review source context before moving to the API — 5,000 free credits, no credit card required.

Parse vs. Extract

The API gives you two request paths, and picking the right one matters:

• Parse returns the full document content as clean Markdown or structured JSON blocks — use it when you need the whole document, structured.
• Extract pulls specific fields you define in a schema — such as invoice number, total amount, or vendor name — use it when you know exactly which fields you need out of a document type.

Most RAG and search use cases run through Parse. Invoice, form, and structured-record workflows usually reach for Extract once the schema is defined.

Markdown vs. spatial JSON

Parse and Extract are about which request you send. Markdown and spatial JSON are about what format the content comes back in — and that choice is independent of Parse vs. Extract; both output formats are available either way.

Markdown is clean, structured content — headings, lists, and tables rendered as Markdown syntax (including real <table> markup for tables). It's positioned as the go-to format for AI and search workflows: RAG ingestion, document Q&A, knowledge base indexing, content migration. It reads like a well-formatted document, not a data structure.

Spatial JSON is the layout-aware format — a typed list of elements, each with its type, its role (heading, body text, footer, and so on), its bounding box, its reading order, and a confidence score. It's the stronger choice when you need element types, bounds, coordinates, and confidence, or when a value has to be traceable back to an exact position on the page — table cell validation, form field mapping, audit trails.

A rule of thumb: if the next step is "feed this to an LLM or a search index," reach for Markdown. If the next step is "validate this value, map it to a database column, or highlight it on the source page," reach for spatial JSON.

They're separate output formats on the same Parse request — check the API docs for the current recommended pattern if a workflow genuinely needs both from one document.

Processing modes: matching depth to the document

Instead of a single one-size-fits-all pass, the API gives you four processing modes:

Mode	When to use it
Text	Born-digital documents where speed and cost matter
Structure	Scans and image-based documents that need OCR and layout
Understand	Complex layouts, tables, handwriting, forms, and higher-quality structure
Agentic	Visually complex or ambiguous documents where deeper reasoning is worth the extra cost — deeper visual reasoning and recovery for documents that need more advanced analysis

You pick the mode per document. A clean digital invoice doesn't need the same processing budget as a scanned, handwritten form.

Schema generation for field extraction

For Extract workflows, you don't have to write a schema from scratch. Schema generator can create a starting schema from a sample document — you then review and refine the fields, types, and instructions before running extraction through the API.

This matters in practice: most teams don't know the exact field list a document type needs until they've looked at a handful of real samples. Starting from a generated schema and refining it is faster and more accurate than guessing upfront.

API example: parsing a document

Here's a minimal Parse request that converts a PDF to Markdown:

curl -X POST https://api.nutrient.io/extraction/parse \
  -H "Authorization: Bearer $NUTRIENT_API_KEY" \
  -F "file=@document.pdf" \
  -F 'instructions={"output":{"format":"markdown"}}'

Swap "format":"markdown" for "format":"json" when you need structured elements with coordinates and confidence instead of clean Markdown — for example, when you need to trace a value back to its exact position on the page, or preserve table cell structure precisely.

A reusable Python script

The curl one-liner is fine for a quick test. For anything you'll run more than once, here's a small Python CLI wrapper around the same endpoint — it takes a PDF path, an output format, and an optional output file:

"""
Extract structured content from a PDF using the Nutrient extraction API.

Setup:
    pip install -r requirements.txt
    Copy .env.example to .env and add your Nutrient API key.

Usage:
    python extract_pdf.py path/to/file.pdf
    python extract_pdf.py path/to/file.pdf --format json
    python extract_pdf.py path/to/file.pdf --output result.md
"""

import argparse
import os
import sys

import requests
from dotenv import load_dotenv

load_dotenv()
API_KEY = os.getenv("NUTRIENT_API_KEY")
URL = "https://api.nutrient.io/extraction/parse"


def extract(pdf_path: str, output_format: str = "markdown") -> dict:
    with open(pdf_path, "rb") as f:
        r = requests.post(
            URL,
            headers={"Authorization": f"Bearer {API_KEY}"},
            files={"file": f},
            data={"instructions": f'{{"output":{{"format":"{output_format}"}}}}'},
            timeout=60,
        )
    r.raise_for_status()
    return r.json()


def main():
    parser = argparse.ArgumentParser(description="Extract content from a PDF via Nutrient")
    parser.add_argument("pdf", help="Path to the PDF file")
    parser.add_argument("--format", choices=["markdown", "json", "text"], default="markdown")
    parser.add_argument("--output", help="Write the extracted content to this file instead of stdout")
    args = parser.parse_args()

    if not API_KEY:
        sys.exit(
            "NUTRIENT_API_KEY not found. Create a .env file next to this script "
            "with the line: NUTRIENT_API_KEY=your_api_key"
        )

    if not os.path.isfile(args.pdf):
        sys.exit(f"File not found: {args.pdf}")

    result = extract(args.pdf, args.format)
    content = result["output"].get(args.format, result["output"])
    if isinstance(content, dict):
        import json
        content = json.dumps(content, indent=2)

    if args.output:
        with open(args.output, "w", encoding="utf-8") as f:
            f.write(content)
        print(f"Saved to {args.output}")
    else:
        print(content)

    credits = result.get("usage", {}).get("data_extraction_credits", {})
    if credits:
        print(f"\n[credits used: {credits.get('cost')}, remaining: {credits.get('remainingCredits')}]", file=sys.stderr)


if __name__ == "__main__":
    main()

Drop this next to a .env file with NUTRIENT_API_KEY=your_api_key, and you have a script you can point at any PDF: python extract_pdf.py invoice.pdf --format json --output invoice.json. It's the same request the curl example makes — just wrapped in something you'll actually reuse.

For schema-defined Extract requests, the exact request shape depends on your schema — check the official API documentation for the current Extract endpoint and payload format, or generate a starting request directly from Studio once your schema is ready.

A JSON response from Parse looks like this — element types, positions, and reading order, not a flat text dump:

{
  "elements": [
    {
      "type": "paragraph",
      "text": "Invoice #2024-0892",
      "boundingBox": { "x": 45, "y": 120, "width": 310, "height": 28 },
      "readingOrder": 0
    },
    {
      "type": "table",
      "boundingBox": { "x": 45, "y": 200, "width": 680, "height": 340 },
      "readingOrder": 1,
      "children": [
        { "type": "tableCell", "text": "Item", "row": 0, "column": 0 },
        { "type": "tableCell", "text": "Amount", "row": 0, "column": 1 }
      ]
    }
  ]
}

Compare that to plain OCR output: "Invoice #2024-0892\nItem Amount\nWidget A $45.00" — with no indication of which value belongs to which column.

What a real response actually looks like

The example above is simplified to make the shape easy to read. Here's what Parse actually returns for a real invoice — anonymized, but otherwise untouched — first as Markdown, then as structured JSON.

Markdown output:

## Invoice

INV-2026-0001 Invoice number

July 3, 2026 Date of issue

## Medium

123 Main Street 94105 San Francisco California USA [phone]

Date due July 17, 2026

## $175.00 USD due July 17, 2026

## Pay online

<table>
 <tr>
  <td>Description</td>
  <td></td>
  <td>Unit price</td>
  <td>Amount</td>
 </tr>
 <tr>
  <td>Content Marketing</td>
  <td>1</td>
  <td>$175.00</td>
  <td>$175.00</td>
 </tr>
 <tr>
  <td></td>
  <td>Subtotal</td>
  <td></td>
  <td>$175.00</td>
 </tr>
 <tr>
  <td></td>
  <td>Total</td>
  <td></td>
  <td>$175.00</td>
 </tr>
 <tr>
  <td></td>
  <td>Amount due</td>
  <td></td>
  <td>$175.00 USD</td>
 </tr>
</table>

## Medium

Bill to

Jamie Chen [e-mail]

Notice the table survives as real HTML <table> markup embedded in the Markdown — not flattened into a line of numbers. That's what makes Markdown output usable for RAG chunking without losing the row/column relationships.

JSON output (trimmed — the real response includes every element, and the table's cell array is shortened here for readability):

{
  "elements": [
    {
      "id": "ad35142f-5426-4995-94e3-56682a3a8c7c",
      "type": "paragraph",
      "role": "SectionHeader",
      "text": "Invoice",
      "confidence": 0.939276,
      "readingOrder": 0,
      "bounds": { "x": 84.03, "y": 92.88, "width": 170.97, "height": 42.12 },
      "page": { "pageNumber": 1, "width": 1700, "height": 2200 }
    },
    {
      "id": "a0d1192d-3d59-4a4b-b10f-16df2233dbbb",
      "type": "paragraph",
      "role": "Text",
      "text": "INV-2026-0001 Invoice number",
      "confidence": 0.904187,
      "readingOrder": 2,
      "bounds": { "x": 82.81, "y": 195.59, "width": 416.19, "height": 23.41 },
      "page": { "pageNumber": 1, "width": 1700, "height": 2200 }
    },
    {
      "id": "7184b67c-468d-42d6-9e00-f0dd10a2a4c3",
      "type": "paragraph",
      "role": "Text",
      "text": "123 Main Street\n94105 San Francisco\nCalifornia\nUSA\n[phone]",
      "confidence": 0.680442,
      "readingOrder": 6,
      "bounds": { "x": 82.46, "y": 394.16, "width": 530.54, "height": 174.84 },
      "page": { "pageNumber": 1, "width": 1700, "height": 2200 }
    },
    {
      "id": "8532e005-9b64-4a04-9745-f1a963a27f4d",
      "type": "table",
      "readingOrder": 13,
      "confidence": 0.928583,
      "rowCount": 5,
      "columnCount": 4,
      "bounds": { "x": 80.54, "y": 825.07, "width": 1537.94, "height": 246.88 },
      "page": { "pageNumber": 1, "width": 1700, "height": 2200 },
      "cells": [
        { "row": 0, "column": 0, "text": "Description", "confidence": 1 },
        { "row": 0, "column": 2, "text": "Unit price", "confidence": 1 },
        { "row": 0, "column": 3, "text": "Amount", "confidence": 1 },
        { "row": 1, "column": 0, "text": "Content Marketing", "confidence": 1 },
        { "row": 1, "column": 3, "text": "$175.00", "confidence": 1 },
        { "row": 4, "column": 1, "text": "Amount due", "confidence": 1 },
        { "row": 4, "column": 3, "text": "$175.00 USD", "confidence": 1 }
      ]
    },
    {
      "id": "c38ff641-a643-4ad5-b9e2-4c370fc6a0b3",
      "type": "paragraph",
      "role": "Text",
      "text": "Jamie Chen\[e-mail]",
      "confidence": 0.686578,
      "readingOrder": 16,
      "bounds": { "x": 695.90, "y": 394.81, "width": 279.40, "height": 65.19 },
      "page": { "pageNumber": 1, "width": 1700, "height": 2200 }
    }
  ],
  "languageDetection": {
    "pages": [
      { "pageNumber": 1, "languages": ["eng"], "textDirection": "lrtb" }
    ]
  }
}

Two things worth pointing out here that don't show up in a toy example:

• role classifies elements, not just types — SectionHeader, Text, and Footer all come back as paragraph type, but role tells you which one is a heading versus body text versus a page footer. That's the field to key off if you're deciding what belongs in a RAG chunk versus what to drop.
• Confidence varies by element, and it's not always near 1.0 — the seller address block above scored 0.68, noticeably lower than the clean, high-contrast text elements around it. That's exactly the kind of value a validation threshold should catch before it moves downstream, not something to assume is correct because the rest of the document extracted cleanly.

The architecture: where extraction sits in the pipeline

It helps to see this as a layer, not a single API call. A production document pipeline usually looks like this:

Document intake (upload / email / scan / hosted URL)
        │
        ▼
Mode selection (Text / Structure / Understand / Agentic)
        │
        ▼
Nutrient Data Extraction API — Parse or Extract
   → structured JSON (elements, coordinates, confidence, reading order)
   → or Markdown (clean, LLM-ready content)
        │
        ▼
Validation layer
   → flag elements below a confidence threshold
   → route flagged elements to human review
        │
        ▼
Downstream systems
   → database / ERP (structured JSON)
   → RAG index / vector store (Markdown)
   → review queue (flagged elements + bounding boxes)

The part teams usually skip is the validation layer. Extraction without a confidence check just moves the trust problem one step downstream — instead of a human misreading a PDF, you get an LLM confidently building on top of a weak extraction. Coordinates and confidence scores make that validation step practical: you set a threshold (say, anything under 85% confidence), route those specific elements to a review queue, and let everything else flow through automatically.

Mode selection also isn't a one-time decision — it's a per-document routing rule. A born-digital invoice from your billing system doesn't need the same processing depth as a photographed, handwritten delivery note. Sending everything through the most expensive mode by default is the most common way teams overpay for extraction.

Pros and cons

Pros

• Source context — confidence scores, page references, and coordinates — comes back alongside the extracted content, not just plain text
• Four processing modes mean you're not paying for AI-heavy extraction on documents that don't need it
• Same API handles PDFs, scans, images, and Office files — no separate OCR pipeline to maintain
• Studio lets you test Parse and Extract, compare modes, and generate a schema before writing any integration code

Cons

• Files are not persistently stored after processing, which is good for compliance but means you handle retention or audit storage yourself if you need it

Best for: teams building document-heavy pipelines — RAG ingestion, invoice/contract automation, compliance workflows — where "we got some text out" isn't good enough and you need to trace values back to their source.

Practical use cases

1. Invoice and statement processing
Route invoices through Structure or Understand mode via Extract with a schema for line items and totals, and flag any amount under your confidence threshold for a human to check before it posts to your accounting system.

2. Contract review pipelines
Use Parse to extract clause-level text with reading order preserved, so a downstream LLM step (clause classification, obligation extraction) works with clauses in the order they actually appear in the contract — not reshuffled by a naive text dump.

3. RAG knowledge base ingestion
Use Markdown output from Parse directly as chunks for your vector store. Because headings, lists, and tables are preserved instead of flattened, retrieval quality improves — the model gets a coherent section, not fragments stitched together.

4. Compliance and audit trails
For regulated workflows (financial statements, medical intake forms, legal filings), the combination of confidence score and page coordinate means extracted values can be traced back to a location in the source document — which is closer to what most audit processes actually require than "we have the data somewhere."

Closing the loop

Structured elements with reading order, confidence, and bounding boxes are what turn "we extracted some text" into "we extracted data we can act on" — whether that means posting a line item to your ERP, indexing a chunk for RAG, or routing a flagged field to a human.

If you're a software or API company looking to explain your product through high-quality educational content (not marketing fluff), feel free to connect with me on LinkedIn.

Key takeaways

• A PDF is a presentation format. Structure has to be extracted, not assumed.
• Source context — confidence scores and page coordinates — is what makes extracted data traceable back to the source, not just readable.
• Match the processing mode (Text / Structure / Understand / Agentic) and request path (Parse / Extract) to the document instead of defaulting to one approach for everything.

How the modes actually perform

The table above tells you when to use each mode. Here's what that difference looks like in numbers — internal benchmark scores (Build #200, July 13, 2026, commit c24e99581f), all on a 0–1 scale where higher is better:

Mode	Overall	NID	NID-S	TEDS	TEDS-S	MHS	MHS-S	Zone-F1
Understand	0.9321	0.9590	0.9567	0.9365	0.9444	0.8671	0.9078	0.6892
Agentic (Vision)	0.9304	0.9572	0.9548	0.9370	0.9444	0.8649	0.9078	0.7006
Structure	0.8923	0.9342	0.9288	0.7386	0.7846	0.8282	0.8884	0.4288
Text	0.8887	0.9263	0.9291	0.7394	0.7902	0.8239	0.8862	N/A

The -S suffix variants (NID-S, TEDS-S, MHS-S) are strict versions of each metric. Zone-F1 measures spatial layout accuracy and doesn't apply to Text mode, which doesn't preserve layout.

What each metric actually measures:

• NID (Normalized Information Distance) — how much information survives from the original document. 0.95+ means almost nothing is lost.
• TEDS (Tree Edit Distance Score) — table structure accuracy specifically: how well row/column relationships in extracted tables match the source. This is the metric that separates Text/Structure from Understand/Agentic the most — TEDS is the hardest metric for complex documents, and scores below 0.80 are common outside these modes.
• MHS (Markdown Heading Score) — whether the heading hierarchy (H1/H2/H3) gets reconstructed correctly.
• Zone-F1 — spatial accuracy: whether bounding-box zones (text regions, tables, figures) are correctly detected and labeled. Higher-depth modes (Understand, Agentic) score noticeably better here.

The practical read: Text and Structure hold up fine on NID (they capture the content), but their TEDS scores drop hard on anything with real tables — 0.74 vs 0.93+ for Understand and Agentic. If your documents are table-heavy, that gap is the one to design around, not overall accuracy.

You can see more details here

FAQs

❓ Can I get structured JSON and Markdown from the same request?
✅ Choose the output format based on the use case — structured JSON when you need layout-aware elements with coordinates and confidence, or Markdown when you need clean content for RAG, search, or document Q&A. Check the API docs for the recommended request pattern if a workflow needs both.

❓ Does this replace OCR, or run on top of it?
✅ It replaces the need for a separate OCR step. Structure and Understand modes handle scanned or image-based documents directly through the same API — you don't build or maintain a separate OCR pipeline before extraction.

❓ How do I know which processing mode to use for a given document?
✅ Start with Text mode for born-digital PDFs (fastest, cheapest). Use Structure mode once scanning or OCR is involved. Reach for Understand mode on complex layouts, handwriting, or documents needing OCR correction. Reserve Agentic mode for visually complex or ambiguous documents where deeper reasoning is worth the extra cost.

❓ What happens to my documents after they're processed?
✅ Files are not persistently stored after processing. If you need retention for audit purposes, that's handled on your side, not the API's.

❓ Is there a free way to test this before integrating it?
✅ Yes. Data Extraction API Studio gives you 5,000 free credits, no credit card required, so you can upload your own documents, compare Parse and Extract, try all four modes, and generate a starting schema before writing any code.

Nutrient Data Extraction API
Turn messy PDFs into structured JSON or Markdown your LLM pipeline can trust — with source context on every request.
→ Get started free