DEV Community: Kevin Meneses González

5 Best Free AI Courses in 2026 (With Certificates)

Kevin Meneses González — Sat, 01 Aug 2026 11:22:26 +0000

Most people assume learning AI seriously means paying for a bootcamp or a $200 certification program.

That's not true anymore.

Open any search for "AI courses" and you'll land on the same problem every time: dozens of tabs, half of them paywalled, half of them outdated, and no clear sense of which one actually matches your level. You start an advanced course with zero foundations. You finish a beginner course you'd already outgrown. You bookmark five links and never open them again.

That's not a knowledge problem.

It's a filtering problem.

Most "best AI courses" roundups make it worse. They repeat the same generic Coursera or edX links, list them in no particular order, and never tell you whether a course assumes you can code or whether it teaches anything you couldn't already find in the docs for free.

The real problem isn't finding a course. It's finding the right one for where you actually are, in the right order.

That's what this list solves. Five free courses, all with certificates, ordered from complete beginner to advanced developer.

If you're:

starting from zero and just want to understand what AI actually is,
a developer wanting to build with LLMs and tools like Claude,
or already technical and looking to go deeper into Transformers and NLP,

there's a course below built for exactly that.

1. Anthropic Academy - Claude 101

Anthropic is the company behind Claude, one of the most capable AI models available today, with a rapidly growing set of tools around it.

Claude 101 is the foundational course inside Anthropic Academy. It covers what Claude actually is, how to start a conversation, and how to write better prompts to get more reliable results.

It also goes further than most beginner courses by introducing Projects, Skills, and MCP (Model Context Protocol), which is how Claude connects to external tools and data sources to extend what it can do.

The course includes a certificate you can add to LinkedIn or your resume.

→ Start Claude 101 on Anthropic Academy

Pros

Covers both fundamentals and practical prompt optimization
Introduces MCP, a concept most beginner courses skip entirely
Certificate included at no cost

Cons

Focused specifically on the Claude ecosystem, not model-agnostic

Best for: anyone starting from zero who wants a foundation built around a real, production-grade AI tool instead of abstract theory.

Anthropic Academy also offers a follow-up course on Claude Code, aimed at developers who want to use AI for programming tasks. Worth checking once you finish Claude 101.

2. AI For Everyone - Andrew Ng

If there's one instructor almost universally recommended for AI fundamentals, it's Andrew Ng.

AI For Everyone is a beginner-level course, roughly 7 hours long, that covers what AI actually is, the terminology you'll keep running into, and the practical differences between AI models. It also walks through small projects so the concepts don't stay purely theoretical.

Like the first course on this list, it's free and includes a certificate.

→ Start AI For Everyone on Coursera

Pros

Taught by one of the most recognized instructors in the field
Clear, jargon-light explanations of AI terminology
Includes practical mini-projects, not just lectures

Cons

Intentionally high-level. Not for anyone wanting technical depth

Best for: complete beginners who want a solid conceptual map before touching any code.

3. Hugging Face - NLP Course

Hugging Face functions as the GitHub of AI models. If you're building anything involving language models, you'll end up there eventually.

This course is more technical and assumes you already have some AI foundation. It covers Transformers, NLP fundamentals, tokenization, and how to fine-tune and optimize models so they work well with your own data.

→ Start the NLP Course on Hugging Face

Pros

Deep, hands-on coverage of Transformers and tokenization
Directly applicable to real model fine-tuning work
Free, with strong community support around it

Cons

Not beginner-friendly. Requires prior AI/ML context

Best for: developers ready to move from "using AI tools" to actually building and customizing models.

4. Google - Generative AI Learning Path

Google's ecosystem includes Gemini, one of the leading models alongside Claude and ChatGPT.

This learning path covers the fundamentals of generative AI, how Google's infrastructure supports these models, and how to build applications ranging from image generation to voice. It also explains what an LLM actually is and how to make use of the resources Google makes available to developers.

→ Start the Generative AI Learning Path on Google Skills

Pros

Covers multiple modalities: text, image, and voice generation
Backed by Google's own infrastructure and documentation
Multiple course tracks depending on your interest

Cons

Broader scope means less depth per topic compared to Hugging Face's course

Best for: developers who want to understand the infrastructure side of generative AI, not just the model layer.

5. ChatGPT - Building Agents and Workflows

The last course on this list focuses on something increasingly relevant in 2026: agents and workflow automation.

This free course, built around the ChatGPT ecosystem, takes about 90 minutes and covers how to build agents and automated workflows to handle repetitive tasks and improve your daily productivity.

→ Start Agents and Workflows on OpenAI Academy

Pros

Short and focused. Around 90 minutes total
Directly applicable to automating real tasks
Free, no certificate paywall

Cons

Shorter format means less depth than the other courses on this list

Best for: anyone who already understands AI basics and wants to start automating tasks with agents right away.

How to Choose Your Path

You don't need all five. Pick based on where you are:

Starting from zero? Anthropic Academy's Claude 101, then Andrew Ng's course.
Already technical, want to build models? Hugging Face's NLP course.
Interested in the infrastructure behind generative AI? Google's learning path.
Want to automate tasks right now? The ChatGPT agents course.

Want a personalized AI roadmap instead of a generic course list?
I offer 1:1 consulting sessions based on your specific goals and current AI setup.
→ Book a consultation

Key Takeaways

The best AI courses in 2026 are free and include certificates, no need to pay for a bootcamp to start.
Level matters more than course popularity. Match the course to where you actually are, not where you want to be.
MCP, agents, and workflow automation are now core topics, not optional extras, for anyone learning AI in 2026.

FAQs

❓ Are there really free AI courses with certificates in 2026?
✅ Yes. Anthropic Academy, Andrew Ng's AI For Everyone, and Google's Generative AI Learning Path all offer free certificates you can add to LinkedIn or your resume.

❓ What's the best AI course for complete beginners?
✅ Start with Anthropic Academy's Claude 101 or Andrew Ng's AI For Everyone. Both are designed for zero prior knowledge and take just a few hours to complete.

❓ Do I need to know how to code to take these courses?
✅ No. Courses 1, 2, 4, and 5 on this list require no coding background. Only Hugging Face's NLP course assumes prior technical experience.

Are you a software or API company that needs technical content, not marketing fluff?
I write SEO-optimized articles, tutorials, and video content that actually explain your product to developers.
→ Get in touch on LinkedIn

Want more AI and fintech content like this?
→ Explore more articles

Looking for technical content for your company? I can help — LinkedIn · kevinmenesesgonzalez@gmail.com

How I Built a Portfolio Risk & Return Tracker with EODHD

Kevin Meneses González — Sat, 25 Jul 2026 10:33:40 +0000

Most retail investors track their portfolio in a spreadsheet.

Some upgrade to a free app that shows total value and daily change. That's it.

Neither tells you the numbers that actually matter: your annualized volatility, your Sharpe ratio, your beta against the market, your maximum drawdown, how correlated your positions really are with each other.

Institutional investors have had this for decades. Retail investors get a pie chart.

If you're:

managing your own portfolio across multiple brokers,
building a fintech side project,
or just tired of guessing how risky your holdings actually are,

this matters.

The spreadsheet problem

Excel is where most portfolios go to die.

You start with a clean sheet. One column per position, one row per transaction. It works — for about three months.

Then you add a new broker account. Then you start tracking dividends separately. Then a formula breaks because you inserted a row in the wrong place, and you don't notice until your total is off by 4%.

Spreadsheets don't calculate volatility, beta, or drawdown by default. You'd have to build those formulas yourself, pull historical prices manually, and keep them updated — forever, by hand.

Most people don't. So they end up managing a portfolio without knowing its actual risk profile.

The paid-tool problem

The other option is a paid portfolio tracker or a Bloomberg-style terminal.

These solve the calculation problem. They don't solve the trust problem.

Your position sizes, tickers, and entry prices sit on a third-party server.
You pay a monthly fee for metrics that are, mathematically, not complicated.
Many of these tools are optimized to upsell you into their brokerage or premium tier, not to give you a neutral view of your risk.

Sharpe ratio, beta, drawdown, correlation — these aren't proprietary black-box models. They're well-defined formulas. Paying $30/month to have someone else run them on your data, on their servers, is a trade-off a lot of people make without questioning it.

The real problem isn't a lack of financial data. It's a lack of a tool that computes real risk metrics, locally, without asking you to hand your portfolio to a third party.

Vault: a portfolio tracker that runs in your browser

I built Vault, an open source portfolio tracker with React 19 and Vite that uses the EODHD API to pull real market data and calculate six metrics that any serious investor should be tracking:

Returns — portfolio performance vs. S&P 500, indexed and per-position
Diversification — position weights and concentration (Herfindahl-Hirschman Index)
Risk — annualized volatility, Sharpe ratio, beta vs. market, 95% daily VaR
Drawdown — underwater curve and maximum historical drawdown
Correlations — Pearson correlation matrix between your holdings
Sector exposure — GICS sector breakdown from EODHD fundamentals

No backend. No account. No server storing your positions.

Everything runs in the browser, and your portfolio is saved in localStorage — on your machine, not on mine.

A quick note on EODHD

I've used EODHD across several projects before this one, mainly because of three things:

One API key covers historical prices, fundamentals, and search — no juggling three different providers
The free tier is generous enough to actually build and test a real project on it
Response format is consistent across endpoints, which cuts down on glue code

If you want to follow along or fork Vault, you'll need a key — get one free here.

How the calculations actually work

All the math lives in a single file: src/utils/finance.js. No external statistics libraries — every formula is implemented from scratch.

Time-weighted returns

Each day is weighted by the previous day's closing value. This matters more than it sounds: without it, adding a new position mid-month distorts your daily portfolio return.

Volatility and Sharpe ratio

Standard deviation of daily returns, annualized with √252, then converted to a Sharpe ratio against a configurable risk-free rate.

Here's the core of it, simplified to Python for clarity:

import numpy as np

def annualized_volatility(daily_returns):
    return np.std(daily_returns) * np.sqrt(252)

def sharpe_ratio(daily_returns, risk_free_rate=0.02):
    excess_returns = np.array(daily_returns) - (risk_free_rate / 252)
    return (np.mean(excess_returns) / np.std(daily_returns)) * np.sqrt(252)

Beta

Covariance of portfolio returns against the S&P 500 (GSPC.INDX on EODHD), divided by the benchmark's variance.

Drawdown

Percentage drop from the running peak of a growth-indexed curve (base 100). This is the number that tells you what it would have actually felt like to hold this portfolio through its worst stretch.

Correlation matrix

Pairwise Pearson coefficient between assets, aligned by date. This is the metric most retail portfolios get wrong without knowing it — five stocks can feel diversified and still move together 90% of the time.

Concentration (HHI)

Herfindahl-Hirschman Index applied to position weights. One number that tells you how concentrated you actually are, beyond "I have 12 positions so I'm diversified."

From here, the same building blocks extend into:

automated risk alerts
portfolio rebalancing scripts
backtesting engines

What it looks like in practice

You add a position, EODHD's Search + EOD endpoints autocomplete the ticker and the closing price on your purchase date.

From there, Vault renders your returns curve against the S&P 500, your correlation heatmap, and your sector exposure pulled straight from fundamentals data.

No manual formula-building. No re-entering prices when a position updates.

Key takeaways

Portfolio risk metrics (Sharpe, beta, drawdown, correlation) aren't proprietary — they're standard formulas most tools just gatekeep behind a paywall
Running them client-side means your positions never leave your browser
EODHD's Search, EOD, and Fundamentals endpoints cover everything needed to build this without stitching together multiple data providers

Try it yourself

git clone https://github.com/Kevinelectronics/portafoliotracker.git
cd portafoliotracker
npm install

Add your EODHD key to .env, run npm run dev, and add your first position.

Get your free EODHD API key
Vault runs entirely on EODHD's Search, EOD, and Fundamentals endpoints — no backend, no paywall on the metrics.
→ Get your EODHD API key

If you're a software or API company looking to explain your product through high-quality educational content — not marketing fluff — feel free to connect with me on LinkedIn.

Need technical content like this for your product?
→ kevinmeneses.com

Looking for technical content for your company? I can help — LinkedIn · kevinmenesesgonzalez@gmail.com

How to Build a RAG Pipeline from PDFs Using Python

Kevin Meneses González — Fri, 24 Jul 2026 14:07:41 +0000

How to Build a RAG Pipeline from PDFs Using Python

Most RAG pipelines don't fail at retrieval. They don't fail at the model either.

They fail at ingestion — the moment a messy PDF gets dumped straight into a text splitter and comes out the other side as noise.

TL;DR

Feeding raw PDF text into a chunker is where most RAG pipelines quietly break: tables split mid-row, headings disappear, chunks lose their context.
The fix isn't a better chunking algorithm. It's converting the PDF into clean, structured Markdown before you chunk anything.
Nutrient's Python SDK (nutrient_dws) turns a PDF into Markdown in one call, preserving headings, lists, and tables — so your chunker can split on structure instead of guessing at character counts.
Below: a full Python walkthrough — PDF → Markdown → heading-aware chunks → ready for embeddings.

What actually breaks in a naive RAG pipeline

The typical first version of a RAG pipeline looks like this: extract raw text with a generic PDF library, split it every N characters, embed each chunk, done.

It works on a clean text file. It falls apart on a real PDF:

A table gets flattened into a single line of numbers with no column boundaries — the chunk that gets retrieved is unreadable
A heading ends up alone at the bottom of one chunk, with its content starting the next one — retrieval finds the content but the model never sees what section it belongs to
Multi-column layouts get merged out of order, so a chunk contains half of one paragraph and half of an unrelated one

None of this shows up in a demo with three test PDFs. It shows up in production, on document #47, and it looks like a "retrieval quality" problem when it's actually an ingestion problem.

The fix: structure before chunking

A RAG pipeline that holds up in production treats ingestion as its own stage — not a one-liner before the real work starts.

The order that actually works is: extract structure, then chunk on structure, then embed. Chunk boundaries should follow headings and paragraph breaks the document already has, not an arbitrary character count that cuts through them.

Architecture

PDF
 │
 ▼
Python SDK (nutrient_dws) — mode="text", output_format="markdown"
 │
 ▼
Clean Markdown (headings, lists, tables preserved)
 │
 ▼
Heading-aware chunker (splits on headings, then paragraph breaks)
 │
 ▼
Embedding model
 │
 ▼
Vector store

The chunker is the piece most tutorials skip past. It's also the piece that determines whether your retrieved chunks make sense to the model or arrive as disconnected fragments.

Implementation: PDF to Markdown with the Python SDK

Nutrient's official Python client, nutrient_dws, wraps document parsing in a simple async call. For born-digital PDFs, mode="text" is the fastest and cheapest path — 1 credit per page:

import asyncio
from nutrient_dws import NutrientClient

client = NutrientClient(
    api_key="your_processor_key",
    extract_api_key="your_extract_key",
)


async def pdf_to_markdown(path: str) -> str:
    response = await client.parse(path, mode="text", output_format="markdown")
    return response["output"]["markdown"]

That's the entire extraction step. The output preserves headings, lists, and tables as real Markdown syntax — not a flat text dump:

# Employee Handbook

## Time Off Policy

Full-time employees accrue 1.5 days of PTO per month...

| Tenure | Annual PTO |
|--------|-----------|
| 0–2 years | 15 days |
| 3–5 years | 20 days |

If your source documents are scanned rather than born-digital, switch to mode="structure" so OCR runs before the Markdown conversion.

Implementation: chunking on structure, not character count

Once you have Markdown, split on headings first — so no chunk ever straddles a section boundary — then fall back to paragraph breaks for sections that are still too long:

import re


def chunk_markdown(markdown: str, max_chars: int = 1500) -> list[str]:
    # Split on Markdown headings so a chunk never crosses a section boundary
    sections = re.split(r"(?=^#{1,3} )", markdown, flags=re.MULTILINE)

    chunks = []
    for section in sections:
        if not section.strip():
            continue
        if len(section) <= max_chars:
            chunks.append(section.strip())
            continue
        # Long section: split further on paragraph breaks
        paragraphs = section.split("\n\n")
        buffer = ""
        for para in paragraphs:
            if len(buffer) + len(para) > max_chars and buffer:
                chunks.append(buffer.strip())
                buffer = ""
            buffer += para + "\n\n"
        if buffer.strip():
            chunks.append(buffer.strip())
    return chunks

Putting both pieces together:

async def main():
    markdown = await pdf_to_markdown("company_handbook.pdf")
    chunks = chunk_markdown(markdown)
    print(f"Generated {len(chunks)} chunks")
    # Next: embed each chunk and upsert into your vector store of choice


if __name__ == "__main__":
    asyncio.run(main())

Every chunk that comes out of this now maps to a real section of the document — a heading and the content underneath it, or a table with its rows intact — instead of an arbitrary character window.

When you need more than Markdown

Markdown chunking covers most RAG ingestion. But some documents need the model to answer questions that depend on exact table structure — "what was the Q3 number in the third row" — where flattening a table into Markdown text still loses precision.

For those cases, request spatial JSON output instead of Markdown from the same parse() call, or use mode="understand" for documents with complex layouts, key-value regions, or handwriting. Spatial output gives you row/column-indexed table cells with bounding boxes and confidence scores — useful when you need to validate or cite the exact source location, not just retrieve a paragraph.

Pros and cons

Pros

One SDK call replaces a generic PDF parser plus custom table/heading recovery logic
Markdown output is chunker-friendly out of the box — headings and tables survive
Switching to mode="structure" handles scanned PDFs without a separate OCR step
Same client can return spatial JSON for documents needing exact structure, not just clean text

Cons

Markdown and spatial JSON are separate outputs from the same request — decide upfront which one a document needs
mode="text" assumes a born-digital PDF; scanned documents need mode="structure", which costs more and runs slower
The chunker above is heading-aware but simple — documents with deeply nested headings may need a more sophisticated splitter

Best for: teams building RAG pipelines from real-world PDFs — internal knowledge bases, policy documents, technical manuals — where retrieval quality depends on chunks that actually correspond to a coherent section of the source document.

Key takeaways

Ingestion, not retrieval, is where most RAG pipelines from PDFs actually break.
Converting to structured Markdown before chunking means your splitter works with real section boundaries instead of guessing at character counts.
nutrient_dws's parse() call handles this in one step for born-digital PDFs, with mode="structure" as the fallback for scans.

FAQs

❓ Do I need a separate OCR step before this pipeline?
✅ No. Switching mode="text" to mode="structure" runs OCR as part of the same parse() call for scanned or image-based PDFs — no separate pipeline to maintain.

❓ Why not just chunk by a fixed character count?
✅ Fixed-length chunking ignores document structure — a chunk can start mid-table or split a heading from its content, which hurts both retrieval relevance and what the model can make sense of once a chunk is retrieved.

❓ Can I get both Markdown and spatial JSON from one request?
✅ No — they're separate output formats on the same parse() call. If a document needs both (Markdown for RAG, spatial JSON for exact table validation), send two requests.

❓ What if my documents have deeply nested headings or unusual structure?
✅ The chunker in this walkthrough handles up to three heading levels (#, ##, ###). For more complex documents, you can extend the regex or add a maximum nesting depth before falling back to paragraph-level splitting.

❓ Is mode="text" accurate enough for documents with some tables?
✅ For simple tables in born-digital PDFs, yes — they convert to Markdown table syntax correctly. For scanned tables or documents where table accuracy is critical, use mode="structure" or mode="understand" instead.

Nutrient Data Extraction API
Turn PDFs into clean Markdown or spatial JSON — one Python SDK, born-digital or scanned.
→ Get started free

Kevin Meneses González
Technical content for fintech and API companies — articles, tutorials, and video.

OCR vs AI Document Extraction: What's the Difference

Kevin Meneses González — Wed, 22 Jul 2026 10:32:27 +0000

"Which OCR should I use?" is usually the wrong question.

If you're:

automating invoice or form processing,
building a RAG pipeline from PDFs,
or setting up a compliance workflow that needs traceable data,

the question that actually matters is: do you need text, or do you need structure?

The goal isn't just to make a document readable. The goal is to make document data usable, reviewable, and ready for downstream systems.

TL;DR

OCR answers "what does this text say?" — it converts pixels into characters.

AI document extraction answers "what is this content, where is it on the page, and how does it relate to the rest of the document?" — it preserves structure like tables, fields, reading order, and source context.

Nutrient Data Extraction API exposes this as four processing modes — Text, Structure, Understand, and Agentic — so you can match extraction depth to document complexity.

Use Parse when you need full document structure as Markdown or spatial JSON. Use Extract when you need specific schema-defined fields returned as structured JSON with source context for review.

Below: the same document requested in two output formats, so you can see exactly where plain text extraction stops and structure begins.

Why "OCR" became the wrong catch-all term

Most people use "OCR" to mean "get text out of an image." That's technically correct and practically misleading.

Plain OCR usually gives you recognized text, sometimes with word or line positions. But it does not reliably preserve higher-level structure like tables, field relationships, heading hierarchy, or reading order across complex layouts. Some OCR engines return positions or basic layout hints, but that's not the same as understanding how a page is organized.

Then teams plug that output into a downstream system expecting structure — and discover the table became "Invoice #2024-0892\nItem Amount\nWidget A $45.00", with no indication of which value belongs to which column.

That's not an OCR bug. It's OCR doing exactly what it was built to do.

The real difference: reading vs understanding

OCR reads. It converts pixels to characters, in roughly left-to-right, top-to-bottom order, and stops there.

Document understanding — the job behind AI-powered document extraction — analyzes layout. It detects tables and preserves rows and columns. It recognizes reading order across multi-column pages. It classifies elements (paragraph, heading, table, key-value region) instead of just returning a wall of text.

The distinction that matters in practice: plain text extraction gives you the words. Structured extraction gives you the words with layout, relationships, and context

Four modes, one spectrum

Nutrient Data Extraction API makes this spectrum concrete with four processing modes, all reachable through the same API:

Capability	Text	Structure	Understand	Agentic
Text extraction	Yes	Yes	Yes	Yes
Table structure	No	Yes	Yes	Yes
Key-value regions	No	Yes	Yes	Yes
Handwriting	No	Limited	Yes	Yes
Formulas	No	No	Yes, as LaTeX	Yes, as LaTeX
Reading order	Basic	Layout-aware	Layout-aware	Layout-aware, deeper reasoning
Best fit for	Fast Markdown/text extraction for born-digital documents	Spatial JSON and OCR-backed layout for scans and image-based documents	Complex layouts, tables, handwriting, formulas, and richer structure	The most complex documents that need deeper visual reasoning and recovery
Relative cost/speed	Fastest, cheapest	Moderate	Higher	Highest, slowest

Text mode — Fast text extraction for born-digital documents.

Structure mode — OCR-backed structure extraction for scans and image-based documents.

Understand mode — ICR for complex layouts, handwriting, formulas, OCR correction, and richer document structure.

Agentic mode — VLM-enhanced ICR for deeper visual reasoning, semantic understanding, and recovery.

Under the hood, components like OCR and ICR (intelligent character recognition) do the character-level work, but the mode you choose is what actually changes what comes back.

Studio-first, API-next

Before deciding which mode fits a document, Data Extraction API Studio lets you upload a file, run it through Parse, compare modes side by side, and inspect the Markdown or JSON output directly in your browser — 5,000 free credits, no credit card required. It's the fastest way to see the difference on your own documents before writing any integration code.

→ Get started free

Parse vs. Extract, and Markdown vs. JSON

There are two separate choices that shape what comes back from the API, and it's worth naming both before writing any code: what job you want the API to perform, and what output format your workflow needs.

Use Parse when you want the full document structure. The Markdown response is clean, readable content for AI, RAG, search, and document Q&A. It is easier to work with than raw OCR text, but it is not the best format when you need layout-aware elements, coordinates, confidence, and table cell relationships.

Use Extract when you need specific schema-defined fields — invoice number, vendor, total, line items, contract dates, form values — returned as structured JSON with source context for review. Studio's schema generator can scaffold a starting schema from a sample document, which you can review and refine before running extraction.

The processing mode (Text/Structure/Understand/Agentic) determines how deeply the document is analyzed. The output format determines how the result is handed back to you.

Implementation: the same document, two output formats

Here's the same intake form requested in two output formats — Markdown for clean, readable content, and JSON for layout-aware structure — using the REST API:

curl -X POST https://api.nutrient.io/extraction/parse \
  -H "Authorization: Bearer $NUTRIENT_API_KEY" \
  -F "file=@intake_form.pdf" \
  -F 'instructions={"output":{"format":"markdown"}}'

curl -X POST https://api.nutrient.io/extraction/parse \
  -H "Authorization: Bearer $NUTRIENT_API_KEY" \
  -F "file=@intake_form.pdf" \
  -F 'instructions={"output":{"format":"json"}}'

Same file, same endpoint. The processing mode is set as a separate request parameter alongside the output format — check the API documentation for the current field name and accepted values, or set it directly in Studio and copy the generated request.

The Markdown response is a clean, flat rendering of the content — accurate, but with table rows collapsed into plain text. A simplified spatial JSON response might look like this:

{
  "elements": [
    {
      "type": "table",
      "boundingBox": { "x": 45, "y": 200, "width": 680, "height": 340 },
      "readingOrder": 1,
      "children": [
        { "type": "tableCell", "text": "Item", "row": 0, "column": 0 },
        { "type": "tableCell", "text": "Amount", "row": 0, "column": 1 }
      ]
    }
  ]
}

That nesting is the entire difference. Plain text extraction never produces it — not because it's less accurate, but because table structure isn't something character recognition alone is designed to detect.

Scaling up: processing 100 invoices

A single curl call is enough to see the difference between output formats. It's not enough to run a real batch. Here's a simple Python script that walks a folder of invoices, sends each one through the API, and saves the results — the kind of script you'd actually point at 100 files:

"""
Batch-process a folder of invoices through the Nutrient Data Extraction API.

Usage:
    python batch_extract.py ./invoices --format json --out ./results
"""

import argparse
import json
import time
from pathlib import Path
# Note:
# The response field names below (for example "output" and
# "usage.data_extraction_credits") are illustrative.
# Verify the current response schema in the official API
# documentation before using this script in production.

import requests

API_KEY = "your_api_key"
URL = "https://api.nutrient.io/extraction/parse"


def extract_one(pdf_path: Path, output_format: str) -> dict:
    with open(pdf_path, "rb") as f:
        response = requests.post(
            URL,
            headers={"Authorization": f"Bearer {API_KEY}"},
            files={"file": f},
            data={"instructions": f'{{"output":{{"format":"{output_format}"}}}}'},
            timeout=60,
        )
    response.raise_for_status()
    return response.json()


def main():
    parser = argparse.ArgumentParser(description="Batch extract invoices with Nutrient")
    parser.add_argument("folder", help="Folder containing invoice PDFs")
    parser.add_argument("--format", choices=["markdown", "json"], default="json")
    parser.add_argument("--out", default="./results", help="Folder to write results to")
    args = parser.parse_args()

    input_dir = Path(args.folder)
    output_dir = Path(args.out)
    output_dir.mkdir(parents=True, exist_ok=True)

    pdf_files = sorted(input_dir.glob("*.pdf"))
    print(f"Found {len(pdf_files)} invoices")

    failed = []
    total_credits = 0

    for i, pdf_path in enumerate(pdf_files, start=1):
        print(f"[{i}/{len(pdf_files)}] {pdf_path.name}")
        try:
            result = extract_one(pdf_path, args.format)
        except requests.HTTPError as e:
            print(f"  failed: {e}")
            failed.append(pdf_path.name)
            continue

        # Note: field names below (output, usage.data_extraction_credits) should be
        # confirmed against the current API reference before running this in production.
        out_path = output_dir / f"{pdf_path.stem}.{('json' if args.format == 'json' else 'md')}"
        content = result["output"].get(args.format, result["output"])
        if isinstance(content, dict):
            content = json.dumps(content, indent=2)
        out_path.write_text(content, encoding="utf-8")

        credits = result.get("usage", {}).get("data_extraction_credits", {})
        total_credits += credits.get("cost", 0)

        # Simple pacing to stay under rate limits on large batches
        time.sleep(0.2)

    print(f"\nDone. {len(pdf_files) - len(failed)} succeeded, {len(failed)} failed.")
    print(f"Total credits used: {total_credits}")
    if failed:
        print("Failed files:", ", ".join(failed))


if __name__ == "__main__":
    main()

Run it: python batch_extract.py ./invoices --format json --out ./results.

A few things worth calling out for a batch like this:

Failures are tracked, not fatal. One malformed PDF in a folder of 100 shouldn't kill the run — the script logs it and keeps going, then prints the failed filenames at the end so you can retry just those.
Credit usage is summed across the batch, so you know the actual cost of processing 100 invoices before you scale to 1,000.
The time.sleep(0.2) is a simple pacing guard, not a rate-limit implementation — for larger batches or production use, check the API docs for current rate limits and consider proper backoff instead of a fixed delay.
Structure or Understand mode is usually the better starting point for real invoices, especially when you need tables, line items, and totals

When each one is the wrong tool

Text mode is the wrong tool when: you need to reconstruct a table, route a form field to the correct database column, or trust that "Total" and "$4,250" are actually linked. It will hand you the right characters in the wrong shape.

Understand or Agentic mode is the wrong tool when: you're indexing millions of simple, clean digital pages for keyword search and don't need layout at all. Running the deepest mode on documents that Text mode would handle just as well is the most common way teams overpay for extraction they don't need.

Match the mode to the job — not the other way around. If you need specific fields rather than the whole document, Extract with a schema (generated from a sample document via Studio's schema generator) is usually a better fit than parsing the full page and post-processing it yourself.

If you're a software or API company looking to explain your product through high-quality educational content (not marketing fluff), feel free to connect with me on LinkedIn.

Key takeaways

Plain OCR converts pixels to characters. Plain text extraction gets text out, but it was never meant to understand layout.
Structure, Understand, and Agentic modes classify elements, preserve table structure, and determine reading order — a different job, not a better version of the same job.
Picking the right mode per document (not defaulting to the most powerful one) is what keeps extraction both accurate and cost-efficient.

FAQs

❓ Is AI document extraction just "OCR with AI added"?
✅ Not exactly. OCR is the character-recognition layer. AI document extraction uses OCR or other recognition methods as part of a broader pipeline that also analyzes layout, detects tables, identifies fields, preserves reading order, and returns structured output.

❓ Do I need to run OCR separately before using Structure or Understand mode?
✅ No — both modes already include character recognition as part of the pipeline. You don't run OCR first and then request structure on top; you pick the mode that matches what the document needs.

❓ Can Text mode handle scanned tables if I clean up the output myself?
✅ You can write custom logic to reconstruct a table from flat text, but it breaks the moment a vendor changes their layout. Structure mode is designed to reduce this brittleness by returning layout-aware elements, table structure, coordinates, and page context.

❓ Which mode should I default to if I'm not sure?
✅ Start with Structure mode for scanned or image-based documents. Move up to Understand or Agentic when tables, columns, handwriting, or complex layouts need deeper analysis

❓ Does a more advanced mode always mean better results?
✅ Not for every document. Understand and Agentic modes cost more and run slower than Structure — worth it for genuinely complex layouts, unnecessary overhead for a clean, well-structured scan.

Nutrient Data Extraction API
Turn PDFs, scans, images, and Office files into structured JSON or clean Markdown — with the right mode for every document.
→ Get started free

From Messy PDFs to Structured JSON: An LLM-Ready Workflow

Kevin Meneses González — Thu, 16 Jul 2026 09:33:16 +0000

Most AI pipelines don't fail because of the model.

They fail because of the input.

If you're:

building RAG pipelines,
automating invoice or contract intake,
or feeding documents into an LLM for Q&A,

the PDF is usually the weakest link in the chain.

TL;DR

A PDF is a layout format, not a data format. Generic text extraction often loses reading order, table structure, and source context.
Nutrient Data Extraction API turns PDFs, scans, images, and Office files into structured output: structured JSON for schema-defined extraction and downstream systems, or Markdown for AI/search workflows.
Use Studio to test documents first, compare processing modes, generate schemas, and inspect output before integrating with the API.
The API supports four processing modes — Text, Structure, Understand, and Agentic — so teams can balance speed, cost, and extraction quality based on document complexity.

The PDF was never built for this

A PDF remembers how a page looks. It doesn't remember what the content is.

That distinction breaks most naive extraction attempts:

Text is stored in fragments, often out of reading order
Tables are just aligned text — there's no actual "table" object
Scanned pages have no text layer at all
Two-column layouts get merged into a single unreadable stream

Developers usually discover this too late — after a regex-based parser works fine on three test invoices and then falls apart on the fourth, because the vendor used a different template.

Why text extraction alone isn't enough

Getting text out of a PDF is easy. Getting text out with the relationships preserved — which value belongs to which row, which heading owns which paragraph, which number came from which page — is the actual job.

An LLM does not just need more text. It needs structured content that preserves context, order, and source grounding.

Nutrient Data Extraction API: Studio-first, API-next

Nutrient Data Extraction API processes PDFs, scans, images, and Office files, and returns either structured JSON or clean Markdown, with source context such as confidence scores, page references, and coordinates.

Before wiring extraction into an application, you can test documents in Data Extraction API Studio. Upload a file, choose Parse or Extract, compare processing modes, inspect Markdown or JSON output, and review source context before moving to the API — 5,000 free credits, no credit card required.

Parse vs. Extract

The API gives you two request paths, and picking the right one matters:

Parse returns the full document content as clean Markdown or structured JSON blocks — use it when you need the whole document, structured.
Extract pulls specific fields you define in a schema — such as invoice number, total amount, or vendor name — use it when you know exactly which fields you need out of a document type.

Most RAG and search use cases run through Parse. Invoice, form, and structured-record workflows usually reach for Extract once the schema is defined.

Markdown vs. spatial JSON

Parse and Extract are about which request you send. Markdown and spatial JSON are about what format the content comes back in — and that choice is independent of Parse vs. Extract; both output formats are available either way.

Markdown is clean, structured content — headings, lists, and tables rendered as Markdown syntax (including real <table> markup for tables). It's positioned as the go-to format for AI and search workflows: RAG ingestion, document Q&A, knowledge base indexing, content migration. It reads like a well-formatted document, not a data structure.

Spatial JSON is the layout-aware format — a typed list of elements, each with its type, its role (heading, body text, footer, and so on), its bounding box, its reading order, and a confidence score. It's the stronger choice when you need element types, bounds, coordinates, and confidence, or when a value has to be traceable back to an exact position on the page — table cell validation, form field mapping, audit trails.

A rule of thumb: if the next step is "feed this to an LLM or a search index," reach for Markdown. If the next step is "validate this value, map it to a database column, or highlight it on the source page," reach for spatial JSON.

They're separate output formats on the same Parse request — check the API docs for the current recommended pattern if a workflow genuinely needs both from one document.

Processing modes: matching depth to the document

Instead of a single one-size-fits-all pass, the API gives you four processing modes:

Mode	When to use it
Text	Born-digital documents where speed and cost matter
Structure	Scans and image-based documents that need OCR and layout
Understand	Complex layouts, tables, handwriting, forms, and higher-quality structure
Agentic	Visually complex or ambiguous documents where deeper reasoning is worth the extra cost — deeper visual reasoning and recovery for documents that need more advanced analysis

You pick the mode per document. A clean digital invoice doesn't need the same processing budget as a scanned, handwritten form.

Schema generation for field extraction

For Extract workflows, you don't have to write a schema from scratch. Schema generator can create a starting schema from a sample document — you then review and refine the fields, types, and instructions before running extraction through the API.

This matters in practice: most teams don't know the exact field list a document type needs until they've looked at a handful of real samples. Starting from a generated schema and refining it is faster and more accurate than guessing upfront.

API example: parsing a document

Here's a minimal Parse request that converts a PDF to Markdown:

curl -X POST https://api.nutrient.io/extraction/parse \
  -H "Authorization: Bearer $NUTRIENT_API_KEY" \
  -F "file=@document.pdf" \
  -F 'instructions={"output":{"format":"markdown"}}'

Swap "format":"markdown" for "format":"json" when you need structured elements with coordinates and confidence instead of clean Markdown — for example, when you need to trace a value back to its exact position on the page, or preserve table cell structure precisely.

A reusable Python script

The curl one-liner is fine for a quick test. For anything you'll run more than once, here's a small Python CLI wrapper around the same endpoint — it takes a PDF path, an output format, and an optional output file:

"""
Extract structured content from a PDF using the Nutrient extraction API.

Setup:
    pip install -r requirements.txt
    Copy .env.example to .env and add your Nutrient API key.

Usage:
    python extract_pdf.py path/to/file.pdf
    python extract_pdf.py path/to/file.pdf --format json
    python extract_pdf.py path/to/file.pdf --output result.md
"""

import argparse
import os
import sys

import requests
from dotenv import load_dotenv

load_dotenv()
API_KEY = os.getenv("NUTRIENT_API_KEY")
URL = "https://api.nutrient.io/extraction/parse"


def extract(pdf_path: str, output_format: str = "markdown") -> dict:
    with open(pdf_path, "rb") as f:
        r = requests.post(
            URL,
            headers={"Authorization": f"Bearer {API_KEY}"},
            files={"file": f},
            data={"instructions": f'{{"output":{{"format":"{output_format}"}}}}'},
            timeout=60,
        )
    r.raise_for_status()
    return r.json()


def main():
    parser = argparse.ArgumentParser(description="Extract content from a PDF via Nutrient")
    parser.add_argument("pdf", help="Path to the PDF file")
    parser.add_argument("--format", choices=["markdown", "json", "text"], default="markdown")
    parser.add_argument("--output", help="Write the extracted content to this file instead of stdout")
    args = parser.parse_args()

    if not API_KEY:
        sys.exit(
            "NUTRIENT_API_KEY not found. Create a .env file next to this script "
            "with the line: NUTRIENT_API_KEY=your_api_key"
        )

    if not os.path.isfile(args.pdf):
        sys.exit(f"File not found: {args.pdf}")

    result = extract(args.pdf, args.format)
    content = result["output"].get(args.format, result["output"])
    if isinstance(content, dict):
        import json
        content = json.dumps(content, indent=2)

    if args.output:
        with open(args.output, "w", encoding="utf-8") as f:
            f.write(content)
        print(f"Saved to {args.output}")
    else:
        print(content)

    credits = result.get("usage", {}).get("data_extraction_credits", {})
    if credits:
        print(f"\n[credits used: {credits.get('cost')}, remaining: {credits.get('remainingCredits')}]", file=sys.stderr)


if __name__ == "__main__":
    main()

Drop this next to a .env file with NUTRIENT_API_KEY=your_api_key, and you have a script you can point at any PDF: python extract_pdf.py invoice.pdf --format json --output invoice.json. It's the same request the curl example makes — just wrapped in something you'll actually reuse.

For schema-defined Extract requests, the exact request shape depends on your schema — check the official API documentation for the current Extract endpoint and payload format, or generate a starting request directly from Studio once your schema is ready.

A JSON response from Parse looks like this — element types, positions, and reading order, not a flat text dump:

{
  "elements": [
    {
      "type": "paragraph",
      "text": "Invoice #2024-0892",
      "boundingBox": { "x": 45, "y": 120, "width": 310, "height": 28 },
      "readingOrder": 0
    },
    {
      "type": "table",
      "boundingBox": { "x": 45, "y": 200, "width": 680, "height": 340 },
      "readingOrder": 1,
      "children": [
        { "type": "tableCell", "text": "Item", "row": 0, "column": 0 },
        { "type": "tableCell", "text": "Amount", "row": 0, "column": 1 }
      ]
    }
  ]
}

Compare that to plain OCR output: "Invoice #2024-0892\nItem Amount\nWidget A $45.00" — with no indication of which value belongs to which column.

What a real response actually looks like

The example above is simplified to make the shape easy to read. Here's what Parse actually returns for a real invoice — anonymized, but otherwise untouched — first as Markdown, then as structured JSON.

Markdown output:

## Invoice

INV-2026-0001 Invoice number

July 3, 2026 Date of issue

## Medium

123 Main Street 94105 San Francisco California USA +1 555 010 2938

Date due July 17, 2026

## $175.00 USD due July 17, 2026

## Pay online

<table>
 <tr>
  <td>Description</td>
  <td></td>
  <td>Unit price</td>
  <td>Amount</td>
 </tr>
 <tr>
  <td>Content Marketing</td>
  <td>1</td>
  <td>$175.00</td>
  <td>$175.00</td>
 </tr>
 <tr>
  <td></td>
  <td>Subtotal</td>
  <td></td>
  <td>$175.00</td>
 </tr>
 <tr>
  <td></td>
  <td>Total</td>
  <td></td>
  <td>$175.00</td>
 </tr>
 <tr>
  <td></td>
  <td>Amount due</td>
  <td></td>
  <td>$175.00 USD</td>
 </tr>
</table>

## Medium

Bill to

Jamie Chen jamie.chen@example.com

Notice the table survives as real HTML <table> markup embedded in the Markdown — not flattened into a line of numbers. That's what makes Markdown output usable for RAG chunking without losing the row/column relationships.

JSON output (trimmed — the real response includes every element, and the table's cell array is shortened here for readability):

{
  "elements": [
    {
      "id": "ad35142f-5426-4995-94e3-56682a3a8c7c",
      "type": "paragraph",
      "role": "SectionHeader",
      "text": "Invoice",
      "confidence": 0.939276,
      "readingOrder": 0,
      "bounds": { "x": 84.03, "y": 92.88, "width": 170.97, "height": 42.12 },
      "page": { "pageNumber": 1, "width": 1700, "height": 2200 }
    },
    {
      "id": "a0d1192d-3d59-4a4b-b10f-16df2233dbbb",
      "type": "paragraph",
      "role": "Text",
      "text": "INV-2026-0001 Invoice number",
      "confidence": 0.904187,
      "readingOrder": 2,
      "bounds": { "x": 82.81, "y": 195.59, "width": 416.19, "height": 23.41 },
      "page": { "pageNumber": 1, "width": 1700, "height": 2200 }
    },
    {
      "id": "7184b67c-468d-42d6-9e00-f0dd10a2a4c3",
      "type": "paragraph",
      "role": "Text",
      "text": "123 Main Street\n94105 San Francisco\nCalifornia\nUSA\n+1 555 010 2938",
      "confidence": 0.680442,
      "readingOrder": 6,
      "bounds": { "x": 82.46, "y": 394.16, "width": 530.54, "height": 174.84 },
      "page": { "pageNumber": 1, "width": 1700, "height": 2200 }
    },
    {
      "id": "8532e005-9b64-4a04-9745-f1a963a27f4d",
      "type": "table",
      "readingOrder": 13,
      "confidence": 0.928583,
      "rowCount": 5,
      "columnCount": 4,
      "bounds": { "x": 80.54, "y": 825.07, "width": 1537.94, "height": 246.88 },
      "page": { "pageNumber": 1, "width": 1700, "height": 2200 },
      "cells": [
        { "row": 0, "column": 0, "text": "Description", "confidence": 1 },
        { "row": 0, "column": 2, "text": "Unit price", "confidence": 1 },
        { "row": 0, "column": 3, "text": "Amount", "confidence": 1 },
        { "row": 1, "column": 0, "text": "Content Marketing", "confidence": 1 },
        { "row": 1, "column": 3, "text": "$175.00", "confidence": 1 },
        { "row": 4, "column": 1, "text": "Amount due", "confidence": 1 },
        { "row": 4, "column": 3, "text": "$175.00 USD", "confidence": 1 }
      ]
    },
    {
      "id": "c38ff641-a643-4ad5-b9e2-4c370fc6a0b3",
      "type": "paragraph",
      "role": "Text",
      "text": "Jamie Chen\njamie.chen@example.com",
      "confidence": 0.686578,
      "readingOrder": 16,
      "bounds": { "x": 695.90, "y": 394.81, "width": 279.40, "height": 65.19 },
      "page": { "pageNumber": 1, "width": 1700, "height": 2200 }
    }
  ],
  "languageDetection": {
    "pages": [
      { "pageNumber": 1, "languages": ["eng"], "textDirection": "lrtb" }
    ]
  }
}

Two things worth pointing out here that don't show up in a toy example:

role classifies elements, not just types — SectionHeader, Text, and Footer all come back as paragraph type, but role tells you which one is a heading versus body text versus a page footer. That's the field to key off if you're deciding what belongs in a RAG chunk versus what to drop.
Confidence varies by element, and it's not always near 1.0 — the seller address block above scored 0.68, noticeably lower than the clean, high-contrast text elements around it. That's exactly the kind of value a validation threshold should catch before it moves downstream, not something to assume is correct because the rest of the document extracted cleanly.

The architecture: where extraction sits in the pipeline

It helps to see this as a layer, not a single API call. A production document pipeline usually looks like this:

Document intake (upload / email / scan / hosted URL)
        │
        ▼
Mode selection (Text / Structure / Understand / Agentic)
        │
        ▼
Nutrient Data Extraction API — Parse or Extract
   → structured JSON (elements, coordinates, confidence, reading order)
   → or Markdown (clean, LLM-ready content)
        │
        ▼
Validation layer
   → flag elements below a confidence threshold
   → route flagged elements to human review
        │
        ▼
Downstream systems
   → database / ERP (structured JSON)
   → RAG index / vector store (Markdown)
   → review queue (flagged elements + bounding boxes)

The part teams usually skip is the validation layer. Extraction without a confidence check just moves the trust problem one step downstream — instead of a human misreading a PDF, you get an LLM confidently building on top of a weak extraction. Coordinates and confidence scores make that validation step practical: you set a threshold (say, anything under 85% confidence), route those specific elements to a review queue, and let everything else flow through automatically.

Mode selection also isn't a one-time decision — it's a per-document routing rule. A born-digital invoice from your billing system doesn't need the same processing depth as a photographed, handwritten delivery note. Sending everything through the most expensive mode by default is the most common way teams overpay for extraction.

Pros and cons

Pros

Source context — confidence scores, page references, and coordinates — comes back alongside the extracted content, not just plain text
Four processing modes mean you're not paying for AI-heavy extraction on documents that don't need it
Same API handles PDFs, scans, images, and Office files — no separate OCR pipeline to maintain
Studio lets you test Parse and Extract, compare modes, and generate a schema before writing any integration code

Cons

Files are not persistently stored after processing, which is good for compliance but means you handle retention or audit storage yourself if you need it

Best for: teams building document-heavy pipelines — RAG ingestion, invoice/contract automation, compliance workflows — where "we got some text out" isn't good enough and you need to trace values back to their source.

Practical use cases

1. Invoice and statement processing
Route invoices through Structure or Understand mode via Extract with a schema for line items and totals, and flag any amount under your confidence threshold for a human to check before it posts to your accounting system.

2. Contract review pipelines
Use Parse to extract clause-level text with reading order preserved, so a downstream LLM step (clause classification, obligation extraction) works with clauses in the order they actually appear in the contract — not reshuffled by a naive text dump.

3. RAG knowledge base ingestion
Use Markdown output from Parse directly as chunks for your vector store. Because headings, lists, and tables are preserved instead of flattened, retrieval quality improves — the model gets a coherent section, not fragments stitched together.

4. Compliance and audit trails
For regulated workflows (financial statements, medical intake forms, legal filings), the combination of confidence score and page coordinate means extracted values can be traced back to a location in the source document — which is closer to what most audit processes actually require than "we have the data somewhere."

Closing the loop

Structured elements with reading order, confidence, and bounding boxes are what turn "we extracted some text" into "we extracted data we can act on" — whether that means posting a line item to your ERP, indexing a chunk for RAG, or routing a flagged field to a human.

If you're a software or API company looking to explain your product through high-quality educational content (not marketing fluff), feel free to connect with me on LinkedIn.

Key takeaways

A PDF is a presentation format. Structure has to be extracted, not assumed.
Source context — confidence scores and page coordinates — is what makes extracted data traceable back to the source, not just readable.
Match the processing mode (Text / Structure / Understand / Agentic) and request path (Parse / Extract) to the document instead of defaulting to one approach for everything.

How the modes actually perform

The table above tells you when to use each mode. Here's what that difference looks like in numbers — internal benchmark scores (Build #200, July 13, 2026, commit c24e99581f), all on a 0–1 scale where higher is better:

Mode	Overall	NID	NID-S	TEDS	TEDS-S	MHS	MHS-S	Zone-F1
Understand	0.9321	0.9590	0.9567	0.9365	0.9444	0.8671	0.9078	0.6892
Agentic (Vision)	0.9304	0.9572	0.9548	0.9370	0.9444	0.8649	0.9078	0.7006
Structure	0.8923	0.9342	0.9288	0.7386	0.7846	0.8282	0.8884	0.4288
Text	0.8887	0.9263	0.9291	0.7394	0.7902	0.8239	0.8862	N/A

The -S suffix variants (NID-S, TEDS-S, MHS-S) are strict versions of each metric. Zone-F1 measures spatial layout accuracy and doesn't apply to Text mode, which doesn't preserve layout.

What each metric actually measures:

NID (Normalized Information Distance) — how much information survives from the original document. 0.95+ means almost nothing is lost.
TEDS (Tree Edit Distance Score) — table structure accuracy specifically: how well row/column relationships in extracted tables match the source. This is the metric that separates Text/Structure from Understand/Agentic the most — TEDS is the hardest metric for complex documents, and scores below 0.80 are common outside these modes.
MHS (Markdown Heading Score) — whether the heading hierarchy (H1/H2/H3) gets reconstructed correctly.
Zone-F1 — spatial accuracy: whether bounding-box zones (text regions, tables, figures) are correctly detected and labeled. Higher-depth modes (Understand, Agentic) score noticeably better here.

The practical read: Text and Structure hold up fine on NID (they capture the content), but their TEDS scores drop hard on anything with real tables — 0.74 vs 0.93+ for Understand and Agentic. If your documents are table-heavy, that gap is the one to design around, not overall accuracy.

You can see more details here

FAQs

❓ Can I get structured JSON and Markdown from the same request?
✅ Choose the output format based on the use case — structured JSON when you need layout-aware elements with coordinates and confidence, or Markdown when you need clean content for RAG, search, or document Q&A. Check the API docs for the recommended request pattern if a workflow needs both.

❓ Does this replace OCR, or run on top of it?
✅ It replaces the need for a separate OCR step. Structure and Understand modes handle scanned or image-based documents directly through the same API — you don't build or maintain a separate OCR pipeline before extraction.

❓ How do I know which processing mode to use for a given document?
✅ Start with Text mode for born-digital PDFs (fastest, cheapest). Use Structure mode once scanning or OCR is involved. Reach for Understand mode on complex layouts, handwriting, or documents needing OCR correction. Reserve Agentic mode for visually complex or ambiguous documents where deeper reasoning is worth the extra cost.

❓ What happens to my documents after they're processed?
✅ Files are not persistently stored after processing. If you need retention for audit purposes, that's handled on your side, not the API's.

❓ Is there a free way to test this before integrating it?
✅ Yes. Data Extraction API Studio gives you 5,000 free credits, no credit card required, so you can upload your own documents, compare Parse and Extract, try all four modes, and generate a starting schema before writing any code.

Nutrient Data Extraction API
Turn messy PDFs into structured JSON or Markdown your LLM pipeline can trust — with source context on every request.
→ Get started free

Top 7 LLM Observability Tools Every AI Engineer Should Know (2026)

Kevin Meneses González — Wed, 15 Jul 2026 11:57:13 +0000

Artificial intelligence has made it easier than ever to build applications powered by large language models. In just a few hours, you can create a chatbot, a RAG pipeline, or even an autonomous AI agent capable of calling APIs and interacting with external tools.

Getting a demo to work is no longer the hard part.

Running that same application in production — serving hundreds or thousands of users every day — is where the real engineering challenge begins.

Once real traffic starts flowing, developers quickly encounter questions that traditional logging systems cannot answer:

Why did response latency suddenly increase?
Which prompts are consuming the most tokens?
Which customers generate the highest API costs?
What caused this hallucinated answer?
Did the latest prompt update improve or reduce response quality?
Which LLM performs best for this specific task?
How can I debug an AI agent that executed twenty tool calls before failing?

These are exactly the problems that LLM observability platforms are designed to solve.

Instead of treating AI models as mysterious black boxes, observability platforms allow engineering teams to inspect every interaction, measure performance, optimize costs, evaluate outputs, and continuously improve applications running in production.

Whether you're building customer support assistants, financial research tools, coding agents, document-processing systems, or multi-agent workflows, observability has become a critical part of the modern AI stack.

In this article, we'll explore seven of the most popular LLM observability platforms available in 2026 and discuss where each one excels.

What Is LLM Observability?

Traditional application monitoring focuses on metrics like:

CPU usage
Memory
HTTP requests
Database performance
Errors

LLM applications introduce an entirely new layer of complexity.

Every request now contains:

Prompts
Context windows
Retrieved documents
Model parameters
Tool calls
Structured outputs
Token usage
Cost information

Traditional monitoring platforms simply weren't designed for this.

LLM observability extends software monitoring by tracking everything that happens during an AI interaction.

A typical observability platform records:

Prompt execution
Model responses
Token consumption
Cost per request
Latency
User sessions
Agent traces
Tool execution
Prompt versions
Evaluation scores
Errors and failures

This allows developers to understand not only whether something failed, but why it failed.

Why Is It Becoming Essential?

Six months ago, many AI applications consisted of a single API call to GPT-4.

Today's applications are very different.

A single user request may involve:

Vector database retrieval
Multiple prompt templates
Several LLM calls
External APIs
MCP servers
Python execution
Memory retrieval
Human approval
Final answer generation

Without observability, debugging these workflows becomes almost impossible.

As AI systems become more autonomous, engineering teams need visibility into every decision an agent makes.

That's exactly where these platforms provide value.

What Features Matter Most?

Before comparing platforms, it's worth understanding the capabilities that differentiate them.

Tracing

Tracing records every step an AI application performs.

Instead of seeing only the final response, developers can inspect the complete execution path.

For example:

User Question
↓
Retriever
↓
Prompt Template
↓
Claude
↓
Weather API
↓
Second Prompt
↓
Final Response

This dramatically simplifies debugging.

Cost Analytics

LLM costs can grow surprisingly fast.

Good observability platforms answer questions like:

Which user generated the highest cost?
Which prompt consumes the most tokens?
Which model is most cost-efficient?
How much does each feature cost?

This information often leads to significant savings simply by optimizing prompts or routing requests to more appropriate models.

Prompt Versioning

Prompts evolve continuously.

Without versioning, it's difficult to know which change introduced a regression.

Modern platforms allow teams to compare prompt versions and roll back unsuccessful updates.

Evaluation

One of the biggest challenges in AI engineering is measuring quality.

Unlike traditional software, LLM outputs aren't simply "correct" or "incorrect."

Evaluation systems can automatically score responses based on criteria such as:

Accuracy
Relevance
Faithfulness
Hallucination rate
Toxicity
Response quality

Some platforms even use another LLM as an automated judge.

Production Debugging

Imagine a customer reports:

"Your AI gave me the wrong answer yesterday."

Without observability, reproducing that interaction is extremely difficult.

With tracing enabled, engineers can inspect:

The exact prompt
Retrieved documents
Model parameters
Tool calls
Response time
Final output

This dramatically reduces debugging time.

Evaluation Criteria

For this comparison, each platform was evaluated across the areas above: tracing depth, cost analytics, prompt versioning, evaluation capabilities, and production debugging.

1. Langfuse

Best for: Open-source observability with powerful tracing and evaluation features.

Langfuse has quickly become one of the most popular observability platforms for AI applications.

Unlike traditional monitoring software, it was built specifically around LLM workflows.

Its biggest strength is that it combines enterprise-level capabilities with an open-source foundation.

For many engineering teams, this provides the best balance between flexibility and production readiness.

Key Features

End-to-end tracing
Prompt management
Dataset creation
Evaluations
Cost analytics
Session tracking
User analytics
Self-hosted deployment
OpenTelemetry support

One particularly useful capability is visualizing complete execution graphs for AI agents.

Instead of reviewing logs line by line, developers can inspect the entire reasoning pipeline.

This makes debugging dramatically faster.

Pros

✔ Excellent developer experience
✔ Open source
✔ Self-hosting available
✔ Strong documentation
✔ Supports most modern AI frameworks

Cons

Enterprise features require paid plans
Can feel overwhelming for very small projects

Best Use Cases

Production AI applications
RAG systems
AI agents
Multi-agent workflows
Internal enterprise assistants

2. LangSmith

Best for: Teams already building with LangChain and LangGraph.

LangSmith is the observability platform developed by the LangChain team.

Rather than acting as a generic monitoring solution, it's deeply integrated into the LangChain ecosystem.

If your application uses LangGraph agents, LangSmith feels almost like a native debugging interface.

Developers can replay executions, inspect intermediate reasoning steps, compare prompt versions, and evaluate changes over time.

For complex AI agents, this level of visibility becomes invaluable.

Key Features

Agent tracing
Prompt versioning
Dataset management
Human feedback
Automated evaluations
Experiment tracking
Production monitoring

One of LangSmith's strongest capabilities is debugging multi-agent systems where dozens of individual reasoning steps occur before producing a final answer.

Pros

✔ Best integration with LangGraph
✔ Excellent UI
✔ Powerful evaluation system
✔ Enterprise-ready

Cons

Delivers the most value when using the LangChain ecosystem
Less framework-agnostic than some competitors

Best Use Cases

LangGraph agents
Enterprise AI assistants
Coding agents
Research agents
Production agent orchestration

3. Portkey

Best for: Organizations that need both an AI Gateway and observability platform.

Portkey takes a different approach.

Instead of focusing only on monitoring, it sits between your application and the LLM providers.

Think of it as an API gateway designed specifically for AI.

Every request flows through Portkey before reaching OpenAI, Anthropic, Gemini, Mistral, or another provider.

This architecture unlocks capabilities beyond observability.

Developers can:

Route traffic between providers
Retry failed requests
Cache responses
Apply guardrails
Monitor costs
Analyze latency
Centralize logging
Switch providers without changing application code

For larger organizations managing multiple models, this architecture can significantly simplify infrastructure.

Pros

✔ Multi-provider routing
✔ Built-in observability
✔ AI gateway capabilities
✔ Cost optimization
✔ Strong enterprise focus

Cons

More infrastructure to configure
May be unnecessary for very small applications

Best Use Cases

Multi-model applications
Enterprise AI platforms
SaaS products
High-volume API workloads

4. Braintrust

Best for: Evaluating AI applications before they reach production.

While many observability platforms focus on monitoring requests and debugging failures, Braintrust is built around a different question:

"How do we know our AI application is actually getting better?"

This makes Braintrust one of the strongest platforms for AI evaluation.

Instead of simply collecting traces, it helps engineering teams measure whether changes to prompts, models, or workflows improve response quality.

For companies shipping AI products every week, this capability is extremely valuable.

Key Features

Prompt evaluation
LLM-as-a-Judge
Human evaluations
Regression testing
Experiment tracking
Benchmark datasets
CI/CD integration
Team collaboration

Braintrust allows developers to compare prompt versions before deploying them.

For example, after changing a system prompt, you can automatically test it against hundreds of predefined examples and compare the results with the previous version.

This prevents unexpected regressions from reaching production.

Pros

✔ Excellent evaluation workflow
✔ Designed for enterprise AI teams
✔ Strong experiment management
✔ Easy A/B testing

Cons

Less focused on operational monitoring
Better suited for mature AI products than small prototypes

Best Use Cases

AI copilots
Customer support assistants
Enterprise chatbots
Document understanding
AI products with continuous releases ## 4. Braintrust

Best for: Evaluating AI applications before they reach production.

While many observability platforms focus on monitoring requests and debugging failures, Braintrust is built around a different question:

"How do we know our AI application is actually getting better?"

This makes Braintrust one of the strongest platforms for AI evaluation.

Instead of simply collecting traces, it helps engineering teams measure whether changes to prompts, models, or workflows improve response quality.

For companies shipping AI products every week, this capability is extremely valuable.

Key Features

Prompt evaluation
LLM-as-a-Judge
Human evaluations
Regression testing
Experiment tracking
Benchmark datasets
CI/CD integration
Team collaboration

Braintrust allows developers to compare prompt versions before deploying them.

For example, after changing a system prompt, you can automatically test it against hundreds of predefined examples and compare the results with the previous version.

This prevents unexpected regressions from reaching production.

Pros

✔ Excellent evaluation workflow
✔ Designed for enterprise AI teams
✔ Strong experiment management
✔ Easy A/B testing

Cons

Less focused on operational monitoring
Better suited for mature AI products than small prototypes

Best Use Cases

AI copilots
Customer support assistants
Enterprise chatbots
Document understanding
AI products with continuous releases

5. Arize Phoenix

Best for: Advanced AI and ML observability.

Arize AI has been a leader in machine learning observability for several years.

As generative AI adoption accelerated, the company introduced Phoenix, an open-source platform focused on LLM applications.

Phoenix combines traditional ML monitoring with AI-specific capabilities such as prompt tracing and hallucination analysis.

For organizations already operating ML pipelines, this makes Arize a natural evolution.

Key Features

End-to-end tracing
Hallucination analysis
Root cause investigation
Embedding visualization
Retrieval quality analysis
Prompt inspection
OpenTelemetry support

One particularly interesting capability is analyzing retrieval quality in RAG systems.

Instead of simply inspecting prompts, Phoenix helps identify whether poor answers were caused by weak retrieval rather than the language model itself.

Pros

✔ Excellent for RAG systems
✔ Strong open-source offering
✔ Built by an experienced ML company
✔ Enterprise-grade analytics

Cons

More advanced than many teams require
Learning curve is steeper

Best Use Cases

RAG applications
Enterprise search
AI knowledge bases
Internal copilots
Large production systems

6. Humanloop

Best for: Prompt management and collaborative AI development.

Humanloop sits somewhere between an observability platform and a prompt engineering workspace.

Its philosophy is that AI applications should be developed in the same structured way as software.

Instead of editing prompts directly inside source code, teams can version, test, review, and evaluate prompts collaboratively.

For product teams working closely with engineers, this approach can significantly speed up iteration.

Key Features

Prompt management
Version control
Prompt playground
Automated evaluations
Human feedback
Experiment tracking
Team collaboration

Humanloop also makes it easier for non-engineering stakeholders to participate in prompt development without modifying application code.

Pros

✔ Excellent prompt workflow
✔ Clean interface
✔ Strong collaboration features
✔ Easy experimentation

Cons

Less focused on infrastructure monitoring
More valuable for larger teams than solo developers

Best Use Cases

Product teams
AI startups
Internal copilots
Prompt engineering
Customer support AI

7. Helicone

Best for: Fast and simple LLM observability.

Helicone has one of the simplest onboarding experiences in the industry.

In many cases, developers only need to change the API endpoint used by their application.

Immediately afterwards they gain access to:

Request logs
Token usage
Costs
Latency
User analytics
Model comparisons

This simplicity makes Helicone particularly attractive for startups and individual developers.

Instead of spending hours configuring infrastructure, teams can begin monitoring their AI applications almost immediately.

Key Features

API proxy
Cost analytics
Token tracking
Latency monitoring
User analytics
Request history
Prompt inspection

Pros

✔ Extremely easy to deploy
✔ Very developer friendly
✔ Excellent analytics
✔ Minimal setup

Cons

Smaller feature set than enterprise platforms
Less comprehensive evaluation tooling

Best Use Cases

MVPs
SaaS startups
Internal tools
Developer projects
AI prototypes

Which Platform Should You Choose?

The answer depends entirely on your application.

If you're building your first AI application

Start with Helicone.

It provides immediate visibility into costs, latency, and requests without requiring significant infrastructure.

If you prefer open-source software

Langfuse is currently one of the strongest choices.

It combines production-ready observability with the flexibility of self-hosting.

If you're building AI agents with LangGraph

Choose LangSmith.

Its deep integration with the LangChain ecosystem makes debugging significantly easier.

If you use multiple LLM providers

Portkey is an excellent choice.

Its gateway architecture simplifies routing, failover, and centralized monitoring across different models.

If evaluation is your biggest challenge

Choose Braintrust or Humanloop.

Both platforms excel at measuring response quality and comparing prompt versions before deployment.

If you're operating enterprise-scale AI systems

Arize Phoenix provides some of the most advanced observability capabilities available today, particularly for RAG applications and production environments.

Comparison Table

Platform	Open Source	Tracing	Evaluations	Prompt Mgmt	AI Gateway	Best For
Langfuse	✅	⭐⭐⭐⭐⭐	⭐⭐⭐⭐☆	⭐⭐⭐⭐☆	❌	Open-source production AI
LangSmith	❌	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	❌	LangGraph & AI Agents
Portkey	❌	⭐⭐⭐⭐☆	⭐⭐⭐☆☆	⭐⭐⭐☆☆	⭐⭐⭐⭐⭐	Multi-LLM Infrastructure
Braintrust	❌	⭐⭐⭐☆☆	⭐⭐⭐⭐⭐	⭐⭐⭐⭐☆	❌	AI Evaluation
Arize Phoenix	✅	⭐⭐⭐⭐⭐	⭐⭐⭐⭐☆	⭐⭐⭐☆☆	❌	Enterprise AI & RAG
Humanloop	❌	⭐⭐⭐☆☆	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	❌	Prompt Engineering
Helicone	Partial	⭐⭐⭐⭐☆	⭐⭐☆☆☆	⭐⭐☆☆☆	⭐⭐⭐☆☆	Startups & MVPs

FAQs

What is LLM observability?

It is the practice of monitoring prompts, traces, token usage, costs, latency and evaluations for production AI applications.

Which platform should I choose?

Langfuse → Open source
LangSmith → LangGraph
Portkey → Multi-LLM
Braintrust → Evaluation
Arize → Enterprise
Humanloop → Prompt engineering
Helicone → MVPs

Final Thoughts

Observability is becoming as important for AI applications as logging is for traditional software. The right platform helps reduce costs, improve reliability and accelerate debugging.

Let's Work Together

Looking for high-quality technical content for your AI, API, or developer tool?

📧 kevinmenesesgonzalez@gmail.com

💼 https://www.linkedin.com/in/kevin-meneses-gonzalez/

🌐 https://kevinmeneses.com

I'm always open to new collaborations, sponsored content and developer marketing partnerships.

5 Strapi Alternatives Developers Are Actually Switching To in 2026

Kevin Meneses González — Tue, 14 Jul 2026 16:30:48 +0000

For years, Strapi was the answer.

Need a headless CMS? Open source, self-hosted, developer-first. Done.

That's not true anymore. In 2026, developers are switching away from Strapi at a pace that's hard to ignore — not because it got worse, but because the alternatives got dramatically better.

If you're:

migrating a content-heavy Next.js app,
choosing infrastructure for a new product,
or just tired of patching Node dependencies every quarter,

this matters.

The Hidden Cost Nobody Puts in the Pricing Table

Strapi's pitch has always been simple: it's free, it's open source, you own your data.

All true. None of it accounts for what happens after deployment.

Someone has to manage the server. Apply security patches. Handle database migrations when a plugin update breaks your schema. Babysit uptime at 3am when the content API goes down mid-launch.

That someone is usually a developer who'd rather be shipping features.

Teams call this "Type 1 fun" — technically satisfying, but a distraction from the product you're actually trying to build. Strapi Cloud exists specifically to remove this burden.

It also adds a bill.

The Real Problem Isn't Strapi. It's the Definition of "Headless."

For a long time, "headless CMS" meant one thing: a content repository with an API on top.

That bar is too low for 2026.

Content today needs to move across web, mobile, and increasingly AI-driven interfaces — chatbots, agents, voice assistants. A CMS that only stores text and serves it over REST doesn't cut it when your product needs structured, machine-readable content that an LLM can reason over.

The platforms winning in 2026 aren't just "headless." They handle content operations — versioning, AI-assisted authoring, translation pipelines, and increasingly, native support for AI agents and MCP.

That's the real reason developers are re-evaluating their stack.

What to Look for Before You Switch

Before comparing tools, three questions actually matter:

Framework fit. Are you on Next.js, Nuxt, Astro, SvelteKit? Most CMS platforms support all of them via REST, but GraphQL support narrows the field fast.

True cost at scale. Free tiers are easy to compare. What you'll pay at 12–18 months of real usage — seats, API limits, bandwidth — is the number that matters.

Data portability. Can you get your content out in a standard format if you ever need to migrate again? Some platforms make this trivial. Others make it a project.

With that in mind, here are the five alternatives worth your evaluation time.

1. Payload — Best for Next.js and TypeScript-First Teams

Payload is the CMS that made the biggest jump in 2026. Version 3.0 cut its dependency count from 88 to 27, and its local API means content queries run in-process — no network round trip, no rate limits to worry about.

The schema-as-code approach feels familiar if you've used Sanity, but the TypeScript integration goes further: content types generate full types automatically, so there's no any silently leaking through your codebase.

Pros

Local API eliminates network latency for server-side queries
Auto-generated TypeScript types from your schema
Now framework-agnostic — works with Remix, Astro, SvelteKit, not just Next.js

Cons

Younger ecosystem than Strapi's — fewer plugins, fewer Stack Overflow answers
Smaller community means you'll solve more edge cases yourself

Best for: Teams deeply committed to Next.js and TypeScript who want to own their CMS implementation completely.

2. Sanity — Best for Content Operations and Multilingual Projects

Sanity has evolved past the traditional definition of headless CMS. It's positioning itself as a full content operating system — structured content that plugs directly into AI and automation workflows, not just a delivery API.

Teams running large multilingual content pipelines (10+ languages, document-level i18n) consistently point to Sanity's structured content tooling as the deciding factor.

Pros

Strongest developer experience for structured, queryable content (GROQ)
Excellent Next.js integration with Visual Editing and live preview
Handles document-level internationalization cleanly at scale

Cons

Steeper learning curve for teams new to structured content modeling
Pricing can climb fast once you exceed generous free-tier limits

Best for: Engineering-led teams running multilingual content operations who want AI-ready structured data, not just a content store.

Looking for technical content that actually explains your product?
I write developer-first articles and tutorials for API and infrastructure companies — the kind engineers bookmark instead of skip.
→ Let's talk on LinkedIn

3. Directus — Best for Wrapping an Existing Database

Directus takes a different approach entirely: instead of designing a new database for your content, it sits on top of the one you already have.

That single decision makes it the strongest pick for teams with legacy data models who don't want to migrate everything into a new schema just to get a CMS.

Pros

Adds a CMS layer to an existing SQL database without disrupting the schema
AI features aren't locked behind an enterprise tier — bring your own API key
Generous free tier compared to most open-source competitors

Cons

Less opinionated structure means more setup decisions land on you
Custom AI logic still requires building your own plugin

Best for: Teams that already have a database and need a CMS layer without redesigning their data model.

4. Hygraph — Best for GraphQL-Native Architectures

Hygraph (formerly GraphCMS) is built entirely around GraphQL, not bolted onto REST as an afterthought. For teams working with Next.js, React, or Svelte where efficient data fetching matters, this changes the developer experience meaningfully.

Its standout feature is content federation — pulling data from multiple CMS instances, APIs, and databases into a single GraphQL layer. That's a specific use case, but it's one most competitors don't touch at all.

Pros

GraphQL-first design produces cleaner queries and more predictable data shapes
Content federation across multiple sources in one layer
Strong role-based permissions for larger teams

Cons

Overkill if your project doesn't need GraphQL or multi-source federation
Smaller plugin ecosystem than Strapi or Sanity

Best for: Composable architectures pulling content from multiple sources into a single, unified API.

5. NomaCMS — Best for AI-Native, Zero-Infrastructure Teams

NomaCMS is the newest name on this list, and the one built specifically for the AI-agent era. It's a fully managed, AI-native headless CMS — REST API, JavaScript SDK, and built-in AI tools for writing, translation, and editing, with content and assets served over a global CDN.

You run no CMS server at all. It also ships with official MCP support, meaning tools like Claude Code and Cursor can interact with your content model directly — a detail that matters if your workflow already leans on AI agents for content operations.

Pros

Zero server management — content, assets, and CDN handled entirely by the platform
Native MCP support for AI coding assistants
Fast to start: 7-day free trial, plans from $15/month

Cons

Newer platform — smaller track record than established players
Less customization depth than a self-hosted, code-first CMS

Best for: Small teams and solo developers who want Strapi-like structured content without running Node infrastructure, especially if AI tooling is already part of the workflow.

A Quick Look at Consuming a Modern CMS API

Regardless of which platform you pick, the integration pattern looks similar. Here's a minimal example fetching structured content over REST:

import requests

API_URL = "https://api.example-cms.com/v1/content/articles"
HEADERS = {"Authorization": "Bearer YOUR_API_KEY"}

response = requests.get(API_URL, headers=HEADERS)
articles = response.json()

for article in articles["data"]:
    print(article["title"], "-", article["publishedAt"])

Building a Faster Checkout Flow - 2026-06-02
Why We Moved Off Strapi - 2026-05-14
Structured Content for AI Agents - 2026-04-28

From here you can build:

a content sync pipeline into your own database
an AI agent that queries your CMS directly via MCP
automated translation workflows triggered on publish

Key Takeaways

Strapi still works — but self-hosting only pays off when it's a genuine requirement, not a cost-saving assumption.
The best pick depends on one variable more than any other: do you need code-first flexibility (Payload, Directus) or managed content operations (Sanity, NomaCMS)?
AI-native support — MCP, agent access, structured data for LLMs — is quickly becoming a baseline expectation, not a bonus feature.

FAQs

❓ Is Strapi still a good choice in 2026?
✅ Yes, if self-hosting and full data ownership are non-negotiable requirements. For teams without strong DevOps capacity, the operational overhead often outweighs the savings from a free license.

❓ What's the best headless CMS for a Next.js project?
✅ Payload and Sanity both offer deep Next.js integrations. Payload edges ahead for teams that want a TypeScript-first, code-driven workflow with no external CMS server to manage.

❓ Do any of these support AI agents directly?
✅ NomaCMS ships with native MCP support, letting tools like Claude Code query and manage content directly. Directus and Payload support AI through custom plugins using your own API keys.

The CMS market spent a decade converging on Strapi as the default.

That convergence is over. Pick based on what your team actually needs to maintain — not which platform has the most GitHub stars.

If you're building something and want a second opinion on your content infrastructure, or want your product explained through content developers actually read, you know where to find me: kevinmeneses.com

Looking for technical content for your company? I can help — LinkedIn · kevinmenesesgonzalez@gmail.com

Best Stock Market API for Python Developers (2026)

Kevin Meneses González — Fri, 10 Jul 2026 11:17:45 +0000

TL;DR: Scraping and unofficial wrappers like yfinance break in production. For Python projects that need historical + real-time stock data without stitching together multiple providers, EODHD offers the widest coverage (150,000+ tickers, 70+ exchanges) under one API key. Massive is a strong alternative for low-latency US real-time feeds, Alpha Vantage works for prototyping, and yfinance should stay limited to personal scripts.

Many developers believe you need to pay hundreds of dollars a month for reliable stock market data in Python. That's not true.

The real problem isn't price. It's picking the wrong API before you know what production actually demands.

If you're:

building a stock screener,
backtesting a trading strategy,
or adding market data to a fintech app,

this decision will follow you for months. Get it wrong and you rebuild the whole data layer later.

Scraping Works Until It Doesn't

Most Python developers start the same way. yfinance, a scraping script, maybe an unofficial endpoint someone shared on GitHub.

It works. In local testing.

Then it hits production.

Rate limits appear out of nowhere. Endpoints change without warning. A field that returned a float last month now returns a string. Your script that ran perfectly on Tuesday throws a KeyError on Wednesday.

Developers often discover this too late — after building an entire pipeline around a source that was never meant to be an API in the first place.

The symptoms are always the same:

Silent failures during market hours
Historical data with random gaps
No SLA, no support, no changelog

None of this is a coding problem. It's an infrastructure problem.

The Real Problem Is Infrastructure, Not Data

Stock data itself isn't scarce. Every exchange publishes it.

The real problem is structure: getting that data through a stable, documented, rate-limit-transparent REST API instead of a scraper held together with try/except blocks.

What to Actually Look For

Before picking any stock market API for a Python project, check for four things:

A real REST API — documented endpoints, not reverse-engineered JSON from a webpage
Clear rate limits — published numbers, not "fair use" vagueness
Both historical and real-time coverage — screeners need history, alerts need live data
Clean JSON responses — no HTML parsing, no regex, no fragile scraping logic

After testing multiple providers for backtesting and screener projects, I consistently reach for EODHD for this type of work. Here's why.

EODHD covers over 150,000 tickers across 70+ exchanges, with end-of-day, intraday, and real-time endpoints under one API key. That means one integration instead of stitching together three providers for three data types.

If you're a software or API company looking to explain your product through high-quality educational content (not marketing fluff), feel free to connect with me on LinkedIn.

Getting Stock Data in Python: A Working Example

Let's pull historical daily prices for a single ticker.

pip install requests pandas

import requests
import pandas as pd

API_TOKEN = "your_api_token"
symbol = "AAPL.US"

url = f"https://eodhd.com/api/eod/{symbol}"
params = {
    "api_token": API_TOKEN,
    "fmt": "json",
    "period": "d"
}

response = requests.get(url, params=params)
data = response.json()

df = pd.DataFrame(data)
df["date"] = pd.to_datetime(df["date"])
print(df[["date", "open", "high", "low", "close", "volume"]].tail())

Output:

        date    open    high     low   close    volume
495  2026-06-24  198.31  200.12  197.85  199.40  48213500
496  2026-06-25  199.50  201.03  198.90  200.77  39871200
497  2026-06-26  200.90  202.44  199.75  201.15  41205300
498  2026-06-29  201.20  203.01  200.44  202.63  36994800
499  2026-06-30  202.70  204.15  201.98  203.29  44012700

From here you can build:

a stock screener that filters by volume or volatility
a backtesting engine for a trading strategy
a real-time alert system on top of the same API key

No scraper to maintain. No parsing HTML. Just a request and a DataFrame.

Comparing the Top Stock Market APIs for Python

1. EODHD — Broad coverage, one API for everything

Covers historical, real-time, and fundamental data across global exchanges through a single REST API.

Pros

150,000+ tickers across 70+ exchanges
Historical, real-time, and fundamentals in one API key
Free tier available for testing before committing

Cons

Real-time data has a short delay on the lowest-tier plans
Some fundamental endpoints require a paid plan

Best for: developers who need historical + real-time + fundamentals without juggling three providers.

2. Alpha Vantage — Good for prototyping

A free-tier-first API popular for quick prototypes and learning projects.

Pros

Generous free tier for low-volume testing
Well-documented technical indicator endpoints

Cons

Rate limits are restrictive (5 calls/minute on the free tier)
Real-time data requires a premium plan

Best for: early-stage prototypes and academic projects, not production systems.

3. Massive — Strong for US real-time data

Focused heavily on US equities and options with fast real-time feeds.

Pros

Low-latency real-time data for US markets
WebSocket support for live streaming

Cons

Limited international exchange coverage
Pricier at the tiers where real-time data actually becomes usable

Best for: US-focused trading applications that need speed over breadth (formerly Polygon.io, now rebranded as Massive).

4. yfinance — Fine for personal projects, risky for production

An unofficial wrapper around Yahoo Finance's internal endpoints.

Pros

Free and instantly usable
No API key required

Cons

Not an official API — Yahoo can change the underlying structure anytime
No SLA, no support, frequent silent breakages

Best for: personal scripts and one-off analysis, never production systems.

How These Fit Together

If you're testing an idea over a weekend, yfinance is fine.

The moment that idea becomes a screener, a dashboard, or anything a client depends on, move to a documented REST API.

EODHD covers the widest range of use cases — historical, real-time, and fundamentals — without forcing you to combine multiple providers just to cover the basics.

Key Takeaways

A stock market API is infrastructure, not just a data source — treat the decision like one
Documented REST endpoints beat scraping every time production matters
Free tiers are enough to validate the integration before paying for anything

FAQs

❓ Is there a free stock market API for Python?
✅ Yes. EODHD, Alpha Vantage, and yfinance all offer free access. EODHD's free tier is the most practical for testing real projects since it includes both historical and limited real-time data under one key.

❓ What's the best stock API for real-time data?
✅ For US-only real-time feeds, Massive is strong. For global coverage combined with real-time data, EODHD covers more exchanges without needing a second provider.

❓ Can I use yfinance in a production app?
✅ Not recommended. It relies on unofficial Yahoo Finance endpoints with no SLA, so it can break without warning. Use it for prototypes only.

→ Get started with EODHD

You'll get access to:

Historical + real-time data across 70+ exchanges
A free tier to test before committing
Fundamentals and technical indicators under the same key

The real question isn't which API has the most features. It's which one you can still trust in six months.

Looking for technical content for your company? I can help — LinkedIn · kevinmenesesgonzalez@gmail.com

I Let Claude Screen 3,000 Stocks Using the EODHD API — Here's What Made the Cut

Kevin Meneses González — Fri, 10 Jul 2026 07:10:00 +0000

TL;DR: I built a 4-stage screening funnel on top of the EODHD Screener API, starting from a universe of 3,000+ US-listed stocks with a market cap above $150M. After filtering for profitability, liquidity, and a fair price relative to earnings, only 10 names survived. No hype, no "buy this now." I also cover two ways to extend this: connecting through EODHD's official MCP server so Claude can run the screen conversationally, and a cron-based script that runs the whole funnel every day and flags what changed. Full list, code, and setup below.

Most "AI picks stocks" content is theater.

Someone asks ChatGPT to "recommend 5 undervalued stocks," the model pulls from stale training data, and the article calls it analysis. No live data. No verifiable filter. No way for the reader to reproduce a single number.

I wanted to do the opposite.

So I built a real screening pipeline: Claude connected directly to the EODHD Stock Screener API, ran a live query against thousands of tickers, and narrowed them down using nothing but hard numeric filters. Every ticker, every price, every EPS figure in this article came from that live call, made on the day this was written.

If you're:

building a quant screening tool,
evaluating financial data APIs for an AI agent,
or just tired of "AI stock picks" content with zero data behind it,

this is for you.

The Problem With Most Stock Screeners

Free screeners on the big finance sites give you a handful of filters: market cap, sector, maybe a P/E range. Fine for a Sunday afternoon. Useless if you're trying to build something programmatic.

Paid terminals solve the flexibility problem but cost more than most independent investors are willing to spend just to test an idea.

And "AI-powered" stock tools usually skip the data problem entirely. They generate plausible-sounding tickers based on pattern matching, not a live query against an actual market database.

The real bottleneck isn't intelligence. It's access to structured, filterable data that an AI model can actually query.

Giving Claude a Real Data Source

EODHD's Screener API exposes exactly what a filtering pipeline needs: market capitalization, EPS, dividend yield, trading volume, sector, and price, across every ticker on the exchange, in one request.

Once Claude can call that endpoint directly, "screen the market" stops being a metaphor. It becomes a real, auditable sequence of API calls.

Here's the funnel I ran:

Stage 1 — Universe: US-listed common stocks with market cap > $150M
Stage 2 — Quality filter: profitable (positive EPS), liquid (200K+ average daily volume), priced above $5
Stage 3 — Value zone: market cap between $1B and $50B, dividend-paying, still liquid
Stage 4 — Ranking: sorted by earnings yield (EPS ÷ price) — the cheapest stocks relative to what they actually earn

Stage 1 alone returned more than 1,000 matches per page and kept going past the API's pagination cap — confirming a universe well north of 3,000 tickers once you include every US exchange EODHD covers at that market cap floor. From there, each stage cuts the field down hard.

Building the Screener

The setup is a single authenticated GET request per stage. No scraping, no manual downloads.

import requests
import pandas as pd

API_TOKEN = "YOUR_API_TOKEN"
BASE_URL = "https://eodhd.com/api/screener"

def run_screen(filters, sort="market_capitalization.desc", limit=100, offset=0):
    params = {
        "api_token": API_TOKEN,
        "filters": filters,
        "sort": sort,
        "limit": limit,
        "offset": offset,
    }
    r = requests.get(BASE_URL, params=params)
    r.raise_for_status()
    return pd.DataFrame(r.json()["data"])

# Stage 2 + 3 combined: profitable, liquid, mid/large-cap, dividend-paying
filters = (
    '[["market_capitalization",">",1000000000],'
    '["market_capitalization","<",50000000000],'
    '["exchange","=","us"],'
    '["earnings_share",">",0],'
    '["avgvol_1d",">",300000],'
    '["adjusted_close",">",5],'
    '["dividend_yield",">",0]]'
)

df = run_screen(filters)
df["earnings_yield"] = df["earnings_share"] / df["adjusted_close"]
df["implied_pe"] = 1 / df["earnings_yield"]

top10 = df.sort_values("earnings_yield", ascending=False).head(10)
print(top10[["code", "name", "sector", "adjusted_close", "earnings_share",
             "earnings_yield", "dividend_yield", "market_capitalization"]])

That's the whole pipeline. No manual screening, no spreadsheet gymnastics — just filters chained on top of each other until the list is short enough to actually read.

What Made the Cut

Out of the 3,000+ stock universe, these 10 names survived every filter: profitable, liquid, mid-to-large cap, and cheapest relative to their own earnings on the day of the screen.

Ticker	Company	Sector	Price	EPS	Earnings Yield	Implied P/E	Dividend Yield	Market Cap
JBS	JBS N.V.	Consumer Defensive (Packaged Foods)	$11.82	$1.62	13.7%	~7.3x	8.2%	$40.1B
EIX	Edison International	Utilities (Regulated Electric)	$74.78	$9.20	12.3%	~8.1x	4.5%	$28.8B
GFI	Gold Fields Ltd ADR	Basic Materials (Gold)	$32.88	$3.94	12.0%	~8.3x	6.9%	$30.0B
PYPL	PayPal Holdings	Financial Services (Credit Services)	$44.53	$5.33	12.0%	~8.4x	0.9%	$40.3B
VICI	VICI Properties	Real Estate (REIT – Gaming)	$26.09	$2.92	11.2%	~8.9x	6.7%	$29.5B
PUK	Prudential PLC ADR	Financial Services (Life Insurance)	$27.36	$3.02	11.0%	~9.1x	1.0%	$34.5B
HIG	Hartford Financial	Financial Services (Insurance)	$138.74	$14.12	10.2%	~9.8x	1.6%	$38.0B
EQT	EQT Corporation	Energy (Oil & Gas E&P)	$51.16	$5.21	10.2%	~9.8x	1.3%	$32.0B
EC	Ecopetrol SA ADR	Energy (Oil & Gas Integrated)	$15.13	$1.40	9.3%	~10.8x	4.4%	$30.2B
KB	KB Financial Group	Financial Services (Banks – Regional)	$116.74	$10.40	8.9%	~11.2x	2.7%	$41.0B

A few things jump out.

No mega-cap tech. Not one FAANG name survived, because none of them clear an earnings yield above 5% at current prices — the filter mechanically excludes anything priced richly relative to its own profits.

Heavy tilt toward energy, financials, and insurance. That's not a bias I introduced. It's what happens when you rank purely by EPS-to-price, in mid-2026, across a market where growth names still carry premium multiples.

Real geographic spread. A Brazilian meat producer, a South African gold miner, a Colombian state oil company, a South Korean bank, and a UK insurer all made a US-dollar-denominated cut. The screener doesn't care about domicile — it cares about the number.

None of this is a buy recommendation. Cheap relative to earnings doesn't mean safe, and several of these names (EIX, in particular, given wildfire litigation exposure; JBS, given historical governance concerns) carry known company-specific risk that a pure numeric filter can't see. This is a starting list for further research, not a portfolio.

Skipping the Code: EODHD's Official MCP Server

Everything above ran through raw HTTP calls, which is the right approach when you're building something you'll productionize. But there's a second way to do this that's worth knowing about, especially if you want to iterate on filter ideas conversationally instead of rewriting Python every time.

EODHD ships an official Model Context Protocol (MCP) server — a standard that lets AI clients like Claude Desktop, Claude Code, and ChatGPT call external tools directly, without you writing the request-handling code yourself. Anthropic created MCP specifically to solve this "every data source needs its own custom integration" problem.

EODHD's server exposes 70+ read-only tools covering the same ground as the REST API: the screener, fundamentals, historical and intraday prices, news and sentiment, technical indicators, macro indicators, US options, and more. It comes in two flavors:

v2 (OAuth) — the simpler setup for Claude Desktop. You paste the server URL into Settings → Extensions, authorize through a consent flow, and you're done. No API key handling.
v1 (API key) — for clients like ChatGPT or Claude Code that expect the key passed directly, or if you'd rather manage auth yourself.

Once it's connected, you don't write a requests.get() call at all. You just ask:

"Screen US stocks with market cap between 1B and 50B, positive EPS, dividend yield above 0, and rank them by earnings yield. Show me the top 10."

Claude translates that into the same stock_screener tool call I used to build this article, executes it against live EODHD data, and returns the table. This is exactly how I pulled the numbers above — no manual API scripting for the exploratory stage, just a direct conversation with the data.

Where this matters in practice: the raw API approach is better once you know exactly what you're building — a cron job, a dashboard, a backtest. The MCP approach is better while you're still deciding what the funnel should even look like, since you can adjust a filter, re-run, and see the new output in seconds without touching code.

Both hit the same underlying EODHD infrastructure and consume the same API quota, so there's no data-quality tradeoff — it's purely a question of whether you want a repeatable script or a fast conversational loop.

From Here You Can Build

This exact funnel took four API calls and about 30 lines of code. From here you can build:

automated weekly screener reports delivered to Slack or email
a Streamlit dashboard where the filters are sliders instead of hardcoded values
a backtesting loop that checks whether "cheap earnings yield + dividend-paying + mid-cap" actually outperforms over rolling 12-month windows
a signal layer on top using EODHD's wallstreet_lo / wallstreet_hi signals to cross-check against analyst targets

Get real-time and historical stock data for your own screeners
EODHD gives you programmatic access to fundamentals, screener queries, and live pricing across global exchanges — the same endpoints used for every number in this article.
→ Start with EODHD's API

Turning This Into a Daily Job

A one-time screen is a snapshot. The actual value shows up when you run it every trading day and track what changes — which names drop out because they got expensive, and which new ones enter because they got cheap or a fresh earnings report shifted the EPS.

Here's a script that runs the full 4-stage funnel, saves the day's result, diffs it against yesterday's, and writes a short summary. Point it at cron and forget about it.

import requests
import pandas as pd
from datetime import date
from pathlib import Path

API_TOKEN = "YOUR_API_TOKEN"
BASE_URL = "https://eodhd.com/api/screener"
OUTPUT_DIR = Path("screener_history")
OUTPUT_DIR.mkdir(exist_ok=True)

FILTERS = (
    '[["market_capitalization",">",1000000000],'
    '["market_capitalization","<",50000000000],'
    '["exchange","=","us"],'
    '["earnings_share",">",0],'
    '["avgvol_1d",">",300000],'
    '["adjusted_close",">",5],'
    '["dividend_yield",">",0]]'
)

def run_screen():
    params = {
        "api_token": API_TOKEN,
        "filters": FILTERS,
        "sort": "market_capitalization.desc",
        "limit": 100,
        "offset": 0,
    }
    r = requests.get(BASE_URL, params=params)
    r.raise_for_status()
    df = pd.DataFrame(r.json()["data"])
    df["earnings_yield"] = df["earnings_share"] / df["adjusted_close"]
    return df.sort_values("earnings_yield", ascending=False).head(10)

def save_and_diff(today_df):
    today_str = date.today().isoformat()
    today_path = OUTPUT_DIR / f"{today_str}.csv"
    today_df.to_csv(today_path, index=False)

    history = sorted(OUTPUT_DIR.glob("*.csv"))
    if len(history) < 2:
        return "First run — no previous data to compare against."

    yesterday_df = pd.read_csv(history[-2])
    today_codes = set(today_df["code"])
    yesterday_codes = set(yesterday_df["code"])

    new_entries = today_codes - yesterday_codes
    dropped = yesterday_codes - today_codes

    summary = []
    if new_entries:
        summary.append(f"New in today's top 10: {', '.join(sorted(new_entries))}")
    if dropped:
        summary.append(f"Dropped out: {', '.join(sorted(dropped))}")
    if not new_entries and not dropped:
        summary.append("No changes in the top 10 today.")

    return "\n".join(summary)

if __name__ == "__main__":
    result = run_screen()
    summary = save_and_diff(result)
    print(summary)

Schedule it with a one-line crontab entry to run every weekday after market close:

0 22 * * 1-5 /usr/bin/python3 /path/to/daily_screener.py >> /path/to/screener.log 2>&1

From there, the summary string is trivial to route anywhere:

Post it to a Slack channel with a webhook (requests.post(SLACK_WEBHOOK_URL, json={"text": summary}))
Email it to yourself using the same pattern as any transactional email script
Append each day's CSV to a running Google Sheet so you get a queryable history instead of a folder of files

The interesting part isn't the top 10 on any single day. It's watching turnover — a name entering the list usually means its price dropped or its earnings just improved enough to make the yield attractive. A name leaving usually means the opposite. That turnover signal is something a one-off screenshot can never give you.

Key Takeaways

A 4-stage numeric funnel turned a 3,000+ stock universe into 10 names in under 30 lines of Python.
Earnings yield (EPS ÷ price) is a simple, brutally objective ranking metric — no sentiment, no narrative, no model hallucination.
The filter mechanically excludes expensive growth names and surfaces cheap, profitable, dividend-paying companies across sectors and countries. That's the point of a rules-based screen: it doesn't know what's "supposed" to be popular.
EODHD's MCP server lets you skip the API scripting entirely while you're still iterating on filter logic — same data, conversational interface.
The real value isn't a single day's list. It's a cron job that runs the funnel daily and tells you what entered or left the top 10, because turnover is the actual signal.

FAQs

❓ Can Claude actually query live financial data, or is this just training data?
✅ Claude called the EODHD Screener API directly for this article, meaning every number reflects the market on the day of the query, not a training snapshot. That distinction matters — most "AI stock pick" content is quietly using stale, memorized data.

❓ Is earnings yield a reliable way to find undervalued stocks?
✅ It's a useful, transparent starting filter, not a complete valuation model. A high earnings yield can mean genuine value or it can mean the market is pricing in a real risk (regulatory, legal, competitive) that the number doesn't capture. Always follow up with qualitative research before acting.

❓ Do I need a paid EODHD plan to run this screener?
✅ The Screener API is available on EODHD's All-In-One and All World Extended plans. Each screener request consumes 5 API calls, so a 4-stage funnel like this one costs 20 calls total — trivial even on a modest plan.

❓ What's the difference between calling the EODHD API directly and using their MCP server?
✅ Same underlying data and the same API quota either way. Direct API calls make sense once you're building a repeatable script, like the daily cron job above. The MCP server makes sense while you're still shaping the filter logic, since you can adjust it in plain language and see results immediately without touching code.

❓ How do I get alerted automatically instead of checking the screener manually?
✅ Run the funnel on a schedule (cron, Task Scheduler, or a serverless function), save each day's output, and diff it against the previous run. Route the diff to Slack, email, or a spreadsheet — the script in this article does exactly that in under 80 lines.

The market doesn't reward the investor with the best opinions.

It rewards the one with the best process for filtering out noise.

Looking for technical content for your company? I can help — LinkedIn · kevinmenesesgonzalez@gmail.com

7 Best WhatsApp APIs for Developers in 2026 (Compared)

Kevin Meneses González — Thu, 09 Jul 2026 10:02:19 +0000

Most developers reach for Twilio or Meta's official Cloud API the second WhatsApp comes up.

That's usually the wrong first move.

The official API wants Business verification, template approval, and per-message billing before you've sent a single test message. Twilio piles its own markup on top. If you're building an AI agent or a support bot that needs to talk to customers this week, you're looking at weeks of paperwork before you write a line of actual logic.

If you're:

Prototyping a WhatsApp AI agent,
shipping a chatbot fast,
or running outbound automation and want to skip Meta's approval process,

keep reading.

Why "official" isn't always the right call

The official WhatsApp Business Platform makes sense if you need the green checkmark, or you're in a regulated industry where Meta's compliance guarantees actually matter.

But getting there is slow.

Business Manager verification. App Review. Template approval that can sit for days. Per-message fees that swing wildly by country and category — marketing messages cost ten times more than utility ones in some markets.

A lot of developers don't need any of that. They just need WhatsApp wired into a backend, an LLM, or an automation pipeline, and that's where unofficial APIs come in. They connect through WhatsApp Web's protocol instead of Meta's Business Platform. You lose some compliance guarantees, you gain instant setup and flat pricing you can actually predict.

I've seen both sides of this go wrong, and not in the way people expect.

A developer picks an unofficial API to ship an MVP fast. It works, customers are happy, volume grows — and then the number gets flagged out of nowhere, because nobody warmed it up or thought about WhatsApp's spam detection. Or it goes the other direction: a team picks the official Cloud API for what's basically an internal notification bot, and burns two weeks on Business Manager and template approval for a use case that never needed that level of compliance in the first place.

Same root mistake both times. Nobody matched the API to their actual message volume and risk tolerance before committing.

TL;DR

This is a rundown of the best WhatsApp APIs for developers right now — official and unofficial, with real pricing, so you can pick the right one for an AI agent, a chatbot, or whatever automation you're building instead of guessing.

Best official WhatsApp API for AI agents: Zernio. No markup on Meta's fees, numbers from $2/month, one API call to get live.
Best flat-rate unofficial option: WAAPI at $10/mo or Whapi.Cloud at $29/mo — no per-message charges, you know your bill in advance.
Best free option if you want to self-host: CodeChat. Open source, the only ongoing cost is your own server.

1. Zernio — the official API minus the paperwork

Zernio runs on the real WhatsApp Business Platform, but somebody on their team clearly got tired of Meta's onboarding and decided to fix it.

Instead of creating a Meta app, passing App Review, and wiring up webhooks by hand, Zernio uses Meta's Embedded Signup flow. You connect a number from a dashboard in one click. No Business Manager rabbit hole.

What you actually get:

WhatsApp numbers in 53 countries, provisioned through the API, starting at $2/month. Zero markup on Meta's per-message fees — you pay what Meta charges, not the 3–5x tax Twilio adds on top. Template CRUD with automatic category tracking, so you don't get blindsided by a template silently getting reclassified into a more expensive tier. Webhooks come back in the same JSON format across WhatsApp, Instagram, and Telegram, which matters if your agent talks to more than one channel.

There's also a hosted MCP server, which means an AI agent (Claude, your own backend, whatever) can manage WhatsApp conversations through plain tool calls instead of you writing glue code for every action.

Pros

Official API, so you're not gambling on an unofficial protocol getting your number banned
Platform fee scales down at low volume, first two connected accounts are free
WhatsApp Calling API support, useful if you want to route voice to an agent

Cons

You still pay Meta's per-message fees on top, Zernio removes the markup, not the underlying cost
No visual dashboard built for non-technical teammates, this is built for developers

Best for: developers building AI agents or SaaS products who want the official API without losing a month to Meta's setup.

👉 Skip the Meta developer portal entirely — Connect a WhatsApp number and start sending messages in one API call, official Business Platform, zero markup.

2. UltraMsg — fastest unofficial setup

UltraMsg connects through WhatsApp Web's protocol. Scan a QR code, grab an instance ID and token, you're sending messages in about five minutes.

Pricing is $39/month per instance, or $390 if you pay yearly.

The upside is flexibility. Any language that can make an HTTP request can talk to it, and there's no hard message cap from UltraMsg's side (WhatsApp's own anti-spam behavior still applies, that part never goes away). The downside is the same one every unofficial API shares: your account lives or dies on WhatsApp Web session stability, not Meta's guarantees. And if you're running more than one number, the per-instance pricing adds up fast.

Good fit for a solo developer or small business that wants a working integration today and has no interest in touching Meta's portal.

3. Green API — the cheapest way to test an idea

Green API splits into two plans: Developer, which is free but capped, and Business, which runs around $8/month per instance for unlimited messaging.

What I like here is that the free tier is actually usable. It's not a three-day trial dressed up as a free plan — you can build and test a real MVP on it before paying anything. There's also a daily-billing "Partner" option if your workload is seasonal.

The tradeoffs: the free plan only handles a handful of individual chats per month, so it's testing-only, not production. And the documentation leans toward Russian-market use cases, though the English docs cover what you need.

Best for validating an idea before you commit to a paid plan anywhere.

4. WAAPI — one flat price, no surprises

WAAPI charges $10/month per instance, flat. Unlimited messages, every feature included, no tier you have to upgrade into.

That simplicity is the whole pitch. You're not parsing a pricing page trying to figure out which plan unlocks webhooks. Everything's already there.

It's a smaller company than UltraMsg or Green API, so there's less of a track record to lean on, and the same unofficial-protocol ban risk applies as everywhere else on this list. But if predictable billing matters more to you than brand history, it's hard to beat.

5. Whapi.Cloud — built for groups and channels

Whapi.Cloud does something most of the others don't bother with well: WhatsApp Groups, Channels, and Status updates. If your agent needs to post to a channel rather than just DM individual users, this is the one that actually handles it properly.

There's a free sandbox (150 messages a day, 5 active conversations a month) and then $29/month per connected number once you outgrow it.

It plugs natively into n8n, Make, and Zapier, which is nice if your team leans no-code. The sandbox is also a genuine free-forever tier, not a countdown timer. The catch is that $29/month is the second-highest flat rate on this list, and the sandbox limits make it useless past the prototype stage.

Best for anyone building Groups or Channels automation specifically, not plain one-to-one messaging.

6. CodeChat — free, open source, your server

CodeChat is for developers who'd rather run their own infrastructure than pay anyone monthly. It's built on the Baileys library, the same WebSocket engine a lot of commercial APIs use under the hood, and ships as a self-hosted REST API.

Cost: nothing, beyond whatever VPS you run it on.

The appeal is obvious if you've ever felt locked into a vendor. Full control over your data, no recurring API bill, and the codebase is battle-tested enough that it later became the foundation for Evolution API. The cost is that you're now the one responsible for uptime, scaling, and keeping the WhatsApp Web layer updated whenever WhatsApp changes something underneath you. There's no support line to call, just community docs.

If you're comfortable with Docker and don't mind owning the ops side, this is the only option on the list with zero recurring cost.

7. Wassenger — built for teams, not just developers

Wassenger is the odd one out here. It's not just an API, it's a full platform: shared team inbox, AI-drafted replies through Claude or ChatGPT, campaign tools.

Pricing runs €39.90 for Professional, €69.90 for Business, €99.90 for Enterprise, all on the official WABA.

If you need a human reviewing AI-drafted replies before they go out, the draft-and-approve workflow is genuinely useful. It also means you're not betting on an unofficial protocol. But it's the most expensive entry point on this list, and if all you need is raw API access for a backend integration, you're paying for a team inbox you'll never open.

Best for teams that want a shared inbox and AI-assisted replies bundled with their API access.

Comparison table

API	Type	Starting price	Best for
Zernio	Official	$2/month	AI agents, SaaS products
UltraMsg	Unofficial	$39/month	Fast solo setup
Green API	Unofficial	Free (capped) / $8/mo	Testing an idea
WAAPI	Unofficial	$10/month flat	Predictable billing
Whapi.Cloud	Unofficial	Free sandbox / $29/mo	Groups & Channels
CodeChat	Unofficial (self-hosted)	Free (server cost only)	Full control, zero recurring fee
Wassenger	Official	€39.90/month	Team inbox + AI replies

How to choose

Need the green checkmark, or operating somewhere regulated? Start with Zernio or Wassenger. Both run on Meta's official platform, but Zernio cuts the setup time and the per-message markup.

Just need something working today and can live with unofficial-protocol risk? WAAPI or Green API get you there for the least money.

Optimizing for zero recurring cost and full control over your stack? CodeChat is the only one here that doesn't send you a bill every month.

FAQs

What's the difference between official and unofficial WhatsApp APIs?
Official APIs like Zernio and Wassenger connect through Meta's WhatsApp Business Platform. You need Business verification and you pay per message. Unofficial APIs (UltraMsg, Green API, WAAPI, Whapi.Cloud, CodeChat) connect through WhatsApp Web's protocol instead. Setup takes minutes and pricing is usually flat per number, but there's real ban risk if you send like a spammer.

Is there a free WhatsApp API for developers?
Green API's free Developer plan is actually usable for testing, not just a teaser. CodeChat is free outright since it's open source and self-hosted — your only cost is the server it runs on.

Which WhatsApp API is best for AI agents?
Zernio is built for this specifically. It ships a hosted MCP server so an AI agent can manage WhatsApp conversations through tool calls, and it's on the official API, so you're not worried about getting banned for sending too much volume.

Can I use an unofficial WhatsApp API without getting banned?
The risk is real but manageable. Warm new numbers up slowly, only message people who opted in, don't blast a fresh number with bulk sends. If you're doing cold outbound at real scale, the official API is the safer bet.

Do these APIs charge per message?
Most unofficial ones on this list (UltraMsg, WAAPI, Whapi.Cloud) charge flat monthly fees, nothing per message. The official ones (Zernio, Wassenger) pass through Meta's per-message fees, which vary by country and message type.

If you're a software or API company looking to explain your product through high-quality educational content (not marketing fluff), feel free to connect with me on LinkedIn: linkedin.com/in/kevin-meneses-gonzalez

Looking for technical content for your company? I can help — LinkedIn · kevinmenesesgonzalez@gmail.com

How LLMs Are Replacing Legacy OCR Workflows

Kevin Meneses González — Wed, 08 Jul 2026 16:21:51 +0000

TL;DR:

OCR digitizes a page. AI document processing turns that page into structured, usable data — no templates, no rules to maintain. Unstract is an open-source, no-code platform that does this with LLMs: define what you want extracted in plain language, deploy it as an API or ETL pipeline, and get clean JSON back regardless of how the document is laid out. Start with the free 14-day trial or read the quick start guide — no credit card required.

OCR was never broken.

It just stopped being enough.

For thirty years, optical character recognition did exactly what it promised: turn pixels into characters. Feed it a scanned invoice, get back a string of text. That was the job, and it did the job well.

But reading text and understanding a document are two different problems. And the industries that depend on documents the most — finance, accounting, insurance, capital markets — have spent years discovering that the hard way.

If you're:

building automation for invoices, statements, or contracts,
maintaining an OCR pipeline that breaks every time a vendor changes their layout,
or evaluating whether to move from rule-based extraction to something smarter,

this is for you.

The 200-Bank Problem

Picture a finance team that receives bank statements from 200 different banks.

Each bank uses its own layout. Different field labels. Different date formats. Some put the account balance top-right; others bury it in a footer table. A few still send scanned PDFs that look like they were faxed twice.

Template-based OCR handles this the only way it knows how: one template per bank. 200 templates. Each one is built by hand, each one breaking the moment a bank redesigns its statement.

Now widen the lens.

An insurance underwriter doesn't just deal with bank statements — they deal with claims forms, KYC files, medical reports, and policy applications, each shaped differently depending on the insurer, the state, and the line of business. An accounting team doesn't process one invoice format — it processes one for every single vendor it works with, and vendors change their templates without asking anyone's permission.

Multiply any of these by real transaction volume, and you get a pattern every ops team eventually recognizes.

A Monday morning that repeats itself:

Someone opens 40 PDFs by hand.
Copy numbers into a spreadsheet, one cell at a time.
Pray the format didn't change since last month.
Finds three that don't match any known template and sets them aside "for later."

That's not automation. That's a workaround wearing automation's clothes.

The Real Problem Was Never Reading Text

Here's the reframe.

The problem was never extracting characters from a page. OCR solved that decades ago, and modern OCR engines hit 99%+ accuracy on printed text and roughly 95% on handwriting.

The problem is that reading text and understanding a document are not the same task.

A human accountant doesn't process an invoice by reading every character in order. They recognize this is an invoice, they know the total usually sits near the bottom, and they adjust instantly when the layout is unfamiliar. That's comprehension, not character recognition.

Legacy systems never had that layer. AI document processing exists because LLMs finally do.

That distinction is also why "just add more OCR accuracy" was never going to solve this. You can push OCR accuracy on printed characters as close to 100% as you like, and it still won't tell you which extracted number is the invoice total versus a line-item subtotal. That's a comprehension task, not a recognition task — and comprehension is what large language models bring to the table that thirty years of OCR research never could.

What Is AI Document Processing, Actually

AI document processing is the use of large language models to read, interpret, and structure the content of unstructured documents — PDFs, scans, images, spreadsheets — without relying on fixed templates or hand-coded rules.

Instead of matching a document against a predefined layout, an LLM-based pipeline reads the document the way a person would: it understands that a number preceded by "Total Due" is different from a number labeled "Subtotal," even if their position on the page changes completely from one document to the next.

Here's how the three generations actually compare:

Traditional OCR isn't obsolete — for one repetitive form, from one source, in one layout, it's still fast and cheap. But finance, insurance, and accounting teams rarely deal with one format. They deal with hundreds of variants of the same document type, arriving in whatever shape a vendor or client happens to send.

How the Pipeline Actually Works, Step by Step

Under the hood, an AI document processing pipeline is doing four distinct things, whether you notice them or not:

Parsing — turning the raw file (scanned PDF, native PDF, image, even a photo of a paper form) into text while preserving layout: tables stay tables, checkboxes stay checkboxes.
Schema definition — describing, in plain language or a JSON schema, exactly what fields matter: customer name, invoice total, claim number, whatever the business needs.
Extraction — the LLM reads the parsed document against that schema and returns structured values.
Validation — checking the extracted values for consistency, flagging low-confidence fields instead of guessing.

Most teams that try to build this themselves get step 3 working in a weekend with a raw LLM API call. Step 1 and step 4 are where things quietly fall apart in production — layout gets flattened, and a hallucinated total slips through with full confidence.

Why Building This In-House Usually Stalls

Plenty of engineering teams start this project the same way: a Python script, an LLM API call, a PDF-to-text library, done in an afternoon.

It works — on the five sample documents used to build it.

Then it hits production, and three things happen, almost every time:

A multi-column layout gets flattened into a single blob of text, and the LLM starts guessing which number belongs to which field.
The model hallucinates a value with full confidence on a document type it hasn't seen before, and nobody notices until the number is already in a report.
Token costs creep up as documents get longer, because nobody built in compression or prompt efficiency from day one.

None of these are LLM problems. They're pipeline problems — the parsing, validation, and cost layers that a raw API call skips entirely. This is precisely the gap purpose-built platforms are designed to close, instead of every team rebuilding the same missing pieces independently.

The Trend: From Templates to Prompts

Something changed in the last 24 months.

Vision-capable LLMs and layout-aware parsers made it possible to feed a document into a pipeline and get structured output back — without teaching the system what that specific document looks like first.

That shift shows up everywhere:

Schema-first extraction. You define what you want (a JSON schema), not how the document is laid out. Even foundation model providers have moved this direction — structured outputs are now a first-class feature in most major LLM APIs.
Agentic workflows. Instead of a human writing extraction rules by hand, an AI agent generates and refines the extraction logic itself, iterating against sample documents until accuracy stabilizes.
Confidence over guesswork. Modern pipelines can flag low-confidence extractions instead of silently returning wrong data — a "no value" is safer than a wrong one on a financial document.
Compliance pressure. New 2026 obligations like the EU AI Act push regulated industries — banking, insurance, life sciences — toward document workflows with audit trails and traceability, something rule-based OCR was never built to provide.

Finance and insurance are leading this shift for a simple reason: they carry the highest document volume and the highest cost of a wrong number. A misread account balance isn't a UX bug — it's a compliance problem. BFSI (banking, financial services, insurance) alone represents close to a third of the entire IDP market in 2026, and that share keeps growing as more of that volume moves to LLM-based extraction.

There's a second force behind this trend worth naming: MCP (Model Context Protocol) and the broader move toward agentic systems. AI agents making decisions — approving a claim, flagging fraud, triggering a payment — need structured, trustworthy input to act on. An agent can't reason well over a wall of unformatted OCR text. It needs the same clean JSON a downstream database would expect. That's pushing document extraction from being a back-office utility to being infrastructure other AI systems depend on directly.

This is exactly the gap Unstract was built to close.

What This Costs vs. What It Replaces

It's worth being blunt about the economics, because "AI is expensive" is the objection that kills most of these projects before they start.

LLM token costs on a single document are real, but they're not the number that matters. The number that matters is the fully-loaded cost of the alternative: a person spending 15–20 minutes per document, multiplied by hundreds of documents a week, multiplied by the error rate of manual re-keying.

Token-efficiency techniques like SinglePass and Summarized Extraction exist specifically because vendors know this comparison only holds up if per-document AI cost stays a small fraction of a person's hourly rate — which, for anything beyond a handful of documents a day, it does.

Where Unstract Fits

Unstract is an open-source, no-code platform purpose-built for extracting structured data from unstructured documents using LLMs — deployable as an API or as an ETL pipeline, without writing extraction rules by hand.

It's not a single model. It's a full pipeline, and each piece solves a specific failure mode that shows up when you try to run LLM extraction in production:

LLMWhisperer — a layout-preserving text extraction engine. It keeps tables, multi-column layouts, checkboxes, and handwritten fields intact before anything reaches the LLM, so the model isn't guessing at structure that got flattened away.
Prompt Studio — a no-code environment where you build extraction logic against real sample documents, compare output and cost across multiple LLMs side by side, and version your prompts instead of losing track of them in a spreadsheet.
LLMChallenge — Unstract's answer to hallucination. Two independent LLMs extract the same field; if they don't agree, you get NULL instead of a confidently wrong number. No value is safer than a wrong one on a financial document.
SinglePass and Summarized Extraction — techniques that cut token usage by up to 7x on large documents, which matters the moment you're processing thousands of statements a month.
API Hub and MCP Server — once a schema is built, deploy it as a callable API in one click, or connect it directly to AI agents that need reliable structured data as an input.

A few concrete benefits this unlocks:

No templates, no per-vendor maintenance. The same extraction project handles bank statements from 200 different banks, because it understands the concept of "account balance," not the position of a specific box.
Built-in hallucination control. LLMChallenge means you get validated data or nothing — never a silent wrong value slipping into a downstream system.
Open-source, no vendor lock-in. Unstract is released under AGPL 3.0. Self-host it, inspect the code, or use the managed cloud edition — your choice, your data.
Deploy without writing a pipeline from scratch. A finished extraction project becomes an API or ETL job in one click, feeding straight into Snowflake, BigQuery, or your existing database.
Document-agnostic by design. No prior training run needed per document type — you point it at samples, define the schema, and it generalizes.

👉 Try Unstract free for 14 days — no credit card required, pre-configured with an LLM, vector database, and LLMWhisperer.

You can read the full breakdown of the platform in the Unstract documentation if you want to see how the pieces connect before touching any code.

Practical Walkthrough #1: Extracting a Bank Statement

Theory is fine, but this only matters if it holds up against a real document. Let's make it concrete.

Say you receive credit card statements as PDFs — same core fields (customer name, issuer, statement date, list of transactions), wildly different formatting depending on the bank. This is one of the most common entry points teams use to test AI document processing, precisely because the pain of maintaining per-bank templates is so immediate and so familiar to anyone who has tried automating reconciliation before.

With a template-based system, this is where things get painful. With Unstract, the workflow looks like this:

Upload a handful of sample statements (from different banks) into Prompt Studio.
Define your schema in plain language — "extract customer name, statement date, total due, and list of transactions with date/description/amount."
Let Prompt Studio generate and test the extraction prompt against your samples, comparing accuracy and cost across LLMs.
Deploy as an API.

Once deployed, calling it looks like a normal REST request:

import requests

url = "https://api.unstract.com/deployment/api/<your-org>/<your-endpoint>/"
headers = {"Authorization": "Bearer YOUR_API_KEY"}
files = {"file": open("statement_chase.pdf", "rb")}

response = requests.post(url, headers=headers, files=files)
print(response.json())

The output is standard, structured JSON — regardless of which bank the statement came from:

{
  "customer_name": "John Doe",
  "issuer": "Chase",
  "statement_date": "2026-06-30",
  "total_due": 1284.55,
  "transactions": [
    {"date": "2026-06-02", "description": "Amazon.com", "amount": 42.99},
    {"date": "2026-06-05", "description": "Uber", "amount": 18.30}
  ]
}

From here you can build:

automated reconciliation pipelines that flag discrepancies
fraud detection models fed by clean transaction data
dashboards that update without anyone touching a spreadsheet

No template. No rule that breaks the next time a bank redesigns its statement.

Practical Walkthrough #2: Automating Vendor Invoice Approval

Here's a second example — a workflow that's easy to replicate if you're on an accounting or AP (accounts payable) team.

The scenario: invoices arrive by email from hundreds of vendors, every one with a different layout. Someone currently opens each one, checks it against a purchase order, and manually keys the total into the accounting system.

The schema you'd define in Prompt Studio:

{
  "vendor_name": "string",
  "invoice_number": "string",
  "invoice_date": "date",
  "due_date": "date",
  "line_items": [
    {"description": "string", "quantity": "number", "unit_price": "number", "line_total": "number"}
  ],
  "subtotal": "number",
  "tax": "number",
  "total_due": "number"
}

Once that schema is validated against a sample of invoices from different vendors, this becomes an ETL pipeline instead of a one-off API call: drop incoming invoices into a shared Google Drive or S3 folder, and Unstract automatically extracts and pushes structured rows into your accounting database.

# Pseudocode for the downstream logic once structured data lands in your DB
for invoice in new_extracted_invoices:
    po = match_purchase_order(invoice["vendor_name"], invoice["total_due"])
    if po and abs(po.amount - invoice["total_due"]) < 0.01:
        approve_for_payment(invoice)
    else:
        flag_for_manual_review(invoice, reason="PO mismatch")

The extraction layer's only job is turning a messy PDF into the JSON above. Everything downstream — matching against POs, approving payment, flagging exceptions — becomes a business logic problem instead of a "can we even read this PDF" problem.

That's the actual unlock: once documents become structured data, the rest of your stack stops needing to know it ever came from a PDF at all.

Where This Pays Off: Use Cases by Industry

1. Finance — Multi-bank statement reconciliation

The pain: A treasury or finance team pulling statements from multiple banking relationships hits the same wall every close cycle — different formats, different field labels, no shared structure. Reconciliation turns into a manual, error-prone task repeated every single month.

How AI document processing helps: One schema handles every bank's statement layout, because the extraction logic understands the concept of "closing balance" or "transaction date" rather than a fixed position on the page. The output feeds directly into reconciliation logic without a human retyping anything.

Best for: Teams closing books monthly across multiple banking relationships, or finance functions managing treasury operations across subsidiaries with different local banks.

2. Insurance — Underwriting and claims processing

The pain: Claims forms, KYC documents, and medical reports vary by insurer, state, and line of business — a nightmare for template-based systems that need a new template for every combination.

How AI document processing helps: Schema-based extraction generalizes across formats instead of requiring a template per variant. Teams running this kind of pipeline report review times dropping from days to minutes, simply because the bottleneck was never document volume — it was the seams between formats that broke automation.

Best for: Underwriting teams processing high volumes of inconsistent intake documents, and claims teams under pressure to cut turnaround time without adding headcount.

3. Accounting — Accounts payable automation

The pain: Every vendor invoices differently, and manual entry doesn't scale past a few dozen suppliers before someone becomes a full-time human OCR machine.

How AI document processing helps: Structured extraction feeds directly into PO-matching and payment approval logic, exactly as shown in Walkthrough #2 above — the invoice becomes a JSON object the moment it lands, and everything downstream is business logic, not document parsing.

Best for: AP teams handling invoices from 50+ vendors with no shared format, or finance ops teams trying to close the "three-way match" (PO, invoice, receipt) faster.

4. Capital markets — KYC and client onboarding

The pain: Onboarding documents (ID scans, proof of address, account applications) arrive in every format imaginable, and compliance requires traceable, auditable extraction — not just a fast one.

How AI document processing helps: Layout-aware parsing handles scanned and handwritten fields that break traditional OCR, while validation layers like dual-LLM consensus create the audit trail regulators increasingly expect under frameworks like the EU AI Act.

Best for: Onboarding teams under regulatory pressure to document exactly how each data point was extracted and verified, not just that it was extracted.

5. Legal and contracts — Clause and obligation extraction

The pain: Contracts share no common template, and finding specific clauses (payment terms, termination conditions, indemnification language) manually doesn't scale across a growing contract portfolio.

How AI document processing helps: Prompt-based extraction can target specific clause types across contracts of any format and length, structuring what used to require a paralegal reading every page line by line.

Best for: Legal ops teams managing large, heterogeneous contract repositories, or M&A teams running due diligence against hundreds of agreements on a deadline.

6. Healthcare-adjacent finance — Medical billing and claims reconciliation

The pain: Medical billing documents mix structured codes (CPT, ICD-10) with unstructured physician notes and inconsistent payer formats, making reconciliation between what was billed and what was paid painfully manual.

How AI document processing helps: Extraction schemas can target both the structured codes and surrounding context simultaneously, flagging discrepancies between billed and reimbursed amounts without a human cross-referencing every line.

Best for: Revenue cycle teams reconciling claims across multiple payers with inconsistent explanation-of-benefits formats.

Across every one of these use cases, the underlying pattern repeats: the documents are different, the pain is identical, and the fix is the same — stop building a template per format and start defining a schema per outcome.

Getting Started, Practically

If you want to test this against your own documents rather than take any of this on faith, the fastest path looks like this:

Pull 5–10 real samples of the document type that causes you the most pain — the messiest ones you have, not the cleanest.
Sign up for the free 14-day trial — it comes pre-configured with an LLM, vector database, embedding model, and LLMWhisperer, so there's nothing to wire up first.
Define your schema in Prompt Studio using plain language, exactly like the JSON examples in Walkthrough #1 and #2 above.
Run it against your samples and compare accuracy and cost across the available LLMs before committing to one.
Deploy as an API or ETL pipeline once the extraction is accurate enough to trust — no separate infrastructure to stand up.

The quick start guide walks through this exact sequence using a credit card statement example, if you want a guided first run before pointing it at your own documents.

Key Takeaways

OCR reads characters. AI document processing understands context — that gap is the entire reason legacy IDP struggles with document variation.
The real cost of template-based systems isn't the software. It's the ongoing human maintenance every new layout demands.
Platforms like Unstract combine layout-aware parsing, schema-based extraction, and hallucination control into one deployable pipeline — so you build the extraction logic once and let it handle the variation, whether that's bank statements, invoices, or insurance claims.

FAQs

❓ What is AI document processing?
✅ It's the use of large language models to read, interpret, and structure data from unstructured documents — PDFs, scans, images — without relying on fixed templates. Unlike OCR, which only extracts text, it extracts meaning: it understands what a field represents, not just where it sits on the page.

❓ Is OCR still useful in 2026?
✅ Yes, for narrow cases: high-volume batches of the exact same form, from the exact same source, where digitizing text is the only goal. For anything with real-world document variation, OCR alone isn't enough — that's the problem AI document processing solves.

❓ Is Unstract open source?
✅ Yes. Unstract is released under the AGPL 3.0 license, with a self-hosted open-source edition and a managed cloud edition that adds enterprise features like LLMChallenge, SSO, and human review workflows.

❓ Do I need to know how to code to use Unstract?
✅ No. Prompt Studio is a no-code environment — you define what to extract in natural language against sample documents. Developers can go further and call the resulting extraction as a REST API or embed it in an ETL pipeline.

❓ Can AI document processing handle handwritten documents?
✅ Yes. Layout-aware parsing engines like LLMWhisperer include handwritten text detection and checkbox/radio button recognition, which is where template-based OCR historically struggled the most.

❓ How is this different from just calling an LLM API directly with a PDF?
✅ A raw LLM call skips the layers that make extraction reliable in production: layout-preserving parsing so tables and forms aren't flattened, schema validation, and hallucination checks like dual-LLM consensus. Those layers are the difference between a demo and a system you can trust with financial data.

❓ How long does it take to go from zero to a working extraction pipeline?
✅ With a no-code platform, most teams get a working schema validated against sample documents within a day or two — the bulk of the time goes into gathering representative samples, not building infrastructure. Deployment as an API or ETL job happens in the same session once the schema is accurate.

❓ Can AI document processing integrate with an existing data warehouse?
✅ Yes. Platforms built for this typically support direct output to destinations like Snowflake, BigQuery, Redshift, or standard databases via ETL pipelines, so structured data lands where your analytics and reporting tools already expect it.

Legacy OCR asked documents to fit a template.

AI document processing asks the system to understand the document instead — and that's the shift finance, accounting, and insurance teams have been waiting for.

👉 Start Unstract's free trial — point it at your messiest documents first and let the comparison speak for itself.

Looking for technical content for your company? I can help — LinkedIn · kevinmenesesgonzalez@gmail.com

How to Backtest a Trading Strategy with Python and EODHD API

Kevin Meneses González — Tue, 07 Jul 2026 12:31:16 +0000

Most backtests lie to you.

Not intentionally. But they lie.

You design a strategy, run it on historical data, and watch the returns look incredible. Then you run it live — and it underperforms a simple buy-and-hold from day one. The math wasn't wrong.

The data was.

If you're:

testing momentum or mean-reversion strategies in Python,
building quant tools for personal or professional use,
or tired of backtests that collapse the moment real execution begins,

This changes how you work.

TL;DR

What this covers: Backtesting trading strategies in Python using EODHD's historical OHLCV data API
Stack: requests, pandas, numpy — no heavy frameworks (no backtrader, no vectorbt)
Scripts included:
- Script 1 — Fetch adjusted historical price data from EODHD
- Script 2 — SMA crossover strategy (20/50-day)
- Script 3 — RSI mean-reversion strategy
- Script 4 — Performance metrics: Sharpe ratio, max drawdown, win rate
EODHD pricing: Free tier available; full access from $19.99/month
Best for: Developers and analysts who need reliable, split/dividend-adjusted data without scraping

The Problem with Free Data

Most developers start with Yahoo Finance or a scraped CSV.

That works fine for a quick prototype. It stops working the moment your strategy includes anything that happened around a stock split, dividend payment, or ticker change.

Non-adjusted price data creates ghost signals. A stock "drops 50%" when it actually split 2:1. Your moving average calculates a crossover that never happened in real life. Your strategy looks profitable because it's trading on a data artifact.

The free path costs you accuracy. And in backtesting, accuracy is the whole point.

The Fix Is Simpler Than You Think

The real bottleneck isn't the strategy logic. It's the data source.

Use split- and dividend-adjusted closing prices from a reliable provider, and half your backtest reliability problems disappear before you write a single signal.

EODHD APIs provides exactly this. Their historical data endpoint returns adjusted OHLCV data for 70,000+ tickers across 50+ exchanges, via a simple REST API. No scraping. No undocumented endpoints that break on weekends.

EODHD Financial Data API
Adjusted historical prices, fundamentals, and real-time data for 70,000+ tickers.
→ Start free at eodhd.com

Setup

Install the required libraries:

pip install requests pandas numpy

Set your API token:

API_TOKEN = "your_eodhd_api_token_here"

You can get a free token at eodhd.com. The free tier includes end-of-day data for US tickers with a 1-year delay — enough to test strategies.

Script 1: Fetch Historical OHLCV Data from EODHD

The foundation of every backtest is the raw price series. This function pulls adjusted daily OHLCV data for any ticker and returns a clean pandas DataFrame.

import requests
import pandas as pd

API_TOKEN = "your_eodhd_api_token_here"

def get_historical_data(symbol: str, exchange: str = "US",
                        start: str = "2020-01-01", end: str = "2024-12-31") -> pd.DataFrame:
    """
    Fetch adjusted EOD OHLCV data from EODHD for a given symbol.
    Returns a DataFrame indexed by date.
    """
    url = f"https://eodhd.com/api/eod/{symbol}.{exchange}"
    params = {
        "api_token": API_TOKEN,
        "fmt": "json",
        "from": start,
        "to": end
    }
    response = requests.get(url, params=params)
    response.raise_for_status()

    df = pd.DataFrame(response.json())
    df["date"] = pd.to_datetime(df["date"])
    df.set_index("date", inplace=True)

    return df[["open", "high", "low", "close", "adjusted_close", "volume"]]


# Example usage
df = get_historical_data("AAPL", start="2020-01-01", end="2024-12-31")
print(df.tail())

Sample output:

            open    high     low   close  adjusted_close      volume
date
2024-12-24  255.5  258.4  254.2  257.9          257.9   32145600
2024-12-26  258.1  261.0  257.3  259.3          259.3   28761300
2024-12-27  258.8  259.4  254.6  255.6          255.6   41232100
2024-12-30  253.1  254.3  250.7  251.8          251.8   39814700
2024-12-31  250.4  252.1  248.9  250.4          250.4   44021900

Note the adjusted_close column. That's what you backtest on — not raw close.

Script 2: SMA Crossover Strategy

The 20/50-day SMA crossover is the classic momentum signal: go long when the short-term average crosses above the long-term, exit when it crosses below.

Simple in theory. The implementation details matter.

def sma_crossover_backtest(df: pd.DataFrame,
                           short_window: int = 20,
                           long_window: int = 50) -> pd.DataFrame:
    """
    SMA crossover strategy using adjusted closing prices.
    Signal:  1 = long, -1 = short, 0 = flat
    Position is shifted by 1 day to prevent look-ahead bias.
    """
    df = df.copy()

    df["sma_short"] = df["adjusted_close"].rolling(short_window).mean()
    df["sma_long"]  = df["adjusted_close"].rolling(long_window).mean()

    # Raw signal
    df["signal"] = 0
    df.loc[df["sma_short"] > df["sma_long"], "signal"] =  1
    df.loc[df["sma_short"] < df["sma_long"], "signal"] = -1

    # Shift by 1 to trade on the *next* day's open — no look-ahead
    df["position"] = df["signal"].shift(1)

    # Returns
    df["market_return"]   = df["adjusted_close"].pct_change()
    df["strategy_return"] = df["position"] * df["market_return"]

    # Cumulative performance
    df["cumulative_market"]   = (1 + df["market_return"]).cumprod()
    df["cumulative_strategy"] = (1 + df["strategy_return"]).cumprod()

    return df


# Run it
df = get_historical_data("AAPL", start="2020-01-01", end="2024-12-31")
result_sma = sma_crossover_backtest(df)

final_market   = result_sma["cumulative_market"].iloc[-1]
final_strategy = result_sma["cumulative_strategy"].iloc[-1]

print(f"Buy & Hold return:   {(final_market - 1):.2%}")
print(f"SMA Strategy return: {(final_strategy - 1):.2%}")

The shift(1) on line 16 is the single most important detail. Without it, you're using today's signal to trade today's close — which is impossible in real life and produces inflated results.

Script 3: RSI Mean-Reversion Strategy

RSI (Relative Strength Index) measures the speed of price changes on a 0–100 scale. Values below 30 signal oversold conditions; above 70 signals overbought.

The mean-reversion hypothesis: when a stock is oversold, it tends to recover. When overbought, it tends to pull back.

import numpy as np

def calculate_rsi(series: pd.Series, period: int = 14) -> pd.Series:
    """Wilder's RSI using simple moving averages of gains and losses."""
    delta = series.diff()
    gain  = delta.clip(lower=0)
    loss  = -delta.clip(upper=0)

    avg_gain = gain.rolling(period).mean()
    avg_loss = loss.rolling(period).mean()

    rs = avg_gain / avg_loss
    return 100 - (100 / (1 + rs))


def rsi_strategy_backtest(df: pd.DataFrame,
                          rsi_period: int = 14,
                          oversold: int = 30,
                          overbought: int = 70) -> pd.DataFrame:
    """
    RSI mean-reversion strategy.
    Enter long when RSI drops below 'oversold'.
    Exit (go flat) when RSI rises above 'overbought'.
    """
    df = df.copy()
    df["rsi"] = calculate_rsi(df["adjusted_close"], rsi_period)

    df["signal"] = 0
    df.loc[df["rsi"] < oversold,   "signal"] =  1   # Buy oversold
    df.loc[df["rsi"] > overbought, "signal"] = -1   # Sell overbought

    # Hold position between signals (forward-fill non-zero values)
    df["position"] = (
        df["signal"]
        .replace(0, np.nan)
        .ffill()
        .fillna(0)
        .shift(1)   # Again: no look-ahead
    )

    df["market_return"]   = df["adjusted_close"].pct_change()
    df["strategy_return"] = df["position"] * df["market_return"]

    df["cumulative_market"]   = (1 + df["market_return"]).cumprod()
    df["cumulative_strategy"] = (1 + df["strategy_return"]).cumprod()

    return df


# Run it
result_rsi = rsi_strategy_backtest(df)

print(f"Buy & Hold return:   {(result_rsi['cumulative_market'].iloc[-1] - 1):.2%}")
print(f"RSI Strategy return: {(result_rsi['cumulative_strategy'].iloc[-1] - 1):.2%}")

Script 4: Performance Metrics (Sharpe, Max Drawdown, Win Rate)

Return alone means nothing. A strategy returning 40% with -60% max drawdown is not a good strategy.

This function calculates the three metrics that matter most for evaluating any backtest:

def calculate_performance(df: pd.DataFrame,
                          risk_free_rate: float = 0.04) -> dict:
    """
    Compute annualized return, Sharpe ratio, max drawdown, and win rate
    for a backtested strategy DataFrame.
    """
    strategy_returns = df["strategy_return"].dropna()

    # Annualized return
    total_days = len(strategy_returns)
    annual_factor = 252 / total_days
    strategy_ann = (df["cumulative_strategy"].iloc[-1] ** annual_factor) - 1
    market_ann   = (df["cumulative_market"].iloc[-1] ** annual_factor) - 1

    # Sharpe ratio (annualized)
    daily_rf = risk_free_rate / 252
    excess   = strategy_returns - daily_rf
    sharpe   = np.sqrt(252) * excess.mean() / excess.std()

    # Maximum drawdown
    cumulative  = df["cumulative_strategy"].dropna()
    rolling_max = cumulative.cummax()
    drawdown    = (cumulative - rolling_max) / rolling_max
    max_dd      = drawdown.min()

    # Win rate (percentage of profitable trading days)
    active   = strategy_returns[strategy_returns != 0]
    win_rate = (active > 0).sum() / len(active)

    return {
        "Strategy Annualized Return": f"{strategy_ann:.2%}",
        "Market Annualized Return":   f"{market_ann:.2%}",
        "Sharpe Ratio":               f"{sharpe:.2f}",
        "Max Drawdown":               f"{max_dd:.2%}",
        "Win Rate":                   f"{win_rate:.2%}",
    }


# Evaluate both strategies
print("=== SMA Crossover ===")
for k, v in calculate_performance(result_sma).items():
    print(f"  {k}: {v}")

print("\n=== RSI Mean-Reversion ===")
for k, v in calculate_performance(result_rsi).items():
    print(f"  {k}: {v}")

Sample output (AAPL, 2020–2024):

=== SMA Crossover ===
  Strategy Annualized Return: 18.4%
  Market Annualized Return:   22.1%
  Sharpe Ratio:               0.91
  Max Drawdown:              -14.3%
  Win Rate:                   53.2%

=== RSI Mean-Reversion ===
  Strategy Annualized Return: 15.7%
  Market Annualized Return:   22.1%
  Sharpe Ratio:               0.78
  Max Drawdown:              -18.6%
  Win Rate:                   51.8%

In this case, buy-and-hold wins on raw return. But look at the max drawdown: the SMA strategy cuts the worst-case scenario from -30%+ to -14%. That's the real value — risk-adjusted performance, not just raw returns.

Putting It All Together

Here's the full pipeline: fetch data, run both strategies, compare metrics.

# Full backtest pipeline
symbols = ["AAPL", "MSFT", "NVDA"]

for ticker in symbols:
    print(f"\n{'='*40}")
    print(f"  {ticker}")
    print(f"{'='*40}")

    data = get_historical_data(ticker, start="2021-01-01", end="2024-12-31")

    sma_result = sma_crossover_backtest(data)
    rsi_result = rsi_strategy_backtest(data)

    print("  SMA Crossover:")
    for k, v in calculate_performance(sma_result).items():
        print(f"    {k}: {v}")

    print("  RSI Mean-Reversion:")
    for k, v in calculate_performance(rsi_result).items():
        print(f"    {k}: {v}")

From here you can build:

a parameter optimization loop (walk-forward testing)
a multi-ticker portfolio backtest with position sizing
a live signal generator using EODHD's real-time endpoints
a dashboard to visualize equity curves and drawdown periods

FAQs

❓ Is EODHD data adjusted for stock splits and dividends?
✅ Yes. The adjusted_close field in the API response accounts for both splits and dividends. Always use this field for backtesting — raw close prices will produce misleading signals around corporate actions.

❓ Can I backtest strategies on non-US markets with EODHD?
✅ EODHD covers 50+ exchanges including LSE, TSX, ASX, Euronext, and major Asian markets. Just change the exchange parameter in the API call (e.g., "LSE" for London, "TO" for Toronto).

❓ Does this approach suffer from survivorship bias?
✅ Potentially yes, if you only test on stocks that still exist today. To minimize it, include delisted tickers in your universe. EODHD provides data for delisted stocks — you can query them using their historical ticker symbols.

❓ What's the difference between a good Sharpe ratio and a bad one?
✅ As a general benchmark: below 0.5 is weak, 0.5–1.0 is acceptable, above 1.0 is considered good. Above 2.0 in a backtest should raise suspicion — it often signals overfitting to historical noise.

❓ Is there a free tier on EODHD to try this?
✅ Yes. EODHD offers a free API key that includes EOD data for US tickers. You can run all the scripts in this article with the free tier. Paid plans start at $19.99/month and unlock real-time data, fundamentals, and full global coverage.

Before You Ship Your Strategy Live

Two things to check before treating any backtest as real:

Transaction costs. Every trade has a spread and a commission. Add a -0.001 cost per trade (0.1%) to your strategy_return calculation and see if the edge survives.

Overfitting. If you tuned your parameters (RSI period, SMA windows) on the same data you're testing on, your results are optimistic. Use a walk-forward split: train on 70% of the data, test on the remaining 30%.

Bad data tells you the strategy works. Good data tells you the truth.

EODHD Financial Data API
Adjusted historical prices for 70,000+ tickers across 50+ global exchanges. REST API, JSON responses, Python-friendly.
→ Start free — no credit card required

Looking for technical content for your company? I can help — LinkedIn · kevinmenesesgonzalez@gmail.com