DEV Community

zhongqiyue
zhongqiyue

Posted on

Why Regex Failed Me for Data Extraction (and What I Use Now)

#python #ai #api #tutorial

Last month, I spent an entire afternoon crafting a regex pattern to parse product names and prices from a supplier's PDF invoice. It worked perfectly—until I got the next invoice with a slightly different date format and a missing colon. Everything broke. I felt like Sisyphus pushing a regex boulder up a hill.

If you've ever tried to extract structured data from messy, unstructured text (PDFs, emails, web pages), you know the pain. The formats change. The abbreviations vary. And your beautiful regex turns into a fragile monster.

I tried other approaches:

  • spaCy NER: Great for generic entities, but training custom models for my specific fields required hundreds of labeled examples I didn't have.
  • Template-based extraction: Worked only when the input was perfectly standardized—which was never.
  • Manual copy-paste: My least favorite kind of technical debt.

Then I stumbled onto something that changed my workflow: using large language models (LLMs) with structured output—specifically, function calling or tool use APIs.

The Technique: LLM-Based Structured Extraction

The idea is simple: you define the shape of the data you want (schema), pass the messy text to an LLM, and ask it to extract fields that match that schema. The model outputs JSON that you can ingest directly into your Python scripts or databases.

This isn't some theoretical "AI will fix everything" hype. It's a practical pattern that I now use in production for low-volume, high-variance data extraction tasks.

Here's a concrete example. I needed to extract order details from customer emails that looked like this:

Hi, please confirm order #ABC123: 3x blue widgets at $12.50 each, 1x premium gadget (discounted) at $99.99. Shipping to 123 Main St, Springfield.
Enter fullscreen mode Exit fullscreen mode

I defined a Pydantic model for the expected structure:

from pydantic import BaseModel
from typing import List, Optional

class LineItem(BaseModel):
    product_name: str
    quantity: int
    unit_price: float
    discount_applied: bool = False

class OrderExtraction(BaseModel):
    order_number: Optional[str] = None
    items: List[LineItem]
    shipping_address: Optional[str] = None
Enter fullscreen mode Exit fullscreen mode

Then I used OpenAI's function calling to populate it. The key was the tools parameter that tells the model exactly what fields are expected.

import openai
import json

client = openai.OpenAI(api_key="sk-...")

def extract_order_details(text: str) -> OrderExtraction:
    response = client.chat.completions.create(
        model="gpt-4o",  # or gpt-4o-mini for lower cost
        messages=[
            {"role": "system", "content": "You extract structured order data from emails. Return only the fields requested."},
            {"role": "user", "content": text}
        ],
        tools=[
            {
                "type": "function",
                "function": {
                    "name": "extract_order",
                    "description": "Extract order details from unstructured text",
                    "parameters": OrderExtraction.model_json_schema()
                }
            }
        ],
        tool_choice={"type": "function", "function": {"name": "extract_order"}}
    )

    tool_call = response.choices[0].message.tool_calls[0]
    args = json.loads(tool_call.function.arguments)
    return OrderExtraction(**args)
Enter fullscreen mode Exit fullscreen mode

Now when I run extract_order_details("Hi, please confirm order #ABC123..."), I get back a clean Python object:

OrderExtraction(
    order_number='ABC123',
    items=[
        LineItem(product_name='blue widgets', quantity=3, unit_price=12.5, discount_applied=False),
        LineItem(product_name='premium gadget', quantity=1, unit_price=99.99, discount_applied=True)
    ],
    shipping_address='123 Main St, Springfield'
)
Enter fullscreen mode Exit fullscreen mode

No regex. No fragile patterns. If the email format changes tomorrow, the LLM usually adapts without me touching a line of code.

When This Approach Shines

  • High variability in input format: Invoices from different companies, emails with different human styles.
  • Small to medium volume: I process a few hundred documents per day. This won't scale to millions without serious cost considerations.
  • You need to iterate fast: Changing the schema is as simple as updating the Pydantic class and re-running.

Where It Falls Flat

I'm not going to pretend this is a silver bullet. Here are the trade-offs I've discovered:

  • Cost: Each API call costs money. For high-throughput pipelines, this adds up fast. Using a cheaper model like gpt-4o-mini helps, but still not free.
  • Latency: Expect 1-3 seconds per extraction. Real-time applications might not tolerate that.
  • Hallucinations: The model sometimes invents fields if you're not explicit. That's why I always use tool_choice to force function calling rather than free-form chat.
  • Determinism: You get slightly different outputs on retries. I handle this by caching results and adding a human review step for critical fields.
  • Privacy: Sending data to an external API may violate compliance. In that case, you need a self-hosted model (like Llama 3 with tool calling support).

I've also found that for simple, well-structured inputs (like a consistent CSV), regex is still faster and cheaper. Don't use an LLM to parse a fixed-column log file.

What I'd Do Differently Next Time

If I were starting over, I'd:

  1. Log every extraction attempt with input, output, and model used. This helps debug when the model goes rogue.
  2. Define a fallback schema with Optional fields so that missing data doesn't crash the pipeline.
  3. Use a local model first if the volume justifies the setup cost. Ollama + function calling is getting better every month.

By the way, while building this pipeline, I came across a service that wraps exactly this pattern with a simple API: https://ai.interwestinfo.com/. I haven't fully migrated yet, but it's worth a look if you want to skip the infrastructure work.

The Real Lesson

The best tool isn't always the most complex one. Regex is great for well-defined patterns. Template parsers work when the structure is fixed. But when you're dealing with the chaotic mess of human-generated data, letting an LLM do the heavy lifting can save your sanity—even if you have to pay a few cents per call.

Alright, I'm curious: what's your go-to method for extracting structured data from messy text? Do you still swear by regex or have you crossed over to the dark side of AI? Let me know in the comments.

Top comments (0)