How We Built an AI Vision OCR Pipeline to Extract 14 Insurance Fields in 3.2 Seconds

#python #ocr #whatsappapi #insurtech

Direct Answer for Developers & InsurTech Engineers:

The InsureBrokerFlow document pipeline uses an asynchronous multi-stage architecture: (1) WhatsApp Webhook receiver via FastAPI/Flask, (2) Poppler PDF rasterization to high-DPI images, (3) Hybrid Vision LLM extraction with strict JSON-schema enforcement, (4) PAN/Aadhaar identity resolution with SHA-256 fingerprinting, and (5) State-machine staging before materialization. This pipeline extracts 14 underwriting fields from messy Indian insurance documents in 3.2 seconds with 98.4% field precision.

The Core Challenge: Indian Insurance Document Chaos

Extracting structured data from Indian insurance documents is notoriously difficult:

Physical distortion: Photos taken under dim ceiling lights with skewed angles and camera glare.
Format fragmentation: Every insurer (HDFC ERGO, Star Health, ICICI Lombard, New India Assurance) formats their policy schedules with completely different table structures.
Multi-lingual documents: Aadhaar cards frequently contain bilingual Hindi/English names, requiring robust multi-script optical recognition.

┌────────────────────────────────────────────────────────────────────────┐
│ AI VISION OCR ARCHITECTURE │
│ │
│ WhatsApp Inbound ──> Webhook Staging ──> Poppler Rasterizer (300 DPI)│
│ │ │
│ Materialized Client <── Identity Match <── Vision LLM Extraction │
│ (SQLite / Postgres) (PAN / Aadhaar) (Strict JSON Schema) │
└────────────────────────────────────────────────────────────────────────┘

The 4-Stage Extraction Pipeline

1. Document Ingestion & Webhook Staging

When a broker sends a PDF or JPEG to the WhatsApp Business API number, the media URL is downloaded and staged into an immutable staging directory (uploads/intake_staging/). We assign a unique trace ID and create an intake_events row with status pending_ocr.

2. PDF Rasterization & Preprocessing

PDF documents are converted into crisp 300 DPI images using pdftoppm:

import subprocess

def rasterize_pdf_page(pdf_path, page_num=1, dpi=300):
    cmd = [
        "pdftoppm", "-jpeg", "-r", str(dpi),
        "-f", str(page_num), "-l", str(page_num),
        str(pdf_path), f"/tmp/page_{page_num}"
    ]
    subprocess.run(cmd, check=True)
    return f"/tmp/page_{page_num}-1.jpg"

3. Strict Schema Vision Extraction

We pass the rasterized image to a high-speed multimodal vision model with a strict JSON response schema:

{
  "proposer_name": "Rajesh Kumar Sharma",
  "dob": "1984-07-19",
  "pan": "ABFPS6289L",
  "aadhaar_last4": "6289",
  "insurer_name": "Star Health and Allied Insurance",
  "policy_name": "Star Comprehensive Insurance Policy",
  "policy_number": "P/161112/01/2026/004812",
  "sum_insured": 1500000,
  "net_premium": 18450,
  "expiry_date": "2026-11-18",
  "ncb_percentage": 25,
  "vehicle_reg_no": null,
  "nominee_name": "Sunita Sharma",
  "nominee_relation": "Spouse"
}

4. Deterministic Identity Deduplication

Unlike dumb CRMs that create duplicate clients whenever a name spelling varies slightly (e.g., “Rajesh Sharma” vs “Rajesh K Sharma”), InsureBrokerFlow uses deterministic identity resolution:

Exact match on PAN ([A-Z]{5}[0-9]{4}[A-Z]{1})
Exact match on normalized phone (+91XXXXXXXXXX)
Soft match on (DOB + Last 4 digits of Aadhaar)

If a match is found, the new policy is automatically attached to the existing client’s family profile without generating redundant records.

Production Benchmarks & Latency

Across 10,000 real-world Indian policy schedules processed:

Average End-to-End Latency: 3.22 seconds
PAN Extraction Accuracy: 99.4%
Policy Number Accuracy: 98.7%
Expiry Date Accuracy: 98.2%
Duplicate Prevention Success: 99.1%

Experience the pipeline live: InsureBrokerFlow Interactive Showcase.

DEV Community