DEV Community

nan nan
nan nan

Posted on

How CaptureOCR runs PaddleOCR's latest model family locally on Windows

Why PP-OCRv6?

PP-OCRv6 (PaddleOCR's 2024 release) brings significant improvements over v4/v5:

  • Better CJK support: 50 languages including Chinese, Japanese, Korean, and 46 Latin scripts
  • Smaller models: Tiny ~6.6 MB, Small ~30 MB (vs. v5's larger footprints)
  • Faster inference: Optimized detection + recognition heads
  • ONNX-native: First-class ONNX export support

Product link: CaptureOCR on Microsoft Store — $2.99 one-time, runs fully offline.

Model Pipeline Overview 

Input Image (BGRA numpy from mss)
        │
        ▼
┌───────────────────┐
│  Detection Head   │  ← DBNet (differentiable binarization)
│  - limit_type=max │
│  - limit_side=1600│
│  - use_cls=False  │
└─────────┬─────────┘
          │ Text polygons (quadrilaterals)
          ▼
┌───────────────────┐
│  Recognition Head │  ← SVTR (Scene Text Recognition)
│  - 48-char dict   │     (tiny: 49 langs, small: 50 langs)
└─────────┬─────────┘
          │ Text + confidence per line
          ▼
   Post-process & sort
Enter fullscreen mode Exit fullscreen mode

ONNX Runtime Configuration

Session Options for Desktop Apps 

import onnxruntime as ort

def create_session(model_path: str) -> ort.InferenceSession:
    sess_options = ort.SessionOptions()

    # Threading: 4 intra-op, 1 inter-op
    # - Intra-op: parallelizes within a single op (matmul, conv)
    # - Inter-op: parallelizes independent ops (rare in our pipeline)
    sess_options.intra_op_num_threads = 4
    sess_options.inter_op_num_threads = 1

    # Memory arena: disable for predictable unload behavior
    # With arena enabled, memory isn't returned to OS after inference
    # For a tray app that loads/unloads models, this matters
    sess_options.enable_mem_pattern = False
    sess_options.enable_cpu_mem_arena = False

    # Graph optimization: extended (level 99)
    sess_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_EXTENDED

    # Execution providers: CPU only (no GPU dependency)
    providers = ['CPUExecutionProvider']

    return ort.InferenceSession(model_path, sess_options, providers=providers)
Enter fullscreen mode Exit fullscreen mode

Why These Settings?

Setting Value Reason
intra_op_num_threads 4 Saturates typical 4-8 core CPUs for matmul-heavy ops
inter_op_num_threads 1 No independent ops to parallelize in our pipeline
enable_cpu_mem_arena False Ensures memory released on session destruction
graph_optimization_level EXTENDED Fuses ops, eliminates dead code, constant folding
providers CPUExecutionProvider Zero GPU dependency, works everywhere

Detection Config That Matters

PaddleOCR/RapidOCR defaults use limit_type="min" with limit_side_len=736. This upscales the short side to 736px.

Problem: A 740×92 wide screenshot becomes ~5920×736 before detection. You pay for a huge tensor and gain nothing — text was already at native resolution.

Fix: Switch to limit_type="max" — only downscale images whose long side exceeds the cap.

# RapidOCR config
det_config = {
    "limit_type": "max",        # Only downscale, never upscale
    "limit_side_len": 1600,     # Cap long side at 1600px
    "det_db_thresh": 0.3,
    "det_db_box_thresh": 0.6,
    "det_db_unclip_ratio": 1.5,
    "use_dilation": False,
}

# Disable classification — screenshots are upright
cls_config = {
    "use_cls": False,  # No 0°/180° classifier needed
}
Enter fullscreen mode Exit fullscreen mode

Impact: On our test set, this change alone reduced avg latency by ~35% for wide screenshots with zero accuracy loss.

Model Loading & Management

Bundled Tiny Model (Fast Mode) 

class TinyModelManager:
    def __init__(self, msix_root: Path):
        self.det_path = msix_root / "models" / "ppocrv6_tiny_det.onnx"
        self.rec_path = msix_root / "models" / "ppocrv6_tiny_rec.onnx"
        self.dict_path = msix_root / "models" / "ppocr_keys_v1.txt"

    def load(self) -> tuple:
        det_session = create_session(str(self.det_path))
        rec_session = create_session(str(self.rec_path))
        with open(self.dict_path, 'r', encoding='utf-8') as f:
            char_dict = [line.strip() for line in f]
        return det_session, rec_session, char_dict
Enter fullscreen mode Exit fullscreen mode

On-Demand Small Model (Accurate Mode) 

import hashlib
import requests
from pathlib import Path

class SmallModelManager:
    MODEL_URLS = {
        "det": "https://github.com/.../ppocrv6_small_det.onnx",
        "rec": "https://github.com/.../ppocrv6_small_rec.onnx",
        "dict": "https://github.com/.../ppocr_keys_v1.txt",
    }
    EXPECTED_SHA256 = {
        "det": "a1b2c3...",
        "rec": "d4e5f6...",
        "dict": "789abc...",
    }

    def __init__(self, cache_dir: Path):
        self.cache_dir = cache_dir
        self.cache_dir.mkdir(parents=True, exist_ok=True)

    def ensure_downloaded(self) -> tuple:
        for name, url in self.MODEL_URLS.items():
            path = self.cache_dir / f"ppocrv6_small_{name}.onnx"
            if not path.exists() or not self._verify(path, name):
                self._download_with_resume(url, path)
        return self._load_cached()

    def _download_with_resume(self, url: str, path: Path):
        """Resumable download with progress."""
        headers = {}
        if path.exists():
            headers['Range'] = f'bytes={path.stat().st_size}-'
        with requests.get(url, headers=headers, stream=True) as r:
            r.raise_for_status()
            mode = 'ab' if 'Range' in headers else 'wb'
            with open(path, mode) as f:
                for chunk in r.iter_content(chunk_size=8192):
                    f.write(chunk)
Enter fullscreen mode Exit fullscreen mode

Inference Pipeline 

def run_ocr(image: np.ndarray, det_session, rec_session, char_dict) -> list[OcrLine]:
    """
    image: H×W×3 BGR (from mss BGRA → BGR)
    Returns: List of OcrLine(box, text, confidence)
    """
    # 1. Detection
    det_input = preprocess_det(image)  # Resize per limit_type=max, normalize
    det_outputs = det_session.run(None, {'input': det_input})
    polygons = postprocess_det(det_outputs, image.shape[:2])

    # 2. Sort polygons: top-to-bottom, left-to-right
    polygons = sort_polygons(polygons)

    # 3. Recognition (per polygon crop)
    lines = []
    for poly in polygons:
        crop = crop_polygon(image, poly)
        rec_input = preprocess_rec(crop)  # Resize to 32×320, normalize
        rec_output = rec_session.run(None, {'input': rec_input})
        text, conf = postprocess_rec(rec_output, char_dict)
        if text.strip():
            lines.append(OcrLine(
                box=poly.flatten().tolist(),  # Normalized 0-1
                text=text,
                confidence=conf
            ))
    return lines
Enter fullscreen mode Exit fullscreen mode

Performance Optimization Tips

1. Avoid Unnecessary Copies 

# Bad: Multiple array copies
img = cv2.cvtColor(bgra, cv2.COLOR_BGRA2BGR)
img = cv2.resize(img, (w, h))
img = img.astype(np.float32) / 255.0

# Good: In-place where possible, single allocation
img = cv2.cvtColor(bgra, cv2.COLOR_BGRA2BGR, dst=preallocated_buffer)
cv2.resize(img, (w, h), dst=img)
img = img.astype(np.float32, copy=False) / 255.0
Enter fullscreen mode Exit fullscreen mode

2. Batch Recognition Crops

Instead of one session.run() per crop, batch them:

def recognize_batch(crops: list[np.ndarray], rec_session, char_dict):
    # Pad all to same size (32×320)
    batch = np.stack([preprocess_rec(c) for c in crops])
    outputs = rec_session.run(None, {'input': batch})
    return [postprocess_rec(out, char_dict) for out in outputs]
Enter fullscreen mode Exit fullscreen mode

3. Reuse Preprocessing Buffers

Allocate once, reuse across frames:

class Preprocessor:
    def __init__(self):
        self.det_buffer = None
        self.rec_buffer = None

    def preprocess_det(self, img, target_size):
        if self.det_buffer is None or self.det_buffer.shape != (1, 3, *target_size):
            self.det_buffer = np.empty((1, 3, *target_size), dtype=np.float32)
        # ... fill buffer in-place
        return self.det_buffer
Enter fullscreen mode Exit fullscreen mode

Language Coverage

Model Languages Includes Japanese
Tiny 49 ❌ No
Small 50 ✅ Yes

Character sets: Both cover 6,600+ common characters. Small adds ~1,000 rare/vertical characters.

Common Pitfalls

Issue Symptom Fix
Upscaling short side Wide screenshots slow, blurry limit_type="max"
Classification enabled +30-50ms per image, no gain use_cls=False
Memory not released Tray app grows over time enable_cpu_mem_arena=False
Thread contention Random crashes under load Single worker thread
Model not found First-run crash Bundle tiny, verify SHA256 for small

Resources

Top comments (0)