DEV Community

nan nan
nan nan

Posted on

Export System Design: Structured Output for OCR Results

How CaptureOCR turns OCR lines into Markdown tables, CSV, and reproducible session packages

Design Goals

  1. Human-readable: Markdown tables paste directly into GitHub, Notion, Jira
  2. Machine-parseable: CSV with coordinates, confidence, metadata for pipelines
  3. Reproducible: Session package (.snipx) enables lossless round-trip and collaboration
  4. Extensible: Schema supports future fields without breaking consumers

Product link: CaptureOCR on Microsoft Store — $2.99 one-time, exports Markdown/CSV/JSON/Session.

Data Model 

from dataclasses import dataclass
from typing import Optional
import uuid

@dataclass
class Selection:
    stable_id: int
    display_index: int
    logical_rect: QRect      # Qt logical coordinates
    physical_rect: QRect     # Screen physical pixels
    screen_index: int
    dpr: float
    image: np.ndarray
    ocr_result: Optional[OcrResult] = None

@dataclass
class OcrLine:
    box: list[float]         # Normalized [x0, y0, x1, y1] relative to region
    text: str
    confidence: float

@dataclass
class OcrResult:
    lines: list[OcrLine]
    model_tier: str          # "fast" or "accurate"
    elapsed_ms: int
    model_hash: str          # SHA256 of model files

@dataclass
class Session:
    session_id: str          # UUID
    created_at: str          # ISO 8601
    selections: list[Selection]
    app_version: str
    settings_snapshot: dict  # Relevant settings at capture time
Enter fullscreen mode Exit fullscreen mode

Export Formats

1. Markdown Table (Human-Friendly) 

| # | Region (x,y,w,h) | Line | Confidence | Text |
|---|---|---|---|---|
| 1 | (120, 45, 380, 120) | 1 | 0.998 | Error 0x80070057: The parameter is incorrect |
| 1 | (120, 45, 380, 120) | 2 | 0.995 | at System.IO.File.Open(String path, FileMode mode) |
| 2 | (520, 45, 300, 80) | 1 | 0.987 | User: john.doe@company.com |
Enter fullscreen mode Exit fullscreen mode

Generation:

def export_markdown(session: Session) -> str:
    lines = ["| # | Region (x,y,w,h) | Line | Confidence | Text |",
             "|---|---|---|---|---|"]

    for sel in sorted(session.selections, key=lambda s: s.display_index):
        if not sel.ocr_result:
            continue
        region = f"({sel.physical_rect.x()},{sel.physical_rect.y()},{sel.physical_rect.width()},{sel.physical_rect.height()})"
        for line_idx, line in enumerate(sel.ocr_result.lines):
            # Escape pipes and newlines for valid Markdown
            text = line.text.replace('|', '\\|').replace('\n', '<br>')
            lines.append(f"| {sel.display_index} | {region} | {line_idx+1} | {line.confidence:.3f} | {text} |")

    return "\n".join(lines)
Enter fullscreen mode Exit fullscreen mode

Key escaping rules:

  • | → \| (breaks table)
  • \n → <br> (preserves line breaks in cell)
  • \\ → \\\\ (literal backslash)

2. CSV (Pipeline-Friendly) 

session_id,index,x,y,w,h,line_index,confidence,text,mode,elapsed_ms,model_hash
a1b2c3d4,1,120,45,380,120,1,0.998,"Error 0x80070057: The parameter is incorrect",fast,78,a1b2c3d4e5f6...
a1b2c3d4,1,120,45,380,120,2,0.995,"at System.IO.File.Open(String path, FileMode mode)",fast,78,a1b2c3d4e5f6...
a1b2c3d4,2,520,45,300,80,1,0.987,"User: john.doe@company.com",fast,65,a1b2c3d4e5f6...
Enter fullscreen mode Exit fullscreen mode

Features:

  • UTF-8 BOM (\ufeff) — Excel opens Chinese correctly
  • Region-level coordinates — Physical pixels, traceable to screen location
  • Per-line metadata — Confidence, timing, model hash for quality gates
  • Stable session_id — Groups related captures

Generation:

import csv
from io import StringIO

def export_csv(session: Session) -> bytes:
    output = StringIO()
    # UTF-8 BOM for Excel
    output.write('\ufeff')

    writer = csv.writer(output)
    writer.writerow([
        'session_id', 'index', 'x', 'y', 'w', 'h',
        'line_index', 'confidence', 'text', 'mode', 
        'elapsed_ms', 'model_hash'
    ])

    for sel in sorted(session.selections, key=lambda s: s.display_index):
        if not sel.ocr_result:
            continue
        for line_idx, line in enumerate(sel.ocr_result.lines):
            writer.writerow([
                session.session_id,
                sel.display_index,
                sel.physical_rect.x(),
                sel.physical_rect.y(),
                sel.physical_rect.width(),
                sel.physical_rect.height(),
                line_idx + 1,
                f"{line.confidence:.3f}",
                line.text,
                sel.ocr_result.model_tier,
                sel.ocr_result.elapsed_ms,
                sel.ocr_result.model_hash
            ])

    return output.getvalue().encode('utf-8-sig')  # utf-8-sig adds BOM
Enter fullscreen mode Exit fullscreen mode

3. Session Package (.snipx) — Reproducible & Collaborative

A ZIP file containing:

session.snipx (ZIP)
├── session.json          # Complete session data (see below)
├── regions/
│   ├── 001.png           # Original capture (lossless PNG)
│   ├── 002.png
│   └── ...
├── export.md             # Markdown export
└── export.csv            # CSV export
Enter fullscreen mode Exit fullscreen mode

session.json (complete round-trip data):

{
  "session_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "created_at": "2026-10-27T14:30:00Z",
  "app_version": "1.0.0.0",
  "settings_snapshot": {
    "ocr_tier": "fast",
    "auto_ocr": true,
    "hotkey": "Ctrl+Shift+X"
  },
  "selections": [
    {
      "stable_id": 0,
      "display_index": 1,
      "logical_rect": [120, 45, 380, 120],
      "physical_rect": [180, 67, 570, 180],
      "screen_index": 0,
      "dpr": 1.5,
      "ocr_result": {
        "lines": [
          {
            "box": [0.1, 0.2, 0.9, 0.4],
            "text": "Error 0x80070057: The parameter is incorrect",
            "confidence": 0.998
          }
        ],
        "model_tier": "fast",
        "elapsed_ms": 78,
        "model_hash": "a1b2c3d4e5f6..."
      }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Why ZIP + JSON + PNG?

  • Lossless: Original pixels preserved for re-OCR with different settings
  • Portable: Single file, no external dependencies
  • Collaborative: Send .snipx to colleague → they open in CaptureOCR → continue editing/exporting
  • Auditable: model_hash detects model drift; settings_snapshot captures intent

Generation:

import zipfile
import json
from pathlib import Path

def export_snipx(session: Session, output_path: Path):
    with zipfile.ZipFile(output_path, 'w', zipfile.ZIP_DEFLATED) as zf:
        # session.json
        zf.writestr('session.json', json.dumps(session.to_dict(), indent=2))

        # Region images
        for sel in session.selections:
            if sel.image is not None:
                img_bytes = cv2.imencode('.png', sel.image)[1].tobytes()
                filename = f"regions/{sel.display_index:03d}.png"
                zf.writestr(filename, img_bytes)

        # Markdown & CSV
        zf.writestr('export.md', export_markdown(session))
        zf.writestr('export.csv', export_csv(session).decode('utf-8-sig'))
Enter fullscreen mode Exit fullscreen mode

Importing .snipx (Round-Trip) 

def import_snipx(file_path: Path) -> Session:
    with zipfile.ZipFile(file_path, 'r') as zf:
        # Load session.json
        session_data = json.loads(zf.read('session.json'))
        session = Session.from_dict(session_data)

        # Load region images
        for sel in session.selections:
            img_name = f"regions/{sel.display_index:03d}.png"
            if img_name in zf.namelist():
                img_bytes = zf.read(img_name)
                sel.image = cv2.imdecode(np.frombuffer(img_bytes, np.uint8), cv2.IMREAD_COLOR)

    return session
Enter fullscreen mode Exit fullscreen mode

After import: User sees all regions, annotations, OCR results exactly as saved. Can re-export with different settings (e.g., switch to Accurate mode).

Format Comparison

Feature Markdown CSV .snipx
Human-readable ✅ ⚠️ ❌
Excel-friendly ❌ ✅ (with BOM) ⚠️ (extract CSV)
Coordinates Region only Region only Region + per-line boxes
Per-line confidence ✅ ✅ ✅
Original images ❌ ❌ ✅
Annotations ❌ ❌ ✅ (in JSON)
Round-trip ❌ ❌ ✅
Collaboration Copy/paste File share Single file
Pipeline automation ❌ ✅ ⚠️ (unzip first)

Deliberate Limitation: Line-Level, Not Table Reconstruction

Current: A 4×6 table becomes text lines with region coordinates, not a 4×6 grid.

Why: Cell merging and column inference need layout analysis — a much bigger project. Line-level is predictable and cheap.

User workaround: Pivot CSV in Excel → reconstruct tables manually.

Future: Optional layout analysis plugin (separate package).

Resources

Top comments (0)