Why Build Another OCR Tool?
Existing OCR tools abound, but few satisfy "fully offline + structured export + desktop-grade UX" simultaneously:
| Pain Point | Status Quo | ReceiptForge Solution |
|---|---|---|
| Privacy leakage | Cloud OCR uploads images | Socket-level network block, zero upload |
| Fixed output | Text/simple JSON only | 22 structured fields + 4 export formats |
| Accuracy black box | Result only, no confidence | Field-level 5-way weighted confidence |
| Failure = loss | Errors = hard failure | Failures still exported, issues collected |
| Model updates | Forced network download | Models ship with package, fully offline |
Technical Constraints
- Fully offline: Socket hook installed at startup, blocks all outbound connections
- Model size: Main models 162MB + multilingual 50MB, total package >300MB
- Cross-platform: Windows 10 19041+ / x64
- Zero-dependency runtime: Python runtime embedded, no system Python needed
Overall Architecture
┌─────────────────────────────────────────────────────────────────┐
│ ReceiptForge │
├─────────────────────────────────────────────────────────────────┤
│ UI Layer (PySide6) │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Import Page │ │ Review Page │ │ Export Page │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
├─────────┼──────────────┼────────────────┼──────────────────────┤
│ ▼ ▼ ▼ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Pipeline Orchestrator │ │
│ │ (QThread + Worker Pattern, Progress/Result Signals) │ │
│ └─────────────────────┬─────────────────────────────────────┘ │
├────────────────────────┼────────────────────────────────────────┤
│ ┌───────────────┴───────────────┐ │
│ ▼ ▼ │
│ ┌───────────────┐ ┌─────────────────┐ │
│ │ Ingestion │ │ Pipeline Core │ │
│ │ (File/Folder │ │ (Classification │ │
│ │ Scan, SHA256 │ │ → Extraction │ │
│ │ Dedup, Split)│ │ → Validation) │ │
│ └───────────────┘ └────────┬────────┘ │
├────────────────────────────────────────┼───────────────────────┤
│ ┌───────────────┬───────────────┴───────────────┐ │
│ ▼ ▼ ▼ │
│ ┌──────────┐ ┌──────────────┐ ┌────────────┐ │
│ │RapidOCR │ │ Validation │ │ Export │ │
│ │(ONNX RT) │ │ (3 Layers) │ │ (4 Formats)│ │
│ └──────────┘ └──────────────┘ └────────────┘ │
└─────────────────────────────────────────────────────────────────┘
Key Architectural Decisions
| Decision Point | Choice | Rationale |
|---|---|---|
| UI Framework | PySide6 (Qt 6) | LGPL dynamic linking, mature, native UX |
| OCR Engine | RapidOCR (ONNX Runtime) | Pure Python, CPU inference, model control |
| Packaging | PyInstaller + MSIX | Official recommendation, Store compatible |
| Concurrency | QThread + Worker | Native Qt signals/slots, no GIL contention |
| Data Storage | SQLite (WAL) + JSON | Lightweight, serverless, concurrent reads |
Core Modules Deep Dive
1. File Ingestion & Preprocessing
# app/pipeline/ingest.py - Core ingestion logic
class FileIngestor:
SUPPORTED_EXTS = {
'image': {'.jpg', '.jpeg', '.png', '.tiff', '.bmp',
'.webp', '.heic', '.heif', '.jfif', '.jpe', '.tif'},
'pdf': {'.pdf'},
'text': {'.txt', '.md', '.csv', '.tsv', '.log', '.text'}
}
def scan(self, root: Path, recursive: bool, dedup: bool) -> List[SourceFile]:
"""Walk directory, return files to process"""
files = self._collect_files(root, recursive)
files = self._filter_supported(files)
if dedup:
files = self._deduplicate_sha256(files) # SHA-256 chunked dedup
return [SourceFile(path=f, kind=self._detect_kind(f)) for f in files]
def _deduplicate_sha256(self, files: List[Path]) -> List[Path]:
"""Streaming SHA-256 dedup, 8MB chunks, no OOM"""
seen = set()
unique = []
for f in files:
h = self._file_sha256(f)
if h not in seen:
seen.add(h)
unique.append(f)
return unique
Key Points:
- Streaming SHA-256, handles GB files without OOM
- PDF text layer quality check: garble rate <30% && ≥20 chars → text layer shortcut
- Multi-receipt auto-split: Otsu + morphology + contour detection, convexity/aspect/line density
2. Classification & Extraction Pipeline
# app/pipeline/pipeline.py - Single file processing main flow
class Pipeline:
def process_file(self, source: SourceFile) -> Receipt:
# 1. Image load + preprocessing
images = self._load_images(source)
# 2. OCR recognition (RapidOCR)
ocr_results = self.ocr_engine.recognize(images)
# 3. Layout classification (17 templates + generic fallback)
ticket_type, type_conf = self.classifier.classify(ocr_results)
# 4. Field extraction (anchor rules + format validators)
fields = self.extractor.extract(ocr_results, ticket_type)
# 5. Confidence fusion (5-way weighted)
for field in fields.values():
field.confidence = self._compute_confidence(field)
# 6. Three-layer validation
alerts = self.validator.validate(receipt)
return receipt
3. Field-Level Confidence Fusion (Core Algorithm)
# app/extract/confidence.py - 5-way weighted fusion
class ConfidenceFuser:
WEIGHTS = {
'ocr': 0.45, # OCR raw confidence
'anchor': 0.20, # Anchor rule match quality
'format': 0.15, # Format validity
'layout': 0.10, # Layout partition consistency
'cross_field': 0.10 # Cross-field arithmetic
}
def fuse(self, field: FieldValue, context: Receipt) -> float:
scores = {
'ocr': self._ocr_score(field),
'anchor': self._anchor_score(field),
'format': self._format_score(field),
'layout': self._layout_score(field),
'cross_field': self._cross_field_score(field, context)
}
return sum(scores[k] * self.WEIGHTS[k] for k in scores)
def _ocr_score(self, field: FieldValue) -> float:
"""0.7×mean + 0.3×min, penalizes long numbers with single low-conf chars"""
if not field.ocr_confidences:
return 0.0
return 0.7 * mean(field.ocr_confidences) + 0.3 * min(field.ocr_confidences)
Confidence Visualization Mapping:
| Range | Color | Icon | Meaning |
|---|---|---|---|
| ≥0.95 | 🟢 Green | ✓ | High confidence / manual |
| 0.85-0.95 | 🔵 Blue | ℹ | Reliable |
| 0.70-0.85 | 🟠 Orange | ⚠ | Needs review |
| <0.70 | 🔴 Red | ✗ | Low confidence / needs manual |
| Empty | ⚪ Gray | — | Unrecognised |
4. Three-Layer Validation System
# app/validate/ - Three-layer validation implementation
# Layer 1: Format Validation
class FormatValidator:
def validate(self, receipt: Receipt) -> List[Alert]:
checks = [
self._check_tax_id_checksum(receipt), # GB 32100-2015 checksum
self._check_invoice_code(receipt), # 10/12-digit code
self._check_invoice_number(receipt), # 8/20-digit number
self._check_amount_parse(receipt), # Amount parseable
self._check_date_parse(receipt), # Date parseable
self._check_tax_rate_whitelist(receipt), # 24 legal tax rates
self._check_required_fields(receipt), # Required fields
]
return [a for c in checks for a in c if a]
# Layer 2: Cross-field Arithmetic
class ReconciliationValidator:
def validate(self, receipt: Receipt) -> List[Alert]:
alerts = []
# Amount + Tax = Total
if not self._check_amount_sum(receipt):
alerts.append(Alert(
code='AMOUNT_MISMATCH',
message='Amount + Tax ≠ Total',
suggestion=self._suggest_fix(receipt) # Reverse-engineer fix
))
# Capitalised vs numeric total
# Line items sum vs subtotal
return alerts
# Layer 3: Business Rules
class BusinessValidator:
def validate(self, receipt: Receipt) -> List[Alert]:
# Date sanity: future = error / >5yr = warning
# Tax rate whitelist: 24 legal rates + 8 tax-exempt terms
# Duplicate fingerprints: code+number=error / merchant+date+amount=warning
# Amount magnitude: 0.01-1 suspicious decimal / ≥10M suspicious large
pass
Key Technical Decisions
Decision 1: Why Not Cloud OCR / Why Not Tesseract?
| Dimension | Cloud OCR | Tesseract | RapidOCR (Chosen) |
|---|---|---|---|
| Privacy | ❌ Upload | ✅ Local | ✅ Local |
| Chinese Support | Good | Poor | Good |
| Maintenance | Low | High | Medium |
| Model Control | ❌ | ✅ | ✅ |
| Speed | Network latency | Slow | Fast (ONNX CPU) |
| License | Commercial | Apache 2.0 | Apache 2.0 |
Verdict: RapidOCR (PP-OCRv6) + ONNX Runtime best fits fully offline, commercial-friendly, Chinese-first requirements.
Decision 2: PyInstaller onedir vs onefile?
# packaging/ReceiptForge.spec - Critical config
COLLECT = COLLECT(
exe,
a.binaries,
a.zipfiles,
a.datas,
strip=False,
upx=True,
upx_exclude=[
'onnxruntime.dll', # UPX breaks ONNX Runtime
'opencv_videoio*.dll', # FFmpeg deps
'Qt6*.dll', # Qt plugin loading
],
name='ReceiptForge'
)
Why onedir?
- Models >200MB, onefile extracts to temp on every launch → slow cold start
- onedir reads
_internal/directly → fast startup, memory efficient - Supports incremental updates (replace only changed files)
Decision 3: Full Offline Network Blocking
# app/engine.py - Runs earliest, before any business imports
import socket
_LOOPBACK_HOSTS = {"127.0.0.1", "::1", "localhost", "0.0.0.0"}
_original_connect = None
_original_create_connection = None
_blocked_count = 0
_blocked_log = []
def _is_loopback(address) -> bool:
if isinstance(address, tuple) and address:
host = str(address[0])
if host in _LOOPBACK_HOSTS:
return True
try:
ip = ipaddress.ip_address(host)
return ip.is_loopback
except ValueError:
pass
return False
def _blocked_connect(self, address, *args, **kwargs):
global _blocked_count, _blocked_log
if not _is_loopback(address):
_blocked_count += 1
_blocked_log.append((time.time(), address))
raise NetworkBlockedError(f"Outbound connection blocked: {address}")
return _original_connect(self, address, *args, **kwargs)
def install_offline_guard():
global _original_connect, _original_create_connection
_original_connect = socket.socket.connect
_original_create_connection = socket.create_connection
socket.socket.connect = _blocked_connect
socket.create_connection = _blocked_create_connection
Verification:
# Runtime verification
python -c "from app.engine import blocked_count; import socket; socket.create_connection(('baidu.com', 80))"
# Expected: NetworkBlockedError raised
Performance Optimization
1. Model Loading Optimization
# app/ocr/rapid_engine.py - Singleton + lazy loading
class RapidEngine:
_instance = None
_lock = threading.Lock()
@classmethod
def get_instance(cls, spec: str = 'small') -> 'RapidEngine':
with cls._lock:
if cls._instance is None or cls._instance.spec != spec:
cls._instance = cls(spec)
return cls._instance
def __init__(self, spec: str):
self.spec = spec
# Load once, reuse session
self.session = ort.InferenceSession(
model_path,
providers=['CPUExecutionProvider'],
sess_options=ort.SessionOptions()
)
2. Batch Processing Pipeline
# app/ui/main_window.py - Background thread + progress signals
class _Worker(QObject):
progress = Signal(dict) # {'done': 5, 'total': 9, 'current': 'xxx.jpg'}
receipt_done = Signal(Receipt)
finished = Signal(list, Progress)
def run(self):
for i, file in enumerate(self.files):
if self._cancelled:
break
receipt = self.pipeline.process_file(file)
self.receipt_done.emit(receipt)
self.progress.emit({'done': i+1, 'total': len(self.files)})
3. Memory Management
| Optimization | Approach | Impact |
|---|---|---|
| Image loading | cv2.imdecode(np.fromfile(...)) |
Fixes Chinese paths + memcpy |
| Large images | Tile / downscale to 1920px | Prevents OOM |
| Temp files |
tempfile.mkstemp + explicit cleanup |
No leftovers |
| Log rotation | 10MB/file × 7 days | Prevents disk full |
Lessons Learned & Pitfalls
Pitfall 1: PyInstaller Can't Find Models at Runtime
# ❌ Wrong: Relative path breaks after packaging
model_path = Path(__file__).parent / 'models' / 'det.onnx'
# ✅ Correct: Use bundle_root() to locate resource root
from app.paths import bundle_root
model_path = bundle_root() / 'models' / 'det.onnx'
# app/paths.py
def bundle_root() -> Path:
"""Resource root: _MEIPASS when packaged, source root in dev"""
env = os.environ.get("RECEIPTFORGE_HOME")
if env:
return Path(env)
if getattr(sys, "frozen", False):
meipass = getattr(sys, "_MEIPASS", None)
if meipass:
return Path(meipass)
return Path(sys.executable).resolve().parent
return Path(__file__).resolve().parent.parent
Pitfall 2: PyMuPDF (AGPL) Commercial Distribution Risk
# app/pipeline/pdf_render.py - BSD-3 alternative path
def _renderer_name() -> str:
return os.environ.get("RECEIPTFORGE_PDF_RENDERER", "pymupdf").lower()
def extract_pdf(path: Path, dpi: int = 300) -> List[PdfPage]:
if _renderer_name() == "pypdfium2":
return _extract_pypdfium2(path, dpi) # BSD-3, safe
return _extract_pymupdf(path, dpi) # AGPL-3.0, risk
Fix: Set RECEIPTFORGE_PDF_RENDERER=pypdfium2 in production, or exclude pymupdf from spec.
Pitfall 3: Windows Long Paths + Chinese Paths
# app/paths.py - Long path prefix + Unicode support
LONG_PATH_THRESHOLD = 240
def long_path(path: str | os.PathLike) -> str:
s = os.fspath(path)
if IS_WINDOWS and len(s) > LONG_PATH_THRESHOLD:
return "\\\\?\\" + os.path.abspath(s)
return s
# Image reading: bypass OpenCV ANSI API
def imread_unicode(path: Path) -> np.ndarray:
data = np.fromfile(str(path), dtype=np.uint8) # Supports Chinese paths
return cv2.imdecode(data, cv2.IMREAD_COLOR)
Pitfall 4: PySide6 High DPI Scaling
# app/main.py - Early setup, before QApplication creation
def _enable_dpi_awareness():
if sys.platform == "win32":
try:
# Per-Monitor DPI Aware V2
ctypes.windll.shcore.SetProcessDpiAwareness(2)
except Exception:
try:
ctypes.windll.user32.SetProcessDPIAware()
except Exception:
pass
Pitfall 5: MSIX Packaging Common Errors
| Error | Cause | Fix |
|---|---|---|
0x80080204 |
Missing Assets | Generate 5 required images (StoreLogo/Square44/150/Wide310/Splash) |
0x80070003 |
Invalid Manifest XML | Check namespaces, Identity fields |
| Publisher Mismatch | Manifest Publisher ≠ Partner Center | Sync CN=...
|
| Version Revision ≠ 0 | 1.0.0.1 invalid | Must be X.Y.Z.0 format |
Open Source & Community
Project Structure
ReceiptForge/
├── app/ # Core business logic
│ ├── ocr/ # OCR engine wrapper
│ ├── pipeline/ # Processing pipeline
│ ├── extract/ # Field extraction
│ ├── validate/ # Three-layer validation
│ ├── ui/ # PySide6 UI
│ └── paths.py # Path management
├── packaging/ # Packaging configs
│ ├── msix/ # MSIX manifest & assets
│ ├── installer/ # Inno Setup script
│ └── ReceiptForge.spec # PyInstaller config
├── scripts/ # Build/tool scripts
├── tests/ # Unit/integration tests
└── docs/ # Documentation
Key Metrics
| Metric | Value |
|---|---|
| Lines of Code | ~15,000 (Python) |
| Test Coverage | 85%+ (918 tests) |
| Supported Formats | 18 (11 img + PDF + 6 text) |
| Receipt Layouts | 17 classes |
| Export Formats | 4 |
| Multilingual Models | 6 (EN/KO/CYR/AR/TH/LAT) |
Conclusion
ReceiptForge isn't chasing "highest accuracy" — it's engineered for "trustworthy, controllable, reviewable".
Engineering = Honesty: Unrecognised = empty, reconciliation fails = suggestions, failures still export, code blocks network — these "clunky" choices are the only responsible stance for real financial workflows.
If you're building offline-first desktop apps, hope these lessons save you detours.
Resources
- 📦 Microsoft Store: ReceiptForge
- 📖 Full Docs:
docs/marketing/store_listing_submission.md - 🛠 Build Scripts:
scripts/build_exe.py|scripts/build_msix.py - 🧪 Test Report:
docs/真实票据测试报告.md - 📜 Privacy Policy:
docs/marketing/privacy_policy.md
Top comments (0)