When building a desktop knowledge workspace that integrates local LLMs (Ollama) and on-device machine learning, the conventional shortcut is obvious: spin up an Electron wrapper, bundle an embedded Chromium runtime, and ship.
After all, almost every major desktop AI app in 2024–2026 is an Electron shell.
However, after running into severe memory ceilings and thermal throttling during heavy research sessions, we decided to build Knovum on a radically different stack: Tauri 2, Vue 3, and a compiled Rust backend.
Here is an honest engineering breakdown of what we gained, the hardest technical problems we had to solve (including running ONNX OCR on CPU without Python), and why we'll never go back to Electron.
1. The Real Cost of Electron: The 800MB Memory Baseline
Local AI introduces a resource constraint that web apps never faced:
Model weights live in physical memory (RAM / VRAM).
If a user runs a 7B or 8B parameter model via Ollama (e.g., Llama 3.2 or Qwen 2.5), that model demands 4.5GB–6GB of RAM. If the user is on an 8GB or 16GB MacBook or standard laptop, every megabyte of desktop overhead directly competes with the model's context window.
| Metric | Typical Electron AI Wrapper | Knovum (Tauri 2 + Rust + Vue 3) |
|---|---|---|
| Idle RAM Usage | 650MB – 950MB | ~42MB – 48MB |
| Cold Boot Latency | 2.8s – 4.5s | ~350ms |
| Binary Installer Size | 180MB – 250MB | ~38MB compressed |
| IPC Serialization | JSON stringify over IPC channel | Zero-copy binary buffers & Rust channels |
By delegating web rendering to the OS-native engine (WKWebView on macOS, WebView2 on Windows) and keeping the heavy lifting in compiled Rust, Knovum leaves 98% of your computer's memory free for local LLMs and vector math.
2. Solving On-Device OCR Without Bundling 4GB of Python
A major use case for desktop research is reading scanned PDFs, receipts, book pages, and whiteboard screenshots.
Most "local" OCR solutions either:
- Shell out to a cloud vision model (destroying user privacy and requiring subscription API keys).
- Bundle a full Python environment with PyTorch or PaddlePaddle, resulting in a multi-gigabyte installer and broken virtual environments.
In Knovum, we solved this by running PP-OCRv6 directly via the ONNX Runtime CPU worker in Rust:
- Fixed & Pinned Model: We pinned the PP-OCRv6 ONNX weights with SHA-256 verification, downloaded once on first run.
- Background Worker Priority: The text recognition job runs asynchronously on a dedicated worker pool with background scheduling priority, consuming zero GPU compute and avoiding any UI lag.
-
Locator Binding: Recognized text lines are bound directly to their physical locator (e.g.
page: 3, line: 12orimage: whiteboard_1). When the local LLM generates an answer, its citations point directly to these coordinates.
3. In-Canvas PDF Reading with Highlight Anchors
When an AI quotes an 80-page document, the biggest UX failure is the "citation disconnect" — reading an abstract text chunk without being able to verify the original context.
We lazy-load an in-app pdf.js viewer directly onto the infinite 2D canvas:
- When a user clicks a citation in an AI response, the viewer opens directly to the exact page and paints an active highlight over the quoted passage.
- 48-Character Anchor Fallback: Because OCR text can occasionally drift by a character or two compared to layout text, we compute 48-character sliding anchor hashes. Even if whitespace or ligatures vary, the highlight reliably lands on the exact visual paragraph.
4. Open Standards: Why We Adopted JSON Canvas (.canvas)
SaaS knowledge tools love locking your notes into proprietary database blobs. If the startup shuts down or hikes pricing, your structured thoughts are trapped.
Knovum stores every visual board directly as an open JSON Canvas (.canvas) file on your local filesystem.
- You can open the exact same canvas file in Obsidian Canvas.
- Notes and relations live in clean, queryable local SQLite tables with numbered migrations.
- Vector search runs locally via embedded LanceDB.
Summary & What's Next
Building with Tauri 2 and Rust requires disciplined architecture — there are no sloppy web shortcuts, and memory lifetimes must be handled deliberately. But the payoff is immense: a fluid, instant, 45MB desktop canvas that respects privacy and runs completely offline.
Knovum is source-available and free for personal & research use.
We’d love to hear your thoughts, feedback, and architecture critiques!
⭐ GitHub: https://github.com/madebyzan/knovum
📦 Releases (macOS DMG & Windows Setup): https://github.com/madebyzan/knovum/releases/latest
Top comments (0)