DEV Community

Zane
Zane

Posted on

Knovum v0.0.3: Building a Local-First Desktop AI Canvas with Tauri 2, Vue 3, and Rust

#ai

Like many developers and researchers, I've spent the past two years experimenting with local LLMs (via Ollama) and local RAG. But existing desktop frontends repeatedly hit two frustrating walls:

  1. The 1D Chat Trap: Linear chat streams work well for simple Q&A, but break down during deep research synthesis. Earlier reasoning turns scroll off the top, and you cannot visually compare multiple document excerpts side-by-side.
  2. The Electron Memory Tax: Most desktop AI wrappers idle at 700MB–1GB RAM before you even load model weights into VRAM.
  3. Modal Dialog Friction: Whenever an AI tool opens a floating modal popup to generate text or images, it hides your active research notes and breaks your creative flow.

To solve this, we built Knovum — a source-available, local-first desktop AI knowledge workspace built on an infinite 2D canvas with Tauri 2, Rust, and Vue 3.

We are preparing our v0.0.3 release, so I wanted to share the engineering decisions behind the architecture, what we learned building it, and what's new.

Core Architecture: 45MB Idle RAM & Local-First Storage 

┌────────────────────────────────────────────────────────┐
│             Vue 3 Spatial Frontend                     │
│  (Konva Infinite 2D Canvas + JSON Canvas Specification) │
└───────────────────────────┬────────────────────────────┘
                            │ Tauri 2 IPC (Channels)
┌───────────────────────────▼────────────────────────────┐
│                    Rust Backend Core                   │
│   ┌────────────────────┐      ┌────────────────────┐   │
│   │   SQLite Storage   │      │ Embedded LanceDB   │   │
│   │  (Notes, Boards,   │      │ (In-Process Vector │   │
│   │   Numbered Migr.)  │      │   Search Engine)   │   │
│   └────────────────────┘      └────────────────────┘   │
│   ┌────────────────────┐      ┌────────────────────┐   │
│   │    Ollama API      │      │ Embedded Whisper   │   │
│   │ (Local LLM Router) │      │ (Speech-to-Text)   │   │
│   └────────────────────┘      └────────────────────┘   │
└────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

1. Why Tauri 2 + Vue 3?

By leveraging native OS WebViews (WKWebView on macOS, WebView2 on Windows) orchestrated by a compiled Rust backend, Knovum cold-boots in sub-second time and idles around 45MB of RAM. This leaves virtually all of your machine's physical memory dedicated to loading model weights.

2. Dual-Engine Embedded Storage

  • SQLite: The canonical single source of truth for notes, cards, and relations, guarded by strict numbered migrations.
  • Embedded LanceDB: Runs in-process on disk for fast vector similarity search without requiring Docker, external daemons, or Python runtimes.

3. Zero Lock-In with JSON Canvas

Every visual workspace is stored directly on your local filesystem using the open JSON Canvas (.canvas) specification. You can open, edit, and inspect any Knovum board directly in Obsidian Canvas or parse it with custom scripts.

What's New in Knovum v0.0.3

Here are the key improvements shipped in our v0.0.3 release:

1. In-Canvas Framed Generators (Goodbye Modal Popups!)

We completely removed floating creation dialogs in favor of in-canvas framed generators:

  • Generative actions (image, video prompt, and document drafting) occur directly inside framed canvas nodes sized to the desired output.
  • The generator frame features an integrated composer beneath it (references, prompt, aspect ratio, duration, executor).
  • When generation finishes, the drafted artifact stands directly on the board in its designated frame.

2. Canvas Reference Pinning

You can now pin existing canvas nodes as multimodal prompt references with a single click (Pick from canvas). The canvas selection gesture attaches reference notes or images directly to the generator, ensuring the AI model has precise spatial context.

3. Side-by-Side Visual Comparison

Derived images and generated documents can be inspected and compared directly alongside their source inputs on an infinite visual board, making multi-iteration refinements effortless.

4. Differential AST Indexing in Rust

In traditional vector RAG setups, updating a document collection often triggers expensive full re-indexes. In Knovum:

  • We track leaf-node blake3 content hashes in Rust AST parsers.
  • Edits update only the specific changed leaf node (~92% single-leaf writes), keeping background LanceDB and SQLite indexing imperceptible.
  • The live index stays fully readable and searchable during async background indexing.

5. Throttled IPC Decoupling (~10Hz)

High-frequency background events (Whisper transcription progress, vector index progress) are emitted through dedicated Tauri Channels to a lightweight shared Task Center throttled to ~10Hz. The 2D Konva canvas layer never subscribes to raw token events, keeping zoom, pan, and drag gestures buttery-smooth at 60 FPS.

Download & Source Code

Knovum is source-available and free for personal/research use under the Knovum Personal Noncommercial License, with transparent open data formats (JSON Canvas, SQLite, Markdown).

If you are interested in local-first software, spatial computing, or running LLMs on your own desktop hardware, check out the repository on GitHub, leave a ⭐ if you like the concept, and share your feedback!

Top comments (0)