DEV Community: Zane

Knovum v0.1.0: Canvas Auto-Layout, In-Canvas Video Scrubbing & Local-First AI Architecture

Zane — Wed, 28 Oct 2026 16:53:27 +0000

Over the past few weeks, we've been building Knovum — a local-first, privacy-respecting desktop AI knowledge workspace built with Tauri 2, Vue 3, and Rust.

Unlike standard chat wrappers or cloud dashboards, Knovum provides an infinite 2D canvas where human notes, PDF excerpts, audio transcriptions, and local LLMs (via Ollama/LanceDB) live together spatially without vendor lock-in.

Today we're preparing our major milestone: v0.1.0. Here is an engineering breakdown of the three biggest technical challenges we solved in this release.

1. Figma-Style Auto-Layout & Tidy-Up on an Open Canvas (`.canvas`)

Spatial canvases are wonderful for brainstorming, but messy card sprawl is a known pain point. When you have dozens of AI generation cards and research notes scattered around, manual alignment becomes tedious.

In v0.1.0, we introduced native Auto-layout Sections and Tidy-Up:

Native Rust Packing (planner::packing): Instead of running heavy geometric calculations in JavaScript, spatial flow (row, column, and balanced grid packing) is calculated by compiled Rust code via Tauri IPC.
Squarest Table Packing: When organizing an arbitrary collection of cards into a Section, the planner computes the optimal aspect ratio to keep the bounding section as close to golden/square proportions as possible without overlapping.
Direct JSON Canvas Compatibility: All positioned cards, section groups, and flow relations are saved directly into the open standard *.canvas format (compatible with Obsidian Canvas), maintaining complete portability.

2. In-Canvas Video Scrubbing & Audio Waveform Streaming

Research often involves multimedia — recorded lectures, technical podcasts, and screen captures. In existing tools, clicking a video either opens a heavy external player or throws you into an isolated modal window.

In v0.1.0, we brought full Video & Audio Workbenches directly onto 2D canvas cards:

Streaming Audio Waveforms: We built a high-performance audio waveform visualizer that decodes and renders audio tracks in real-time without buffering whole gigabyte files into RAM.
Filmstrip & Frame-Accurate Scrubbing: Hovering over the timeline scrubs through visual filmstrip keyframes rendered directly within the Konva-based canvas node.
Whisper Integration with Diagnostics: Paired with our embedded whisper.cpp sidecar, audio from video cards can be transcribed locally into time-stamped note cards with zero audio bytes leaving your computer.

3. Preflight Diagnostics: Eliminating the "Silent AI Failure" Problem

One of the most frustrating aspects of local AI desktop apps is silent runtime failure:

Is Ollama running in the background?
Did the user pull the requested model (e.g. qwen2.5 or llama3.2)?
Does the system have sufficient VRAM available?
Are the speech sidecar binaries executable?

In v0.1.0, we implemented an AI Readiness Preflight Engine (features/ai-preflight):
Before firing off any generation or vector embedding pipeline, Knovum conducts non-blocking health checks:

Pings the local Ollama daemon and inspects installed model tags.
Verifies local Whisper sidecar binary permissions and model weights (ggml-base.bin).
Validates LanceDB vector index integrity and SQLite schema version.

If any check fails, the UI surfaces actionable, inline recovery prompts (e.g., "Ollama is reachable, but qwen2.5:7b is missing — run ollama pull qwen2.5:7b") instead of freezing or failing silently.

Performance Snapshot

Idle RAM: ~45MB (thanks to Tauri 2 native OS WebViews vs 800MB+ for Electron).
Storage: 100% local SQLite for notes + LanceDB for embeddings. No cloud subscription, no SaaS telemetry.
License: Source-available and free for personal & research use.

We’d love to hear your thoughts, feedback, and architecture critiques!

🔗 GitHub Repository: https://github.com/madebyzan/knovum

Knovum v0.0.3: Building a Local-First Desktop AI Canvas with Tauri 2, Vue 3, and Rust

Zane — Mon, 26 Oct 2026 16:20:07 +0000

Like many developers and researchers, I've spent the past two years experimenting with local LLMs (via Ollama) and local RAG. But existing desktop frontends repeatedly hit two frustrating walls:

The 1D Chat Trap: Linear chat streams work well for simple Q&A, but break down during deep research synthesis. Earlier reasoning turns scroll off the top, and you cannot visually compare multiple document excerpts side-by-side.
The Electron Memory Tax: Most desktop AI wrappers idle at 700MB–1GB RAM before you even load model weights into VRAM.
Modal Dialog Friction: Whenever an AI tool opens a floating modal popup to generate text or images, it hides your active research notes and breaks your creative flow.

To solve this, we built Knovum — a source-available, local-first desktop AI knowledge workspace built on an infinite 2D canvas with Tauri 2, Rust, and Vue 3.

We are preparing our v0.0.3 release, so I wanted to share the engineering decisions behind the architecture, what we learned building it, and what's new.

Core Architecture: 45MB Idle RAM & Local-First Storage

┌────────────────────────────────────────────────────────┐
│             Vue 3 Spatial Frontend                     │
│  (Konva Infinite 2D Canvas + JSON Canvas Specification) │
└───────────────────────────┬────────────────────────────┘
                            │ Tauri 2 IPC (Channels)
┌───────────────────────────▼────────────────────────────┐
│                    Rust Backend Core                   │
│   ┌────────────────────┐      ┌────────────────────┐   │
│   │   SQLite Storage   │      │ Embedded LanceDB   │   │
│   │  (Notes, Boards,   │      │ (In-Process Vector │   │
│   │   Numbered Migr.)  │      │   Search Engine)   │   │
│   └────────────────────┘      └────────────────────┘   │
│   ┌────────────────────┐      ┌────────────────────┐   │
│   │    Ollama API      │      │ Embedded Whisper   │   │
│   │ (Local LLM Router) │      │ (Speech-to-Text)   │   │
│   └────────────────────┘      └────────────────────┘   │
└────────────────────────────────────────────────────────┘

1. Why Tauri 2 + Vue 3?

By leveraging native OS WebViews (WKWebView on macOS, WebView2 on Windows) orchestrated by a compiled Rust backend, Knovum cold-boots in sub-second time and idles around 45MB of RAM. This leaves virtually all of your machine's physical memory dedicated to loading model weights.

2. Dual-Engine Embedded Storage

SQLite: The canonical single source of truth for notes, cards, and relations, guarded by strict numbered migrations.
Embedded LanceDB: Runs in-process on disk for fast vector similarity search without requiring Docker, external daemons, or Python runtimes.

3. Zero Lock-In with JSON Canvas

Every visual workspace is stored directly on your local filesystem using the open JSON Canvas (.canvas) specification. You can open, edit, and inspect any Knovum board directly in Obsidian Canvas or parse it with custom scripts.

What's New in Knovum v0.0.3

Here are the key improvements shipped in our v0.0.3 release:

1. In-Canvas Framed Generators (Goodbye Modal Popups!)

We completely removed floating creation dialogs in favor of in-canvas framed generators:

Generative actions (image, video prompt, and document drafting) occur directly inside framed canvas nodes sized to the desired output.
The generator frame features an integrated composer beneath it (references, prompt, aspect ratio, duration, executor).
When generation finishes, the drafted artifact stands directly on the board in its designated frame.

2. Canvas Reference Pinning

You can now pin existing canvas nodes as multimodal prompt references with a single click (Pick from canvas). The canvas selection gesture attaches reference notes or images directly to the generator, ensuring the AI model has precise spatial context.

3. Side-by-Side Visual Comparison

Derived images and generated documents can be inspected and compared directly alongside their source inputs on an infinite visual board, making multi-iteration refinements effortless.

4. Differential AST Indexing in Rust

In traditional vector RAG setups, updating a document collection often triggers expensive full re-indexes. In Knovum:

We track leaf-node blake3 content hashes in Rust AST parsers.
Edits update only the specific changed leaf node (~92% single-leaf writes), keeping background LanceDB and SQLite indexing imperceptible.
The live index stays fully readable and searchable during async background indexing.

5. Throttled IPC Decoupling (~10Hz)

High-frequency background events (Whisper transcription progress, vector index progress) are emitted through dedicated Tauri Channels to a lightweight shared Task Center throttled to ~10Hz. The 2D Konva canvas layer never subscribes to raw token events, keeping zoom, pan, and drag gestures buttery-smooth at 60 FPS.

Download & Source Code

Knovum is source-available and free for personal/research use under the Knovum Personal Noncommercial License, with transparent open data formats (JSON Canvas, SQLite, Markdown).

GitHub Repository: https://github.com/madebyzan/knovum
Releases & Prebuilt Binaries: https://github.com/madebyzan/knovum/releases
- macOS (Apple Silicon .dmg & Intel .dmg)
- Windows (x64-setup.exe)

If you are interested in local-first software, spatial computing, or running LLMs on your own desktop hardware, check out the repository on GitHub, leave a ⭐ if you like the concept, and share your feedback!