TL;DR
- I built VoiceVault, an open-source macOS app that does what Wispr Flow and Granola do — hold-a-key dictation anywhere, and manual-start meeting recording with an AI summary — with audio, transcription, and summarization staying on the machine.
- The two products don't need two apps. Dictation and meeting notes share the same Whisper transcription pipeline; the only real difference is trigger (hotkey vs. a Record button) and downstream processing (paste vs. summarize).
- Local-first isn't hardcoded — Whisper and Ollama sit behind a provider abstraction: a common interface with five other STT backends and three other LLM backends behind it, auto-discovered by health check, swappable without touching the dictation or meeting pipeline.
- On a 48GB M-series Mac, VoiceVault plus its dashboard run around 1.8GB; the number to actually plan around is Ollama's model weights, which spike to ~9.2GB while
llama3.1:8bis loaded and are governed by Ollama's own idle timeout, not VoiceVault's. - It doesn't match Wispr Flow or Granola on breadth — no speaker diarization, no mobile app yet, no calendar integration, macOS-only. What it matches is the specific workflow I actually use daily, with nothing leaving the machine.
Wispr Flow and Granola solve two halves of the same problem — talk instead of type, and let AI handle the notes — and both do it well. Neither runs the audio, the transcription, or the summarization on your machine. I wanted the same workflow with a hard constraint: nothing I say leaves the laptop. So I built VoiceVault, a macOS menu bar app that reproduces both experiences using local models only — mlx-whisper for speech-to-text and Ollama running llama3.1:8b for summarization.
The Wiring
flowchart LR
HK["Fn hotkey<br/>(CGEventTap)"] --> MIC["Mic capture"]
REC["Record button<br/>(Electron dashboard)"] -->|"HTTP, 127.0.0.1"| API["Local FastAPI server"]
API --> MM["MeetingManager<br/>(shared singleton)"]
MM --> MIC
MIC --> STT{"STT provider registry"}
STT -->|default| WHISPER["mlx-whisper<br/>(in-process)"]
STT -.->|swap| CLOUDSTT["Deepgram / AssemblyAI /<br/>Speechmatics / Rev.ai / AWS"]
WHISPER -->|dictation| PASTE["Paste at cursor"]
WHISPER -->|meeting transcript| LLM{"LLM provider registry"}
LLM -->|default| OLLAMA["Ollama service<br/>(llama3.1:8b, own lifecycle)"]
LLM -.->|swap| CLOUDLLM["Anthropic / OpenAI / OpenRouter"]
OLLAMA --> SUMMARY["Meeting summary"]
SUMMARY --> OBS[("Obsidian export")]
WHISPER --> INDEX[("SQLite: sqlite-vec + FTS5")]
INDEX --> ASK["Ask tab (RAG)"]
ASK --> LLM
Two triggers, one funnel: a hotkey or a Record click both end up feeding the same mic capture, through the same STT provider registry, defaulting to the in-process Whisper model. Dictation output goes straight to the cursor; meeting transcripts continue on to the LLM provider registry for a summary, an Obsidian export, and a slot in the search index the Ask tab queries later. The dotted arrows are the swap points — same funnel, different provider, no code change on either side of it.
One Pipeline, Two Triggers
Wispr Flow and Granola read as separate products, but underneath they're the same operation triggered differently. VoiceVault treats them that way: one Whisper pipeline, two entry points.
Dictation is push-to-talk. Hold Fn, speak, release — the audio streams to mlx-whisper and the transcribed text lands at the cursor in whatever app has focus. A double-tap switches to hands-free mode for longer dictations, closing with a second tap instead of a held key. Every dictation appends to a daily Markdown log, so nothing is lost even if the paste target was wrong.
Meeting mode is manual-start recording, not always-on capture. Click Start Recording in the dashboard, and VoiceVault captures the microphone (plus system audio through BlackHole if you route it) while you take inline timestamped notes. Click Stop, and the same Whisper pipeline transcribes with timestamps, Ollama generates a summary, and the whole thing exports to Obsidian as Markdown with YAML frontmatter — summary, your notes, and the full transcript in one file.
The text injection path for dictation has three fallbacks in order: simulated Cmd+V through the clipboard (primary), AppleScript keystroke injection if clipboard access isn't available, and a clipboard-only mode as the last resort where you paste manually. Three fallbacks for one operation is more code than a single reliable path would need — the tradeoff is that dictation degrades instead of silently failing when a target app blocks synthetic keystrokes.
The Provider Layer Is What Makes It Scalable
"Local-first" describes VoiceVault's defaults, not a constraint baked into its architecture. Both the speech-to-text and summarization stages sit behind a provider pattern — the same pattern also known as a strategy pattern or provider abstraction: a single interface, multiple interchangeable implementations behind it, and the calling code never knows which one is live.
VoiceVault's STT side ships six providers behind that interface — local Whisper, Deepgram, AssemblyAI, Speechmatics, Rev.ai, AWS Transcribe — and the LLM side ships four: Ollama, Anthropic, OpenAI, OpenRouter. Every provider implements the same base interface and is auto-discovered at startup by a health check, not a hardcoded config switch. Add an API key for one of the cloud options and it becomes available without touching the dictation or meeting-recording code at all; remove it and VoiceVault falls back to the local default. An evaluation harness runs every provider you have keys for against the same reference audio, scores the output with an LLM-as-judge, and produces a ranked cost/quality report — so swapping isn't a leap of faith, it's a measured decision.
That structure is also why VoiceVault isn't a single, monolithic Electron app with a Dock icon end to end. The hotkey listener that drives dictation runs as a native macOS CGEventTap inside the menu bar process, independent of whether the Electron dashboard window is even open — Electron's own globalShortcut module shares the same OS-level limitation every web runtime hits on macOS: it cannot reliably capture the physical Fn/Globe key. Wispr Flow solves the identical problem the same way, with a native listener outside its web layer. Splitting the hotkey listener from the dashboard UI is the same instinct as the provider layer — isolate the part that has to be swappable or platform-specific from the part that doesn't.
What's Actually Running, and What It Costs
The dashboard is a second, separable piece: an Electron app for browsing every past dictation and meeting, hybrid search (semantic via sqlite-vec plus keyword via SQLite FTS5, combined with reciprocal rank fusion) over everything you've recorded, and an Ask tab that retrieves relevant notes and answers questions against them with llama3.1:8b, citing which note it used and saying plainly when the notes don't contain an answer.
Two things run independently of each other and of VoiceVault itself:
| Component | Runs when |
|---|---|
| VoiceVault menu bar app + Whisper | Loads on first dictation, resident until you quit VoiceVault |
| Electron dashboard | Only while its window is open |
| Ollama server | Background service since login, regardless of VoiceVault |
llama3.1:8b weights |
Loaded on first request from anything, unloaded after Ollama's idle timeout |
Measured on a 48GB M-series Mac with VoiceVault and the dashboard both open: VoiceVault plus the resident Whisper model runs about 1.4GB, the Electron dashboard about 360MB, Ollama idle about 150MB. The number that actually matters is the llama3.1:8b weights — about 9.2GB while loaded, controlled entirely by Ollama's OLLAMA_KEEP_ALIVE setting (five minutes by default), not by anything VoiceVault does. I lowered it to two minutes in a login LaunchAgent that runs launchctl setenv at boot, which frees that memory faster between meetings without touching VoiceVault's own code.
What's Missing
VoiceVault doesn't cover Wispr Flow and Granola's full surface. No speaker diarization — the transcript is timestamped but not attributed to a speaker. No mobile app yet, though the architecture has a planned path: an on-device STT preview on the phone, full reprocessing on the desktop once it syncs. No calendar integration to auto-join and auto-title meetings. macOS only, and the default Whisper backend needs Apple Silicon — Intel Macs have to switch to a CPU-based faster-whisper backend, which I have not benchmarked side by side with the default.
The two-process architecture — a Python menu bar app plus a separate Electron dashboard, talking over a local HTTP API — is also more moving parts than either Wispr Flow's or Granola's single-process apps. It exists because the menu bar app owns the hotkey listener and needs to run headless without a Dock icon, while the dashboard needs a real window and a browser runtime for search and chat UI. That split works, but I haven't found a way to collapse it into one process without losing either the headless hotkey listener or the dashboard's UI capabilities — that's the open problem, not a solved one.
So What
The privacy tradeoff in most AI dictation and meeting tools isn't a limitation of the underlying models — Whisper and a capable local LLM can do the transcription and summarization work on a modern Mac. It's a product decision to route through the cloud. VoiceVault is proof that the same daily workflow — talk instead of type, record a meeting and get a summary — runs entirely offline, on hardware that already sits on your desk, with the source open enough to verify that claim yourself.
Top comments (0)