I dictate prompts to coding agents (Claude Code, Cursor) all day. Typing paragraph-long prompts was my bottleneck, and I wanted the speech side fully local - push-to-talk, CPU-only, nothing leaving the machine. The model choices surprised me, so here's the stack.
What I ended up with
Built in Rust on sherpa-onnx:
- NVIDIA Parakeet TDT v3 for European languages: one model covers 25 languages with automatic detection, and TDT decoding is noticeably lighter on CPU than Whisper-class encoder-decoder models in my use. It also emits its own punctuation.
- Paraformer (zh-en bilingual) for Mandarin: code-switching ("??? function ?? async") is where general multilingual models fall apart, and a dedicated bilingual model handles it far better. Punctuation comes from a separate local CT-Transformer pass.
- Models download on demand (a few hundred MB each); everything runs offline afterwards.
Why not Whisper
Nothing wrong with Whisper - but for live push-to-talk on CPU, the latency/quality point of TDT worked better for me, and Paraformer beat everything I tried on bilingual Mandarin. I deliberately don't publish latency numbers (too hardware-dependent to be honest about), so treat this as one person's experience, not a benchmark.
The part nobody talks about: insertion
Text insertion is harder than recognition. Getting clean text into whatever field has focus - terminal, editor, browser - across Windows and macOS is its own rabbit hole. The worst one: Windows silently stops delivering low-level keyboard hooks (WH_KEYBOARD_LL) when a Chromium-based window is focused. It's on record against Tauri and CEF too, and nothing you write inside the hook can fix it - the procedure is never entered. The fix is architectural: the focused window answers its own keydown events, and the global hook only owns the unfocused case. That one cost me a week.
Acronym handling matters more than WER for this use case: "m c p" has to come out as MCP, and a user-editable replacement dictionary catches project jargon the model can't know.
Disclosure
This shipped as a small paid app - VocalCode, USD 4.99 one-time with a 30-day free trial, at vocalcode.app - so I'm affiliated. The model discussion is the point of this post, though: if you're running TDT-family models for live dictation, or you've gotten good zh-en code-switching out of anything else local, I'd genuinely like to compare notes in the comments.
Top comments (1)
The WH_KEYBOARD_LL Chromium hole is the part that actually matches what I hit: hold-to-talk looks fine in a native editor, then silently dies the moment VS Code or a Chrome tab has focus. Routing keydown through the focused window is the right architecture. Curious whether that path still drops the last character when the caret is inside an Electron webview rather than a native control.