I dictate prompts to coding agents (Claude Code, Cursor) all day. Typing paragraph-long prompts was my bottleneck, and I wanted the speech side fully local - push-to-talk, CPU-only, nothing leaving the machine. The model choices surprised me, so here's the stack.
What I ended up with
Built in Rust on sherpa-onnx:
- NVIDIA Parakeet TDT v3 for European languages: one model covers 25 languages with automatic detection, and TDT decoding is noticeably lighter on CPU than Whisper-class encoder-decoder models in my use. It also emits its own punctuation.
- Paraformer (zh-en bilingual) for Mandarin: code-switching ("??? function ?? async") is where general multilingual models fall apart, and a dedicated bilingual model handles it far better. Punctuation comes from a separate local CT-Transformer pass.
- Models download on demand (a few hundred MB each); everything runs offline afterwards.
Why not Whisper
Nothing wrong with Whisper - but for live push-to-talk on CPU, the latency/quality point of TDT worked better for me, and Paraformer beat everything I tried on bilingual Mandarin. I deliberately don't publish latency numbers (too hardware-dependent to be honest about), so treat this as one person's experience, not a benchmark.
The part nobody talks about: insertion
Text insertion is harder than recognition. Getting clean text into whatever field has focus - terminal, editor, browser - across Windows and macOS is its own rabbit hole. The worst one: Windows silently stops delivering low-level keyboard hooks (WH_KEYBOARD_LL) when a Chromium-based window is focused. It's on record against Tauri and CEF too, and nothing you write inside the hook can fix it - the procedure is never entered. The fix is architectural: the focused window answers its own keydown events, and the global hook only owns the unfocused case. That one cost me a week.
Acronym handling matters more than WER for this use case: "m c p" has to come out as MCP, and a user-editable replacement dictionary catches project jargon the model can't know.
Disclosure
This shipped as a small paid app - VocalCode, USD 4.99 one-time with a 30-day free trial, at vocalcode.app - so I'm affiliated. The model discussion is the point of this post, though: if you're running TDT-family models for live dictation, or you've gotten good zh-en code-switching out of anything else local, I'd genuinely like to compare notes in the comments.
Top comments (0)