DEV Community: VoxRT

Four on-device TTS models on one CPU core: LinnetTTS, Piper, Kokoro, pocket-tts

VoxRT — Fri, 06 Nov 2026 09:40:44 +0000

Most CPU TTS writeups benchmark one model in isolation. When you are building a voice product, that is the wrong question. The real one is which small model earns its slot on your CPU budget, and what you give up compared to its neighbours.

Four open-source TTS models fit under 100 M parameters and target consumer CPUs: LinnetTTS 41 M (VoxRT), Piper medium 16 M (OHF-Voice, originally Rhasspy), Kokoro-82M (hexgrad), and pocket-tts 100 M (Kyutai Labs). Each has a different stance on voice cloning, streaming, and multilingual coverage. Here is how they land on the same desktop CPU with the same text.

The four models

LinnetTTS 41 M (VoxRT, shipped 2026-11-04) is an English-only autoregressive flow LM over a 25 Hz continuous-latent neural codec with a Vocos-style decoder. It clones a voice from a short audio prompt and emits 80 ms of audio per step, so playback can start before the full sentence is generated. Runtime is in Rust, no PyTorch at inference. LinnetTTS is an open-source on-device TTS runtime (Apache-2.0) with openly available model weights under a free license that permits commercial use; voice cloning and model training are licensed separately. Model card on Hugging Face: VoxRT/linnet-en-35m.

Piper medium 16 M is a VITS-style single-speaker English TTS, one voice per checkpoint, no voice cloning. It processes a sentence as one unit with no in-sentence streaming. GPL-3.0 (the original rhasspy/piper under MIT is now archived, active development is at OHF-Voice/piper1-gpl and pip install piper-tts resolves there). Voices and model files on Hugging Face: rhasspy/piper-voices. Primary audience is offline voice assistants and Home Assistant setups.

Kokoro-82M (hexgrad, published early 2025) is a multilingual TTS covering 8 languages with a bank of preset voices. Audio is generated per full utterance with no streaming. Apache-2.0 license. The release has strong quality on short English utterances and has become a common on-device peer.

pocket-tts 100 M (version 4.0.1 tested) targets real-time on 2-core CPUs with voice prompts and 80 ms streaming. It is similar in shape to Linnet but with a wider parameter budget and a more aggressive multi-core assumption. MIT license. The pip package is pocket-tts, project homepage at kyutai.org/pocket-tts. An English model is used here, with 7 languages available as separate models.

Methodology

Same hardware for all four: a desktop Core i9-13900K running Linux (WSL2) with processes pinned to performance cores via taskset. One-thread runs use one core, four-thread runs use four physical cores. The same three English test texts go through each model via its own Python package (pip install linnet-tts, piper-tts, kokoro-onnx, pocket-tts). Peak RSS is ru_maxrss of the Python process, interpreter and framework included, so these are process-level memory numbers, not raw model-weight sizes on disk.

One caveat the numbers hide: pocket-tts is designed to run across two cores and reports RTF on two cores. Linnet, Piper, and Kokoro are reported on one core. I kept both configurations because squashing pocket-tts to one core underreports the design intent.

Speed and memory only. Audio quality is not measured here. You will need to listen to decide that for yourself.

The numbers

Model	RTF 1-core	RTF 4-core	Peak RSS	First audio	Streaming	Voice cloning	Languages
Piper medium 16 M	5.8%	2.1%	503 MB	180 ms	no	no	1
LinnetTTS 41 M	10.6%	3.4%	187 MB	87 ms	80 ms steps	yes	1
pocket-tts 4.0.1 (100 M)	n/a	24.1%	968 MB	71 ms	80 ms steps	yes	1 (7 separate)
Kokoro-82M	51.7%	17.2%	1246 MB	1.5 s	no	no	8

Piper is the fastest here, partly because it is also the smallest and has the simplest decoding path (no voice prompts, no streaming). LinnetTTS is second fastest and the smallest in RSS by a wide margin. pocket-tts hits its design point on 2 cores at 23.8% RTF, and adding more cores barely moves the number (24.1% on four), so plan the budget around 2 cores. It also has the lowest first-audio latency at 71 ms, 16 ms ahead of LinnetTTS at 87 ms. Kokoro is the slowest in this group with 1.2 GB resident, which is the price its multilingual bank of voices pays.

What to pick and when

Pick Piper if you need a single English voice, no cloning, no in-sentence streaming, and the smallest install on disk (.onnx 64 MB for the medium voice). This is the right call for a static TTS backend on constrained devices, including small Raspberry Pis, where extras are overhead you do not need.

Pick LinnetTTS if you need voice cloning from a short prompt, 80 ms streaming so playback starts before the sentence ends, and a smaller resident footprint (187 MB vs 968 MB for pocket-tts, mostly a Rust-vs-PyTorch runtime difference as noted in Known limits). The trade is English only and a hybrid license (Apache code, Linnet Weights License for weights).

Pick pocket-tts if your target is a laptop or desktop with at least 2 performance cores available, you want voice prompts, and you accept around 1 GB of process memory. Fully permissive under MIT if license matters to you.

Pick Kokoro if you need more than one language. It is the only one here with 8-language coverage. The cost is the highest memory and the slowest speed, and no in-sentence streaming.

Rule of thumb for single-English on CPU: Piper if no cloning, Linnet if cloning. For multilingual CPU: Kokoro, with pocket-tts as an alternate when English-only is enough and memory is not tight.

Known limits of this benchmark

Three things this does not tell you. First, audio quality is not scored. Mean opinion scores on a shared test set would need a listening study, which is a different write-up. Second, this is one desktop CPU. Phones and Raspberry Pis will reorder the ranking because the models respond differently to thermal and memory pressure. Third, each model was run through its own Python package, so the numbers include whatever overhead each library imposes. For Piper and Kokoro that is ONNX Runtime, for pocket-tts it is PyTorch, for Linnet it is a thin Rust-backed wrapper. Much of the RSS gap between models is the runtime itself, not the model weights. Raw model-weight files on disk tell a different story: Piper 64 MB, Linnet 83 MB fp16, pocket-tts 236 MB, Kokoro 326 MB.

Run it yourself

The code, Rust runtime, and benchmark script are at github.com/VoxRT/LinnetTTS. If you want to hear Linnet alongside the other three on the same text, the Compare tab on the demo Space runs all four side by side. Model card and weights are at huggingface.co/VoxRT/linnet-en-35m.

Your turn

Which of these four is sitting in your product right now, and did the switch to a smaller model buy you what you expected, or did you trade quality for speed in a way that bit you later?

Wake-word vs KWS vs streaming ASR: when to pick what for voice interfaces

VoxRT — Tue, 03 Nov 2026 07:36:50 +0000

Why architecture choice matters

Three different voice ML architectures solve three different problems, but they get mixed up constantly. Picking the wrong one means either overpaying on compute and storage (using full ASR where KWS would do) or underwhelming the UX (trying to scale a wake-word into a task that needed KWS).

This article is a short decision guide. Three architectures (wake-word, KWS, streaming ASR), a profile of each with real numbers, hybrid flows, a decision tree by use case, and common mistakes we see in projects.

Numbers below are VoxRT reference measurements (our shipping SDKs). Competitor profiles vary. The point of this article is architectural fit, not vendor benchmark.

The three architectures

1. Wake-word (always-on trigger detection)

Model size: 100 KB (verified voxrt-wake-word-linux)
CPU: 5.3% RTF on Raspberry Pi Zero 2 W (one A53 core at 1 GHz)
Latency: under 100 ms detection
Vocabulary: 1 fixed trigger phrase ("Hey Assistant" or similar)
Use case: entry trigger for voice interaction (24/7 listening at minimal load)
Trade-off: cannot do anything beyond detecting the one phrase

The wake-word model is the smallest and cheapest of the three. Its job is to listen 24/7 on a minimal CPU budget, decide whether the trigger phrase was said, and hand off to another model or wake up the app. Not used standalone for actual commands.

Real projects sometimes ship 2 or 3 wake-word models in parallel (an entry trigger plus a barge-in stop phrase, for example). Each adds its own CPU share, but at 5% per model this is affordable on all but the most constrained hardware.

2. KWS (keyword spotting)

Model size: 1.3 MB (verified voxrt-kws-linux)
CPU: 32.2% RTF on Pi Zero 2 W
Latency: 120 ms sustained detection (3 consecutive detection frames at 25 fps) plus output
Vocabulary: 14 fixed commands (yes/no/cancel/play/pause/next/previous/up/down/back/on/off plus 2 wake-like)
Use case: closed-set voice control (media controls, UI navigation, binary switches, confirmation flows)
Trade-off: vocabulary is fixed. Custom words require retraining, not runtime config

KWS sits in the middle ground between wake-word and ASR. It knows a limited command set (14 in our case), runs real-time on cheap hardware like Pi Zero 2 W, and is more accurate for its vocabulary than open-vocab ASR because the confusion space is smaller. Underrated for DIY voice-control projects. Most projects reach for ASR when KWS would do the job.

Beyond raw detection, real KWS deployments spend time tuning per-command thresholds, FAR (false accept rate) vs FRR (false reject rate), and sometimes applying downstream confidence filtering. Default threshold 0.9 is a starting point, not a final answer.

3. Streaming ASR (open-vocabulary transcription)

Model size: 60.4 MB (verified voxrt-asr-models v0.1.2)
CPU: does not fit Pi Zero 2 W (overload). Works on Pi 4 and up, plus mobile class hardware
Latency: 200 to 500 ms first partial transcript, more to end of utterance
Vocabulary: open English speech (any word)
Use case: dictation, voice assistant queries, free-form user input, transcription
Trade-off: large model, requires meaningful hardware (mobile or Pi 4 minimum)

Streaming ASR is the "say anything" architecture. It accepts arbitrary speech and emits text transcripts incrementally. Expensive on compute and storage compared to WW and KWS, but the only path when you need to handle arbitrary user speech.

Many ASR engines also support hot-word biasing: a runtime list of words to weight higher in decoding (useful for custom product names, people names, domain vocabulary). Sherpa-onnx and whisper.cpp both support this. It is a middle path between the fully open vocabulary and the fully fixed KWS set.

Hybrid flows

Real production voice systems rarely use just one architecture. Common combinations:

Wake-word plus ASR

The voice assistant classic (Alexa, Siri, Google Assistant pattern). Wake-word listens 24/7 cheaply and triggers streaming ASR for the complex query. Battery efficient because ASR only runs when needed, not as a sustained drain.

Wake-word plus KWS

For constrained UIs: smart display, kiosk, embedded controls, hands-free lab setup. Wake-word gates attention and KWS handles the limited command set. Faster than ASR for known actions because you do not need to load a large model. A good fit for Pi Zero or Pi 4 with media or IoT controls.

Wake-word plus KWS plus ASR (full hybrid)

Pi 4 class or mobile hardware. Wake-word triggers, KWS tries a fast path for common commands, ASR falls back for anything else. Best latency for predictable commands with graceful fallback for complex phrases.

Edge-cloud hybrid

Dominant pattern in shipping consumer products: on-device wake-word triggers cloud ASR. Cheap always-on listening with heavyweight transcription offloaded. Trade-offs on privacy, latency, and ongoing cost. Not an on-device-only architecture, but worth naming because readers often pick it without realizing they have.

Orchestration note: most SDKs do not ship a turnkey multi-stage orchestrator with VAD + audio buffer carryover + state machine for free. Picovoice ships Porcupine + Rhino + Cheetah with documented reference flows. Sherpa-onnx has KeywordSpotter plus streaming ASR with sample orchestration code in the repo. Everyone else leaves the state machine ("waiting for trigger", "listening for command", "transcribing") to the integrator. Expect to spend meaningful time on orchestration code even if the model integration is simple.

Decision tree by use case

You need a single trigger phrase ("Hey X"):
→ Wake-word only. 100 KB, lowest CPU, works on everything down to microcontrollers.

You need 5 to 14 fixed commands (UI control, media, IoT):
→ KWS. Faster and more accurate than ASR for this scope. Works on Pi Zero 2 W, Pi 4, mobile. 1.3 MB model.

You need open vocabulary (dictation, user queries, free-form input):
→ Streaming ASR. Mobile class hardware minimum (Pi 4 or above). 60.4 MB model. No way around it for truly open vocabulary.

You need both commands and dictation:
→ Wake-word plus KWS plus ASR hybrid. Pi 4 or modern mobile minimum. Plan to write orchestration code yourself.

You need custom commands not in the 14-list:
→ Three options: ASR with hot-word biasing for a few extra terms, custom KWS training for a few dozen fixed commands, or open-vocabulary ASR for anything beyond that.

You need multi-language support:
→ Streaming ASR with multilingual models. Whisper multilingual, sherpa-onnx multilingual Zipformer, Vosk 20+ language packs. Wake-word and KWS vocabularies are language-locked to the model you ship.

Target is a noisy environment (room-wide, car cabin, 2+ meters):
→ Architecture choice is secondary. Microphone and audio frontend matter more (beamforming array, AEC, noise suppression) than WW vs KWS vs ASR. Expect to spend most engineering time on the acoustic pipeline.

Hardware is a microcontroller (ESP32 class):
→ Wake-word only. KWS and ASR do not fit in MCU budgets today. Picovoice Porcupine runs on Cortex-M. TensorFlow Lite Micro has reference keyword-spotting examples. Expect to build the acoustic frontend yourself (PDM mic to MFCC), budget around 10 mW total, and plan for per-device calibration.

Browser (WASM):
→ Wake-word via WASM works (see our browser wake-word article). KWS has limited WASM support across vendors. Streaming ASR is possible: sherpa-onnx has WASM builds, whisper.cpp has a WASM port (non-streaming). VoxRT itself does not ship a browser streaming ASR today.

Always-on considerations

A reader on our recent KWS tutorial raised an important point: benchmark RTF may be measured over a short run, but always-on behavior can differ. Sustained 24/7 inference stacks heat, especially in small cases on Pi Zero 2 W (no stock heatsink, PCB with limited copper heatspreading, RAM PoP-stacked on top of the SoC).

Rule of thumb:

Wake-word at 5% RTF: comfortable 24/7 on the cheapest hardware
KWS at 32% RTF: works 24/7 but worth monitoring vcgencmd get_throttled and measure_temp in production, plus watching for thermal throttling that kicks in mid-inference and drops frames
Streaming ASR: usually NOT run always-on (triggered by WW). If sustained, hardware choice is critical (Pi 4 with heatsink minimum)

Duty-cycling (sleep between inference bursts), CPU governor settings, and memory fragmentation over multi-day runs are the next layer of concern once the model fits.

Common mistakes

1. Using ASR when KWS would do. Overpaying 60.4 MB and mobile hardware for a 10-command UI. KWS at 1.3 MB does the same job with equal or better accuracy on target vocabulary. We see this regularly. "We just want voice control for 10 things" often ends up in ASR territory when it should not.

2. Using only wake-word when multiple commands are needed. Single trigger plus "did it fire?" does not scale beyond entry. Do not bolt separate WW models per command. Use KWS.

3. Not budgeting always-on CPU and thermal. Wake-word at 5% sustained sounds free. 24/7 means real battery drain and heat buildup. Measure, do not assume.

4. No VAD pre-gate before KWS or ASR. Running inference on silence or non-speech wastes compute and makes false-positive rates look worse than they are. A cheap VAD (Silero, WebRTC VAD) in front of KWS and ASR is the single biggest wasted-compute fix we see.

5. Audio buffer sizing and format mismatches. Ring buffer too small drops audio under load. Device-native 48 kHz float versus the 16 kHz mono int16 that voice models want is the number-one integration bug on fresh projects. Capture format mismatches silently produce garbage inference. Test the audio path end-to-end before trusting any accuracy number.

Picking the right fit

A short honest check. What is the actual voice input need?

Specific closed command set → KWS
Open phrases or dictation → streaming ASR
Just an entry trigger → wake-word
Mix of commands plus dictation → hybrid (orchestration cost is real)

One caveat on "pick the smallest architecture that handles the use case". If the product roadmap is likely to grow from fixed commands to dictation within a year, shipping KWS first and then re-integrating ASR later means two SDK integrations, two sets of user retraining expectations, and sometimes two license agreements. Future-proofing has a real cost, but so does over-engineering for a hypothetical. Judgment call.

All three architectures ship as production SDKs in 2026: VoxRT (hybrid license), Picovoice (commercial), sherpa-onnx (Apache-2.0 open), Vosk (Apache-2.0), whisper.cpp (streaming via sliding window, MIT). A prior comparison article On-device streaming ASR in 2026 goes deeper on per-vendor trade-offs for ASR specifically.

If a project does not fit cleanly into one category, you probably need a hybrid. If it does fit, pick the smallest architecture that handles the use case, not the biggest.

14 voice commands on a $15 Raspberry Pi Zero 2 W (no cloud, no ASR)

VoxRT — Fri, 30 Oct 2026 07:36:43 +0000

Why KWS instead of full ASR

For most DIY voice-control projects, a full open-vocabulary ASR model is overkill. If you want to control a lamp, a slideshow, a media player, or a simple UI by voice, you do not need a model that can transcribe arbitrary speech. You need to reliably recognize 10 to 20 specific commands.

That is the job of KWS (keyword spotting), a separate class of models trained on a fixed closed set of commands. Trade-offs versus full ASR:

Model size. Around 1 MB versus 60 MB or more for streaming ASR.
CPU footprint. One third of a core on a Pi Zero 2 W versus full-load or overload for streaming ASR on that hardware.
Accuracy on target vocabulary. Higher than ASR on the same words because the confusion space is smaller.
Trade-off. Vocabulary is fixed. Custom words require retraining.
Latency. Tens of milliseconds versus hundreds for streaming ASR.

For voice-control projects where the command set is small and known in advance, KWS wins on every axis except vocabulary flexibility.

This article is a tutorial for KWS on a Pi Zero 2 W ($15 hardware, 1 GHz A53). We cover the 14 supported commands, basic detection code, two practical integration examples (GPIO LED and HTTP webhook dispatcher), and honest KWS limits.

All numbers are verified against the voxrt-kws-linux release artifacts.

The 14 commands

The KWS model supports 14 phrases, grouped by purpose:

Confirmation and control (3):

yes: confirm
no: deny
cancel: cancel action

Media playback (5):

play: start playback
pause: pause
next: next track or slide
previous: previous
back: back navigation

Direction (2):

up: up (volume, scroll)
down: down

Switch (2):

on: turn on
off: turn off

Wake-like (2):

voxrt: brand trigger phrase
hey_vox: alternative trigger phrase

Twelve functional commands cover most DIY use cases: media control, UI navigation, binary switches, confirmation flows. The two wake-like commands can serve as an entry trigger for multi-step flows (first hey_vox, then a command).

The modelled vocabulary is fixed. If you need the word "lights", it is not in the vocab and will not be recognized. Workaround: map semantically (on and off to a lamp), or wait for custom training as a feature.

Hardware setup

What you need:

Raspberry Pi Zero 2 W ($15, 1 GHz quad-core A53, 512 MB RAM, see raspberrypi.com/products/raspberry-pi-zero-2-w)
USB microphone (see mic quality note below)
MicroSD card 8 GB or more with Raspberry Pi OS Lite (32-bit)
Power supply 5V 2.5A
Optional: GPIO breadboard, LED, and 220 Ohm resistor for the hardware integration example

Mic quality matters more than you'd expect. A $5 USB lavalier works if the speaker is 0.5 to 1 meter away in a quiet room. For reliable far-field detection (2+ meters) or noisy rooms, consider a ReSpeaker 4-mic array or Matrix Voice ($35 to $80 extra). Cheap single-element USB mics have poor AGC and no beamforming, which measurably hurts KWS recall in realistic conditions.

Why Pi Zero 2 W:
It is the cheapest Pi that handles the KWS workload in realtime. The original Pi Zero is too weak (single-core 1 GHz, 32-bit ARM11, no NEON SIMD). Pi 3, 4, and 5 run faster but cost $35 to $80. Pi Zero 2 W is the sweet spot for always-on voice-control projects where price and power efficiency matter.

OS setup:
Flash the standard Pi image through Raspberry Pi Imager. After boot, configure Wi-Fi via sudo raspi-config and enable SSH. For audio, arecord -l should show the USB mic as a card and the device index (card number). Note that index for the next step.

Install the SDK and hello world

System dependencies first (fresh Bookworm Lite does not have portaudio headers):

sudo apt update
sudo apt install -y portaudio19-dev python3-dev python3-venv

Then the Python package:

# Create venv (required on Bookworm 2023+ per PEP 668)
python3 -m venv ~/kws-venv
source ~/kws-venv/bin/activate

# Install the SDK (Python 3.9 to 3.13 supported, auto-selects arch wheel)
pip install voxrt-kws
pip install pyaudio

# Download the model file from the voxrt-kws-models repo (~1.3 MB)
wget https://github.com/VoxRT/voxrt-kws-models/releases/latest/download/voxrt_kws.vxrt

Hello world: detect and print:

import pyaudio
from voxrt_kws import KwsEngine

# Load model (single argument)
engine = KwsEngine.from_path("voxrt_kws.vxrt")

# Configure as attributes (mutable at any time, change safely between chunks)
engine.threshold = 0.9             # 0.0-1.0, higher = stricter
engine.consecutive_frames = 3      # 120 ms sustained before fire
engine.cooldown = 25               # ~1 s between repeated emits

# Audio capture setup: 16 kHz mono PCM int16, 512-sample chunks
# Replace MIC_INDEX with the card number from `arecord -l`
MIC_INDEX = 1

p = pyaudio.PyAudio()
stream = p.open(
    format=pyaudio.paInt16,
    channels=1,
    rate=16000,
    input=True,
    input_device_index=MIC_INDEX,
    frames_per_buffer=512,
)

print("Listening... say one of the 14 commands")
try:
    while True:
        pcm = stream.read(512, exception_on_overflow=False)
        # push_pcm_i16 is a generator. Iterate to consume detections.
        for d in engine.push_pcm_i16(pcm):
            print(f"heard: {d.class_name} (score {d.score:.2f})")
finally:
    stream.stop_stream()
    stream.close()
    p.terminate()

Run with python hello_kws.py. Say yes and you see heard: yes (score 0.95) or similar in the terminal. Latency on my Pi Zero 2 W is around 150 ms end-to-end from end of pronunciation to the output line (120 ms consecutive-frame buffer plus inference and output).

Important notes:

MIC_INDEX. PyAudio defaults to the system default device, which on Pi is often the onboard audio (broken for mic input). Always set input_device_index explicitly from arecord -l output.
Audio format is strict. 16 kHz mono, PCM int16 signed little-endian, 512-sample chunks (32 ms). Other formats require resampling via SoX, libsndfile, or similar.
exception_on_overflow=False silently drops audio on overrun. On Pi Zero 2 W under system load, this can fire occasionally. For debugging, change to True to see when.

Integration example 1: GPIO LED control

The most maker-friendly example is turning an LED on and off by voice through GPIO. For real use, swap the LED for a relay module and the relay drives a lamp, fan, or anything else on mains power.

Wiring:

GPIO 17 to a 220 Ohm resistor to the LED anode
LED cathode to GND

Install gpiozero:

pip install gpiozero

Code:

import pyaudio
from gpiozero import LED
from voxrt_kws import KwsEngine

MIC_INDEX = 1  # CHANGE THIS to card number from `arecord -l`

led = LED(17)
engine = KwsEngine.from_path("voxrt_kws.vxrt")
engine.threshold = 0.9

p = pyaudio.PyAudio()
stream = p.open(
    format=pyaudio.paInt16, channels=1, rate=16000,
    input=True, input_device_index=MIC_INDEX, frames_per_buffer=512,
)

try:
    while True:
        pcm = stream.read(512, exception_on_overflow=False)
        for d in engine.push_pcm_i16(pcm):
            if d.class_name == "on":
                led.on()
                print("LED on")
            elif d.class_name == "off":
                led.off()
                print("LED off")
finally:
    # try/finally guarantees cleanup even on unexpected exceptions
    stream.stop_stream()
    stream.close()
    p.terminate()
    led.off()

Done. Say on and the LED lights up. Say off and it goes dark. The other 12 commands are ignored (not mapped in the handler).

Scaling to real hardware:

Replace LED with OutputDevice for a 5V relay module
Multiple devices with a dict mapping like {"on": [lamp_relay.on, speaker.on], "off": [...]}
Scenes by combining commands (play turns on lights and starts music via a sequence of actions)

Integration example 2: HTTP webhook dispatcher

More flexible than hardware control is a dispatcher that fires HTTP requests to your home automation server. This works on Pi OS Lite (headless, no X11 needed) and adapts to Home Assistant, OpenHAB, Node-RED, or any custom webhook endpoint.

Install requests:

pip install requests

Code:

import pyaudio
import requests
from voxrt_kws import KwsEngine

MIC_INDEX = 1  # CHANGE THIS to card number from `arecord -l`

# Point to your home automation webhook (adapt URL/payload to your system)
WEBHOOK_URL = "http://your-automation-server:8080/voice"

def trigger(action):
    try:
        requests.post(WEBHOOK_URL, json={"action": action}, timeout=1.0)
    except requests.exceptions.RequestException:
        pass  # non-blocking, next command still fires

ACTIONS = {
    "on": lambda: trigger("lights_on"),
    "off": lambda: trigger("lights_off"),
    "play": lambda: trigger("music_play"),
    "pause": lambda: trigger("music_pause"),
    "next": lambda: trigger("music_next"),
    "previous": lambda: trigger("music_prev"),
    "up": lambda: trigger("volume_up"),
    "down": lambda: trigger("volume_down"),
}

engine = KwsEngine.from_path("voxrt_kws.vxrt")
engine.threshold = 0.9

p = pyaudio.PyAudio()
stream = p.open(
    format=pyaudio.paInt16, channels=1, rate=16000,
    input=True, input_device_index=MIC_INDEX, frames_per_buffer=512,
)

try:
    while True:
        pcm = stream.read(512, exception_on_overflow=False)
        for d in engine.push_pcm_i16(pcm):
            action = ACTIONS.get(d.class_name)
            if action:
                action()
                print(f"triggered: {d.class_name}")
finally:
    stream.stop_stream()
    stream.close()
    p.terminate()

This pattern works with any HTTP-reachable automation layer. For Home Assistant you POST to /api/services/light/turn_on with a Bearer token. For MQTT instead of HTTP, swap requests.post for paho.mqtt.publish.single. The dispatcher dict keeps the mapping readable and easy to extend from YAML or JSON config.

Two things to know. First, only 8 of the 14 commands are mapped above (cancel, back, yes, no, voxrt, hey_vox fall through silently). Add them to ACTIONS to handle more. Second, requests.post blocks the loop for up to the timeout. If your webhook endpoint is slow, wrap it in a thread or queue to avoid audio buffer drops while waiting.

Benchmarks on Pi Zero 2 W

Measurements on my Pi Zero 2 W (1 GHz A53, 32-bit Raspberry Pi OS Bookworm):

Metric	Value
RTF (CPU usage for realtime)	32.2% of one A53 core
Inference latency per chunk	around 10 ms (p99 13 ms)
End-to-end detection latency	120 ms (3 consecutive frames at 32 ms) plus output time
Model file on disk	1.3 MB (fp16, encrypted .vxrt)
Power under KWS load	approximately 1 W (community-measured, varies with mic and setup)

RTF 32.2% on one core means you can run KWS 24/7 on a Pi Zero 2 W and still have most of the quad-core available for your app logic (relay control, HTTP server, display rendering). For $15 of hardware and roughly 1 W of power, you get honest realtime always-on voice control.

For comparison: full streaming ASR on the same hardware does not fit (CPU overload). Wake-word only (covered in a previous article about Pi Zero) takes 5.3% on the same core but is limited to a single trigger. KWS is the middle ground.

Battery estimate: at roughly 1 W total draw, a 10000 mAh USB power bank (around 37 Wh usable after conversion losses) runs the setup for approximately 30 hours. Enough for multi-day field deployments if you add solar charging or battery swap.

Honest KWS limits

Where KWS fails and when to pick a different architecture:

1. Fixed vocabulary. 14 commands is the ceiling. If a project needs "turn on the bedroom light", the word "bedroom" is not in the vocab and will not be recognized. Workaround: a two-step flow (hey_vox as a trigger for an ASR path that handles complex phrases). Note that the handoff between KWS and ASR requires custom orchestration (audio buffer carryover, ASR instantiation timing, VAD gating). It is not a prebuilt SDK feature today.

2. Far-field performance. KWS works best at close range (0.5 to 1 meter from the mic). Beyond 2 meters accuracy drops noticeably. For room-wide voice control, use a better mic array (ReSpeaker 4-mic at ~$35 or Matrix Voice at ~$70) or place the mic closer.

3. Background noise. The default threshold of 0.9 is tuned for quiet environments. In a room with TV or music in the background you have to raise the threshold (fewer false positives, more missed detections) or add a VAD pre-gate to filter non-speech sections.

4. Accent variability. The model was trained primarily on American English pronunciations. Non-native English speakers and strong regional accents can see noticeably lower recall. Pronunciation tweaks (saying pause more like "pawz") sometimes help marginally but are not a real fix. Realistic options: raise or lower the threshold per-speaker, accept degraded recall, or wait for custom training support.

5. No custom commands currently. If a project needs specific words outside the 14-list, custom training is required and that is not shipping as a public SDK feature today. Options: stay inside the 14-list, use yes and no and cancel plus an ASR path for complex interactions, or look at vendor options like Picovoice Rhino intents, Snips, or Rhasspy.

When KWS versus alternatives

A short decision guide:

Pick KWS if:

The project needs up to 14 fixed commands
Pi Zero or similar low-power hardware
Always-on operation matters (CPU budget is tight)
Latency under 150 ms is a requirement

Pick wake-word (not KWS) if:

You only need 1 trigger phrase (not 14)
Hardware is even more constrained (microcontroller or original Pi Zero)
A 100 KB model is preferred over 1.3 MB

Pick streaming ASR if:

Vocabulary is open or over 20 words
Arbitrary user phrases
Platform is at least Pi 4 or mobile

Pick a KWS plus ASR hybrid if:

You need both simple commands (fast path via KWS) and complex phrases (fallback via ASR)
Hardware allows both (Pi 4 and up)
You are willing to build the orchestration code yourself (not a prebuilt feature)

What to build next

This tutorial is a minimal setup. From here you can extend:

Multi-room setup: several Pi Zero units publishing commands via MQTT to a central controller
Voice-controlled dashboard: display state plus voice navigation for kiosk mode (requires Pi OS Desktop)
Accessibility tool: voice shortcuts for users with mobility limitations
Hands-free lab setup: voice commands for photo capture (3D scanning rigs), timer start/stop (chemistry experiments)
Battery-powered portable device: Pi Zero 2 W plus a USB power bank plus a USB mic gives around 30 hours on a 10000 mAh pack

All code from this tutorial is available in the examples folder of voxrt-kws-linux. If you build something interesting, drop a comment with what you made.

On-device streaming ASR in 2026: five approaches and when to pick what

VoxRT — Thu, 22 Oct 2026 08:35:11 +0000

Why this topic right now

In 2023 there were two options. Take OpenAI Whisper and live with offline batch inference (3+ GB large-v3, not streaming), or wait for cloud ASR with its privacy, cost, and latency trade-offs. On-device streaming ASR as a production option was fringe.

In 2026 there are at least five viable approaches on the table, each optimized for a different use case and different constraints. This post is an ecosystem overview and decision guide, not a head-to-head benchmark with a single winner.

What is being compared, and what is not

In scope:

Streaming. The model emits partial transcripts incrementally, not waiting for end of utterance. First-partial latency measured in hundreds of milliseconds.
On-device. Inference runs locally, no network round-trip, no cloud dependency.
Production-ready. Shipping in real mobile and edge apps today, not research prototypes.
Five options that meet all of these criteria.

Out of scope:

Cloud ASR APIs (Deepgram, Whisper API, Google USM via API). Different trade-off entirely.
Batch-only on-device models (Whisper offline without a streaming wrapper). Not streaming.
Wake-word and keyword spotting. Narrower task scope, different SDKs.
ASR at research stage without a production SDK.
A few worth naming but excluded for specific reasons: NVIDIA Riva (Jetson-tier and server, not mobile-first), Moonshine from Useful Sensors (recent 2024 release, short production track record), SeamlessStreaming from Meta (speech translation primary, not pure ASR). All legitimate for adjacent use cases.

Methodology for the numbers:
All size, platform, and license claims come from each vendor's own published sources (model cards, release artifacts, official docs). There are no cross-benchmarks of our test subset against someone else's full set. Apples-to-apples comparison is only honest when the same test methodology is applied to every cell. Where specific accuracy numbers are needed, the post names the test set and whose claim it is.

The five options

1. VoxRT

Model size: 60.4 MB (verified against the voxrt-asr-models v0.1.2 release artifact)
Architecture: NeMo FastConformer family, RNN-T decoding
Platforms: iOS, Android, Linux (aarch64, tested on Raspberry Pi)
License: hybrid. Apache-2.0 on the SDK wrapper, proprietary runtime binary, CC-BY-4.0 for model weights.
Language: English, open-vocabulary
Streaming: native, 200 ms chunk size (3200 samples at 16 kHz)
Source: closed runtime, open SDK wrappers
Repos: voxrt-asr-linux, voxrt-asr-ios, voxrt-asr-android

Shipping since 2026, targeting iOS, Android, and Linux (no desktop or browser builds today). The hybrid license covers three layers. Apache-2.0 on the wrapper SDK means you can fork and modify integration code. The proprietary runtime binary cannot be audited, forked, patched, or deployed outside VoxRT's distribution. CC-BY-4.0 model weights can be redistributed with attribution.

2. Picovoice Cheetah

Model size: compact (vendor does not publish an exact figure)
Architecture: end-to-end neural network (closed-source, specifics not published)
Platforms: iOS, Android, Linux, macOS, Windows, Raspberry Pi, WebAssembly
License: commercial. Free tier for personal and non-commercial use, paid tier for production commercial use.
Language: 8 languages per vendor docs (English, French, German, Italian, Japanese, Korean, Spanish, Portuguese)
Streaming: native
Source: closed-source, commercial product
Site: picovoice.ai/platform/cheetah

Picovoice is a commercial on-device voice AI vendor with wide platform coverage. The company separately offers microcontroller support for wake-word and keyword spotting (Porcupine and Rhino SDKs), but Cheetah streaming ASR itself targets mobile, desktop, Raspberry Pi, and web. Business model is SaaS-style licensing with a free tier for exploration.

3. sherpa-onnx (k2-fsa project)

Model size: roughly 40 MB (20M-param streaming Zipformer int8) to 340 MB (full fp32), depending on model configuration and quantization
Architecture: streaming Zipformer, deployed via ONNX runtime
Platforms: Linux, macOS, Windows, Android, iOS, HarmonyOS, embedded (very wide)
License: Apache-2.0 (code and most models)
Language: multi-language. English, Chinese, and many others depending on the model.
Streaming: native. Zipformer was designed specifically for streaming.
Source: open-source
Repo: github.com/k2-fsa/sherpa-onnx

Part of the broader k2-fsa project (Next-generation Kaldi). Very active open-source community under the k2-fsa umbrella. The model zoo keeps growing, with streaming Zipformer models for different languages and different sizes. Using ONNX runtime for inference keeps deployment portable.

4. Vosk

Model size: small models around 40-50 MB (vosk-model-small-en-us-0.15 = 40 MB), standard models up to ~4.4 GB (largest is vosk-model-de-tuda-0.6-900k) per the Vosk models page
Architecture: Kaldi-based (TDNN acoustic model plus n-gram or RNNLM language model)
Platforms: Python, Node.js, Java, Android, iOS, C#, Go, C++ (wide binding support)
License: Apache-2.0
Language: 20+ languages with pretrained models
Streaming: native
Source: open-source
Site: alphacephei.com/vosk

Long-running open-source project (Alpha Cephei). Kaldi at the core is proven technology, with pre-conformer architecture (TDNN plus HMM-style decoding). One of the smallest production-ready options in this list through the 40 MB small model.

5. whisper.cpp (with the streaming example)

Model size: tiny 75 MB, base 142 MB, small 466 MB, medium 1.5 GB, large 2.9+ GB (per the OpenAI Whisper README)
Architecture: OpenAI Whisper (encoder-decoder transformer), ported to C/C++
Platforms: Linux, macOS, Windows, iOS, Android, Web (via WASM)
License: MIT (whisper.cpp code), MIT (Whisper weights from OpenAI)
Language: 99 languages (multilingual Whisper models)
Streaming: not native. Whisper is a batch model, and whisper.cpp has a stream example built on a sliding-window pattern.
Source: open-source
Repo: github.com/ggml-org/whisper.cpp

An important caveat. Whisper is architecturally a batch model that processes 30-second windows end-to-end. The streaming example in whisper.cpp creates streaming-like behavior through overlapping sliding windows, but first-partial latency is higher than for models designed around streaming, and accuracy is reported to degrade on short chunks. For use cases where accuracy is critical and latency is secondary, this is a valid approach. For real-time dictation with a <500 ms first-partial requirement, the streaming-first models in items 1-4 above are a closer fit.

Comparison matrix

Axis	VoxRT	Picovoice Cheetah	sherpa-onnx	Vosk	whisper.cpp
Model size	60 MB	Compact (not published)	~40 MB int8 lightweight to ~340 MB fp32	40 MB small, up to 4.4 GB	75 MB tiny, 466 MB small, 1.5 GB med
Streaming	Native	Native	Native	Native	Sliding-window
Platforms	iOS, Android, Linux	iOS, Android, Linux, Mac, Win, RPi, WASM	Linux, Mac, Win, iOS, Android, HarmonyOS, embedded	Py, Node, Java, Android, iOS, C#, Go, C++	Linux, Mac, Win, iOS, Android, Web
License	Hybrid (Apache SDK, proprietary runtime, CC-BY model)	Commercial (free tier plus paid)	Apache 2.0	Apache 2.0	MIT
Languages	English	8 languages	Multi-language	20+	99
Source	Runtime closed	Closed	Open	Open	Open
First-partial latency	Native streaming	Native streaming	Native streaming	Native streaming	Higher (sliding window)

Platform notes: Picovoice separately offers MCU support (Cortex-M) for wake-word and keyword spotting (Porcupine, Rhino), not for Cheetah streaming ASR. sherpa-onnx supports a wide embedded range including RISC-V.

Decision tree by use case

Smallest model size matters above all other considerations:
→ Vosk small (40 MB) or Picovoice Cheetah (compact). Trade-off: Vosk is pre-conformer accuracy, Picovoice is commercial license.

Open-source plus production-ready streaming plus platform maturity:
→ sherpa-onnx (Zipformer well-supported, large community) or Vosk (older but stable).

Multi-language from day one:
→ whisper.cpp (99 languages) or Vosk (20+). VoxRT and Picovoice are English-focused today.

Target is embedded or microcontroller:
→ For embedded Linux (ARM, RISC-V), sherpa-onnx has the widest support. For microcontrollers (Cortex-M), full ASR is not realistic today. Picovoice's Porcupine/Rhino offer wake-word and keyword spotting at that tier, which is a different task than streaming ASR.

Mobile voice input, English, want a simple SDK integration and streaming-first behavior:
→ VoxRT or Picovoice Cheetah. Trade-off: VoxRT has a hybrid license (proprietary runtime), Picovoice uses a commercial subscription.

Whisper quality required, latency secondary:
→ whisper.cpp (streaming is a sliding-window pattern but accuracy is preserved).

Browser inference (WASM):
→ whisper.cpp (WASM supported) or sherpa-onnx (if the WASM path works for your model configuration, verify before integrating). VoxRT does not ship ASR for the browser today.

Nothing fits cleanly:
→ Download your top two candidates based on size, license, and platform fit. Run them on your actual audio data. Measure RTF and accuracy. Decide on measured numbers, not predicted ones.

Honest trade-offs. Where someone else beats us.

I work on VoxRT, but a fair comparison needs an honest picture of where we do not win:

Vosk small is smaller than our model (around 40 MB versus our 60 MB). If bytes-on-disk is the primary constraint, Vosk small wins on that axis.
sherpa-onnx has wider platform coverage for embedded Linux targets (ARM, RISC-V). Browser (WASM) is also not something we ship. For those targets we are not an option.
sherpa-onnx has significantly wider language coverage. Chinese, Japanese, European languages with active model development. We are English-only.
whisper.cpp handles 99 languages through OpenAI multilingual models. For truly multilingual products, the Whisper family is still the baseline.
sherpa-onnx and Vosk are fully open-source (Apache-2.0). If your organization will not accept a proprietary runtime binary, our hybrid license does not fit.
Picovoice has a longer track record as an established commercial vendor. For teams that want a paid support contract and SLA, Picovoice fits traditional procurement better.
VoxRT is less than a year old. Picovoice has multi-year production use, Vosk and whisper.cpp are each years into shipping. If track record is a procurement gate, we lack it today.
We do not publish WER numbers currently. Picovoice, Vosk, and sherpa-onnx all publish WER evaluations on standard benchmarks. If your decision gates on specific accuracy numbers from the vendor, that absence is a real gap for us.
Closed runtime means you cannot patch bugs, fork for custom modifications, or audit the inference path. For teams that require full-source transparency, our hybrid license disqualifies us.

VoxRT was built around a specific trade-off: streaming-first behavior, mobile focus, hybrid license acceptable, English voice input. For that slice we think the engineering is sound. For other slices, the options above are a better fit.

The ecosystem is healthier than it was before 2024

Three years ago this would have been a much shorter post. Whisper had just shipped (2022), on-device streaming ASR was at research stage, Vosk was the best open-source option but had a visible accuracy gap versus cloud, and Picovoice was the commercial choice but with narrower vocabulary coverage.

In 2026 there are five production-ready options, with different architectural choices, different license models, and different target hardware. The one-size-fits-all approach does not work anymore because use cases are not one size.

Pick based on actual constraints:

License tolerance (open-only? hybrid acceptable? commercial subscription fine?)
Platform requirements (mobile only? MCU too? browser?)
Language requirements (English only? multi-language needed?)
Model size budget (bytes on disk? runtime memory?)
Latency requirements (real-time dictation? post-utterance OK?)
Support expectations (self-service open-source? paid vendor contract?)

All five options in this post are production-ready for shipping in 2026. If you cannot pick between options, download your top two candidates by size, license, and platform fit, and run them on your real data. Measure. Decide on the numbers, not the marketing (including mine).

Why we built voice AI for the device, not the cloud

VoxRT — Tue, 13 Oct 2026 08:23:06 +0000

The default is to send voice to the cloud

If you are shipping a voice feature in your app today, odds are you are sending audio to a server. That is how most existing solutions work. The mobile app captures sound from the microphone, packages it as WAV or opus, streams it to an API endpoint, and gets back JSON with the recognized text. Your business logic then operates on text.

This pattern dominates not because it is the best approach, but because historically it was the only one that worked at real production model sizes. Open ASR models were too big. On-device model quality was noticeably worse than the cloud equivalents. Mobile hardware could not sustain ML inference in the background without destroying battery life inside an hour.

The trade-off did not satisfy us for very long. We built the VoxRT stack for on-device from day one. We did not migrate from the cloud. We picked a different path from the start. This article covers why that decision keeps paying off, what trade-offs we accepted, and where cloud still legitimately wins.

The economics that changed our thinking

Picture a hypothetical voice-notes app with 100,000 monthly active users. The average user dictates 20 minutes a month. That is 2 million minutes of audio to transcribe.

Take a typical cloud ASR provider rate of roughly $0.004 per minute, which is on the lower end of mainstream pricing today. The arithmetic is simple. 2,000,000 x $0.004 = $8,000 a month. $96,000 a year. For transcription alone. No wake-word, no keyword spotting, no TTS.

Now consider on-device. The engineering work is upfront, not per-user. Integrating an on-device SDK is usually 2 to 6 weeks for a straightforward single-platform start, longer when you ship across multiple platforms or need custom optimization. There is ongoing cost too: SDK upgrades when model versions improve, device-specific regressions to debug, new OS releases to validate against. None of it scales linearly with user minutes. The per-unit variable cost drops to zero.

One factor we do not want to hide: on-device SDKs are not universally free. Our stack ships under a hybrid license (open wrappers, proprietary runtime), and other on-device voice vendors use per-seat, per-deployment, or revenue-share pricing. For most teams the engineering cost plus any license fee combined still beats per-minute cloud pricing at scale, but the full picture requires factoring both sides, not just the per-minute delta.

The breakeven point depends on the user base. For our hypothetical app, a few thousand paying users are enough for on-device to beat cloud economics. If you expect your app to reach hundreds of thousands or millions of users, on-device wins over the long run almost without exception.

The calculation also ignores second-order effects. No cold-start latency from the network. No rate limits during peak load. No scaling cost when you grow. No backend team to monitor the uptime of your ASR pipeline. Those effects compound differently per product. For a voice-notes app running at scale, the first two matter most. For an occasional-use feature in a bigger app, the engineering commitment to a mobile SDK may not pay off even if the per-minute math eventually does.

Three cases where on-device wins cleanly

Privacy is a legal requirement. Healthcare apps with voice notes. Legal tools recording client meetings. Personal journals. For these use cases, sending audio to a third-party server is not a performance question. It is a compliance question. HIPAA, GDPR, and the EU AI Act are all simpler to operate under when the audio never physically leaves the device. It is easier to prove to a lawyer that user voice was never sent to a server when the stack is architecturally incapable of making network calls.

The app has to work offline. Field-work tools for inspectors, construction workers, and surveyors. Travel apps on airplanes or in countries without data roaming. Workflows for medics and first responders in remote areas. If a user needs voice input in these contexts, cloud-only is not an option. You can build an offline fallback through a cache, but if you are already building the on-device part, why keep the cloud path running in parallel?

Latency has to be below 100 ms. For wake-word and hot-word detection, the cycle "user said it, app responded" must feel instant. Any network round-trip breaks the UX. Even on perfect 5G, you lose 50 to 150 ms to RTT alone, plus server-side inference. On-device wake-word inference typically lands between 50 and 200 ms end-to-end. For a wake-detection flow, that is the difference between "this works" and "this is annoying."

These three cases are not edge cases. They are a substantial slice of real voice use cases: privacy-sensitive apps, offline-first tools, and always-listening trigger layers inside a larger voice stack.

What is genuinely hard about on-device

This article would not be honest without a section on the trade-offs we accepted.

Model size versus quality. To fit a model into 100 KB (our wake-word), you cut the architecture deep. For our wake-word we accepted a fixed trigger phrase ("Hey Assistant") and ship one proven model, not a custom-trained one per user. That worked for our scenario, but it means we cannot offer custom trigger phrases without a training pipeline that we have not released publicly. The streaming ASR model is larger (around 60 MB of weights for the medium model), still well under what cloud providers run server-side on their multi-gigabyte models.

Hardware fragmentation is real. We ship across iOS, Android, browser (via WASM), and Linux (Raspberry Pi, Jetson, servers). The shared Rust runtime compiles to each target, but every target still needs its own benchmark run and build tuning for local SIMD extensions, memory layout, and audio API conventions. Publishing a single "RTF of 0.05" number across all platforms is not honest. The reality is 5 to 10 benchmark points on different devices, each with its own caveats. That is more work than cloud providers face, because their infrastructure is uniform while ours is not.

Updates ship through the SDK release cycle. If you have a bug in a cloud ASR, you push a fix to the server and everything is current tomorrow. For us, a bug means a new SDK version, npm/pip/CocoaPods updates, and users have to upgrade. Mobile developers know this pain from any native library. A stable C ABI between the runtime and the SDK bindings reduces the breaking-change frequency, but it does not remove the user-upgrade step. That is a real friction we have not solved.

Benchmarking requires discipline. It is easy to cherry-pick one device and publish a pretty number. It is hard to publish a multi-device matrix with measured RTF, different CPU budgets on different cores, and p99 chunk latency. We make this harder on ourselves than we have to because we think the alternative misleads integrators who then build on expectations that do not match their hardware. The problem compounds every time a new flagship phone ships.

On-device debugging in production is thinner. When a cloud ASR regresses on a specific user, the server team has logs, metrics, and the ability to replay the audio. On-device, your telemetry surface is whatever you can get back from the user's device without violating the privacy premise that made you pick on-device in the first place. We ship RTF counters and detection scores back via opt-in telemetry only, and even that cuts off at what a user is comfortable sharing. Diagnosing a regression on one specific chipset family often means buying that device and reproducing the issue locally.

Where this trade-off actually lands

We share the numbers below not as a pitch but as evidence that the on-device path has a real engineering shape in 2026, not just an aspirational one. Each of these came out of an actual measurement that we had to defend internally before publishing.

A wake-word model that fits in 100 KB and sustains 5.3% of one A53 core on a $15 Raspberry Pi Zero 2 W is on the extreme end of what on-device looks like today. On a Pi 4 B or Pi 5 it drops to 1 to 2% CPU. The constraints that forced the 100 KB target (single trigger phrase, fixed, no custom training in-SDK) are real limitations, not features. We shipped them because the alternative was not shipping at all on hardware this cheap.

A voice activity detector running at 3% CPU on a mid-range Android (Snapdragon 662 class) is the ceiling that lets us run always-on microphone capture without destroying battery. We only ship VAD on mobile, not in browser, because the WASM profile for continuous audio capture is different enough that we have not justified the engineering investment yet.

Keyword spotting across 14 fixed English commands at 0.97% RTF in Chrome WASM and 16% RTF on Snapdragon 662 is what makes a browser voice-UX feel instant. The full browser stack (runtime included) bundles to about 1.47 MB. That is small enough to not matter for most apps and large enough to matter for landing pages optimized for mobile 3G.

Streaming ASR (NeMo FastConformer, 60 MB weights) runs real-time on mobile. Partial transcripts arrive while the user is still speaking, with finalization after a silence timeout. This is the model that would have been impossible to ship on-device three years ago. Today it is a shipping choice.

These numbers live in the READMEs at github.com/VoxRT. We publish them with the test set, the hardware, the methodology, and the caveats. We do not publish subset measurements next to another vendor's full-set measurements. If our wake-word gets compared with another approach, both numbers need to be on the same test set, or it is not a comparison.

Where cloud still wins honestly

Cloud ASR is the right call for several scenarios:

Multilingual coverage across 50-plus languages. Open ASR models for rarer languages are often dramatically worse than cloud equivalents. If you need voice in Swahili, Thai, or Tagalog, cloud providers are still ahead. On-device multilingual coverage is catching up, but slower.

Fast iteration on models. A cloud team can push an improved model today, and tomorrow every user gets better accuracy without any change to their app. On-device apps have to re-ship. For products that iterate fast on model quality, that is a meaningful velocity difference.

Enterprise compliance tooling around the API. Major cloud ASR providers come with SOC2, HIPAA, and PCI-DSS compliance infrastructure, audit logs, and enterprise SLA agreements. If you sell into the Fortune 500, buying compliance through a vendor is sometimes easier than proving it yourself for on-device deployment.

Open-ended complex workflows. If your voice UX is "ask AI a complex question via voice, run it through ASR, pipe it to an LLM, get a voice response," the entire pipeline already lives on a server. Adding transcription on the same server costs you very little extra latency. The on-device win on the first step is small when every other step already runs in the cloud.

These cases are real. On-device is not a universal answer. It is the right answer for a specific set of product scenarios that are becoming more common.

Where this is going

To understand why we think on-device picks up more market share, it helps to see how much ground has already shifted. Capabilities that were impractical or research-only three years ago are standard engineering choices today:

Capability	2022-2023 state	2026 state
Streaming ASR on mobile	Whisper-class models (1-3 GB, batch architecture) impractical for real-time mobile streaming	FastConformer-class models (~60 MB weights) run real-time on commodity mobile hardware
Browser voice inference	Pre-SIMD128 WASM, 2-4x realtime typical, Safari lacked SIMD128 until March 2023	SIMD128 across all major browsers, sub-1% RTF achievable
Wake-word on cheap hardware	Reproducible open benchmarks rare on Pi Zero class, mostly proprietary paid tiers	Open benchmarks published, 5.3% RTF measured on $15 Pi Zero 2 W
Voice-cloning TTS on-device	On-device voice-cloning was research-stage, production voice-cloning was cloud-only	Shipping products exist, single-step flow sampling made fast synthesis realistic

Three trends we see on the two-to-three year horizon:

Mobile silicon for ML becomes standard. Apple Neural Engine in iPhones, Google Tensor TPU, Qualcomm Hexagon NPU in Snapdragon. Every flagship now has dedicated silicon for ML inference. What was a feat in 2022 (running a transformer on a phone) is a standard capability in 2026.

Browser as a first-class target. WASM SIMD128 (available in Chrome 91+, Firefox 89+, and Safari 16.4+) opened the door to serious ML inference in the browser. Our 100 KB wake-word runs at 0.16% RTF in Chrome M4. A few years ago that was not possible without a native wrapper. Fully-in-browser voice UX is now a real option.

Privacy becomes a compliance requirement. The EU AI Act for GPAI models started applying with enforcement teeth. Regulators in the US and Asia are following. For voice-app companies, proving to a regulator that user voice is never sent to the cloud is easier when the architecture makes that impossible by construction. On-device moves from nice-to-have to a compliance posture.

Our prediction is that cloud ASR stays the default for broad applications over the next 2-3 years, but on-device picks up market share in privacy-sensitive, offline-first, and latency-critical segments. This is not "on-device will kill cloud." It is a reasonable coexistence of two architectural approaches for different use cases.

Repos and resources

Our runtime and SDKs are on github.com/VoxRT. Platform-specific bindings:

Linux (Pi, Jetson, servers): voxrt-wake-word-linux, voxrt-kws-linux, voxrt-asr-linux
Browser (WASM): voxrt-wake-word-browser, voxrt-kws-browser
Mobile: iOS and Android bindings in the respective repos

More technical articles from our team on on-device voice are on dev.to/voxrtio.

If you looked at on-device voice recently and chose cloud instead, what was the deciding factor? The honest answers are the ones worth hearing.

Build a voice-controlled music player on Raspberry Pi 4 (fully offline)

VoxRT — Mon, 28 Sep 2026 07:15:52 +0000

What we are building

A media player that listens for voice. The user says "Hey Assistant", then one of six commands (play, pause, next, previous, up, down), and the track plays, pauses, or the volume moves. All on a Raspberry Pi 4. No cloud, no API keys, no external services.

Three components make it work:

Wake-word (VoxRT) listens to the microphone 24/7. About 2% of one CPU core on Pi 4.
Keyword spotter (VoxRT) recognizes a command in a 2-second window after the wake-word triggers. About 15% CPU during the burst.
MPD (Music Player Daemon) plays the actual music. Commands go through mpc, the standard CLI client. Our script runs the right subprocess for each recognized command.

The whole thing sets up in about 30 minutes on a Pi 4 with a USB microphone and a pair of speakers. Six of our built-in KWS commands cover music control cleanly, so no custom model training is needed.

Why WW plus KWS fits music control

The VoxRT KWS model recognizes 14 fixed English commands out of the box:
yes, no, up, down, on, off, back, play, pause, next, previous, cancel, voxrt, hey_vox.

Six of those map to music control:

Voice	Action
`play`	start playback
`pause`	pause
`next`	next track
`previous`	previous track
`up`	volume +5
`down`	volume -5

No custom model training, no Whisper-style open-vocabulary ASR, no cloud service. All you need is a mapping from six voice labels to six shell commands.

The triage pattern is the same as our full Pi 4 voice assistant tutorial, only simpler. The ASR fallback is not needed here because KWS covers every command in this scope.

Hardware

Raspberry Pi 4 (2GB or more) running 64-bit Raspberry Pi OS (aarch64)
USB microphone (any class-compliant device, around $10 to $15) or an I2S HAT
Speakers or headphones via the 3.5mm jack or USB audio
Some music files in .mp3 / .flac / .ogg. Your own collection or a few downloaded tracks.
Python 3.9+ on the Pi

The tutorial assumes you are comfortable with basic Linux commands (apt, pip, editing a config file). No previous MPD experience is required. We cover everything you need.

Install

Three packages via apt (MPD, mpc, and PortAudio for the mic):

sudo apt-get update
sudo apt-get install mpd mpc libportaudio2

Three Python libraries. On current Raspberry Pi OS (Bookworm and later) bare pip install refuses to touch the system Python because of PEP 668. Use a venv:

python3 -m venv ~/voxrt-env
source ~/voxrt-env/bin/activate
pip install voxrt-wake-word voxrt-kws sounddevice

Everything from here on runs inside that venv. If you prefer pipx, or want to install globally with --break-system-packages, either works. The rest of this tutorial uses the venv path.

VoxRT model files. Download them to the directory you plan to run the script from (or use absolute paths in WakeWordEngine.from_path() and KwsEngine.from_path() further down).

curl -LO https://github.com/VoxRT/voxrt-wake-word-models/releases/download/v0.1.0/voxrt_wake_word.vxrt
curl -LO https://github.com/VoxRT/voxrt-kws-models/releases/download/v0.1.0/voxrt_kws.vxrt

Check the exact versions in each repo README. They update periodically.

Configure MPD

The default MPD config on Raspberry Pi OS almost works. You need to point it at your music library and (optionally) adjust the audio output.

Open /etc/mpd.conf:

sudo nano /etc/mpd.conf

Check or edit these lines:

music_directory     "/home/YOUR_USER/Music"
playlist_directory  "/var/lib/mpd/playlists"
db_file             "/var/lib/mpd/tag_cache"

audio_output {
    type            "alsa"
    name            "Default ALSA"
    device          "hw:0,0"   # verify via `aplay -l`
}

Replace YOUR_USER with your actual Pi username. Recent Raspberry Pi OS (Bookworm and later) does not ship a default pi user, so hardcoding /home/pi/Music will not work unless you specifically created that account. Alternative is /var/lib/mpd/music, which the packaged mpd user already owns.

MPD runs as its own mpd system user. If you point it at a folder inside your home directory, mpd needs read access. Common first-run trap is an empty mpc listall because MPD cannot see the files. Fix with sudo usermod -a -G $(whoami) mpd (add mpd to your group) and make sure the Music folder is group-readable.

Drop your music into that directory. Restart MPD and reindex the library:

sudo systemctl restart mpd
mpc update
mpc listall | head    # should print track paths

Verify playback works manually:

mpc add /
mpc play
mpc pause

If music plays, you are ready. If there is no sound, run aplay -l to find your audio device and update the device "hw:X,Y" line in mpd.conf.

The Pi-side pipeline

Around 70 lines of Python. Wake-word listens all the time, KWS activates after the trigger, matched commands run through mpc:

import subprocess
import time
from enum import Enum
from queue import Queue, Empty

import numpy as np
import sounddevice as sd

from voxrt_wake_word import WakeWordEngine
from voxrt_kws import KwsEngine

# --- Config
KWS_WINDOW_SEC = 2.0
SAMPLE_RATE = 16000
CHUNK_FRAMES = 512   # 32 ms

# --- Command mapping
COMMANDS = {
    "play":     ["mpc", "play"],
    "pause":    ["mpc", "pause"],
    "next":     ["mpc", "next"],
    "previous": ["mpc", "prev"],
    "up":       ["mpc", "volume", "+5"],
    "down":     ["mpc", "volume", "-5"],
}

# --- Engines
ww = WakeWordEngine.from_path("voxrt_wake_word.vxrt")
ww.threshold = 0.9
ww.cooldown_frames = 100

kws = KwsEngine.from_path(
    "voxrt_kws.vxrt",
    threshold=0.9,
    consecutive_frames_required=3,
    cooldown_frames=25,
)

# --- State
class State(Enum):
    IDLE = 1
    KWS_CHECK = 2

state = State.IDLE
kws_deadline = 0.0

# --- Audio capture
audio_queue: "Queue[np.ndarray]" = Queue(maxsize=64)

def on_audio(indata, frames, time_info, status):
    if status:
        print(f"audio: {status}")
    audio_queue.put(indata.copy())

stream = sd.InputStream(
    samplerate=SAMPLE_RATE,
    channels=1,
    dtype="int16",
    blocksize=CHUNK_FRAMES,
    callback=on_audio,
)
stream.start()

# --- Main loop
try:
    while stream.active:
        try:
            chunk = audio_queue.get(timeout=0.1)
        except Empty:
            continue

        samples = chunk.flatten().tolist()

        if state == State.IDLE:
            for det in ww.push_pcm_i16(samples):
                print(f"[WW] wake score={det.score:.3f}")
                state = State.KWS_CHECK
                kws_deadline = time.time() + KWS_WINDOW_SEC

        elif state == State.KWS_CHECK:
            matched = False
            for det in kws.push_pcm_i16(samples):
                cmd = det.class_name
                if cmd in COMMANDS:
                    print(f"[KWS] command={cmd} score={det.score:.3f}")
                    subprocess.run(COMMANDS[cmd], check=False)
                    state = State.IDLE
                    matched = True
                    break

            if not matched and time.time() > kws_deadline:
                print("[KWS] no matching command, back to idle")
                state = State.IDLE
finally:
    stream.stop()
    stream.close()

Run it:

python3 voice_music.py

Say "Hey Assistant", then one of the six commands. The player should react.

A few details worth calling out:

The COMMANDS dict is the single place where you change the mapping. Want to add stop? Add "cancel": ["mpc", "stop"] and it will work immediately (the KWS vocabulary already contains cancel).
Commands from the KWS vocabulary that are not in the dict (yes, no, on, off, back, voxrt, hey_vox) are silently ignored during the 2-second window. If one gets recognized, the script does nothing and returns to idle after the timeout.
subprocess.run(..., check=False) does not raise on a non-zero exit code. If MPD is not running or the library is empty, the script keeps working instead of crashing.
Wake-word cooldown_frames = 100 means about 3 seconds of silence after a detection before the next trigger. Useful to prevent multiple triggers if the phrase "Hey Assistant" lands twice in the buffer.

Performance on Pi 4

Numbers from the official VoxRT READMEs, extrapolated to Pi 4 from a measured Pi Zero 2 W baseline:

WW always-on. RTF 0.018 to 0.024, which works out to about 2% of one A72 core on Pi 4 B / 400 at 1.5 to 1.8 GHz.
KWS in the 2-second burst. RTF 13% to 16%, roughly one-sixth of one core for a short window.
mpc command exec. Sub-millisecond. Negligible.

Voice-to-music latency. You say "Hey Assistant, play", the wake-word fires about 100 to 200 ms after the phrase ends, KWS recognizes the command in the following 500 to 1500 ms window, mpc play runs in under 50 ms, MPD starts playback. Typical end-to-end is 700 to 1700 ms. It feels instant in practice.

RAM. Roughly 50 to 70 MB for both VoxRT engines combined, plus another 20 to 40 MB for MPD. Small fraction of a 2GB Pi 4. Your exact numbers depend on your Python version, MPD version, and how much of your music library MPD has indexed.

Power draw on a Pi 4 B running this stack is around 3.5W total (bare Pi 4 idle is closer to 2.7W, add ~0.1 to 0.2W for VoxRT plus the USB mic overhead). Over a month of 24/7 operation that is around 2.5 to 2.6 kWh. Under a dollar in most regions.

Extending

A few directions past the first working version:

More commands. MPD has plenty of useful operations via mpc:

mpc stop stops playback
mpc shuffle shuffles the queue
mpc repeat on/off toggles loop mode
mpc clear && mpc add / clears and reloads the full library

Extend the COMMANDS dict with more mappings from the KWS vocabulary. For example, "cancel": ["mpc", "stop"], "off": ["mpc", "clear"], "on": ["mpc", "shuffle"].

Playlists. MPD supports named playlists as .m3u files in /var/lib/mpd/playlists/. Load one with mpc load workout (loads workout.m3u). Adding voice commands for specific playlists is possible, but the 14-word vocabulary does not include custom names. A workaround is to bind one voice command to "cycle through the next playlist" and rotate.

Multi-room. If you have several Pis around the house, each with its own MPD and voice script, you get independent voice zones by default. Synchronised multi-room playback is a separate topic. See MPD satellite mode or Snapcast.

Alternative players. Not a fan of MPD? The same mapping pattern works with:

mpv: mpv --input-ipc-server=/tmp/mpv.sock plus echo cycle pause | socat - /tmp/mpv.sock
VLC: cvlc --extraintf rc
spotifyd (Spotify Connect): dbus-send commands to the MediaPlayer2 interface

You only change the contents of COMMANDS. The rest of the code stays the same.

Wake-word threshold tuning. The default threshold = 0.9 is conservative. If music is playing loud and the wake-word gets missed, drop it to 0.7 or 0.8. If a TV or a conversation triggers false wakes, raise it to 0.95.

Wrap up

You now have a fully local voice-driven MPD music player. No audio, no recognition, no metadata leaves the Pi. It runs without an internet connection, which makes it viable for a cabin, a garage, or anywhere WiFi is unreliable.

Three things this article does not cover:

TTS feedback ("playing next track"). The VoxRT TTS SDK is in development. That will get its own article when it ships.
Custom wake-word phrase. Only "Hey Assistant" ships as a ready-to-use model today. Training a custom trigger is not yet a public feature.
Voice search ("play something jazz"). That needs open-vocabulary ASR. We have a separate SDK for it, and the pattern is covered in our full Pi 4 voice assistant tutorial.

Repos:

WW Linux SDK: github.com/VoxRT/voxrt-wake-word-linux
KWS Linux SDK: github.com/VoxRT/voxrt-kws-linux
All SDKs: github.com/VoxRT

More on-device voice tutorials from our team on dev.to/voxrtio.

Curious to hear what you would build with a setup like this. Which player do you use on your Pi (MPD, mpv, Volumio, something else)? And which voice commands beyond play, pause, next would you add first?

Wake word + voice commands in the browser: a full offline pipeline

VoxRT — Mon, 21 Sep 2026 06:20:28 +0000

Voice interfaces in the browser have always had the same friction: Web Speech API is patchy across browsers, ships your audio to a vendor, and often needs network to respond. If you want a private, offline pipeline that works the same everywhere, you have had to wire it yourself. That got easier when WebAssembly SIMD128 shipped in every modern browser.

A few weeks ago I wrote a piece on adding a wake word to a web app in five minutes. This is the natural next step. A wake word alone tells you when the user wants attention. To do anything useful, you also need to know what they said. This post wires up the second half: a 14-command keyword spotter that activates on wake and dispatches commands to your app. Everything runs on-device, in about 1.75 MB of assets total, at roughly 1% of one CPU core on a modern laptop.

What you will need

A modern browser: Chrome/Edge 91+, Firefox 89+, Safari 16.4+. WASM SIMD128 is required.
A microphone.
Any static file server on HTTPS or localhost. Browsers block getUserMedia on plain HTTP.

The pattern: cheap wake, activated commands

You could run keyword spotting continuously and skip the wake word. Don't. Two reasons.

First, wake-word models are trained for one job: sit in a hot loop for hours, fire on a single phrase, ignore everything else. They are tiny (100 KB of weights, ~275 KB total bundle) and hardened against false triggers from radio, TV, and background conversation. Command spotters are heavier (~1.47 MB with a 14-word vocabulary) and, running always-on, will occasionally fire when someone in the room says "play" or "next" without meaning to talk to your app.

Second, the split matches how users actually interact. A wake word says "I am about to talk to you." A command says what to do. Gating the command recognizer on the wake cuts false triggers and CPU when the user is not addressing your app.

The pattern:

[Microphone]
     ↓
[Wake word engine] ── always on, ~275 KB, ~0.16% RTF
     ↓ (fires on "Hey Assistant")
[Listening window ~3 seconds]
     ↓
[KWS engine] ── activated during window, ~1.47 MB, ~0.97% RTF
     ↓
[Command: play / pause / next / ...]
     ↓
[Your app]

Both engines read the same 16 kHz mono PCM stream from the microphone. The wake engine sees every frame. The KWS engine only sees frames inside a listening window opened by the wake trigger. When a command fires (or the window times out), listening closes and the pipeline goes quiet again.

Step 1: install the SDKs

npm install @voxrt/wake-word-browser @voxrt/kws-browser

Or via CDN if you skip the bundler:

import initWake, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";
import initKws, { KwsEngine } from
  "https://unpkg.com/@voxrt/kws-browser@0.1.0/voxrt-kws-browser.js";

Both packages ship two things: a WebAssembly runtime (.js + .wasm) and a .vxrt model file. The model files download separately at runtime and get memory-mapped by the engine.

Step 2: HTML shell

Minimum needed to show what is happening:

<!DOCTYPE html>
<html>
<head><meta charset="utf-8"><title>Voice control demo</title></head>
<body>
  <button id="start">Start listening</button>
  <p id="status">Idle</p>
  <p>Last command: <span id="last">none</span></p>
  <script type="module" src="./app.js"></script>
</body>
</html>

Serve over HTTPS or from localhost.

Step 3: initialize both engines

Create app.js:

import initWake, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";
import initKws, { KwsEngine } from
  "https://unpkg.com/@voxrt/kws-browser@0.1.0/voxrt-kws-browser.js";

await Promise.all([initWake(), initKws()]);

const [wakeBytes, kwsBytes] = await Promise.all([
  fetch("https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt_wake_word.vxrt")
    .then(r => r.arrayBuffer()).then(b => new Uint8Array(b)),
  fetch("https://unpkg.com/@voxrt/kws-browser@0.1.0/voxrt_kws.vxrt")
    .then(r => r.arrayBuffer()).then(b => new Uint8Array(b)),
]);

const wake = WakeWordEngine.fromBytes(wakeBytes);
const kws = KwsEngine.fromBytes(kwsBytes);

Two engines, two model files, both loaded before we touch the microphone. Downloads happen in parallel. On a warm cache the whole init is under 100 ms.

Default thresholds are 0.9 in sigmoid space for both. Lower them to increase sensitivity (more false accepts). Raise them to be pickier. We will tune later if needed.

Step 4: capture microphone audio

Both engines want 16 kHz mono. We ask AudioContext for it and verify, because Firefox and Safari sometimes ignore the hint:

document.getElementById("start").onclick = async () => {
  const stream = await navigator.mediaDevices.getUserMedia({audio: true});
  const audioContext = new AudioContext({sampleRate: 16000});
  if (audioContext.sampleRate !== 16000) {
    console.warn(`Got ${audioContext.sampleRate} Hz. Need 16000. Resample or reject.`);
    return;
  }
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(512, 1, 1);
  const mute = audioContext.createGain();
  mute.gain.value = 0;

  // detection loop goes here (Step 5)

  source.connect(processor);
  processor.connect(mute).connect(audioContext.destination);
  document.getElementById("status").textContent = "Listening for wake word";
};

Two gotchas from the wake-word post still apply here:

ScriptProcessorNode is deprecated. Fine for a demo, swap for AudioWorkletNode in production.
The muted GainNode between processor and destination is a Web Audio quirk. Without connecting the processor to the graph, audioprocess never fires. Muting the gain prevents the microphone from being routed straight to the speakers.

Step 5: detection loop with a listening window

Here is the actual logic. The wake engine sees every frame. On a wake hit, we open a 3-second window during which frames also go to the KWS engine. First command detection closes the window early.

const LISTEN_WINDOW_MS = 3000;

const pcmBuffer = new Int16Array(512);
let listeningUntil = 0; // performance.now() timestamp; 0 means closed

processor.addEventListener("audioprocess", (e) => {
  const inputFloat = e.inputBuffer.getChannelData(0);

  // Float32 [-1, 1] to Int16
  for (let i = 0; i < inputFloat.length; i++) {
    const s = Math.max(-1, Math.min(1, inputFloat[i]));
    pcmBuffer[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
  }

  // Wake word always runs
  for (const hit of wake.pushPcmI16(pcmBuffer)) {
    console.log(`Wake at ${hit.timestampSec.toFixed(3)}s, score ${hit.score.toFixed(3)}`);
    listeningUntil = performance.now() + LISTEN_WINDOW_MS;
    document.getElementById("status").textContent = "Listening for command...";
  }

  // KWS only runs while the listening window is open
  if (performance.now() < listeningUntil) {
    for (const command of kws.pushPcmI16(pcmBuffer)) {
      if (command.class === "hey_vox" || command.class === "voxrt") {
        // Also in KWS vocab. Treat as a second wake: re-arm the window, do not dispatch.
        listeningUntil = performance.now() + LISTEN_WINDOW_MS;
        continue;
      }
      console.log(`Command "${command.class}" score ${command.score.toFixed(3)}`);
      document.getElementById("last").textContent = command.class;
      dispatch(command.class);
      listeningUntil = 0; // close window after first command
      document.getElementById("status").textContent = "Listening for wake word";
    }
  } else if (listeningUntil !== 0) {
    // window expired without a command
    listeningUntil = 0;
    document.getElementById("status").textContent = "Listening for wake word";
  }
});

function dispatch(command) {
  switch (command) {
    case "play":  /* start media playback */  break;
    case "pause": /* pause media playback */  break;
    case "next":  /* skip to next item */     break;
    case "previous": /* previous item */      break;
    case "up":    /* volume up */             break;
    case "down":  /* volume down */           break;
    case "on":    /* enable something */      break;
    case "off":   /* disable something */     break;
    case "yes":   /* confirm */               break;
    case "no":    /* dismiss */               break;
    case "cancel": /* cancel current action */ break;
    case "back":  /* go back / navigate up */ break;
  }
}

One note on the vocabulary overlap: voxrt and hey_vox are both in KWS's 14-word set. Once the listening window is open, the user can say them and KWS will fire. The filter at the top of the loop treats those two as a "second wake" (re-arm the window) rather than a command. Without that filter, saying "hey vox" mid-listening would silently close the window on a no-op dispatch. Either behaviour is defensible, but the reader should see the choice.

That is the whole runtime. Say "Hey Assistant", then within 3 seconds say one of the 14 supported commands, and dispatch fires. Open DevTools (F12, Console) to see the timing and scores.

Numbers

Measured on a MacBook Pro M4 running Chrome, single-threaded WASM with SIMD128:

Wake word: RTF 0.16% (625× real time), always on
KWS: RTF 0.97% (103× real time), only during listening windows
Combined footprint: ~1% of one CPU core when actively listening, closer to 0.2% between windows

Model quality:

Wake word ("Hey Assistant"): ROC AUC 0.9966, precision 0.993 and recall 0.982 at threshold 0.9 on the reference test split (11,656 utterances)
KWS (14 commands): F1 macro 0.9671 at threshold 0.9 on a speaker-disjoint test set. Per-class ROC AUC is ≥0.99 for every command

Assets shipped to the browser:

Wake-word runtime + model: ~275 KB
KWS runtime + model: ~1.47 MB
Total pipeline: ~1.75 MB

That is smaller than the average site's hero image.

The 14-word vocabulary

Fixed in v0.1.0: yes, no, cancel, play, pause, next, previous, up, down, back, on, off, voxrt, hey_vox.

It covers three classes of interaction:

Confirmation and cancellation: yes, no, cancel, back
Media control: play, pause, next, previous
State toggles and navigation: up, down, on, off

Those 12 cover most "wake, then dispatch" flows: a recipe app in the kitchen, an accessibility layer for hands-free navigation, a dashboard that toggles panels by voice, a car UI. The remaining two (voxrt, hey_vox) exist as alternate wake-style phrases if you want a second entry point.

If you need product-specific words (custom brand names, unique verbs, domain jargon), the vocabulary is currently fixed at these 14. Custom command sets are a paid tier: help@voxrt.com.

Where this fits

Two-stage voice control fits a specific shape of use case:

Hands are busy or dirty. Kitchen recipe apps, workshop tool controllers, medical stations.
Accessibility. Users with motor limitations get an entry-level voice control layer that does not depend on Apple or Google account state.
In-car dashboards. Voice shortcuts on top of a touch UI, without shipping cabin audio to a cloud provider.
Kiosks and public terminals. No microphone stream leaves the machine.

If you need continuous transcription (arbitrary sentences, not fixed commands), you want streaming ASR instead. That is not in browser yet: our streaming ASR runtime is Android, iOS, and Linux only. In-browser ASR is a real question we get and the honest answer today is "not shipped."

Wrap up

Full pipeline, on-device, in ~1.75 MB. Two SDKs, one shared .vxrt runtime, no cloud, no per-request cost. Wake word optimized for always-on, command spotter activated on demand.

Repos and packages:

Wake word: github.com/VoxRT/voxrt-wake-word-browser. npm: @voxrt/wake-word-browser
KWS: github.com/VoxRT/voxrt-kws-browser. npm: @voxrt/kws-browser
Model files: served from unpkg alongside the JS, or from GitHub releases
VoxRT overview: voxrt.com

If you build something with this, the vocabulary limits (14 words, English) will be the first thing that pinches. That is next on our roadmap.

Building a fully offline voice assistant on Raspberry Pi 4

VoxRT — Mon, 14 Sep 2026 08:16:09 +0000

What we are building

A voice assistant that runs entirely on a Raspberry Pi 4. No cloud, no API keys, no per-minute pricing. The user says "Hey Assistant", the device wakes up, and then handles whatever was said next.

Two interaction paths:

The user says one of 14 fixed commands (turn on lights, play music, stop, and so on). The device recognises it instantly and runs the action.
The user says something outside that set (read me the news, what time does the store open). The device transcribes the speech into text and processes the open-ended query.

Three models work together. A wake-word detector listens to the microphone 24/7. It is cheap and always on. A keyword spotter takes over for the next couple of seconds and tries to match a fixed vocabulary. And a streaming ASR transcribes open-ended speech if the keyword spotter finds no match. It is heavier but only runs when actually needed.

All three install with three pip install calls and run on a $35 Raspberry Pi 4 without an internet connection. This article walks through the full Python code, shows equivalent snippets in Node.js, Go and C, and explains why splitting this into three SDKs on one runtime keeps the always-on cost low without giving up open-ended speech.

Why the triage pattern (WW then KWS then ASR)

The core idea is to avoid running the heaviest model until it is actually needed.

The wake-word detector is the lightest piece of the pipeline. Our model is about 100 KB and uses about 5.3% of one A53 core on a $15 Raspberry Pi Zero 2 W (24/7 sustained, per our earlier benchmarks, link below). On a Pi 4 the cost drops further because the Cortex-A72 cores are faster and run in 64-bit mode. That means the wake-word can listen to the environment continuously without stealing meaningful CPU from anything else on the device.

But the wake-word only knows one thing. Did the trigger phrase ("Hey Assistant" in our case) get said. To understand what the user actually wants, you need another model.

The obvious option is to fire up ASR as soon as the wake-word triggers. That works but it is overkill for most real requests. In a voice assistant, users usually say one of a small set of commands. Turn on lights. Set a timer. Play music. Running full open-vocabulary ASR on that is like using a regex when strcmp would do.

KWS (keyword spotter) solves this cheaply. Our model processes 16 kHz mono audio in 32 ms frames and classifies each into one of 14 fixed command classes. It does not transcribe arbitrary speech, but it does quickly answer "yes, this is command X" or "no match".

The triage logic looks like this.

WW listening state. Wake-word runs continuously, waiting for "Hey Assistant".
WW triggered. Switch to KWS. Give the user about 2 seconds to say a command.
KWS matched. Run the command action (turn on lights, stop, and so on). Go back to state 1.
KWS did not match. Fall back to ASR. Transcribe the open-ended speech until silence or a timeout.
ASR finished. Process the text (route to a local LLM, a parser, or send it to your backend). Go back to state 1.

In compute terms this is a switch with branches for known cases and a default for everything else. WW costs almost nothing all the time. KWS runs only after a wake-word (typically 1 to 2 seconds). ASR runs only when the command missed the KWS vocabulary. In practice, most assistant requests fall inside a small command vocabulary, though the exact fraction depends heavily on your product.

Hardware

Minimum setup:

Raspberry Pi 4 (4GB). Four gigabytes of RAM give comfortable headroom for the ASR model, which keeps its weights in memory. The 2GB variant will boot the whole pipeline but leaves little room for anything else running on the device. Pi 5 and Pi 3B+ also work, but we tested on Pi 4 since that is the most common Pi model developers reach for right now.
A USB microphone ($10 to $15, any class-compliant device) or an I2S mic HAT such as the ReSpeaker 2-Mics Pi HAT (about $10) for cleaner audio.
A 32GB microSD card with 64-bit Raspberry Pi OS (aarch64). The 64-bit build is required. Our Linux SDKs ship aarch64 wheels only, and the 32-bit armhf variant is not supported.
Optional: a speaker or headphones if you want to add a TTS response. TTS is in development. This tutorial does not cover it.

Wake-word alone (without the rest of the pipeline) works even on a Pi Zero 2 W. We tested that in an earlier article, wake-word on a $15 Pi Zero 2 W. Adding KWS and ASR bumps the compute budget, so this tutorial uses a Pi 4.

Installing the three SDKs

Each model ships as a separate package.

pip install voxrt-wake-word voxrt-kws voxrt-asr

The three libraries share the same VoxRT runtime under the hood. That runtime is a Rust inference engine with a C ABI. We do not ship a single voxrt-all package, because many integrators only need part of the pipeline (wake-word only for a kiosk device, ASR only for a dictation app) and carrying weights they never load makes no sense.

Model weights download separately as files.

# Wake-word "Hey Assistant"
curl -LO https://github.com/VoxRT/voxrt-wake-word-models/releases/download/v0.1.0/voxrt_wake_word.vxrt

# Keyword spotter (14 commands)
curl -LO https://github.com/VoxRT/voxrt-kws-models/releases/download/v0.1.0/voxrt_kws.vxrt

# Streaming ASR (medium PC model)
curl -LO https://github.com/VoxRT/voxrt-asr-models/releases/download/v0.1.2/streaming_medium_pc.vxrt

(Check the exact versions in each repo README. They update periodically.)

The Node.js, Go and C install options are covered in the "Same pipeline in other languages" section below.

Code: the triage pipeline in Python

The full example breaks into three parts.

Microphone capture. Shared across all three SDKs.
State machine. Controls which engine gets audio at any given moment.
Wiring. The event loop that puts it all together.

Audio capture

We use sounddevice (a PortAudio wrapper), which is the standard way to read a microphone in Python on Linux. It provides a callback-based API. The PortAudio audio thread calls our function with each new chunk of samples. We push them into a queue and the main thread drains it.

import sounddevice as sd
import numpy as np
from queue import Queue

SAMPLE_RATE = 16000   # all three models expect 16 kHz mono
CHUNK_FRAMES = 512    # 32 ms, matches WW and KWS internal chunk size

audio_queue: "Queue[np.ndarray]" = Queue(maxsize=64)

def on_audio(indata, frames, time_info, status):
    if status:
        print(f"audio status: {status}")
    # indata is (frames, 1) int16. Copy the buffer since sounddevice
    # reuses the same backing storage between callbacks.
    audio_queue.put(indata.copy())

stream = sd.InputStream(
    samplerate=SAMPLE_RATE,
    channels=1,
    dtype="int16",
    blocksize=CHUNK_FRAMES,
    callback=on_audio,
)
stream.start()

After this call, 32 ms blocks of int16 samples land in audio_queue steadily. The next question is which engine handles them at any given moment.

State machine

We keep one state enum and a main loop that pulls chunks off the queue and routes them into the right engine.

from enum import Enum

class State(Enum):
    IDLE = 1            # only WW is listening
    KWS_CHECK = 2       # WW fired, KWS is checking for 2 seconds
    ASR_TRANSCRIBE = 3  # KWS missed, ASR is transcribing

state = State.IDLE

Initialise the three engines

from voxrt_wake_word import WakeWordEngine
from voxrt_kws import KwsEngine
from voxrt_asr import AsrStreamingEngine, DecodeMode

ww = WakeWordEngine.from_path("voxrt_wake_word.vxrt")
ww.threshold = 0.9
ww.cooldown_frames = 100  # ignore 100 frames (~3.2 s) after a detection

kws = KwsEngine.from_path(
    "voxrt_kws.vxrt",
    threshold=0.9,
    consecutive_frames_required=3,
    cooldown_frames=25,
)
print(f"KWS commands: {kws.class_names}")

asr = AsrStreamingEngine.from_path(
    "streaming_medium_pc.vxrt",
    mode=DecodeMode.RNNT,
)

The main event loop

This is where the triage happens. Broken down by state:

import time
from queue import Empty

KWS_WINDOW_SEC = 2.0    # window after WW for a KWS attempt
ASR_SILENCE_SEC = 1.5   # silence at the end of an ASR utterance


def handle_command(name):
    print(f"running fixed command: {name}")
    # wire this into GPIO, home automation API, whatever you need


def handle_open_query(text):
    print(f"open-ended query: {text}")
    # send this to a local LLM, a parser, wherever


kws_check_deadline = 0.0
asr_buffer_f32 = []
asr_last_speech_time = 0.0

try:
    while stream.active:
        try:
            chunk_i16 = audio_queue.get(timeout=0.1)
        except Empty:
            continue

        samples_i16 = chunk_i16.flatten().tolist()

        if state == State.IDLE:
            # WW listens all the time.
            for det in ww.push_pcm_i16(samples_i16):
                print(f"[WW] wake detected, score={det.score:.3f}")
                state = State.KWS_CHECK
                kws_check_deadline = time.time() + KWS_WINDOW_SEC

        elif state == State.KWS_CHECK:
            # Try to match a fixed command in the next 2 seconds.
            matched = False
            for det in kws.push_pcm_i16(samples_i16):
                print(f"[KWS] command: {det.class_name} score={det.score:.3f}")
                handle_command(det.class_name)
                state = State.IDLE
                matched = True
                break

            if not matched and time.time() > kws_check_deadline:
                print("[KWS] no match, falling back to ASR")
                state = State.ASR_TRANSCRIBE
                asr_buffer_f32 = []
                asr_last_speech_time = time.time()

        elif state == State.ASR_TRANSCRIBE:
            # ASR expects f32 samples. Convert from int16.
            f32 = [s / 32768.0 for s in samples_i16]
            asr_buffer_f32.extend(f32)

            # ASR wants 200 ms chunks (3200 samples).
            while len(asr_buffer_f32) >= 3200:
                chunk_200ms = asr_buffer_f32[:3200]
                asr_buffer_f32 = asr_buffer_f32[3200:]
                partial = asr.push_audio(chunk_200ms)
                if partial:
                    print(f"[ASR partial] {partial}")
                    asr_last_speech_time = time.time()

            # If silence lasts more than 1.5 seconds, finalise.
            if time.time() - asr_last_speech_time > ASR_SILENCE_SEC:
                final = asr.stop()
                print(f"[ASR final] {final}")
                handle_open_query(final)
                # asr.reset() keeps the loaded weights and resets the streaming
                # state. Cheaper than re-loading ~60 MB via from_path().
                # If your voxrt-asr version does not expose reset(), fall back
                # to AsrStreamingEngine.from_path(...) here, but expect a
                # multi-second stall while weights re-load.
                asr.reset()
                state = State.IDLE
finally:
    stream.stop()
    stream.close()

One key detail. WW and KWS want int16 samples. ASR wants float32 normalised to [-1.0, 1.0]. The conversion is one line (s / 32768.0) but it is easy to forget. Chunk sizes also differ. 32 ms for WW and KWS. 200 ms for ASR (matching the ASR model's internal attention window).

This is not "one API for everything". It is "consistent integration pattern (from_path then push_something then read the result), with input format matched to each model's requirements". Wake-word and KWS have the same shape internally (both are event detectors on short chunks), so they share a signature. ASR works differently (it is a streaming decoder), so it has a different signature.

Same pipeline in other languages

Full quickstart examples for Node.js, Go and C are in the repos:

github.com/VoxRT/voxrt-wake-word-linux/tree/main/examples/{nodejs,go,c}/
github.com/VoxRT/voxrt-kws-linux/tree/main/examples/{nodejs,go,c}/
github.com/VoxRT/voxrt-asr-linux/tree/main/examples/{nodejs,go,c}/

Below are the critical snippets. How push_pcm_i16 and push_audio look in each language. Follow the repo links for the full runnable examples.

Node.js

const { WakeWordEngine } = require("@voxrt/wake-word");
const { KwsEngine } = require("@voxrt/kws");
const { AsrStreamingEngine, DecodeMode } = require("@voxrt/asr");

const ww = WakeWordEngine.fromPath("voxrt_wake_word.vxrt");
ww.threshold = 0.9;

// mic capture via any ALSA/PortAudio Node binding available on Pi.
mic.on("data", (buffer) => {
  // Buffer is a slice of a larger ArrayBuffer. Use byteOffset and
  // byteLength to build a view over just the samples we received.
  const samples = new Int16Array(
    buffer.buffer,
    buffer.byteOffset,
    buffer.length / 2,
  );
  for (const det of ww.pushPcmI16(samples)) {
    console.log(`[WW] score=${det.score}`);
    // switch state to KWS_CHECK here
  }
});

Go

import (
    "log"

    wakeword "github.com/VoxRT/voxrt-wake-word-linux/go"
    // kws     "github.com/VoxRT/voxrt-kws-linux/go"
    // asr     "github.com/VoxRT/voxrt-asr-linux/go"
)

engine, err := wakeword.OpenFromPath("voxrt_wake_word.vxrt")
if err != nil {
    log.Fatal(err)
}
defer engine.Close()

_ = engine.SetThreshold(0.9)
_ = engine.SetCooldownFrames(100)

// mic capture via any ALSA/PortAudio Go binding available on Pi.
for chunk := range micChan { // chunk is []int16 of length 512
    for _, d := range engine.PushPcmI16(chunk) {
        log.Printf("[WW] score=%.3f", d.Score)
        // switch state here
    }
}

C

#include <stdint.h>
#include <stddef.h>
#include <stdio.h>

#include <voxrt_wake_word.h>

// load_vxrt_model() mmaps the model file and returns (bytes, len).
// See examples/c/alsa-mic-quickstart/main.c in the repo for the exact
// implementation, about 20 lines around open() + fstat() + mmap().
// alsa_read_chunk() is a thin wrapper over snd_pcm_readi().

const uint8_t *model_bytes = NULL;
size_t model_len = 0;
if (load_vxrt_model("voxrt_wake_word.vxrt", &model_bytes, &model_len) != 0) {
    fprintf(stderr, "failed to mmap the wake-word model file\n");
    return 1;
}

voxrt_wake_word_t *engine = NULL;
voxrt_status_t rc = voxrt_wake_word_create(model_bytes, model_len, &engine);
if (rc != VOXRT_OK) {
    fprintf(stderr, "voxrt_wake_word_create failed: %d\n", (int)rc);
    return 2;
}
voxrt_wake_word_set_threshold(engine, 0.9f);
voxrt_wake_word_set_cooldown_frames(engine, 100);

// ALSA capture, 16 kHz mono int16, 512 samples per chunk.
int16_t chunk[512];
voxrt_wake_word_detection_t dets[8];
while (alsa_read_chunk(pcm_handle, chunk, 512) == 0) {
    size_t written = 0;
    rc = voxrt_wake_word_push_pcm_i16(engine, chunk, 512, dets, 8, &written);
    // VOXRT_ERR_BUFFER_TOO_SMALL just means more detections would fit;
    // the ones we got are still valid.
    if (rc != VOXRT_OK && rc != VOXRT_ERR_BUFFER_TOO_SMALL) {
        fprintf(stderr, "push failed: %d\n", (int)rc);
        break;
    }
    for (size_t i = 0; i < written; i++) {
        printf("[WW] score=%.3f\n", dets[i].score);
        // switch state here
    }
}

voxrt_wake_word_destroy(engine);
// remember to munmap(model_bytes, model_len)

Each SDK's quickstart file already does this work. It opens the mic, feeds the engine, and prints detections. Our triage loop is the state machine on top of that.

Expected performance on Pi 4

Important disclaimer. The numbers below extrapolate from benchmarks on other hardware. We have not run the full triage loop on our Pi 4 yet. Your numbers should land in this range but you will only know exactly after you measure on your own board.

IDLE state (WW only). About 1 to 2% of one CPU core. Extrapolated from 5.3% on Pi Zero 2 W (A53 at 1 GHz running the default 32-bit Raspberry Pi OS). Pi 4 uses a 64-bit A72 at 1.5 GHz with better IPC and more efficient NEON when the OS runs in aarch64 mode.
KWS_CHECK state (2 seconds after a wake). About 15 to 25% CPU during the burst. Extrapolated from 16% RTF on Snapdragon 662, but Pi 4's A72 is a generation older than the Kryo 260 Gold cores in SD662, so expect somewhat higher RTF. KWS only runs in short windows.
ASR_TRANSCRIBE state. Roughly 40 to 70% CPU during active transcription. ASR is the heaviest of the three, and NEON-specialised int8 kernels tuned for mobile Snapdragons will not extract the same efficiency on A72. Real-time streaming (RTF under 1.0) should still be achievable on Pi 4 aarch64, but the headroom is the tightest of the three. Measure on your device before assuming.

RAM footprint with all three engines initialised: around 200 to 300 MB. (model weights plus Python runtime, audio buffers, and engine overhead). ASR is the biggest single component at roughly 60 MB of weights. That is why we recommend Pi 4 with 4 GB. The 2 GB variant also works but leaves little room for the rest of your software.

If we run real Pi 4 numbers after publish, we will do a follow-up article with actual measurements.

What this article does not cover

TTS (speaking back). In development. A separate on-device voice-cloning model. Will get its own article when it ships.
Custom wake-word phrase. We currently only ship "Hey Assistant" as a pre-trained model. Training your own trigger phrase requires a separate training pipeline that we have not released publicly yet.
Speaker verification / voice biometrics. Not included. The models do not tell you who is speaking.
Multi-language support. Wake-word and KWS are English-only at the moment. The ASR model is multilingual in general, but we have only tested it on English.

Everything above is on the VoxRT roadmap. If any of it is critical for your use case, mention it in the comments so we can prioritise.

Repos and resources

Wake-word Linux SDK: github.com/VoxRT/voxrt-wake-word-linux
KWS Linux SDK: github.com/VoxRT/voxrt-kws-linux
ASR Linux SDK: github.com/VoxRT/voxrt-asr-linux
All SDKs: github.com/VoxRT
Our earlier on-device voice articles: dev.to/voxrtio

If you are building anything voice-driven and want the audio to stay on the device, all three SDKs are on GitHub with the documentation in the README. Full quickstart examples in Python, Node.js, Go and C are in every repo.

How we prepared voice cloning TTS to run on-device

VoxRT — Mon, 07 Sep 2026 14:05:00 +0000

We trained a text-to-speech model with voice cloning that fits into 100 MB (fp16) and runs on-device for synthesis, from smartphones to a Raspberry Pi. The model is in a file, generation happens on the device. No API, no cloud backend at inference time.

Early tests on a budget Android phone hit faster-than-realtime on a single core. Full benchmarks with numbers and different devices are coming in a separate post. Here is how we got there.

The model is about 45M parameters (a language model plus an audio decoder). For comparison, typical server-side TTS systems are hundreds of MB to several GB of weights. We trained both stages from scratch and packaged them into a single binary.

The road here was not smooth. This post is a short architecture overview (without it, it is hard to explain what broke) and two concrete stories of what broke and how we fixed it. If you end up training something similar, this should save you a few surprises.

Architecture in two paragraphs

We use the classic two-stage scheme. The first stage is a language model (LM) that, given text and a short voice sample, generates audio latents autoregressively, roughly the way an LLM generates text, except the outputs are continuous audio representations instead of words. The second stage is the decoder of our own neural codec (24 kHz, compact continuous latent space), which turns those latents back into a waveform.

The key design decision is a single-step flow sampler instead of iterative diffusion. Classical diffusion TTS often needs many forward passes per frame to sound right. That is fine on a server. On mobile or a Raspberry Pi, that does not work. One step per frame gives you speed on weak hardware. But you pay for it in training complexity, described below.

Two more things for context. The architecture is causal, so it streams by construction. You do not have to wait for the end of the input, and audio comes out with sub-second time-to-first-audio. Voice cloning works from a short sample of around 5 seconds. The cloning part runs off-device, the synthesis is on-device.

What broke #1: single-step sampler collapses speech to averaged sound

Diffusion models work well partly because stochasticity accumulates over many steps. Each step adds a bit of noise and the model learns to remove it. By the end of the process you get a quality sample.

When you do this in a single step, that stochasticity disappears. A naively trained one-step flow sampler tends to converge to an "averaged" sound. The model finds an output that minimizes loss on average across the training set, rather than a diverse, characteristic sound.

We saw this on early iterations as a lifeless "muddled" voice, identical in intonation regardless of input. Test recordings sounded like a robotic monotone from the nineties.

What helped: a special sampler training scheme. I will skip the architectural details (that part of the recipe stays internal), but the general idea is to force the sampler to account for the target variance of the distribution, not just its mean. Verification was through a separate synthetic test set specifically for this failure mode, where voice diversity is expected. The model passed, it started to sound like real speech.

The takeaway: single-step sampling is not free. You save on inference and pay in training complexity.

What broke #2: mixed precision, one poisoned step costs 5000 iterations

We trained in mixed precision, some operations in fp16 for speed, some in fp32 for stability. Standard practice for large neural nets on GPUs.

The problem is that fp16 has a narrow range of representable numbers. Sometimes gradients overflow and you get a NaN. The usual protection (loss scaling and gradient clipping) works most of the time. But sometimes a rare combination of input data plus current weight state causes one operation to produce a NaN. That NaN gradient then poisons every parameter it touches at the next optimizer step.

One such "poisoned" step can invalidate 5000 previous training iterations. The model starts emitting noise, loss shoots to infinity, and everything since the last checkpoint is gone.

The fix was multi-layer protection. Per-batch NaN check, per-epoch sanity check, running-loss anomaly detection, then automatic rollback to the last good checkpoint plus a human alert. Not free compute-wise, but cheaper than losing a week of training. Mixed precision is a real engineering discipline, not a flag you enable and forget, especially on long runs where accumulated risk grows linearly with time.

What we optimized for (and what we did not)

The design constraints were tight.

Streaming by construction, with sub-second time-to-first-audio on weak hardware. No "wait a moment while the model thinks".

Numbers, dates, and currency without special-case hacks. The model reads "March 21st, 2026" and "$19.99" correctly. To get there, we added synthetic speech to the training mix with quality control specifically on those edge cases.

Around 45M parameters, because 100 MB is what you can bundle into an app without a noticeable impact on bundle size. Any bigger and it gets uncomfortable for mobile apps.

What we did not prioritize: language diversity (English only for now, the training corpus was around 3000 hours for the codec and 1700 hours for the LM), and extreme quality benchmarks for professional use. We aim for natural-sounding output on default use cases, not to beat top commercial TTS in blind studio tests.

Quality is judged via blind listening tests on a fixed text set: numbers, dates, tongue twisters, long narratives. Not perfect, but reproducible, and raters agree with each other.

Where we go next

The model is going through final polishing (we are fine-tuning the decoder for the production generation mode) and packaging into the VoxRT runtime. The delivery format will be the same as our other SDKs: a single binary model file plus native SDKs for iOS, Android, and Raspberry Pi.

Release in September 2026. Code, model, and release notes will land on GitHub. Star the repo if you want a ping when it drops.

More about our other on-device SDKs (VAD, wake-word, KWS, streaming ASR) at voxrt.com. Models and SDKs on GitHub.

Add a wake-word to your web app in 5 minutes

VoxRT — Wed, 02 Sep 2026 08:00:03 +0000

Voice interfaces are getting easier to add to the web, but wake-words have been the hard part. Cloud APIs mean latency, per-request billing, and audio leaving the user's device. On-device wake-words in the browser were awkward until WebAssembly SIMD128 shipped in every major browser.

You will have a working "Hey Assistant" detector in about five minutes. Everything runs on-device, no cloud, no per-request cost.

What you will need

A modern browser (Chrome/Edge 91+, Firefox 89+, or Safari 16.4+). WASM SIMD128 is a hard requirement
A microphone
Any static file server on HTTPS or localhost. Browsers block getUserMedia on plain HTTP
Node.js if you want npm install. Otherwise CDN works fine

The full download is ~275 KB (170 KB WASM runtime + ~100 KB model). That is smaller than most icon fonts.

Step 1: install the SDK

Two options.

via npm:

npm install @voxrt/wake-word-browser

via CDN (no build step):

import init, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";

Use CDN to skip the bundler.

Step 2: HTML shell

Nothing fancy. A single button and a status line will do:

<!DOCTYPE html>
<html>
<head><meta charset="utf-8"><title>Wake-word demo</title></head>
<body>
  <button id="start">Start listening</button>
  <p id="status">Idle</p>
  <script type="module" src="./app.js"></script>
</body>
</html>

Serve this over HTTPS or localhost. Microphone access does not work on plain http:// except for localhost.

Step 3: initialize the engine and load the model

Create app.js:

import init, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";

await init();

const modelBytes = new Uint8Array(await (await fetch(
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt_wake_word.vxrt"
)).arrayBuffer());

const engine = WakeWordEngine.fromBytes(modelBytes);
engine.threshold = 0.9;

That is the whole "load the model" step. The .vxrt file is the pre-trained "Hey Assistant" model at ~100 KB. init() bootstraps the WebAssembly runtime.

The threshold is in sigmoid space [0, 1]. Default is 0.9. Lower it and you get more sensitive detection (more false accepts). Raise it and you get fewer false accepts but might miss quieter or accented pronunciations. We will tune this later.

Step 4: capture microphone audio

Wake-word models need 16 kHz mono audio. Modern browsers give you higher sample rates by default, so we ask AudioContext for 16 kHz explicitly. Not all browsers honor the request, so we also check:

document.getElementById("start").onclick = async () => {
  const stream = await navigator.mediaDevices.getUserMedia({audio: true});
  const audioContext = new AudioContext({sampleRate: 16000});
  if (audioContext.sampleRate !== 16000) {
    console.warn(`Got ${audioContext.sampleRate} Hz instead of 16000. Detection quality will drop. Resample or check the SDK docs.`);
  }
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(512, 1, 1);
  const mute = audioContext.createGain();
  mute.gain.value = 0;

  // detection loop goes here (Step 5)

  source.connect(processor);
  processor.connect(mute).connect(audioContext.destination);
  document.getElementById("status").textContent = "Listening";
};

Two notes on the code above:

ScriptProcessorNode is deprecated in favor of AudioWorkletNode for production. For a 5-minute demo, ScriptProcessor is simpler. Swap it out when you productionize
The muted GainNode between processor and destination is a Web Audio quirk. Without connecting the processor to something, audioprocess never fires. Setting gain to 0 avoids sending your microphone straight to the speakers (which would cause feedback)

Step 5: push audio frames and detect

The engine wants Int16 PCM samples in fixed-size chunks. Here is the detection loop:

const pcmBuffer = new Int16Array(512);

processor.addEventListener("audioprocess", (e) => {
  const inputFloat = e.inputBuffer.getChannelData(0);

  // convert Float32 [-1, 1] to Int16
  for (let i = 0; i < inputFloat.length; i++) {
    const s = Math.max(-1, Math.min(1, inputFloat[i]));
    pcmBuffer[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
  }

  // push to engine, iterate detections
  for (const detection of engine.pushPcmI16(pcmBuffer)) {
    console.log(`Wake detected at ${detection.timestampSec.toFixed(3)}s, score ${detection.score.toFixed(4)}`);
    document.getElementById("status").textContent = "Wake-word detected";
  }
});

That is it. Open DevTools (F12, then Console), click the button, say "Hey Assistant", and you should see a log line. The detection.score is the sigmoid probability at the moment of detection.

What just happened

You loaded a ~100 KB model into WebAssembly, streamed 16 kHz mono audio through it in 512-sample frames (32 ms each), and got detections when the score crossed threshold.

The engine handles its own cooldown so you do not get spam detections on the same utterance. Default cooldown is 100 frames (about 1 second at 10 ms hop). You can tune with engine.cooldownFrames.

Two quick customizations

Prefer Float32 over Int16? Skip the conversion:

processor.addEventListener("audioprocess", (e) => {
  const inputFloat = e.inputBuffer.getChannelData(0);
  for (const detection of engine.pushPcmF32(inputFloat)) {
    console.log(`Detected: ${detection.score.toFixed(4)}`);
  }
});

Tuning threshold: Watch engine.currentScore() to see live confidence. If your users often speak quietly, lower the threshold to 0.85. If you get false triggers from radio or TV, raise it to 0.93.

setInterval(() => {
  console.log(`Live score: ${engine.currentScore().toFixed(3)}`);
}, 100);

At the default threshold of 0.9, the model has precision 0.993 and recall 0.982 on the reference test split (11,656 utterances, ROC AUC 0.9966).

Full code (copy-paste starter)

Here is everything above, glued together:

import init, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";

await init();

const modelBytes = new Uint8Array(await (await fetch(
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt_wake_word.vxrt"
)).arrayBuffer());

const engine = WakeWordEngine.fromBytes(modelBytes);
engine.threshold = 0.9;

document.getElementById("start").onclick = async () => {
  const stream = await navigator.mediaDevices.getUserMedia({audio: true});
  const audioContext = new AudioContext({sampleRate: 16000});
  if (audioContext.sampleRate !== 16000) {
    console.warn(`Got ${audioContext.sampleRate} Hz instead of 16000. Detection quality will drop.`);
  }
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(512, 1, 1);
  const mute = audioContext.createGain();
  mute.gain.value = 0;
  const pcmBuffer = new Int16Array(512);

  processor.addEventListener("audioprocess", (e) => {
    const inputFloat = e.inputBuffer.getChannelData(0);
    for (let i = 0; i < inputFloat.length; i++) {
      const s = Math.max(-1, Math.min(1, inputFloat[i]));
      pcmBuffer[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
    }
    for (const detection of engine.pushPcmI16(pcmBuffer)) {
      console.log(`Wake at ${detection.timestampSec.toFixed(3)}s score ${detection.score.toFixed(4)}`);
      document.getElementById("status").textContent = "Wake-word detected";
    }
  });

  source.connect(processor);
  processor.connect(mute).connect(audioContext.destination);
  document.getElementById("status").textContent = "Listening";
};

Serve this alongside the HTML from Step 2 and you have a working wake-word demo.

Where to go from here

Custom phrase or another language: the free tier ships with "Hey Assistant" only. Custom phrases and additional languages are available on a paid tier. We train the model for you and swap the .vxrt file
Native mobile: same SDK exists for iOS (Swift) and Android (Kotlin). Wake-word runs at 1.5% RTF on iPhone A15, 2.1% on Snapdragon 662
Linux and edge: available for Linux aarch64 with bindings for Python, Node.js, Go, and C. Holds at 5.3% RTF sustained on a $15 Raspberry Pi Zero 2 W
Custom on-device pipeline: wake-word into ASR into your app logic works well for privacy-critical voice UIs

One note on licensing

The wrapper (Rust crate, wasm-bindgen bindings, examples) is Apache-2.0 so you can freely integrate it. The compiled WebAssembly runtime and the model weights are proprietary. Redistribution is allowed only as an unmodified part of the SDK package. In practical terms: install and use it in your product, no additional legal setup beyond reading the license.

Live demo: voxrt.com/wake-word-demo

Repo: github.com/VoxRT/voxrt-wake-word-browser

Wake-word on a $15 Raspberry Pi Zero 2 W: 5.3% RTF always-on

VoxRT — Fri, 21 Aug 2026 13:15:02 +0000

The problem: always-on wake-word is hard and expensive

A wake-word detector is the part of the voice stack that listens to the mic 24/7 and triggers the rest of the pipeline when it hears a specific phrase like "Alexa", "Hey Siri", or "Hey Google". Because it is always-on, wake-word dominates the energy and compute budget of any voice-first device.

Ready-made options for hobbyists are limited, and all have trade-offs:

Alexa/Google Nest/Apple HomePod work well but are locked to their vendor platforms, require cloud connectivity in some scenarios, and use closed hardware.
Picovoice / Porcupine is an on-device SDK that works offline after activation. The model is proprietary, and the free tier was deprecated as of 30 June 2026 (now a 7-day trial followed by paid).
openWakeWord is open source (Apache-2.0) with community models, but training a custom wake phrase is non-trivial. You need to gather data, train the model, and iterate.
Custom Python + TF Lite from scratch requires ML expertise and hours of work.

The problem: hobbyists want a simple recipe. Grab a $15 Pi, install a package with one line, hear a custom wake word, do not pay per activation.

This post covers our solution (proprietary, but with a free tier "Hey Assistant" by default). The headline number: 5.3% CPU on one A53 core of a $15 Raspberry Pi Zero 2 W, 24/7 sustained.

What a wake-word detector is: quick technical background

Three components usually connect:

VAD (Voice Activity Detection) is the lightest. It answers "is there any speech in this audio?" (regardless of what kind). Runs at ~1% CPU. Used as a pre-filter for heavier stages.
Wake-word detector answers "is there a specific phrase?". Heavier than VAD (needs actual pattern matching) but still light (typically 10-100 KB model). Runs 1-10% CPU in always-on mode.
ASR (Automatic Speech Recognition) does full transcription: "what exactly was said?". Heavy (~60 MB model, 30-40% CPU on mid-range Android). Only fires after wake-word success.

Wake-word sits in the sweet spot. Not as light as VAD (VAD does not distinguish "Hey Assistant" from "hello world"), not as heavy as ASR (no full vocabulary needed). Cache-friendly architecture, sub-100 ms decisions, runs on any ARM SoC from 2020+ without a DSP.

Why Pi Zero 2 W as target

Raspberry Pi Zero 2 W is the most popular "minimal" Pi (~$15). Specs:

ARM Cortex-A53 quad-core @ 1 GHz: same class as midrange smartphones from 2016-2018
512 MB RAM: enough for small ML models, not for heavy Whisper
No GPU: CPU-only inference
~$15 retail: cheap enough for DIY smart speakers, doorbell notifications, custom voice UI

The logic: if it works on Pi Zero 2 W, it works everywhere. More powerful Pi 4/5 and Jetson Nano are 4-10x faster, embedded ARM SoCs (Rockchip, Allwinner) are comparable class. So Pi Zero 2 W sets the lower bound: if the numbers hold there, they hold everywhere else.

Wake-word across the full Pi ladder:

Device	RTF (Real-Time Factor)	CPU budget
Pi Zero 2 W (Cortex-A53)	0.053 (5.3%)	one core
Pi 3 A+/B/B+ (Cortex-A53)	~0.038-0.044 (3.8-4.4%)	one core
Pi 4 B / 400 (Cortex-A72)	~0.018-0.024 (1.8-2.4%)	one core
Pi 5 (Cortex-A76)	~0.008-0.012 (0.8-1.2%)	one core

Pi Zero 2 W number is measured (60-second sustained live-mic push). Pi 3/4/5 numbers are estimates scaled by clock speed and architecture generation, not independently benchmarked.

RTF 0.053 means the processor spends 53 milliseconds per 1 second of audio. Wake-word takes ~5% of one core on Pi Zero 2 W, leaving 95% plus the other 3 cores for the application (UI, LEDs, LLM inference, storage). On Pi 5, wake-word is essentially free (<1%).

Model architecture

Key characteristics of our wake-word model:

~48 thousand parameters: very small neural network by modern standards (for comparison, Whisper base.en has ~74 million parameters, roughly 1500x larger).
File size: ~100 KB (encrypted .vxrt format).
Sampling rate: 16 kHz mono, 32 ms frames: standard for speech recognition.
Trained on 100% synthetic data with augmentation: no human voice samples in the training set, which addresses licensing concerns for commercial deployment.
Free tier phrase: "Hey Assistant": works out of the box.
Custom phrases: paid tier: training pipeline available on request (help@voxrt.com).

Accuracy metrics (measured on a test set of 5,240 positive + 6,416 negative utterances):

ROC AUC: 0.9966: very clean separation between positive and negative.
PR AUC (Average Precision): 0.9899.
Threshold 0.90: precision 0.993 / recall 0.982.

In practice: at threshold 0.90 the model produces ~1 false positive per 100+ triggers and misses ~2% of real wake-words. Acceptable for an always-on hot loop.

Benchmark: 5.3% RTF sustained on Pi Zero 2 W

RTF (Real-Time Factor) is CPU time divided by audio duration. RTF 5.3% means the processor spends 53 milliseconds of CPU time per 1 second of audio.

How we measured:

Pi Zero 2 W, Cortex-A53 @ 1 GHz, aarch64 Linux
60-second continuous loop with microphone input
Used one core out of four
Sustained RTF (not peak) because sustained better reflects real-world thermal throttling

Why sustained matters more than peak: Pi Zero 2 W without a heat sink can throttle CPU after 2-3 minutes of sustained load. Peak RTF (first 5 seconds) is typically 10-20% better than sustained. We publish the sustained number, 5.3%, because that is what a hobbyist actually gets in production.

What to do with the remaining 95% CPU plus 3 cores: UI (LEDs / display), Bluetooth / WiFi for sending triggers to home assistant, local logic (dimmer, thermostat), or on a mid-tier board like Pi 4, running a smaller LLM for command interpretation.

Why NEON matters: 8.7x speedup

The wake-word runtime is written in Rust. But "pure Rust" is not enough for these RTF numbers. The key ingredient is ARM NEON SIMD intrinsics in the inner loops (matrix multiply, activation functions).

Measured on Snapdragon 662, Cortex-A73 big cluster pinned via HIGH_PERF affinity (SD662 is a big.LITTLE SoC with 4x A73 + 4x A53, and the perf cluster is used for stable clock):

Implementation	RTF
Rust scalar (no SIMD)	0.182 (18.2%)
Rust + NEON SIMD	0.021 (2.1%)

Speedup: 8.7x.

What this means for the reader: even a small neural network (48K params) on CPU-only inference depends critically on SIMD acceleration. Writing your own solution in pure Python + NumPy gives you an RTF that is 10-20x higher than an optimized SIMD implementation.

On Pi Zero 2 W (A53, one generation older than A73), the NEON speedup ratio is similar, but A53 has fewer cycles per second per core. That is why we see 5.3% RTF on A53 vs 2.1% on A73 with the same code.

Practical takeaway: if you plan to write a custom wake-word, targeting ARM NEON is mandatory. Otherwise it will be too slow for always-on mode on any embedded ARM.

How to run: setup + code (4 languages)

The SDK is available for Python, Node.js, Go, and C with an identical API pattern: load a .vxrt file, set threshold and cooldown_frames, push PCM int16 chunks (16 kHz mono), receive detection events with frame_index, timestamp_sec, and score fields.

The voxrt_wake_word.vxrt model (~100 KB, free tier "Hey Assistant") for all examples below can be downloaded from the releases page or HuggingFace.

All examples below are simplified versions of the quickstarts in examples/{python,nodejs,go,c}/ in the repository. Full versions (with WAV parsing, ALSA live-mic, performance timing) live there too.

Python

pip install voxrt-wake-word

from voxrt_wake_word import WakeWordEngine

engine = WakeWordEngine.from_path("voxrt_wake_word.vxrt")
engine.threshold = 0.9
engine.cooldown_frames = 100

for chunk in mic_iter():  # int16 mono @ 16 kHz, 512-sample chunks
    for d in engine.push_pcm_i16(list(chunk)):
        print(f"wake! t={d.timestamp_sec:.3f}s score={d.score:.4f}")

For live mic, install sounddevice (pip install sounddevice) and feed sd.InputStream(samplerate=16000, channels=1, dtype='int16', blocksize=512, callback=...) chunks into engine.push_pcm_i16 inside the callback.

Node.js

npm install @voxrt/wake-word

const { WakeWordEngine } = require("@voxrt/wake-word");

const engine = WakeWordEngine.fromPath("voxrt_wake_word.vxrt");
engine.threshold = 0.9;
engine.cooldownFrames = 100;

for (const chunk of micIter()) {  // Int16Array, 512 samples, 16 kHz mono
  for (const d of engine.pushPcmI16(chunk)) {
    console.log(`wake! t=${d.timestampSec.toFixed(3)}s score=${d.score.toFixed(4)}`);
  }
}

For live mic, use the mic or node-record-lpcm16 npm packages. Both return Node Buffer objects, so convert to Int16Array before pushing: new Int16Array(buf.buffer, buf.byteOffset, buf.length / 2).

Go

go get github.com/VoxRT/voxrt-wake-word-linux/go

package main

import (
    "fmt"
    "log"
    wakeword "github.com/VoxRT/voxrt-wake-word-linux/go"
)

func main() {
    engine, err := wakeword.OpenFromPath("voxrt_wake_word.vxrt")
    if err != nil {
        log.Fatal(err)
    }
    defer engine.Close()
    engine.SetThreshold(0.9)
    engine.SetCooldownFrames(100)

    // micIter returns <-chan []int16 (512-sample chunks, 16 kHz mono)
    for chunk := range micIter() {
        for _, d := range engine.PushPcmI16(chunk) {
            fmt.Printf("wake! t=%.3fs score=%.4f\n", d.TimestampSec, d.Score)
        }
    }
}

For live mic on Linux, use github.com/gordonklaus/portaudio or pipe from arecord (simpler for a quickstart). Have your mic reader emit <-chan []int16 for the range loop above.

C

Install headers and shared library from the tarball release, then build with pkg-config for flags:

gcc main.c $(pkg-config --cflags --libs voxrt-wake-word) -lasound -o wake-word-app

#include <stdint.h>
#include <stdio.h>
#include <voxrt_wake_word.h>

int main(int argc, char **argv) {
    // Load model bytes via mmap. See full example in repo for the ~30-line
    // helper (open + fstat + mmap). Placeholders here for brevity:
    const uint8_t *model_bytes = NULL;   // TODO: mmap voxrt_wake_word.vxrt
    size_t model_len = 0;                // TODO: file size

    voxrt_wake_word_t *engine = NULL;
    voxrt_status_t rc = voxrt_wake_word_create(model_bytes, model_len, &engine);
    if (rc != VOXRT_OK) {
        fprintf(stderr, "voxrt_wake_word_create failed: %d\n", rc);
        return 1;
    }
    voxrt_wake_word_set_threshold(engine, 0.9f);
    voxrt_wake_word_set_cooldown_frames(engine, 100);

    int16_t buf[512];
    voxrt_wake_word_detection_t dets[8];
    for (;;) {
        // Read 512 int16 samples from ALSA / mic into buf.
        size_t written = 0;
        voxrt_wake_word_push_pcm_i16(engine, buf, 512, dets, 8, &written);
        for (size_t i = 0; i < written; ++i) {
            printf("wake! t=%.3fs score=%.4f\n",
                   dets[i].timestamp_sec, dets[i].score);
        }
    }
    voxrt_wake_word_destroy(engine);
    return 0;
}

The full C example with all ALSA setup (~150 lines) is at examples/c/alsa-mic-quickstart/main.c. There is also a C++ variant (alsa-mic-quickstart-cpp) and a CMake consumer example for integrating into existing projects.

Parameters and what is not shown

All four languages use identical parameters:

threshold (default 0.5): confidence threshold for firing. 0.9 is strict (fewer false positives). For always-on in a noisy environment, 0.85 to 0.90 is a reasonable baseline.
cooldown_frames (default 50, ~1.6 sec): how many silence frames after a detection before the next trigger. Prevents double-fire.

Not shown in the snippets (standard boilerplate): microphone init, error handling, action on detection (starting ASR, HTTP webhook to Home Assistant, turning on an LED).

Wake-word alternatives: honest comparison

Three main options for a hobbyist in 2026:

	Porcupine (Picovoice)	openWakeWord	VoxRT wake-word
Model size	~200 KB - 1 MB (varies by tier)	~50-400 KB per model + ~3 MB shared runtime	~100 KB
RTF on Pi Zero 2 W	~3.8% on Pi 3 per Picovoice docs, no Pi Zero 2 W numbers published (likely 4-8% scaled by clock)	reported CPU-heavy on Pi Zero 2 W, no public RTF, on Pi 4 <5 ms per 80 ms chunk	5.3% (measured)
Runtime	proprietary C/C++ SDK	Python + ONNX Runtime	proprietary Rust SDK
License	commercial (7-day trial then paid, no free tier since 30 June 2026)	Apache-2.0 (fully open)	proprietary (LICENSE-BINARY), free default phrase, paid custom
Custom wake phrase	commercial console (paid)	DIY training pipeline (Python, data collection, hours of work)	paid tier (email us)
Setup complexity (out-of-the-box "Hey X")	medium (auth flow)	high (need to train or download a community model)	low (pip install + download)

Each tool has its fit:

Porcupine: if you need a large catalog of pre-trained wake phrases and you are fine with commercial licensing.
openWakeWord: if 100% open source is critical and you have time for DIY training.
VoxRT wake-word: if you need the smallest footprint and simplest deployment ("Hey Assistant" out of the box, custom phrase via paid tier).

No tool is universally "better" than the others. It depends on requirements.

Licensing model

Important disclosure: VoxRT wake-word is a proprietary tool (unlike our Silero VAD packaging, which is MIT).

Runtime + model: LICENSE-BINARY (Elephant Enterprises LLC). Distributable as part of an unmodified SDK, no forking, no rebuilding.
Free tier phrase: "Hey Assistant" works out of the box without activation.
Custom wake phrases: paid tier. We train a custom model on your phrase (typical 2-3 weeks turnaround). Contact: help@voxrt.com.
No per-activation billing: unlike Porcupine, no ongoing fees. A one-time custom-phrase fee covers redistribution in your product.

This is a trade-off: you get the smallest footprint and simplest deployment, but you cannot modify the runtime or train your own model. For a 100% open-source requirement, use openWakeWord (Apache-2.0).

WASM bonus: 0.16% RTF in Chrome

The same ~100 KB .vxrt file runs via WASM SIMD128 in the browser:

Environment	RTF
Chrome / MacBook Pro M4 (WASM SIMD128)	0.16%
Safari / iPhone A15 (WASM)	0.23%

Essentially free. The full npm bundle @voxrt/wake-word-browser is ~275 KB (model + WASM runtime).

Not the primary story of this post, but worth mentioning because it is unusual. Voice AI in the browser without a backend, integrated into a web app via npm install. The same model runs on Pi, iPhone, Android, and in a user's Chrome tab.

Wrapping up

Wake-word detection on a $15 Raspberry Pi Zero 2 W, always on, 5.3% CPU is not an abstract idea. One-line install with pip install voxrt-wake-word, add a microphone reader loop, done.

Three things worth remembering:

RTF sustained (60+ sec) matters more than peak. Thermal throttling on a Pi Zero 2 W without a heat sink makes peak numbers misleading.
ARM NEON SIMD is critical for CPU inference. 8.7x speedup vs scalar Rust. Custom implementations without SIMD will be too slow for always-on.
Wake-word is not VAD, not ASR. Three tiers with different CPU budgets. Wake-word is the middle tier: heavier than VAD, light enough for always-on.

Tools from this post:

github.com/VoxRT/voxrt-wake-word-linux: Linux SDK (aarch64)
github.com/VoxRT/voxrt-wake-word-android: Android SDK
github.com/VoxRT/voxrt-wake-word-ios: iOS SDK
github.com/VoxRT/voxrt-wake-word-browser: WASM browser SDK
huggingface.co/VoxRT/wake-word-hey-assistant-vxrt: free tier model

Previous posts in this voice AI series:

Building Private Voice Notes for iOS and Android Without the Cloud: full tutorial for a VAD + ASR pipeline
Streaming ASR vs Whisper on mobile: when to switch: architectural comparison for live voice apps

Learn more about the runtime and product family: voxrt.com.

Custom wake phrase for your product: help@voxrt.com.

Streaming ASR vs Whisper on mobile: when to switch

VoxRT — Tue, 18 Aug 2026 08:45:10 +0000

The problem: Whisper feels laggy for live apps

Voice-first apps became mainstream in 2024-2026. Voice agents, real-time captions, live meeting transcription, voice notes with instant feedback. Developers building this new generation of apps typically look at Whisper first (whisper.cpp, faster-whisper). It is the de facto default for on-device speech-to-text.

And for batch transcription (upload audio file, wait, get transcript), Whisper is genuinely great. Interview transcription, meeting notes, lecture recordings on MacBook or in an Android app. Whisper handles all these accurately and reliably.

But as soon as you try Whisper for live scenarios (user speaks, expects instant response), something feels off. A 300-500 ms delay minimum shows up, often up to 1-2 seconds. Words get "sliced" at chunk boundaries. Transcript comes out in jerks. For a voice agent that should respond like a human in conversation, this is a frustrating UX.

This is not a Whisper bug. It is a design mismatch: Whisper is a batch model retrofitted for streaming, while a real-time scenario fundamentally needs a different architecture.

This post is about what native streaming ASR is, how it differs from Whisper's approach, what alternatives exist, and when you should switch (or stay).

What "streaming" really means in ASR

Before comparing options, terminology first.

Batch ASR processes audio as a whole: the model receives the entire file (or a fixed-length chunk), runs inference in one pass, returns the full transcript. Classic architectures (Whisper, original Wav2Vec2, seq2seq encoder-decoder) are batch by nature.

Streaming ASR processes audio continuously: the model takes a stream of samples, emits text incrementally as speech arrives. Key streaming parameters: lookahead (how many milliseconds of future audio the model "sees" before emitting text) and chunk latency (how many milliseconds the model buffers between emissions).

Classification by lookahead:

Infinite lookahead: batch (the whole file is known upfront). Whisper, original NeMo Conformer non-streaming.
Large lookahead (1000ms+): pseudo-streaming batch with large chunks. Whisper.cpp streaming mode, faster-whisper VAD-based.
Small lookahead (80-320 ms): native streaming. NeMo FastConformer streaming, streaming Conformer variants, Parakeet streaming, some Kaldi TDNN variants.
Zero lookahead: causal streaming. Fastest response but usually strictly worse accuracy.

The difference between "pseudo-streaming" and "native streaming" is fundamental but often confused. Both technically "stream." But native streaming is trained with a constraint to see only limited future audio, while pseudo-streaming is a hack on top of a batch model.

Whisper's approach: chunked batch retrofit

Whisper is trained on 30-second audio chunks. To get streaming behavior, developers use one of two workarounds.

Whisper.cpp streaming mode: rolling window of 500 ms to 3 s, each chunk is processed as a mini-batch, outputs are concatenated. It works, but has several side effects:

Chunk boundary artifacts: words at chunk borders can be cut, repeated, or misrecognized.
Latency equals chunk size plus inference time. On SD662 with Whisper base.en at 500 ms chunks, perceived latency is 700-1500 ms.
VAD and endpointing need separate tuning (Whisper has no native VAD).
Punctuation can be inconsistent between chunks (the model "loses context" at borders).

VAD-based streaming wrappers (e.g., Whisper-Streaming or WhisperLive built on faster-whisper): VAD triggers, the accumulated segment gets transcribed, output emits. Cleaner output (no chunk boundary artifacts), but latency is even higher (you have to wait for end of speech for VAD trigger).

Neither of these approaches delivers what native streaming does: continuous text stream with sub-100 ms lookahead. This is an architectural limitation, not a bug.

Whisper is not bad. Whisper is not for live. These are two different task categories.

Native streaming ASR: how it works

Native streaming architectures (NeMo FastConformer streaming, streaming Conformer, Parakeet streaming variants) work differently.

Cache-aware inference. The model is trained knowing it will see only limited future audio (typically 80-320 ms lookahead). Internal hidden states cache between chunks. No need to re-process previous audio for each new segment.

Continuous emission. The model emits tokens incrementally as speech arrives. No "chunk boundaries" in perceived output. Text simply prints in real time.

Trade-off exists, but minimal. Native streaming models are usually slightly less accurate than their batch counterparts (the limited future audio constraint reduces context understanding). But the gap is small, typically 0.2-0.5% WER on LibriSpeech test-clean.

Example architecture: NeMo FastConformer streaming (stt_en_fastconformer_hybrid_medium_streaming_80ms_pc by NVIDIA). Roughly 32 million parameters, hybrid decoder (RNN-T + CTC), 80 ms cache-aware lookahead, WER 3.267% on LibriSpeech test-clean with the RNN-T decoder (our WER-500 measurement, see footnote in the comparison table).

Similar variants:

Parakeet streaming: same NeMo family, different size variants.
Streaming Conformer: general architectural family, different implementations from Google, Microsoft, and NVIDIA.
Kaldi TDNN-F: old-school, but streaming, still used in Vosk.

If the task is live, these models deliver a qualitatively different experience than chunked Whisper.

Our implementation: VoxRT, a Rust runtime around NeMo streaming

We (VoxRT) built a Rust runtime that packages NeMo FastConformer streaming (the same cache-aware architecture described above) into ready-made SPM (iOS) and Gradle(Android) packages. The model is stt_en_fastconformer_hybrid_medium_streaming_80ms_pc from NVIDIA, ported into our .vxrt format without architectural changes (fp16 quantization). Our .vxrt format supports AES-256-GCM encryption for proprietary models, but the NeMo streaming ASR model here remains under CC-BY-4.0, not encrypted.

What this gives a developer compared to the "write your own wrapper" approach:

One-line integration in Gradle/SPM. No custom Python + ONNX bridge to write for JNI or Objective-C interop.
Symmetric API surface between iOS and Android. Kotlin and Swift code is nearly identical, same lifecycle pattern (init -> processPcm -> stop).
Native inference via Rust runtime. No PyTorch, no ONNX runtime dependency, minimal APK/IPA footprint (native binary ~424 KB stripped on Android, ~500 KB framework compressed on iOS).
Bundled Silero VAD for pre-filtering. No separate VAD integration to write, the 1.2 MB model works out of the box paired with ASR.

Honest trade-off: our runtime is proprietary (LICENSE-BINARY, Elephant Enterprises LLC), though the wrapper is Apache-2.0. The model is under CC-BY-4.0 (NVIDIA), which requires attribution in your app's credits. For commercial closed-source apps, no compliance issues. Pure GPL/AGPL projects will hit a license conflict with our proprietary runtime. In that case, writing your own wrapper around the open-source NeMo runtime is a better path.

From here on, all "streaming ASR" numbers in the post come from our implementation of this architecture. Whisper and Vosk numbers come from their upstream benchmarks (source cited).

Comparison table

Extended comparison versus the first post: added axes for streaming approach, chunk latency, and ready SDK availability. All numbers cited.

	whisper.cpp (base.en)	whisper.cpp (small.en)	Vosk (small-en-0.15)	VoxRT NeMo FastConformer streaming
Model type	Batch (encoder-decoder Transformer)	Batch	Streaming (Kaldi TDNN-F)	Native streaming (Conformer + cache)
Model size	~149 MB ¹	~489 MB ¹	40 MB (+ 1.6 GB for punctuation) ²	60.4 MB ³
WER (LibriSpeech test-clean)	4.27% ⁴	3.05% ⁴	9.85% ²	3.267% (RNN-T) / 4.895% (CTC) ⁵
Streaming approach	Chunked batch (~500 ms blocks)	Chunked batch	Native streaming	Native cache-aware (80 ms lookahead)
Latency pattern	500-1500 ms per chunk (persistent)	500-1500 ms per chunk	300-500 ms per chunk	1.12 s initial buffer, then continuous ~100-200 ms
Punctuation by default	✓ (in-model)	✓	✗ (needs +1.6 GB model)	✓ (in-model)
Ready iOS SPM package	✗ (only community `ggml-org/whisper.spm`)	✗	✗	✓
Ready Android Gradle package	✗	✗	✗	✓
License	MIT (code + weights)	MIT	Apache-2.0 (code + weights)	Apache-2.0 wrapper / proprietary runtime / CC-BY-4.0 model

¹ whisper.cpp README, ggml format disk size (source lists 142 MiB and 466 MiB, converted to MB).
² alphacephei.com/vosk/models, evaluation table.
³ voxrt-asr-models v0.1.2 downloads table.
⁴ HuggingFace model cards openai/whisper-base.en and openai/whisper-small.en, evaluation section.
⁵ Our WER-500 measurements on LibriSpeech test-clean.

Important latency nuance: whisper.cpp has persistent per-chunk latency (each chunk 500-1500 ms). VoxRT NeMo streaming has an initial buffer of 1.12 s (for cache warmup), then sustained latency of ~100-200 ms perceived. Meaning:

For short queries (under 3 s): whisper.cpp can feel faster (VoxRT's initial buffer is just overhead).
For longer live sessions (over 3 s): VoxRT streaming feels smoother (continuous), whisper.cpp remains "jerky."

Real numbers on mobile

Everything above is architectural comparison. But any dev reading this will ask one question: how heavy is this on the device?

VoxRT NeMo FastConformer streaming:

Snapdragon 662 (mid-range Android from 2020): RTF 0.302 (file replay) / 0.353 (live-mic). That is 300-350 milliseconds of CPU time per 1 second of audio.
iPhone 13 Pro Max (Apple A15): RTF 0.08-0.10. Processor is roughly 10× faster than real time.
Runtime memory: ~150 MB steady-state.
APK/IPA impact: 60.4 MB (model) + ~500 KB (native framework) = ~61 MB added to app size.

VoxRT Silero VAD (used as pre-filter):

SD662: RTF 3.05%, latency ~1 ms per 32-ms frame.
iPhone A15: RTF 1.85%, latency ~0.6 ms.
Model size: 1.2 MB.

Whisper.cpp benchmarks on mobile: no public RTF numbers for Snapdragon or iOS in the whisper.cpp README, only Apple Silicon benchmarks. Community measurements (posts in r/androiddev, r/iOSProgramming) usually put Whisper base.en RTF in the 40-70% range on mid-range Android SoCs from 2020-2022. That is roughly 2× heavier than VoxRT NeMo for comparable accuracy.

Practical conclusion: for always-on live transcription on mid-range Android, Whisper base.en often burns enough CPU to noticeably heat up the device and drain battery. Streaming ASR under 40% RTF is sustainable without visible performance impact.

Decision framework: when to switch, when to stay

Practical rules for the reader.

Stay with Whisper if:

Batch transcription of files (audio, wait, transcript). Live UX not required. Whisper accuracy is great, community is huge, plenty of ready tools.
Short one-shot voice notes (user hits record, speaks 10 seconds, releases, gets transcript). Streaming ASR initial buffer is not justified here.
You do not care about battery or CPU load (e.g., transcription runs only on-demand, not always-on).
You want maximum community support, example code, and integrations.

Switch to streaming ASR if:

Live voice agents (chatbot that responds while user speaks). Every 500-1500 ms delay equals broken conversation.
Real-time captions for accessibility (hearing-impaired users). Here latency directly impacts UX.
Live meeting transcription with on-screen text visible as people speak.
Always-on background listening (voice notes 24/7, wake-word context). CPU load matters.

Go with a hosted API (Deepgram, AssemblyAI, etc.) if:

Concurrency: thousands of parallel sessions. Self-hosted infrastructure becomes overhead.
Diarization required out of the box (who is speaking when). Native streaming ASR usually has no built-in diarization, you need a separate model (pyannote and similar).
Multi-language mixed audio. Streaming models are usually English-focused, hosted APIs have multilingual support.
Budget allows and privacy is not a hard requirement.

Go with self-hosted streaming ASR (native streaming architecture) if:

Privacy is required (data cannot leave device or your infrastructure).
Mobile deployment: apps must work offline.
Scale exists, but budget matters. Hosted APIs get expensive at high volume.
You want low sustained latency without a hosted API dependency.

Wrapping up

Streaming vs batch in ASR is not a runtime tweak, it is an architectural choice. Whisper for batch is a valid default. For live it is a mismatch. Native streaming ASR (NeMo FastConformer, streaming Conformer, Parakeet variants) is a different tool for a different job.

Three things worth remembering when picking a stack:

Latency pattern matters more than absolute latency. Persistent 500 ms per chunk (Whisper chunked) subjectively feels worse than 1 s initial buffer + continuous ~100 ms (native streaming).
Punctuation by default is critical for UX. Whisper and native streaming provide it. Vosk requires a separate 1.6 GB model.
Ready-made mobile SDK. The only project that ships symmetric SPM + Gradle packages for streaming ASR is us (VoxRT). Whisper.cpp has a community SPM (ggml-org/whisper.spm) but no Gradle. Vosk offers only bindings, wrap yourself.

We build a Rust runtime for on-device voice models. Our ASR repos: github.com/orgs/VoxRT/repositories?q=asr+sort:stars. If you are building a live voice app for iOS, Android, or Linux, this stack plus a native streaming model give you everything needed.

Our first post covered building a full on-device voice notes pipeline (VAD + ASR + local storage) with working Kotlin and Swift snippets: dev.to/voxrtio/building-private-voice-notes-for-ios-and-android-without-the-cloud-54a8.

Learn more about the runtime and product family: voxrt.com.

Attribution: the streaming ASR model is a derivative of NVIDIA NeMo stt_en_fastconformer_hybrid_medium_streaming_80ms_pc, released under CC-BY-4.0.