DEV Community

Cover image for 14 voice commands on a $15 Raspberry Pi Zero 2 W (no cloud, no ASR)
VoxRT
VoxRT

Posted on

14 voice commands on a $15 Raspberry Pi Zero 2 W (no cloud, no ASR)

Why KWS instead of full ASR

For most DIY voice-control projects, a full open-vocabulary ASR model is overkill. If you want to control a lamp, a slideshow, a media player, or a simple UI by voice, you do not need a model that can transcribe arbitrary speech. You need to reliably recognize 10 to 20 specific commands.

That is the job of KWS (keyword spotting), a separate class of models trained on a fixed closed set of commands. Trade-offs versus full ASR:

  • Model size. Around 1 MB versus 60 MB or more for streaming ASR.
  • CPU footprint. One third of a core on a Pi Zero 2 W versus full-load or overload for streaming ASR on that hardware.
  • Accuracy on target vocabulary. Higher than ASR on the same words because the confusion space is smaller.
  • Trade-off. Vocabulary is fixed. Custom words require retraining.
  • Latency. Tens of milliseconds versus hundreds for streaming ASR.

For voice-control projects where the command set is small and known in advance, KWS wins on every axis except vocabulary flexibility.

This article is a tutorial for KWS on a Pi Zero 2 W ($15 hardware, 1 GHz A53). We cover the 14 supported commands, basic detection code, two practical integration examples (GPIO LED and HTTP webhook dispatcher), and honest KWS limits.

All numbers are verified against the voxrt-kws-linux release artifacts.

The 14 commands

The KWS model supports 14 phrases, grouped by purpose:

Confirmation and control (3):

  • yes: confirm
  • no: deny
  • cancel: cancel action

Media playback (5):

  • play: start playback
  • pause: pause
  • next: next track or slide
  • previous: previous
  • back: back navigation

Direction (2):

  • up: up (volume, scroll)
  • down: down

Switch (2):

  • on: turn on
  • off: turn off

Wake-like (2):

  • voxrt: brand trigger phrase
  • hey_vox: alternative trigger phrase

Twelve functional commands cover most DIY use cases: media control, UI navigation, binary switches, confirmation flows. The two wake-like commands can serve as an entry trigger for multi-step flows (first hey_vox, then a command).

The modelled vocabulary is fixed. If you need the word "lights", it is not in the vocab and will not be recognized. Workaround: map semantically (on and off to a lamp), or wait for custom training as a feature.

Hardware setup

What you need:

  • Raspberry Pi Zero 2 W ($15, 1 GHz quad-core A53, 512 MB RAM, see raspberrypi.com/products/raspberry-pi-zero-2-w)
  • USB microphone (see mic quality note below)
  • MicroSD card 8 GB or more with Raspberry Pi OS Lite (32-bit)
  • Power supply 5V 2.5A
  • Optional: GPIO breadboard, LED, and 220 Ohm resistor for the hardware integration example

Mic quality matters more than you'd expect. A $5 USB lavalier works if the speaker is 0.5 to 1 meter away in a quiet room. For reliable far-field detection (2+ meters) or noisy rooms, consider a ReSpeaker 4-mic array or Matrix Voice ($35 to $80 extra). Cheap single-element USB mics have poor AGC and no beamforming, which measurably hurts KWS recall in realistic conditions.

Why Pi Zero 2 W:
It is the cheapest Pi that handles the KWS workload in realtime. The original Pi Zero is too weak (single-core 1 GHz, 32-bit ARM11, no NEON SIMD). Pi 3, 4, and 5 run faster but cost $35 to $80. Pi Zero 2 W is the sweet spot for always-on voice-control projects where price and power efficiency matter.

OS setup:
Flash the standard Pi image through Raspberry Pi Imager. After boot, configure Wi-Fi via sudo raspi-config and enable SSH. For audio, arecord -l should show the USB mic as a card and the device index (card number). Note that index for the next step.

Install the SDK and hello world

System dependencies first (fresh Bookworm Lite does not have portaudio headers):

sudo apt update
sudo apt install -y portaudio19-dev python3-dev python3-venv
Enter fullscreen mode Exit fullscreen mode

Then the Python package:

# Create venv (required on Bookworm 2023+ per PEP 668)
python3 -m venv ~/kws-venv
source ~/kws-venv/bin/activate

# Install the SDK (Python 3.9 to 3.13 supported, auto-selects arch wheel)
pip install voxrt-kws
pip install pyaudio

# Download the model file from the voxrt-kws-models repo (~1.3 MB)
wget https://github.com/VoxRT/voxrt-kws-models/releases/latest/download/voxrt_kws.vxrt
Enter fullscreen mode Exit fullscreen mode

Hello world: detect and print:

import pyaudio
from voxrt_kws import KwsEngine

# Load model (single argument)
engine = KwsEngine.from_path("voxrt_kws.vxrt")

# Configure as attributes (mutable at any time, change safely between chunks)
engine.threshold = 0.9             # 0.0-1.0, higher = stricter
engine.consecutive_frames = 3      # 120 ms sustained before fire
engine.cooldown = 25               # ~1 s between repeated emits

# Audio capture setup: 16 kHz mono PCM int16, 512-sample chunks
# Replace MIC_INDEX with the card number from `arecord -l`
MIC_INDEX = 1

p = pyaudio.PyAudio()
stream = p.open(
    format=pyaudio.paInt16,
    channels=1,
    rate=16000,
    input=True,
    input_device_index=MIC_INDEX,
    frames_per_buffer=512,
)

print("Listening... say one of the 14 commands")
try:
    while True:
        pcm = stream.read(512, exception_on_overflow=False)
        # push_pcm_i16 is a generator. Iterate to consume detections.
        for d in engine.push_pcm_i16(pcm):
            print(f"heard: {d.class_name} (score {d.score:.2f})")
finally:
    stream.stop_stream()
    stream.close()
    p.terminate()
Enter fullscreen mode Exit fullscreen mode

Run with python hello_kws.py. Say yes and you see heard: yes (score 0.95) or similar in the terminal. Latency on my Pi Zero 2 W is around 150 ms end-to-end from end of pronunciation to the output line (120 ms consecutive-frame buffer plus inference and output).

Important notes:

  • MIC_INDEX. PyAudio defaults to the system default device, which on Pi is often the onboard audio (broken for mic input). Always set input_device_index explicitly from arecord -l output.
  • Audio format is strict. 16 kHz mono, PCM int16 signed little-endian, 512-sample chunks (32 ms). Other formats require resampling via SoX, libsndfile, or similar.
  • exception_on_overflow=False silently drops audio on overrun. On Pi Zero 2 W under system load, this can fire occasionally. For debugging, change to True to see when.

Integration example 1: GPIO LED control

The most maker-friendly example is turning an LED on and off by voice through GPIO. For real use, swap the LED for a relay module and the relay drives a lamp, fan, or anything else on mains power.

Wiring:

  • GPIO 17 to a 220 Ohm resistor to the LED anode
  • LED cathode to GND

Install gpiozero:

pip install gpiozero
Enter fullscreen mode Exit fullscreen mode

Code:

import pyaudio
from gpiozero import LED
from voxrt_kws import KwsEngine

MIC_INDEX = 1  # CHANGE THIS to card number from `arecord -l`

led = LED(17)
engine = KwsEngine.from_path("voxrt_kws.vxrt")
engine.threshold = 0.9

p = pyaudio.PyAudio()
stream = p.open(
    format=pyaudio.paInt16, channels=1, rate=16000,
    input=True, input_device_index=MIC_INDEX, frames_per_buffer=512,
)

try:
    while True:
        pcm = stream.read(512, exception_on_overflow=False)
        for d in engine.push_pcm_i16(pcm):
            if d.class_name == "on":
                led.on()
                print("LED on")
            elif d.class_name == "off":
                led.off()
                print("LED off")
finally:
    # try/finally guarantees cleanup even on unexpected exceptions
    stream.stop_stream()
    stream.close()
    p.terminate()
    led.off()
Enter fullscreen mode Exit fullscreen mode

Done. Say on and the LED lights up. Say off and it goes dark. The other 12 commands are ignored (not mapped in the handler).

Scaling to real hardware:

  • Replace LED with OutputDevice for a 5V relay module
  • Multiple devices with a dict mapping like {"on": [lamp_relay.on, speaker.on], "off": [...]}
  • Scenes by combining commands (play turns on lights and starts music via a sequence of actions)

Integration example 2: HTTP webhook dispatcher

More flexible than hardware control is a dispatcher that fires HTTP requests to your home automation server. This works on Pi OS Lite (headless, no X11 needed) and adapts to Home Assistant, OpenHAB, Node-RED, or any custom webhook endpoint.

Install requests:

pip install requests
Enter fullscreen mode Exit fullscreen mode

Code:

import pyaudio
import requests
from voxrt_kws import KwsEngine

MIC_INDEX = 1  # CHANGE THIS to card number from `arecord -l`

# Point to your home automation webhook (adapt URL/payload to your system)
WEBHOOK_URL = "http://your-automation-server:8080/voice"

def trigger(action):
    try:
        requests.post(WEBHOOK_URL, json={"action": action}, timeout=1.0)
    except requests.exceptions.RequestException:
        pass  # non-blocking, next command still fires

ACTIONS = {
    "on": lambda: trigger("lights_on"),
    "off": lambda: trigger("lights_off"),
    "play": lambda: trigger("music_play"),
    "pause": lambda: trigger("music_pause"),
    "next": lambda: trigger("music_next"),
    "previous": lambda: trigger("music_prev"),
    "up": lambda: trigger("volume_up"),
    "down": lambda: trigger("volume_down"),
}

engine = KwsEngine.from_path("voxrt_kws.vxrt")
engine.threshold = 0.9

p = pyaudio.PyAudio()
stream = p.open(
    format=pyaudio.paInt16, channels=1, rate=16000,
    input=True, input_device_index=MIC_INDEX, frames_per_buffer=512,
)

try:
    while True:
        pcm = stream.read(512, exception_on_overflow=False)
        for d in engine.push_pcm_i16(pcm):
            action = ACTIONS.get(d.class_name)
            if action:
                action()
                print(f"triggered: {d.class_name}")
finally:
    stream.stop_stream()
    stream.close()
    p.terminate()
Enter fullscreen mode Exit fullscreen mode

This pattern works with any HTTP-reachable automation layer. For Home Assistant you POST to /api/services/light/turn_on with a Bearer token. For MQTT instead of HTTP, swap requests.post for paho.mqtt.publish.single. The dispatcher dict keeps the mapping readable and easy to extend from YAML or JSON config.

Two things to know. First, only 8 of the 14 commands are mapped above (cancel, back, yes, no, voxrt, hey_vox fall through silently). Add them to ACTIONS to handle more. Second, requests.post blocks the loop for up to the timeout. If your webhook endpoint is slow, wrap it in a thread or queue to avoid audio buffer drops while waiting.

Benchmarks on Pi Zero 2 W

Measurements on my Pi Zero 2 W (1 GHz A53, 32-bit Raspberry Pi OS Bookworm):

Metric Value
RTF (CPU usage for realtime) 32.2% of one A53 core
Inference latency per chunk around 10 ms (p99 13 ms)
End-to-end detection latency 120 ms (3 consecutive frames at 32 ms) plus output time
Model file on disk 1.3 MB (fp16, encrypted .vxrt)
Power under KWS load approximately 1 W (community-measured, varies with mic and setup)

RTF 32.2% on one core means you can run KWS 24/7 on a Pi Zero 2 W and still have most of the quad-core available for your app logic (relay control, HTTP server, display rendering). For $15 of hardware and roughly 1 W of power, you get honest realtime always-on voice control.

For comparison: full streaming ASR on the same hardware does not fit (CPU overload). Wake-word only (covered in a previous article about Pi Zero) takes 5.3% on the same core but is limited to a single trigger. KWS is the middle ground.

Battery estimate: at roughly 1 W total draw, a 10000 mAh USB power bank (around 37 Wh usable after conversion losses) runs the setup for approximately 30 hours. Enough for multi-day field deployments if you add solar charging or battery swap.

Honest KWS limits

Where KWS fails and when to pick a different architecture:

1. Fixed vocabulary. 14 commands is the ceiling. If a project needs "turn on the bedroom light", the word "bedroom" is not in the vocab and will not be recognized. Workaround: a two-step flow (hey_vox as a trigger for an ASR path that handles complex phrases). Note that the handoff between KWS and ASR requires custom orchestration (audio buffer carryover, ASR instantiation timing, VAD gating). It is not a prebuilt SDK feature today.

2. Far-field performance. KWS works best at close range (0.5 to 1 meter from the mic). Beyond 2 meters accuracy drops noticeably. For room-wide voice control, use a better mic array (ReSpeaker 4-mic at ~$35 or Matrix Voice at ~$70) or place the mic closer.

3. Background noise. The default threshold of 0.9 is tuned for quiet environments. In a room with TV or music in the background you have to raise the threshold (fewer false positives, more missed detections) or add a VAD pre-gate to filter non-speech sections.

4. Accent variability. The model was trained primarily on American English pronunciations. Non-native English speakers and strong regional accents can see noticeably lower recall. Pronunciation tweaks (saying pause more like "pawz") sometimes help marginally but are not a real fix. Realistic options: raise or lower the threshold per-speaker, accept degraded recall, or wait for custom training support.

5. No custom commands currently. If a project needs specific words outside the 14-list, custom training is required and that is not shipping as a public SDK feature today. Options: stay inside the 14-list, use yes and no and cancel plus an ASR path for complex interactions, or look at vendor options like Picovoice Rhino intents, Snips, or Rhasspy.

When KWS versus alternatives

A short decision guide:

Pick KWS if:

  • The project needs up to 14 fixed commands
  • Pi Zero or similar low-power hardware
  • Always-on operation matters (CPU budget is tight)
  • Latency under 150 ms is a requirement

Pick wake-word (not KWS) if:

  • You only need 1 trigger phrase (not 14)
  • Hardware is even more constrained (microcontroller or original Pi Zero)
  • A 100 KB model is preferred over 1.3 MB

Pick streaming ASR if:

  • Vocabulary is open or over 20 words
  • Arbitrary user phrases
  • Platform is at least Pi 4 or mobile

Pick a KWS plus ASR hybrid if:

  • You need both simple commands (fast path via KWS) and complex phrases (fallback via ASR)
  • Hardware allows both (Pi 4 and up)
  • You are willing to build the orchestration code yourself (not a prebuilt feature)

What to build next

This tutorial is a minimal setup. From here you can extend:

  • Multi-room setup: several Pi Zero units publishing commands via MQTT to a central controller
  • Voice-controlled dashboard: display state plus voice navigation for kiosk mode (requires Pi OS Desktop)
  • Accessibility tool: voice shortcuts for users with mobility limitations
  • Hands-free lab setup: voice commands for photo capture (3D scanning rigs), timer start/stop (chemistry experiments)
  • Battery-powered portable device: Pi Zero 2 W plus a USB power bank plus a USB mic gives around 30 hours on a 10000 mAh pack

All code from this tutorial is available in the examples folder of voxrt-kws-linux. If you build something interesting, drop a comment with what you made.

Top comments (13)

Collapse
 
crtdrops profile image
Israel Negrete Lepe •

Thanks for putting the p99 next to the per-chunk latency; that's the number I look for first. Was it measured over a long always-on run? That's where temperature catches up. On a Pi 5 in my lab, a 30-minute sustained inference run hit the soft temperature limit at minute 15 (vcgencmd get_throttled flagged it), and yet latency only moved from 127.9 to 128.8 ms. The flag and the slowdown weren't the same thing, so now I log both.

A Pi Zero 2 W in a small case running 24/7 may behave differently. Mine idles at 44–47 °C with no throttle flags, but I haven't put a sustained load on it yet. Have you logged get_throttled alongside the RTF during a long run?

Collapse
 
voxrtio profile image
VoxRT •

Good catch on the flag/throttle decoupling. Honest answer: no, I didn't log get_throttled alongside the RTF in the article. The 32.2% figure comes from our benchmark run, likely short duration not sustained. I don't have duration metadata for that specific number.

For sustained always-on, your approach of logging both signals is smarter. On the Pi 5 you tested, the flag firing at minute 15 with latency barely moving sounds like soft-limit scaling at 80 C (bit 3), which applies mild clock reduction before the hard throttle at 85 C where latency actually moves. On Pi Zero 2 W you don't have a stock heatsink, the PCB is much smaller with less copper heatspreading, and RAM is PoP-stacked on top of the SoC trapping heat. A small sealed case also kills convection. All of that likely narrows the gap between flag and actual latency hit vs what you see on Pi 5.

If you run a long sustained test on your Pi Zero 2 W, would love to see the data. 32.2% RTF looks like a lot of headroom on paper, but headroom disappears quickly without airflow.

Collapse
 
crtdrops profile image
Israel Negrete Lepe •

Thanks for the straight answer; a short run is still a useful number as long as it says it's short.

The Zero 2 W details make sense: no heatsink, PoP RAM on top of the SoC and a sealed case all push the throttle point closer. I'll run a sustained test on mine, logging get_throttled, temperature and RTF together, and post the numbers here.

Thread Thread
 
voxrtio profile image
VoxRT •

Looking forward to seeing those numbers. If possible, decoding the bitmask into specific bits (bit 2 hard throttle, bit 3 soft limit) rather than just checking non-zero would help separate the two kinds of events. That distinction is where your Pi 5 observation came from.

Thread Thread
 
crtdrops profile image
Israel Negrete Lepe •

Will do. I'll log the raw value and decode it in two halves: bits 0–3 for what is happening at the moment of the sample, and bits 16–19 for what has happened since boot. A sampler that reads every few seconds can miss a short throttle event; the sticky bits can't, so a run won't come back clean just because the sample landed at the wrong moment.

Thread Thread
 
voxrtio profile image
VoxRT •

The sticky-bit split is better than what I suggested. Looking forward to the numbers.

Thread Thread
 
crtdrops profile image
Israel Negrete Lepe •

Numbers from today, Pi 5 8 GB with a fan heatsink in a metal case, YOLO26n on ONNX Runtime, one process for 30 minutes, get_throttled read once a second:

  • Soft temperature limit: the sticky bit 19 set at 7.8 min, at 80.7 °C. The first sample that caught bit 3 live came a minute later, and from then on it flickered with the temperature, on in about 60% of the samples.
  • Actual throttling: sticky bits 17 and 18 set at 17.5 min. Bits 1 and 2 were live in only 3 of 1,747 samples, the first one 4.4 minutes after the sticky bits said it had already happened. Reading only the low half, this run would look almost clean. Your split is what made it visible.
  • Linux reported 2,400 MHz the whole time; the firmware's clock went as low as 1,500 MHz. p50 went from 128.7 to 133.3 ms.

One thing I didn't expect: the same model as four single-thread processes topped out at 81.8 °C and never set a bit. The one-process run is the hot one.

Thread Thread
 
voxrtio profile image
VoxRT •

Thank you for running this. The 3 of 1,747 is the ratio I will be citing from now on when someone asks why sticky bits matter. The firmware 1,500 MHz while Linux reported 2,400 is the other half of that lesson: Linux-level monitoring cannot catch what lives in the VideoCore firmware.

This changes how we benchmark Pi going forward. Adding get_throttled sticky logging to our Pi benchmark script and re-running Pi Zero 2 W in sustained mode is on our list. If the numbers we published for always-on workloads do not hold up to 30+ min runs with thermal monitoring, we will update them.

The multi-process result is interesting too and probably worth its own test on our side.

Thread Thread
 
crtdrops profile image
Israel Negrete Lepe •

Glad it helps. Since you're re-running the Zero 2 W in sustained mode anyway: I have one here too. If a second data point is useful, I can run voxrt-kws-linux with the 14-command model for 30 minutes straight, with the same logger (temperature, the firmware's clock next to the one Linux reports, get_throttled once a second), and send you the numbers. Whether they get published is your call. No cost: I'd like Zero 2 W data next to the Pi 5 run anyway.

Thread Thread
 
voxrtio profile image
VoxRT •

Thanks, yes please. voxrt-kws-linux with the 14-command model is the exact setup behind our 5.3% RTF claim, so your sustained run would cover the case our bench did not. We will run our own sustained test and update if the numbers need it.

Thread Thread
 
crtdrops profile image
Israel Negrete Lepe •

Great, I'm setting it up on the Zero 2 W today: voxrt-kws-linux with the 14-command model, 30 minutes straight, logging temperature, the firmware clock next to the one Linux reports, and get_throttled once a second. I'll post the numbers here so you can put them next to your own sustained run.

Thread Thread
 
crtdrops profile image
Israel Negrete Lepe •

Numbers from today. Zero 2 W as a bare board, no case and no heatsink, 64-bit Raspberry Pi OS, voxrt-kws 0.1.0 from PyPI with voxrt_kws.vxrt from your models release. Your bench method (silence in 512-sample chunks after a 100-chunk warmup, one thread, as fast as it goes), but for 30 minutes instead of 60 s, with the logger once a second, 2 minutes idle before and 5 of cool-down after.

  • RTF 32.9% over the whole run (170,638 chunks), and flat: 32.90 to 32.92% in each of the six 5-minute blocks. Your 32.2% reproduces within a point.
  • Per chunk: mean 10.5 ms, p99 13.4 ms, worst 14.6 ms against the 32 ms budget. Four chunks take about 13 ms, then one takes 0.6 ms.
  • Temperature: 47.2 °C idle, 60 °C by minute 10, peak 61.8 °C, 49.9 °C after cool-down.
  • get_throttled: 0x0 in every sample, sticky bits included, so nothing happened between samples either.
  • The firmware clock and the one Linux reports agreed the whole time: 1,000 MHz, never more than 1 MHz apart.

So on this board the 14-command model holds 30 minutes at full load without reaching the soft limit, and the number doesn't drift. One board, one run, office ambient not logged. One detail: your last comment says 5.3% for this setup, but your article and README give 32.2% for the 14-command model (5.3% is the wake-word post), and 32.2% is the one I reproduced.

Thread Thread
 
voxrtio profile image
VoxRT •

You're right, I mixed up 5.3 (wake-word) and 32.2 (14-command KWS) in the previous comment. Thanks for the sustained run. Your 32.9% reproduces 32.2% within a point, and sticky bits staying 0x0 for 30 minutes answers the question short runs couldn't.