DEV Community: VoxRT

Wake word + voice commands in the browser: a full offline pipeline

VoxRT — Mon, 21 Sep 2026 06:20:28 +0000

Voice interfaces in the browser have always had the same friction: Web Speech API is patchy across browsers, ships your audio to a vendor, and often needs network to respond. If you want a private, offline pipeline that works the same everywhere, you have had to wire it yourself. That got easier when WebAssembly SIMD128 shipped in every modern browser.

A few weeks ago I wrote a piece on adding a wake word to a web app in five minutes. This is the natural next step. A wake word alone tells you when the user wants attention. To do anything useful, you also need to know what they said. This post wires up the second half: a 14-command keyword spotter that activates on wake and dispatches commands to your app. Everything runs on-device, in about 1.75 MB of assets total, at roughly 1% of one CPU core on a modern laptop.

What you will need

A modern browser: Chrome/Edge 91+, Firefox 89+, Safari 16.4+. WASM SIMD128 is required.
A microphone.
Any static file server on HTTPS or localhost. Browsers block getUserMedia on plain HTTP.

The pattern: cheap wake, activated commands

You could run keyword spotting continuously and skip the wake word. Don't. Two reasons.

First, wake-word models are trained for one job: sit in a hot loop for hours, fire on a single phrase, ignore everything else. They are tiny (100 KB of weights, ~275 KB total bundle) and hardened against false triggers from radio, TV, and background conversation. Command spotters are heavier (~1.47 MB with a 14-word vocabulary) and, running always-on, will occasionally fire when someone in the room says "play" or "next" without meaning to talk to your app.

Second, the split matches how users actually interact. A wake word says "I am about to talk to you." A command says what to do. Gating the command recognizer on the wake cuts false triggers and CPU when the user is not addressing your app.

The pattern:

[Microphone]
     ↓
[Wake word engine] ── always on, ~275 KB, ~0.16% RTF
     ↓ (fires on "Hey Assistant")
[Listening window ~3 seconds]
     ↓
[KWS engine] ── activated during window, ~1.47 MB, ~0.97% RTF
     ↓
[Command: play / pause / next / ...]
     ↓
[Your app]

Both engines read the same 16 kHz mono PCM stream from the microphone. The wake engine sees every frame. The KWS engine only sees frames inside a listening window opened by the wake trigger. When a command fires (or the window times out), listening closes and the pipeline goes quiet again.

Step 1: install the SDKs

npm install @voxrt/wake-word-browser @voxrt/kws-browser

Or via CDN if you skip the bundler:

import initWake, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";
import initKws, { KwsEngine } from
  "https://unpkg.com/@voxrt/kws-browser@0.1.0/voxrt-kws-browser.js";

Both packages ship two things: a WebAssembly runtime (.js + .wasm) and a .vxrt model file. The model files download separately at runtime and get memory-mapped by the engine.

Step 2: HTML shell

Minimum needed to show what is happening:

<!DOCTYPE html>
<html>
<head><meta charset="utf-8"><title>Voice control demo</title></head>
<body>
  <button id="start">Start listening</button>
  <p id="status">Idle</p>
  <p>Last command: <span id="last">none</span></p>
  <script type="module" src="./app.js"></script>
</body>
</html>

Serve over HTTPS or from localhost.

Step 3: initialize both engines

Create app.js:

import initWake, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";
import initKws, { KwsEngine } from
  "https://unpkg.com/@voxrt/kws-browser@0.1.0/voxrt-kws-browser.js";

await Promise.all([initWake(), initKws()]);

const [wakeBytes, kwsBytes] = await Promise.all([
  fetch("https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt_wake_word.vxrt")
    .then(r => r.arrayBuffer()).then(b => new Uint8Array(b)),
  fetch("https://unpkg.com/@voxrt/kws-browser@0.1.0/voxrt_kws.vxrt")
    .then(r => r.arrayBuffer()).then(b => new Uint8Array(b)),
]);

const wake = WakeWordEngine.fromBytes(wakeBytes);
const kws = KwsEngine.fromBytes(kwsBytes);

Two engines, two model files, both loaded before we touch the microphone. Downloads happen in parallel. On a warm cache the whole init is under 100 ms.

Default thresholds are 0.9 in sigmoid space for both. Lower them to increase sensitivity (more false accepts). Raise them to be pickier. We will tune later if needed.

Step 4: capture microphone audio

Both engines want 16 kHz mono. We ask AudioContext for it and verify, because Firefox and Safari sometimes ignore the hint:

document.getElementById("start").onclick = async () => {
  const stream = await navigator.mediaDevices.getUserMedia({audio: true});
  const audioContext = new AudioContext({sampleRate: 16000});
  if (audioContext.sampleRate !== 16000) {
    console.warn(`Got ${audioContext.sampleRate} Hz. Need 16000. Resample or reject.`);
    return;
  }
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(512, 1, 1);
  const mute = audioContext.createGain();
  mute.gain.value = 0;

  // detection loop goes here (Step 5)

  source.connect(processor);
  processor.connect(mute).connect(audioContext.destination);
  document.getElementById("status").textContent = "Listening for wake word";
};

Two gotchas from the wake-word post still apply here:

ScriptProcessorNode is deprecated. Fine for a demo, swap for AudioWorkletNode in production.
The muted GainNode between processor and destination is a Web Audio quirk. Without connecting the processor to the graph, audioprocess never fires. Muting the gain prevents the microphone from being routed straight to the speakers.

Step 5: detection loop with a listening window

Here is the actual logic. The wake engine sees every frame. On a wake hit, we open a 3-second window during which frames also go to the KWS engine. First command detection closes the window early.

const LISTEN_WINDOW_MS = 3000;

const pcmBuffer = new Int16Array(512);
let listeningUntil = 0; // performance.now() timestamp; 0 means closed

processor.addEventListener("audioprocess", (e) => {
  const inputFloat = e.inputBuffer.getChannelData(0);

  // Float32 [-1, 1] to Int16
  for (let i = 0; i < inputFloat.length; i++) {
    const s = Math.max(-1, Math.min(1, inputFloat[i]));
    pcmBuffer[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
  }

  // Wake word always runs
  for (const hit of wake.pushPcmI16(pcmBuffer)) {
    console.log(`Wake at ${hit.timestampSec.toFixed(3)}s, score ${hit.score.toFixed(3)}`);
    listeningUntil = performance.now() + LISTEN_WINDOW_MS;
    document.getElementById("status").textContent = "Listening for command...";
  }

  // KWS only runs while the listening window is open
  if (performance.now() < listeningUntil) {
    for (const command of kws.pushPcmI16(pcmBuffer)) {
      if (command.class === "hey_vox" || command.class === "voxrt") {
        // Also in KWS vocab. Treat as a second wake: re-arm the window, do not dispatch.
        listeningUntil = performance.now() + LISTEN_WINDOW_MS;
        continue;
      }
      console.log(`Command "${command.class}" score ${command.score.toFixed(3)}`);
      document.getElementById("last").textContent = command.class;
      dispatch(command.class);
      listeningUntil = 0; // close window after first command
      document.getElementById("status").textContent = "Listening for wake word";
    }
  } else if (listeningUntil !== 0) {
    // window expired without a command
    listeningUntil = 0;
    document.getElementById("status").textContent = "Listening for wake word";
  }
});

function dispatch(command) {
  switch (command) {
    case "play":  /* start media playback */  break;
    case "pause": /* pause media playback */  break;
    case "next":  /* skip to next item */     break;
    case "previous": /* previous item */      break;
    case "up":    /* volume up */             break;
    case "down":  /* volume down */           break;
    case "on":    /* enable something */      break;
    case "off":   /* disable something */     break;
    case "yes":   /* confirm */               break;
    case "no":    /* dismiss */               break;
    case "cancel": /* cancel current action */ break;
    case "back":  /* go back / navigate up */ break;
  }
}

One note on the vocabulary overlap: voxrt and hey_vox are both in KWS's 14-word set. Once the listening window is open, the user can say them and KWS will fire. The filter at the top of the loop treats those two as a "second wake" (re-arm the window) rather than a command. Without that filter, saying "hey vox" mid-listening would silently close the window on a no-op dispatch. Either behaviour is defensible, but the reader should see the choice.

That is the whole runtime. Say "Hey Assistant", then within 3 seconds say one of the 14 supported commands, and dispatch fires. Open DevTools (F12, Console) to see the timing and scores.

Numbers

Measured on a MacBook Pro M4 running Chrome, single-threaded WASM with SIMD128:

Wake word: RTF 0.16% (625× real time), always on
KWS: RTF 0.97% (103× real time), only during listening windows
Combined footprint: ~1% of one CPU core when actively listening, closer to 0.2% between windows

Model quality:

Wake word ("Hey Assistant"): ROC AUC 0.9966, precision 0.993 and recall 0.982 at threshold 0.9 on the reference test split (11,656 utterances)
KWS (14 commands): F1 macro 0.9671 at threshold 0.9 on a speaker-disjoint test set. Per-class ROC AUC is ≥0.99 for every command

Assets shipped to the browser:

Wake-word runtime + model: ~275 KB
KWS runtime + model: ~1.47 MB
Total pipeline: ~1.75 MB

That is smaller than the average site's hero image.

The 14-word vocabulary

Fixed in v0.1.0: yes, no, cancel, play, pause, next, previous, up, down, back, on, off, voxrt, hey_vox.

It covers three classes of interaction:

Confirmation and cancellation: yes, no, cancel, back
Media control: play, pause, next, previous
State toggles and navigation: up, down, on, off

Those 12 cover most "wake, then dispatch" flows: a recipe app in the kitchen, an accessibility layer for hands-free navigation, a dashboard that toggles panels by voice, a car UI. The remaining two (voxrt, hey_vox) exist as alternate wake-style phrases if you want a second entry point.

If you need product-specific words (custom brand names, unique verbs, domain jargon), the vocabulary is currently fixed at these 14. Custom command sets are a paid tier: help@voxrt.com.

Where this fits

Two-stage voice control fits a specific shape of use case:

Hands are busy or dirty. Kitchen recipe apps, workshop tool controllers, medical stations.
Accessibility. Users with motor limitations get an entry-level voice control layer that does not depend on Apple or Google account state.
In-car dashboards. Voice shortcuts on top of a touch UI, without shipping cabin audio to a cloud provider.
Kiosks and public terminals. No microphone stream leaves the machine.

If you need continuous transcription (arbitrary sentences, not fixed commands), you want streaming ASR instead. That is not in browser yet: our streaming ASR runtime is Android, iOS, and Linux only. In-browser ASR is a real question we get and the honest answer today is "not shipped."

Wrap up

Full pipeline, on-device, in ~1.75 MB. Two SDKs, one shared .vxrt runtime, no cloud, no per-request cost. Wake word optimized for always-on, command spotter activated on demand.

Repos and packages:

Wake word: github.com/VoxRT/voxrt-wake-word-browser. npm: @voxrt/wake-word-browser
KWS: github.com/VoxRT/voxrt-kws-browser. npm: @voxrt/kws-browser
Model files: served from unpkg alongside the JS, or from GitHub releases
VoxRT overview: voxrt.com

If you build something with this, the vocabulary limits (14 words, English) will be the first thing that pinches. That is next on our roadmap.

Building a fully offline voice assistant on Raspberry Pi 4

VoxRT — Mon, 14 Sep 2026 08:16:09 +0000

What we are building

A voice assistant that runs entirely on a Raspberry Pi 4. No cloud, no API keys, no per-minute pricing. The user says "Hey Assistant", the device wakes up, and then handles whatever was said next.

Two interaction paths:

The user says one of 14 fixed commands (turn on lights, play music, stop, and so on). The device recognises it instantly and runs the action.
The user says something outside that set (read me the news, what time does the store open). The device transcribes the speech into text and processes the open-ended query.

Three models work together. A wake-word detector listens to the microphone 24/7. It is cheap and always on. A keyword spotter takes over for the next couple of seconds and tries to match a fixed vocabulary. And a streaming ASR transcribes open-ended speech if the keyword spotter finds no match. It is heavier but only runs when actually needed.

All three install with three pip install calls and run on a $35 Raspberry Pi 4 without an internet connection. This article walks through the full Python code, shows equivalent snippets in Node.js, Go and C, and explains why splitting this into three SDKs on one runtime keeps the always-on cost low without giving up open-ended speech.

Why the triage pattern (WW then KWS then ASR)

The core idea is to avoid running the heaviest model until it is actually needed.

The wake-word detector is the lightest piece of the pipeline. Our model is about 100 KB and uses about 5.3% of one A53 core on a $15 Raspberry Pi Zero 2 W (24/7 sustained, per our earlier benchmarks, link below). On a Pi 4 the cost drops further because the Cortex-A72 cores are faster and run in 64-bit mode. That means the wake-word can listen to the environment continuously without stealing meaningful CPU from anything else on the device.

But the wake-word only knows one thing. Did the trigger phrase ("Hey Assistant" in our case) get said. To understand what the user actually wants, you need another model.

The obvious option is to fire up ASR as soon as the wake-word triggers. That works but it is overkill for most real requests. In a voice assistant, users usually say one of a small set of commands. Turn on lights. Set a timer. Play music. Running full open-vocabulary ASR on that is like using a regex when strcmp would do.

KWS (keyword spotter) solves this cheaply. Our model processes 16 kHz mono audio in 32 ms frames and classifies each into one of 14 fixed command classes. It does not transcribe arbitrary speech, but it does quickly answer "yes, this is command X" or "no match".

The triage logic looks like this.

WW listening state. Wake-word runs continuously, waiting for "Hey Assistant".
WW triggered. Switch to KWS. Give the user about 2 seconds to say a command.
KWS matched. Run the command action (turn on lights, stop, and so on). Go back to state 1.
KWS did not match. Fall back to ASR. Transcribe the open-ended speech until silence or a timeout.
ASR finished. Process the text (route to a local LLM, a parser, or send it to your backend). Go back to state 1.

In compute terms this is a switch with branches for known cases and a default for everything else. WW costs almost nothing all the time. KWS runs only after a wake-word (typically 1 to 2 seconds). ASR runs only when the command missed the KWS vocabulary. In practice, most assistant requests fall inside a small command vocabulary, though the exact fraction depends heavily on your product.

Hardware

Minimum setup:

Raspberry Pi 4 (4GB). Four gigabytes of RAM give comfortable headroom for the ASR model, which keeps its weights in memory. The 2GB variant will boot the whole pipeline but leaves little room for anything else running on the device. Pi 5 and Pi 3B+ also work, but we tested on Pi 4 since that is the most common Pi model developers reach for right now.
A USB microphone ($10 to $15, any class-compliant device) or an I2S mic HAT such as the ReSpeaker 2-Mics Pi HAT (about $10) for cleaner audio.
A 32GB microSD card with 64-bit Raspberry Pi OS (aarch64). The 64-bit build is required. Our Linux SDKs ship aarch64 wheels only, and the 32-bit armhf variant is not supported.
Optional: a speaker or headphones if you want to add a TTS response. TTS is in development. This tutorial does not cover it.

Wake-word alone (without the rest of the pipeline) works even on a Pi Zero 2 W. We tested that in an earlier article, wake-word on a $15 Pi Zero 2 W. Adding KWS and ASR bumps the compute budget, so this tutorial uses a Pi 4.

Installing the three SDKs

Each model ships as a separate package.

pip install voxrt-wake-word voxrt-kws voxrt-asr

The three libraries share the same VoxRT runtime under the hood. That runtime is a Rust inference engine with a C ABI. We do not ship a single voxrt-all package, because many integrators only need part of the pipeline (wake-word only for a kiosk device, ASR only for a dictation app) and carrying weights they never load makes no sense.

Model weights download separately as files.

# Wake-word "Hey Assistant"
curl -LO https://github.com/VoxRT/voxrt-wake-word-models/releases/download/v0.1.0/voxrt_wake_word.vxrt

# Keyword spotter (14 commands)
curl -LO https://github.com/VoxRT/voxrt-kws-models/releases/download/v0.1.0/voxrt_kws.vxrt

# Streaming ASR (medium PC model)
curl -LO https://github.com/VoxRT/voxrt-asr-models/releases/download/v0.1.2/streaming_medium_pc.vxrt

(Check the exact versions in each repo README. They update periodically.)

The Node.js, Go and C install options are covered in the "Same pipeline in other languages" section below.

Code: the triage pipeline in Python

The full example breaks into three parts.

Microphone capture. Shared across all three SDKs.
State machine. Controls which engine gets audio at any given moment.
Wiring. The event loop that puts it all together.

Audio capture

We use sounddevice (a PortAudio wrapper), which is the standard way to read a microphone in Python on Linux. It provides a callback-based API. The PortAudio audio thread calls our function with each new chunk of samples. We push them into a queue and the main thread drains it.

import sounddevice as sd
import numpy as np
from queue import Queue

SAMPLE_RATE = 16000   # all three models expect 16 kHz mono
CHUNK_FRAMES = 512    # 32 ms, matches WW and KWS internal chunk size

audio_queue: "Queue[np.ndarray]" = Queue(maxsize=64)

def on_audio(indata, frames, time_info, status):
    if status:
        print(f"audio status: {status}")
    # indata is (frames, 1) int16. Copy the buffer since sounddevice
    # reuses the same backing storage between callbacks.
    audio_queue.put(indata.copy())

stream = sd.InputStream(
    samplerate=SAMPLE_RATE,
    channels=1,
    dtype="int16",
    blocksize=CHUNK_FRAMES,
    callback=on_audio,
)
stream.start()

After this call, 32 ms blocks of int16 samples land in audio_queue steadily. The next question is which engine handles them at any given moment.

State machine

We keep one state enum and a main loop that pulls chunks off the queue and routes them into the right engine.

from enum import Enum

class State(Enum):
    IDLE = 1            # only WW is listening
    KWS_CHECK = 2       # WW fired, KWS is checking for 2 seconds
    ASR_TRANSCRIBE = 3  # KWS missed, ASR is transcribing

state = State.IDLE

Initialise the three engines

from voxrt_wake_word import WakeWordEngine
from voxrt_kws import KwsEngine
from voxrt_asr import AsrStreamingEngine, DecodeMode

ww = WakeWordEngine.from_path("voxrt_wake_word.vxrt")
ww.threshold = 0.9
ww.cooldown_frames = 100  # ignore 100 frames (~3.2 s) after a detection

kws = KwsEngine.from_path(
    "voxrt_kws.vxrt",
    threshold=0.9,
    consecutive_frames_required=3,
    cooldown_frames=25,
)
print(f"KWS commands: {kws.class_names}")

asr = AsrStreamingEngine.from_path(
    "streaming_medium_pc.vxrt",
    mode=DecodeMode.RNNT,
)

The main event loop

This is where the triage happens. Broken down by state:

import time
from queue import Empty

KWS_WINDOW_SEC = 2.0    # window after WW for a KWS attempt
ASR_SILENCE_SEC = 1.5   # silence at the end of an ASR utterance


def handle_command(name):
    print(f"running fixed command: {name}")
    # wire this into GPIO, home automation API, whatever you need


def handle_open_query(text):
    print(f"open-ended query: {text}")
    # send this to a local LLM, a parser, wherever


kws_check_deadline = 0.0
asr_buffer_f32 = []
asr_last_speech_time = 0.0

try:
    while stream.active:
        try:
            chunk_i16 = audio_queue.get(timeout=0.1)
        except Empty:
            continue

        samples_i16 = chunk_i16.flatten().tolist()

        if state == State.IDLE:
            # WW listens all the time.
            for det in ww.push_pcm_i16(samples_i16):
                print(f"[WW] wake detected, score={det.score:.3f}")
                state = State.KWS_CHECK
                kws_check_deadline = time.time() + KWS_WINDOW_SEC

        elif state == State.KWS_CHECK:
            # Try to match a fixed command in the next 2 seconds.
            matched = False
            for det in kws.push_pcm_i16(samples_i16):
                print(f"[KWS] command: {det.class_name} score={det.score:.3f}")
                handle_command(det.class_name)
                state = State.IDLE
                matched = True
                break

            if not matched and time.time() > kws_check_deadline:
                print("[KWS] no match, falling back to ASR")
                state = State.ASR_TRANSCRIBE
                asr_buffer_f32 = []
                asr_last_speech_time = time.time()

        elif state == State.ASR_TRANSCRIBE:
            # ASR expects f32 samples. Convert from int16.
            f32 = [s / 32768.0 for s in samples_i16]
            asr_buffer_f32.extend(f32)

            # ASR wants 200 ms chunks (3200 samples).
            while len(asr_buffer_f32) >= 3200:
                chunk_200ms = asr_buffer_f32[:3200]
                asr_buffer_f32 = asr_buffer_f32[3200:]
                partial = asr.push_audio(chunk_200ms)
                if partial:
                    print(f"[ASR partial] {partial}")
                    asr_last_speech_time = time.time()

            # If silence lasts more than 1.5 seconds, finalise.
            if time.time() - asr_last_speech_time > ASR_SILENCE_SEC:
                final = asr.stop()
                print(f"[ASR final] {final}")
                handle_open_query(final)
                # asr.reset() keeps the loaded weights and resets the streaming
                # state. Cheaper than re-loading ~150 MB via from_path().
                # If your voxrt-asr version does not expose reset(), fall back
                # to AsrStreamingEngine.from_path(...) here, but expect a
                # multi-second stall while weights re-load.
                asr.reset()
                state = State.IDLE
finally:
    stream.stop()
    stream.close()

One key detail. WW and KWS want int16 samples. ASR wants float32 normalised to [-1.0, 1.0]. The conversion is one line (s / 32768.0) but it is easy to forget. Chunk sizes also differ. 32 ms for WW and KWS. 200 ms for ASR (matching the ASR model's internal attention window).

This is not "one API for everything". It is "consistent integration pattern (from_path then push_something then read the result), with input format matched to each model's requirements". Wake-word and KWS have the same shape internally (both are event detectors on short chunks), so they share a signature. ASR works differently (it is a streaming decoder), so it has a different signature.

Same pipeline in other languages

Full quickstart examples for Node.js, Go and C are in the repos:

github.com/VoxRT/voxrt-wake-word-linux/tree/main/examples/{nodejs,go,c}/
github.com/VoxRT/voxrt-kws-linux/tree/main/examples/{nodejs,go,c}/
github.com/VoxRT/voxrt-asr-linux/tree/main/examples/{nodejs,go,c}/

Below are the critical snippets. How push_pcm_i16 and push_audio look in each language. Follow the repo links for the full runnable examples.

Node.js

const { WakeWordEngine } = require("@voxrt/wake-word");
const { KwsEngine } = require("@voxrt/kws");
const { AsrStreamingEngine, DecodeMode } = require("@voxrt/asr");

const ww = WakeWordEngine.fromPath("voxrt_wake_word.vxrt");
ww.threshold = 0.9;

// mic capture via any ALSA/PortAudio Node binding available on Pi.
mic.on("data", (buffer) => {
  // Buffer is a slice of a larger ArrayBuffer. Use byteOffset and
  // byteLength to build a view over just the samples we received.
  const samples = new Int16Array(
    buffer.buffer,
    buffer.byteOffset,
    buffer.length / 2,
  );
  for (const det of ww.pushPcmI16(samples)) {
    console.log(`[WW] score=${det.score}`);
    // switch state to KWS_CHECK here
  }
});

Go

import (
    "log"

    wakeword "github.com/VoxRT/voxrt-wake-word-linux/go"
    // kws     "github.com/VoxRT/voxrt-kws-linux/go"
    // asr     "github.com/VoxRT/voxrt-asr-linux/go"
)

engine, err := wakeword.OpenFromPath("voxrt_wake_word.vxrt")
if err != nil {
    log.Fatal(err)
}
defer engine.Close()

_ = engine.SetThreshold(0.9)
_ = engine.SetCooldownFrames(100)

// mic capture via any ALSA/PortAudio Go binding available on Pi.
for chunk := range micChan { // chunk is []int16 of length 512
    for _, d := range engine.PushPcmI16(chunk) {
        log.Printf("[WW] score=%.3f", d.Score)
        // switch state here
    }
}

C

#include <stdint.h>
#include <stddef.h>
#include <stdio.h>

#include <voxrt_wake_word.h>

// load_vxrt_model() mmaps the model file and returns (bytes, len).
// See examples/c/alsa-mic-quickstart/main.c in the repo for the exact
// implementation, about 20 lines around open() + fstat() + mmap().
// alsa_read_chunk() is a thin wrapper over snd_pcm_readi().

const uint8_t *model_bytes = NULL;
size_t model_len = 0;
if (load_vxrt_model("voxrt_wake_word.vxrt", &model_bytes, &model_len) != 0) {
    fprintf(stderr, "failed to mmap the wake-word model file\n");
    return 1;
}

voxrt_wake_word_t *engine = NULL;
voxrt_status_t rc = voxrt_wake_word_create(model_bytes, model_len, &engine);
if (rc != VOXRT_OK) {
    fprintf(stderr, "voxrt_wake_word_create failed: %d\n", (int)rc);
    return 2;
}
voxrt_wake_word_set_threshold(engine, 0.9f);
voxrt_wake_word_set_cooldown_frames(engine, 100);

// ALSA capture, 16 kHz mono int16, 512 samples per chunk.
int16_t chunk[512];
voxrt_wake_word_detection_t dets[8];
while (alsa_read_chunk(pcm_handle, chunk, 512) == 0) {
    size_t written = 0;
    rc = voxrt_wake_word_push_pcm_i16(engine, chunk, 512, dets, 8, &written);
    // VOXRT_ERR_BUFFER_TOO_SMALL just means more detections would fit;
    // the ones we got are still valid.
    if (rc != VOXRT_OK && rc != VOXRT_ERR_BUFFER_TOO_SMALL) {
        fprintf(stderr, "push failed: %d\n", (int)rc);
        break;
    }
    for (size_t i = 0; i < written; i++) {
        printf("[WW] score=%.3f\n", dets[i].score);
        // switch state here
    }
}

voxrt_wake_word_destroy(engine);
// remember to munmap(model_bytes, model_len)

Each SDK's quickstart file already does this work. It opens the mic, feeds the engine, and prints detections. Our triage loop is the state machine on top of that.

Expected performance on Pi 4

Important disclaimer. The numbers below extrapolate from benchmarks on other hardware. We have not run the full triage loop on our Pi 4 yet. Your numbers should land in this range but you will only know exactly after you measure on your own board.

IDLE state (WW only). About 1 to 2% of one CPU core. Extrapolated from 5.3% on Pi Zero 2 W (A53 at 1 GHz running the default 32-bit Raspberry Pi OS). Pi 4 uses a 64-bit A72 at 1.5 GHz with better IPC and more efficient NEON when the OS runs in aarch64 mode.
KWS_CHECK state (2 seconds after a wake). About 15 to 25% CPU during the burst. Extrapolated from 16% RTF on Snapdragon 662, but Pi 4's A72 is a generation older than the Kryo 260 Gold cores in SD662, so expect somewhat higher RTF. KWS only runs in short windows.
ASR_TRANSCRIBE state. Roughly 40 to 70% CPU during active transcription. ASR is the heaviest of the three, and NEON-specialised int8 kernels tuned for mobile Snapdragons will not extract the same efficiency on A72. Real-time streaming (RTF under 1.0) should still be achievable on Pi 4 aarch64, but the headroom is the tightest of the three. Measure on your device before assuming.

RAM footprint with all three engines initialised: around 200 to 300 MB. ASR dominates with its roughly 150 MB of weights. That is why we recommend Pi 4 with 4 GB. The 2 GB variant also works but leaves little room for the rest of your software.

If we run real Pi 4 numbers after publish, we will do a follow-up article with actual measurements.

What this article does not cover

TTS (speaking back). In development. A separate on-device voice-cloning model. Will get its own article when it ships.
Custom wake-word phrase. We currently only ship "Hey Assistant" as a pre-trained model. Training your own trigger phrase requires a separate training pipeline that we have not released publicly yet.
Speaker verification / voice biometrics. Not included. The models do not tell you who is speaking.
Multi-language support. Wake-word and KWS are English-only at the moment. The ASR model is multilingual in general, but we have only tested it on English.

Everything above is on the VoxRT roadmap. If any of it is critical for your use case, mention it in the comments so we can prioritise.

Repos and resources

Wake-word Linux SDK: github.com/VoxRT/voxrt-wake-word-linux
KWS Linux SDK: github.com/VoxRT/voxrt-kws-linux
ASR Linux SDK: github.com/VoxRT/voxrt-asr-linux
All SDKs: github.com/VoxRT
Our earlier on-device voice articles: dev.to/voxrtio

If you are building anything voice-driven and want the audio to stay on the device, all three SDKs are on GitHub with the documentation in the README. Full quickstart examples in Python, Node.js, Go and C are in every repo.

How we prepared voice cloning TTS to run on-device

VoxRT — Mon, 07 Sep 2026 14:05:00 +0000

We trained a text-to-speech model with voice cloning that fits into 100 MB (fp16) and runs on-device for synthesis, from smartphones to a Raspberry Pi. The model is in a file, generation happens on the device. No API, no cloud backend at inference time.

Early tests on a budget Android phone hit faster-than-realtime on a single core. Full benchmarks with numbers and different devices are coming in a separate post. Here is how we got there.

The model is about 45M parameters (a language model plus an audio decoder). For comparison, typical server-side TTS systems are hundreds of MB to several GB of weights. We trained both stages from scratch and packaged them into a single binary.

The road here was not smooth. This post is a short architecture overview (without it, it is hard to explain what broke) and two concrete stories of what broke and how we fixed it. If you end up training something similar, this should save you a few surprises.

Architecture in two paragraphs

We use the classic two-stage scheme. The first stage is a language model (LM) that, given text and a short voice sample, generates audio latents autoregressively, roughly the way an LLM generates text, except the outputs are continuous audio representations instead of words. The second stage is the decoder of our own neural codec (24 kHz, compact continuous latent space), which turns those latents back into a waveform.

The key design decision is a single-step flow sampler instead of iterative diffusion. Classical diffusion TTS often needs many forward passes per frame to sound right. That is fine on a server. On mobile or a Raspberry Pi, that does not work. One step per frame gives you speed on weak hardware. But you pay for it in training complexity, described below.

Two more things for context. The architecture is causal, so it streams by construction. You do not have to wait for the end of the input, and audio comes out with sub-second time-to-first-audio. Voice cloning works from a short sample of around 5 seconds. The cloning part runs off-device, the synthesis is on-device.

What broke #1: single-step sampler collapses speech to averaged sound

Diffusion models work well partly because stochasticity accumulates over many steps. Each step adds a bit of noise and the model learns to remove it. By the end of the process you get a quality sample.

When you do this in a single step, that stochasticity disappears. A naively trained one-step flow sampler tends to converge to an "averaged" sound. The model finds an output that minimizes loss on average across the training set, rather than a diverse, characteristic sound.

We saw this on early iterations as a lifeless "muddled" voice, identical in intonation regardless of input. Test recordings sounded like a robotic monotone from the nineties.

What helped: a special sampler training scheme. I will skip the architectural details (that part of the recipe stays internal), but the general idea is to force the sampler to account for the target variance of the distribution, not just its mean. Verification was through a separate synthetic test set specifically for this failure mode, where voice diversity is expected. The model passed, it started to sound like real speech.

The takeaway: single-step sampling is not free. You save on inference and pay in training complexity.

What broke #2: mixed precision, one poisoned step costs 5000 iterations

We trained in mixed precision, some operations in fp16 for speed, some in fp32 for stability. Standard practice for large neural nets on GPUs.

The problem is that fp16 has a narrow range of representable numbers. Sometimes gradients overflow and you get a NaN. The usual protection (loss scaling and gradient clipping) works most of the time. But sometimes a rare combination of input data plus current weight state causes one operation to produce a NaN. That NaN gradient then poisons every parameter it touches at the next optimizer step.

One such "poisoned" step can invalidate 5000 previous training iterations. The model starts emitting noise, loss shoots to infinity, and everything since the last checkpoint is gone.

The fix was multi-layer protection. Per-batch NaN check, per-epoch sanity check, running-loss anomaly detection, then automatic rollback to the last good checkpoint plus a human alert. Not free compute-wise, but cheaper than losing a week of training. Mixed precision is a real engineering discipline, not a flag you enable and forget, especially on long runs where accumulated risk grows linearly with time.

What we optimized for (and what we did not)

The design constraints were tight.

Streaming by construction, with sub-second time-to-first-audio on weak hardware. No "wait a moment while the model thinks".

Numbers, dates, and currency without special-case hacks. The model reads "March 21st, 2026" and "$19.99" correctly. To get there, we added synthetic speech to the training mix with quality control specifically on those edge cases.

Around 45M parameters, because 100 MB is what you can bundle into an app without a noticeable impact on bundle size. Any bigger and it gets uncomfortable for mobile apps.

What we did not prioritize: language diversity (English only for now, the training corpus was around 3000 hours for the codec and 1700 hours for the LM), and extreme quality benchmarks for professional use. We aim for natural-sounding output on default use cases, not to beat top commercial TTS in blind studio tests.

Quality is judged via blind listening tests on a fixed text set: numbers, dates, tongue twisters, long narratives. Not perfect, but reproducible, and raters agree with each other.

Where we go next

The model is going through final polishing (we are fine-tuning the decoder for the production generation mode) and packaging into the VoxRT runtime. The delivery format will be the same as our other SDKs: a single binary model file plus native SDKs for iOS, Android, and Raspberry Pi.

Release in September 2026. Code, model, and release notes will land on GitHub. Star the repo if you want a ping when it drops.

More about our other on-device SDKs (VAD, wake-word, KWS, streaming ASR) at voxrt.com. Models and SDKs on GitHub.

Add a wake-word to your web app in 5 minutes

VoxRT — Wed, 02 Sep 2026 08:00:03 +0000

Voice interfaces are getting easier to add to the web, but wake-words have been the hard part. Cloud APIs mean latency, per-request billing, and audio leaving the user's device. On-device wake-words in the browser were awkward until WebAssembly SIMD128 shipped in every major browser.

You will have a working "Hey Assistant" detector in about five minutes. Everything runs on-device, no cloud, no per-request cost.

What you will need

A modern browser (Chrome/Edge 91+, Firefox 89+, or Safari 16.4+). WASM SIMD128 is a hard requirement
A microphone
Any static file server on HTTPS or localhost. Browsers block getUserMedia on plain HTTP
Node.js if you want npm install. Otherwise CDN works fine

The full download is ~275 KB (170 KB WASM runtime + ~100 KB model). That is smaller than most icon fonts.

Step 1: install the SDK

Two options.

via npm:

npm install @voxrt/wake-word-browser

via CDN (no build step):

import init, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";

Use CDN to skip the bundler.

Step 2: HTML shell

Nothing fancy. A single button and a status line will do:

<!DOCTYPE html>
<html>
<head><meta charset="utf-8"><title>Wake-word demo</title></head>
<body>
  <button id="start">Start listening</button>
  <p id="status">Idle</p>
  <script type="module" src="./app.js"></script>
</body>
</html>

Serve this over HTTPS or localhost. Microphone access does not work on plain http:// except for localhost.

Step 3: initialize the engine and load the model

Create app.js:

import init, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";

await init();

const modelBytes = new Uint8Array(await (await fetch(
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt_wake_word.vxrt"
)).arrayBuffer());

const engine = WakeWordEngine.fromBytes(modelBytes);
engine.threshold = 0.9;

That is the whole "load the model" step. The .vxrt file is the pre-trained "Hey Assistant" model at ~100 KB. init() bootstraps the WebAssembly runtime.

The threshold is in sigmoid space [0, 1]. Default is 0.9. Lower it and you get more sensitive detection (more false accepts). Raise it and you get fewer false accepts but might miss quieter or accented pronunciations. We will tune this later.

Step 4: capture microphone audio

Wake-word models need 16 kHz mono audio. Modern browsers give you higher sample rates by default, so we ask AudioContext for 16 kHz explicitly. Not all browsers honor the request, so we also check:

document.getElementById("start").onclick = async () => {
  const stream = await navigator.mediaDevices.getUserMedia({audio: true});
  const audioContext = new AudioContext({sampleRate: 16000});
  if (audioContext.sampleRate !== 16000) {
    console.warn(`Got ${audioContext.sampleRate} Hz instead of 16000. Detection quality will drop. Resample or check the SDK docs.`);
  }
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(512, 1, 1);
  const mute = audioContext.createGain();
  mute.gain.value = 0;

  // detection loop goes here (Step 5)

  source.connect(processor);
  processor.connect(mute).connect(audioContext.destination);
  document.getElementById("status").textContent = "Listening";
};

Two notes on the code above:

ScriptProcessorNode is deprecated in favor of AudioWorkletNode for production. For a 5-minute demo, ScriptProcessor is simpler. Swap it out when you productionize
The muted GainNode between processor and destination is a Web Audio quirk. Without connecting the processor to something, audioprocess never fires. Setting gain to 0 avoids sending your microphone straight to the speakers (which would cause feedback)

Step 5: push audio frames and detect

The engine wants Int16 PCM samples in fixed-size chunks. Here is the detection loop:

const pcmBuffer = new Int16Array(512);

processor.addEventListener("audioprocess", (e) => {
  const inputFloat = e.inputBuffer.getChannelData(0);

  // convert Float32 [-1, 1] to Int16
  for (let i = 0; i < inputFloat.length; i++) {
    const s = Math.max(-1, Math.min(1, inputFloat[i]));
    pcmBuffer[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
  }

  // push to engine, iterate detections
  for (const detection of engine.pushPcmI16(pcmBuffer)) {
    console.log(`Wake detected at ${detection.timestampSec.toFixed(3)}s, score ${detection.score.toFixed(4)}`);
    document.getElementById("status").textContent = "Wake-word detected";
  }
});

That is it. Open DevTools (F12, then Console), click the button, say "Hey Assistant", and you should see a log line. The detection.score is the sigmoid probability at the moment of detection.

What just happened

You loaded a ~100 KB model into WebAssembly, streamed 16 kHz mono audio through it in 512-sample frames (32 ms each), and got detections when the score crossed threshold.

The engine handles its own cooldown so you do not get spam detections on the same utterance. Default cooldown is 100 frames (about 1 second at 10 ms hop). You can tune with engine.cooldownFrames.

Two quick customizations

Prefer Float32 over Int16? Skip the conversion:

processor.addEventListener("audioprocess", (e) => {
  const inputFloat = e.inputBuffer.getChannelData(0);
  for (const detection of engine.pushPcmF32(inputFloat)) {
    console.log(`Detected: ${detection.score.toFixed(4)}`);
  }
});

Tuning threshold: Watch engine.currentScore() to see live confidence. If your users often speak quietly, lower the threshold to 0.85. If you get false triggers from radio or TV, raise it to 0.93.

setInterval(() => {
  console.log(`Live score: ${engine.currentScore().toFixed(3)}`);
}, 100);

At the default threshold of 0.9, the model has precision 0.993 and recall 0.982 on the reference test split (11,656 utterances, ROC AUC 0.9966).

Full code (copy-paste starter)

Here is everything above, glued together:

import init, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";

await init();

const modelBytes = new Uint8Array(await (await fetch(
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt_wake_word.vxrt"
)).arrayBuffer());

const engine = WakeWordEngine.fromBytes(modelBytes);
engine.threshold = 0.9;

document.getElementById("start").onclick = async () => {
  const stream = await navigator.mediaDevices.getUserMedia({audio: true});
  const audioContext = new AudioContext({sampleRate: 16000});
  if (audioContext.sampleRate !== 16000) {
    console.warn(`Got ${audioContext.sampleRate} Hz instead of 16000. Detection quality will drop.`);
  }
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(512, 1, 1);
  const mute = audioContext.createGain();
  mute.gain.value = 0;
  const pcmBuffer = new Int16Array(512);

  processor.addEventListener("audioprocess", (e) => {
    const inputFloat = e.inputBuffer.getChannelData(0);
    for (let i = 0; i < inputFloat.length; i++) {
      const s = Math.max(-1, Math.min(1, inputFloat[i]));
      pcmBuffer[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
    }
    for (const detection of engine.pushPcmI16(pcmBuffer)) {
      console.log(`Wake at ${detection.timestampSec.toFixed(3)}s score ${detection.score.toFixed(4)}`);
      document.getElementById("status").textContent = "Wake-word detected";
    }
  });

  source.connect(processor);
  processor.connect(mute).connect(audioContext.destination);
  document.getElementById("status").textContent = "Listening";
};

Serve this alongside the HTML from Step 2 and you have a working wake-word demo.

Where to go from here

Custom phrase or another language: the free tier ships with "Hey Assistant" only. Custom phrases and additional languages are available on a paid tier. We train the model for you and swap the .vxrt file
Native mobile: same SDK exists for iOS (Swift) and Android (Kotlin). Wake-word runs at 1.5% RTF on iPhone A15, 2.1% on Snapdragon 662
Linux and edge: available for Linux aarch64 with bindings for Python, Node.js, Go, and C. Holds at 5.3% RTF sustained on a $15 Raspberry Pi Zero 2 W
Custom on-device pipeline: wake-word into ASR into your app logic works well for privacy-critical voice UIs

One note on licensing

The wrapper (Rust crate, wasm-bindgen bindings, examples) is Apache-2.0 so you can freely integrate it. The compiled WebAssembly runtime and the model weights are proprietary. Redistribution is allowed only as an unmodified part of the SDK package. In practical terms: install and use it in your product, no additional legal setup beyond reading the license.

Live demo: voxrt.com/wake-word-demo

Repo: github.com/VoxRT/voxrt-wake-word-browser

Wake-word on a $15 Raspberry Pi Zero 2 W: 5.3% RTF always-on

VoxRT — Fri, 21 Aug 2026 13:15:02 +0000

The problem: always-on wake-word is hard and expensive

A wake-word detector is the part of the voice stack that listens to the mic 24/7 and triggers the rest of the pipeline when it hears a specific phrase like "Alexa", "Hey Siri", or "Hey Google". Because it is always-on, wake-word dominates the energy and compute budget of any voice-first device.

Ready-made options for hobbyists are limited, and all have trade-offs:

Alexa/Google Nest/Apple HomePod work well but are locked to their vendor platforms, require cloud connectivity in some scenarios, and use closed hardware.
Picovoice / Porcupine is an on-device SDK that works offline after activation. The model is proprietary, and the free tier was deprecated as of 30 June 2026 (now a 7-day trial followed by paid).
openWakeWord is open source (Apache-2.0) with community models, but training a custom wake phrase is non-trivial. You need to gather data, train the model, and iterate.
Custom Python + TF Lite from scratch requires ML expertise and hours of work.

The problem: hobbyists want a simple recipe. Grab a $15 Pi, install a package with one line, hear a custom wake word, do not pay per activation.

This post covers our solution (proprietary, but with a free tier "Hey Assistant" by default). The headline number: 5.3% CPU on one A53 core of a $15 Raspberry Pi Zero 2 W, 24/7 sustained.

What a wake-word detector is: quick technical background

Three components usually connect:

VAD (Voice Activity Detection) is the lightest. It answers "is there any speech in this audio?" (regardless of what kind). Runs at ~1% CPU. Used as a pre-filter for heavier stages.
Wake-word detector answers "is there a specific phrase?". Heavier than VAD (needs actual pattern matching) but still light (typically 10-100 KB model). Runs 1-10% CPU in always-on mode.
ASR (Automatic Speech Recognition) does full transcription: "what exactly was said?". Heavy (~60 MB model, 30-40% CPU on mid-range Android). Only fires after wake-word success.

Wake-word sits in the sweet spot. Not as light as VAD (VAD does not distinguish "Hey Assistant" from "hello world"), not as heavy as ASR (no full vocabulary needed). Cache-friendly architecture, sub-100 ms decisions, runs on any ARM SoC from 2020+ without a DSP.

Why Pi Zero 2 W as target

Raspberry Pi Zero 2 W is the most popular "minimal" Pi (~$15). Specs:

ARM Cortex-A53 quad-core @ 1 GHz: same class as midrange smartphones from 2016-2018
512 MB RAM: enough for small ML models, not for heavy Whisper
No GPU: CPU-only inference
~$15 retail: cheap enough for DIY smart speakers, doorbell notifications, custom voice UI

The logic: if it works on Pi Zero 2 W, it works everywhere. More powerful Pi 4/5 and Jetson Nano are 4-10x faster, embedded ARM SoCs (Rockchip, Allwinner) are comparable class. So Pi Zero 2 W sets the lower bound: if the numbers hold there, they hold everywhere else.

Wake-word across the full Pi ladder:

Device	RTF (Real-Time Factor)	CPU budget
Pi Zero 2 W (Cortex-A53)	0.053 (5.3%)	one core
Pi 3 A+/B/B+ (Cortex-A53)	~0.038-0.044 (3.8-4.4%)	one core
Pi 4 B / 400 (Cortex-A72)	~0.018-0.024 (1.8-2.4%)	one core
Pi 5 (Cortex-A76)	~0.008-0.012 (0.8-1.2%)	one core

Pi Zero 2 W number is measured (60-second sustained live-mic push). Pi 3/4/5 numbers are estimates scaled by clock speed and architecture generation, not independently benchmarked.

RTF 0.053 means the processor spends 53 milliseconds per 1 second of audio. Wake-word takes ~5% of one core on Pi Zero 2 W, leaving 95% plus the other 3 cores for the application (UI, LEDs, LLM inference, storage). On Pi 5, wake-word is essentially free (<1%).

Model architecture

Key characteristics of our wake-word model:

~48 thousand parameters: very small neural network by modern standards (for comparison, Whisper base.en has ~74 million parameters, roughly 1500x larger).
File size: ~100 KB (encrypted .vxrt format).
Sampling rate: 16 kHz mono, 32 ms frames: standard for speech recognition.
Trained on 100% synthetic data with augmentation: no human voice samples in the training set, which addresses licensing concerns for commercial deployment.
Free tier phrase: "Hey Assistant": works out of the box.
Custom phrases: paid tier: training pipeline available on request (help@voxrt.com).

Accuracy metrics (measured on a test set of 5,240 positive + 6,416 negative utterances):

ROC AUC: 0.9966: very clean separation between positive and negative.
PR AUC (Average Precision): 0.9899.
Threshold 0.90: precision 0.993 / recall 0.982.

In practice: at threshold 0.90 the model produces ~1 false positive per 100+ triggers and misses ~2% of real wake-words. Acceptable for an always-on hot loop.

Benchmark: 5.3% RTF sustained on Pi Zero 2 W

RTF (Real-Time Factor) is CPU time divided by audio duration. RTF 5.3% means the processor spends 53 milliseconds of CPU time per 1 second of audio.

How we measured:

Pi Zero 2 W, Cortex-A53 @ 1 GHz, aarch64 Linux
60-second continuous loop with microphone input
Used one core out of four
Sustained RTF (not peak) because sustained better reflects real-world thermal throttling

Why sustained matters more than peak: Pi Zero 2 W without a heat sink can throttle CPU after 2-3 minutes of sustained load. Peak RTF (first 5 seconds) is typically 10-20% better than sustained. We publish the sustained number, 5.3%, because that is what a hobbyist actually gets in production.

What to do with the remaining 95% CPU plus 3 cores: UI (LEDs / display), Bluetooth / WiFi for sending triggers to home assistant, local logic (dimmer, thermostat), or on a mid-tier board like Pi 4, running a smaller LLM for command interpretation.

Why NEON matters: 8.7x speedup

The wake-word runtime is written in Rust. But "pure Rust" is not enough for these RTF numbers. The key ingredient is ARM NEON SIMD intrinsics in the inner loops (matrix multiply, activation functions).

Measured on Snapdragon 662, Cortex-A73 big cluster pinned via HIGH_PERF affinity (SD662 is a big.LITTLE SoC with 4x A73 + 4x A53, and the perf cluster is used for stable clock):

Implementation	RTF
Rust scalar (no SIMD)	0.182 (18.2%)
Rust + NEON SIMD	0.021 (2.1%)

Speedup: 8.7x.

What this means for the reader: even a small neural network (48K params) on CPU-only inference depends critically on SIMD acceleration. Writing your own solution in pure Python + NumPy gives you an RTF that is 10-20x higher than an optimized SIMD implementation.

On Pi Zero 2 W (A53, one generation older than A73), the NEON speedup ratio is similar, but A53 has fewer cycles per second per core. That is why we see 5.3% RTF on A53 vs 2.1% on A73 with the same code.

Practical takeaway: if you plan to write a custom wake-word, targeting ARM NEON is mandatory. Otherwise it will be too slow for always-on mode on any embedded ARM.

How to run: setup + code (4 languages)

The SDK is available for Python, Node.js, Go, and C with an identical API pattern: load a .vxrt file, set threshold and cooldown_frames, push PCM int16 chunks (16 kHz mono), receive detection events with frame_index, timestamp_sec, and score fields.

The voxrt_wake_word.vxrt model (~100 KB, free tier "Hey Assistant") for all examples below can be downloaded from the releases page or HuggingFace.

All examples below are simplified versions of the quickstarts in examples/{python,nodejs,go,c}/ in the repository. Full versions (with WAV parsing, ALSA live-mic, performance timing) live there too.

Python

pip install voxrt-wake-word

from voxrt_wake_word import WakeWordEngine

engine = WakeWordEngine.from_path("voxrt_wake_word.vxrt")
engine.threshold = 0.9
engine.cooldown_frames = 100

for chunk in mic_iter():  # int16 mono @ 16 kHz, 512-sample chunks
    for d in engine.push_pcm_i16(list(chunk)):
        print(f"wake! t={d.timestamp_sec:.3f}s score={d.score:.4f}")

For live mic, install sounddevice (pip install sounddevice) and feed sd.InputStream(samplerate=16000, channels=1, dtype='int16', blocksize=512, callback=...) chunks into engine.push_pcm_i16 inside the callback.

Node.js

npm install @voxrt/wake-word

const { WakeWordEngine } = require("@voxrt/wake-word");

const engine = WakeWordEngine.fromPath("voxrt_wake_word.vxrt");
engine.threshold = 0.9;
engine.cooldownFrames = 100;

for (const chunk of micIter()) {  // Int16Array, 512 samples, 16 kHz mono
  for (const d of engine.pushPcmI16(chunk)) {
    console.log(`wake! t=${d.timestampSec.toFixed(3)}s score=${d.score.toFixed(4)}`);
  }
}

For live mic, use the mic or node-record-lpcm16 npm packages. Both return Node Buffer objects, so convert to Int16Array before pushing: new Int16Array(buf.buffer, buf.byteOffset, buf.length / 2).

Go

go get github.com/VoxRT/voxrt-wake-word-linux/go

package main

import (
    "fmt"
    "log"
    wakeword "github.com/VoxRT/voxrt-wake-word-linux/go"
)

func main() {
    engine, err := wakeword.OpenFromPath("voxrt_wake_word.vxrt")
    if err != nil {
        log.Fatal(err)
    }
    defer engine.Close()
    engine.SetThreshold(0.9)
    engine.SetCooldownFrames(100)

    // micIter returns <-chan []int16 (512-sample chunks, 16 kHz mono)
    for chunk := range micIter() {
        for _, d := range engine.PushPcmI16(chunk) {
            fmt.Printf("wake! t=%.3fs score=%.4f\n", d.TimestampSec, d.Score)
        }
    }
}

For live mic on Linux, use github.com/gordonklaus/portaudio or pipe from arecord (simpler for a quickstart). Have your mic reader emit <-chan []int16 for the range loop above.

C

Install headers and shared library from the tarball release, then build with pkg-config for flags:

gcc main.c $(pkg-config --cflags --libs voxrt-wake-word) -lasound -o wake-word-app

#include <stdint.h>
#include <stdio.h>
#include <voxrt_wake_word.h>

int main(int argc, char **argv) {
    // Load model bytes via mmap. See full example in repo for the ~30-line
    // helper (open + fstat + mmap). Placeholders here for brevity:
    const uint8_t *model_bytes = NULL;   // TODO: mmap voxrt_wake_word.vxrt
    size_t model_len = 0;                // TODO: file size

    voxrt_wake_word_t *engine = NULL;
    voxrt_status_t rc = voxrt_wake_word_create(model_bytes, model_len, &engine);
    if (rc != VOXRT_OK) {
        fprintf(stderr, "voxrt_wake_word_create failed: %d\n", rc);
        return 1;
    }
    voxrt_wake_word_set_threshold(engine, 0.9f);
    voxrt_wake_word_set_cooldown_frames(engine, 100);

    int16_t buf[512];
    voxrt_wake_word_detection_t dets[8];
    for (;;) {
        // Read 512 int16 samples from ALSA / mic into buf.
        size_t written = 0;
        voxrt_wake_word_push_pcm_i16(engine, buf, 512, dets, 8, &written);
        for (size_t i = 0; i < written; ++i) {
            printf("wake! t=%.3fs score=%.4f\n",
                   dets[i].timestamp_sec, dets[i].score);
        }
    }
    voxrt_wake_word_destroy(engine);
    return 0;
}

The full C example with all ALSA setup (~150 lines) is at examples/c/alsa-mic-quickstart/main.c. There is also a C++ variant (alsa-mic-quickstart-cpp) and a CMake consumer example for integrating into existing projects.

Parameters and what is not shown

All four languages use identical parameters:

threshold (default 0.5): confidence threshold for firing. 0.9 is strict (fewer false positives). For always-on in a noisy environment, 0.85 to 0.90 is a reasonable baseline.
cooldown_frames (default 50, ~1.6 sec): how many silence frames after a detection before the next trigger. Prevents double-fire.

Not shown in the snippets (standard boilerplate): microphone init, error handling, action on detection (starting ASR, HTTP webhook to Home Assistant, turning on an LED).

Wake-word alternatives: honest comparison

Three main options for a hobbyist in 2026:

	Porcupine (Picovoice)	openWakeWord	VoxRT wake-word
Model size	~200 KB - 1 MB (varies by tier)	~50-400 KB per model + ~3 MB shared runtime	~100 KB
RTF on Pi Zero 2 W	~3.8% on Pi 3 per Picovoice docs, no Pi Zero 2 W numbers published (likely 4-8% scaled by clock)	reported CPU-heavy on Pi Zero 2 W, no public RTF, on Pi 4 <5 ms per 80 ms chunk	5.3% (measured)
Runtime	proprietary C/C++ SDK	Python + ONNX Runtime	proprietary Rust SDK
License	commercial (7-day trial then paid, no free tier since 30 June 2026)	Apache-2.0 (fully open)	proprietary (LICENSE-BINARY), free default phrase, paid custom
Custom wake phrase	commercial console (paid)	DIY training pipeline (Python, data collection, hours of work)	paid tier (email us)
Setup complexity (out-of-the-box "Hey X")	medium (auth flow)	high (need to train or download a community model)	low (pip install + download)

Each tool has its fit:

Porcupine: if you need a large catalog of pre-trained wake phrases and you are fine with commercial licensing.
openWakeWord: if 100% open source is critical and you have time for DIY training.
VoxRT wake-word: if you need the smallest footprint and simplest deployment ("Hey Assistant" out of the box, custom phrase via paid tier).

No tool is universally "better" than the others. It depends on requirements.

Licensing model

Important disclosure: VoxRT wake-word is a proprietary tool (unlike our Silero VAD packaging, which is MIT).

Runtime + model: LICENSE-BINARY (Elephant Enterprises LLC). Distributable as part of an unmodified SDK, no forking, no rebuilding.
Free tier phrase: "Hey Assistant" works out of the box without activation.
Custom wake phrases: paid tier. We train a custom model on your phrase (typical 2-3 weeks turnaround). Contact: help@voxrt.com.
No per-activation billing: unlike Porcupine, no ongoing fees. A one-time custom-phrase fee covers redistribution in your product.

This is a trade-off: you get the smallest footprint and simplest deployment, but you cannot modify the runtime or train your own model. For a 100% open-source requirement, use openWakeWord (Apache-2.0).

WASM bonus: 0.16% RTF in Chrome

The same ~100 KB .vxrt file runs via WASM SIMD128 in the browser:

Environment	RTF
Chrome / MacBook Pro M4 (WASM SIMD128)	0.16%
Safari / iPhone A15 (WASM)	0.23%

Essentially free. The full npm bundle @voxrt/wake-word-browser is ~275 KB (model + WASM runtime).

Not the primary story of this post, but worth mentioning because it is unusual. Voice AI in the browser without a backend, integrated into a web app via npm install. The same model runs on Pi, iPhone, Android, and in a user's Chrome tab.

Wrapping up

Wake-word detection on a $15 Raspberry Pi Zero 2 W, always on, 5.3% CPU is not an abstract idea. One-line install with pip install voxrt-wake-word, add a microphone reader loop, done.

Three things worth remembering:

RTF sustained (60+ sec) matters more than peak. Thermal throttling on a Pi Zero 2 W without a heat sink makes peak numbers misleading.
ARM NEON SIMD is critical for CPU inference. 8.7x speedup vs scalar Rust. Custom implementations without SIMD will be too slow for always-on.
Wake-word is not VAD, not ASR. Three tiers with different CPU budgets. Wake-word is the middle tier: heavier than VAD, light enough for always-on.

Tools from this post:

github.com/VoxRT/voxrt-wake-word-linux: Linux SDK (aarch64)
github.com/VoxRT/voxrt-wake-word-android: Android SDK
github.com/VoxRT/voxrt-wake-word-ios: iOS SDK
github.com/VoxRT/voxrt-wake-word-browser: WASM browser SDK
huggingface.co/VoxRT/wake-word-hey-assistant-vxrt: free tier model

Previous posts in this voice AI series:

Building Private Voice Notes for iOS and Android Without the Cloud: full tutorial for a VAD + ASR pipeline
Streaming ASR vs Whisper on mobile: when to switch: architectural comparison for live voice apps

Learn more about the runtime and product family: voxrt.com.

Custom wake phrase for your product: help@voxrt.com.

Streaming ASR vs Whisper on mobile: when to switch

VoxRT — Tue, 18 Aug 2026 08:45:10 +0000

The problem: Whisper feels laggy for live apps

Voice-first apps became mainstream in 2024-2026. Voice agents, real-time captions, live meeting transcription, voice notes with instant feedback. Developers building this new generation of apps typically look at Whisper first (whisper.cpp, faster-whisper). It is the de facto default for on-device speech-to-text.

And for batch transcription (upload audio file, wait, get transcript), Whisper is genuinely great. Interview transcription, meeting notes, lecture recordings on MacBook or in an Android app. Whisper handles all these accurately and reliably.

But as soon as you try Whisper for live scenarios (user speaks, expects instant response), something feels off. A 300-500 ms delay minimum shows up, often up to 1-2 seconds. Words get "sliced" at chunk boundaries. Transcript comes out in jerks. For a voice agent that should respond like a human in conversation, this is a frustrating UX.

This is not a Whisper bug. It is a design mismatch: Whisper is a batch model retrofitted for streaming, while a real-time scenario fundamentally needs a different architecture.

This post is about what native streaming ASR is, how it differs from Whisper's approach, what alternatives exist, and when you should switch (or stay).

What "streaming" really means in ASR

Before comparing options, terminology first.

Batch ASR processes audio as a whole: the model receives the entire file (or a fixed-length chunk), runs inference in one pass, returns the full transcript. Classic architectures (Whisper, original Wav2Vec2, seq2seq encoder-decoder) are batch by nature.

Streaming ASR processes audio continuously: the model takes a stream of samples, emits text incrementally as speech arrives. Key streaming parameters: lookahead (how many milliseconds of future audio the model "sees" before emitting text) and chunk latency (how many milliseconds the model buffers between emissions).

Classification by lookahead:

Infinite lookahead: batch (the whole file is known upfront). Whisper, original NeMo Conformer non-streaming.
Large lookahead (1000ms+): pseudo-streaming batch with large chunks. Whisper.cpp streaming mode, faster-whisper VAD-based.
Small lookahead (80-320 ms): native streaming. NeMo FastConformer streaming, streaming Conformer variants, Parakeet streaming, some Kaldi TDNN variants.
Zero lookahead: causal streaming. Fastest response but usually strictly worse accuracy.

The difference between "pseudo-streaming" and "native streaming" is fundamental but often confused. Both technically "stream." But native streaming is trained with a constraint to see only limited future audio, while pseudo-streaming is a hack on top of a batch model.

Whisper's approach: chunked batch retrofit

Whisper is trained on 30-second audio chunks. To get streaming behavior, developers use one of two workarounds.

Whisper.cpp streaming mode: rolling window of 500 ms to 3 s, each chunk is processed as a mini-batch, outputs are concatenated. It works, but has several side effects:

Chunk boundary artifacts: words at chunk borders can be cut, repeated, or misrecognized.
Latency equals chunk size plus inference time. On SD662 with Whisper base.en at 500 ms chunks, perceived latency is 700-1500 ms.
VAD and endpointing need separate tuning (Whisper has no native VAD).
Punctuation can be inconsistent between chunks (the model "loses context" at borders).

VAD-based streaming wrappers (e.g., Whisper-Streaming or WhisperLive built on faster-whisper): VAD triggers, the accumulated segment gets transcribed, output emits. Cleaner output (no chunk boundary artifacts), but latency is even higher (you have to wait for end of speech for VAD trigger).

Neither of these approaches delivers what native streaming does: continuous text stream with sub-100 ms lookahead. This is an architectural limitation, not a bug.

Whisper is not bad. Whisper is not for live. These are two different task categories.

Native streaming ASR: how it works

Native streaming architectures (NeMo FastConformer streaming, streaming Conformer, Parakeet streaming variants) work differently.

Cache-aware inference. The model is trained knowing it will see only limited future audio (typically 80-320 ms lookahead). Internal hidden states cache between chunks. No need to re-process previous audio for each new segment.

Continuous emission. The model emits tokens incrementally as speech arrives. No "chunk boundaries" in perceived output. Text simply prints in real time.

Trade-off exists, but minimal. Native streaming models are usually slightly less accurate than their batch counterparts (the limited future audio constraint reduces context understanding). But the gap is small, typically 0.2-0.5% WER on LibriSpeech test-clean.

Example architecture: NeMo FastConformer streaming (stt_en_fastconformer_hybrid_medium_streaming_80ms_pc by NVIDIA). Roughly 32 million parameters, hybrid decoder (RNN-T + CTC), 80 ms cache-aware lookahead, WER 3.267% on LibriSpeech test-clean with the RNN-T decoder (our WER-500 measurement, see footnote in the comparison table).

Similar variants:

Parakeet streaming: same NeMo family, different size variants.
Streaming Conformer: general architectural family, different implementations from Google, Microsoft, and NVIDIA.
Kaldi TDNN-F: old-school, but streaming, still used in Vosk.

If the task is live, these models deliver a qualitatively different experience than chunked Whisper.

Our implementation: VoxRT, a Rust runtime around NeMo streaming

We (VoxRT) built a Rust runtime that packages NeMo FastConformer streaming (the same cache-aware architecture described above) into ready-made SPM (iOS) and Gradle(Android) packages. The model is stt_en_fastconformer_hybrid_medium_streaming_80ms_pc from NVIDIA, ported into our .vxrt format without architectural changes (fp16 quantization). Our .vxrt format supports AES-256-GCM encryption for proprietary models, but the NeMo streaming ASR model here remains under CC-BY-4.0, not encrypted.

What this gives a developer compared to the "write your own wrapper" approach:

One-line integration in Gradle/SPM. No custom Python + ONNX bridge to write for JNI or Objective-C interop.
Symmetric API surface between iOS and Android. Kotlin and Swift code is nearly identical, same lifecycle pattern (init -> processPcm -> stop).
Native inference via Rust runtime. No PyTorch, no ONNX runtime dependency, minimal APK/IPA footprint (native binary ~424 KB stripped on Android, ~500 KB framework compressed on iOS).
Bundled Silero VAD for pre-filtering. No separate VAD integration to write, the 1.2 MB model works out of the box paired with ASR.

Honest trade-off: our runtime is proprietary (LICENSE-BINARY, Elephant Enterprises LLC), though the wrapper is Apache-2.0. The model is under CC-BY-4.0 (NVIDIA), which requires attribution in your app's credits. For commercial closed-source apps, no compliance issues. Pure GPL/AGPL projects will hit a license conflict with our proprietary runtime. In that case, writing your own wrapper around the open-source NeMo runtime is a better path.

From here on, all "streaming ASR" numbers in the post come from our implementation of this architecture. Whisper and Vosk numbers come from their upstream benchmarks (source cited).

Comparison table

Extended comparison versus the first post: added axes for streaming approach, chunk latency, and ready SDK availability. All numbers cited.

	whisper.cpp (base.en)	whisper.cpp (small.en)	Vosk (small-en-0.15)	VoxRT NeMo FastConformer streaming
Model type	Batch (encoder-decoder Transformer)	Batch	Streaming (Kaldi TDNN-F)	Native streaming (Conformer + cache)
Model size	~149 MB ¹	~489 MB ¹	40 MB (+ 1.6 GB for punctuation) ²	60.4 MB ³
WER (LibriSpeech test-clean)	4.27% ⁴	3.05% ⁴	9.85% ²	3.267% (RNN-T) / 4.895% (CTC) ⁵
Streaming approach	Chunked batch (~500 ms blocks)	Chunked batch	Native streaming	Native cache-aware (80 ms lookahead)
Latency pattern	500-1500 ms per chunk (persistent)	500-1500 ms per chunk	300-500 ms per chunk	1.12 s initial buffer, then continuous ~100-200 ms
Punctuation by default	✓ (in-model)	✓	✗ (needs +1.6 GB model)	✓ (in-model)
Ready iOS SPM package	✗ (only community `ggml-org/whisper.spm`)	✗	✗	✓
Ready Android Gradle package	✗	✗	✗	✓
License	MIT (code + weights)	MIT	Apache-2.0 (code + weights)	Apache-2.0 wrapper / proprietary runtime / CC-BY-4.0 model

¹ whisper.cpp README, ggml format disk size (source lists 142 MiB and 466 MiB, converted to MB).
² alphacephei.com/vosk/models, evaluation table.
³ voxrt-asr-models v0.1.2 downloads table.
⁴ HuggingFace model cards openai/whisper-base.en and openai/whisper-small.en, evaluation section.
⁵ Our WER-500 measurements on LibriSpeech test-clean.

Important latency nuance: whisper.cpp has persistent per-chunk latency (each chunk 500-1500 ms). VoxRT NeMo streaming has an initial buffer of 1.12 s (for cache warmup), then sustained latency of ~100-200 ms perceived. Meaning:

For short queries (under 3 s): whisper.cpp can feel faster (VoxRT's initial buffer is just overhead).
For longer live sessions (over 3 s): VoxRT streaming feels smoother (continuous), whisper.cpp remains "jerky."

Real numbers on mobile

Everything above is architectural comparison. But any dev reading this will ask one question: how heavy is this on the device?

VoxRT NeMo FastConformer streaming:

Snapdragon 662 (mid-range Android from 2020): RTF 0.302 (file replay) / 0.353 (live-mic). That is 300-350 milliseconds of CPU time per 1 second of audio.
iPhone 13 Pro Max (Apple A15): RTF 0.08-0.10. Processor is roughly 10× faster than real time.
Runtime memory: ~150 MB steady-state.
APK/IPA impact: 60.4 MB (model) + ~500 KB (native framework) = ~61 MB added to app size.

VoxRT Silero VAD (used as pre-filter):

SD662: RTF 3.05%, latency ~1 ms per 32-ms frame.
iPhone A15: RTF 1.85%, latency ~0.6 ms.
Model size: 1.2 MB.

Whisper.cpp benchmarks on mobile: no public RTF numbers for Snapdragon or iOS in the whisper.cpp README, only Apple Silicon benchmarks. Community measurements (posts in r/androiddev, r/iOSProgramming) usually put Whisper base.en RTF in the 40-70% range on mid-range Android SoCs from 2020-2022. That is roughly 2× heavier than VoxRT NeMo for comparable accuracy.

Practical conclusion: for always-on live transcription on mid-range Android, Whisper base.en often burns enough CPU to noticeably heat up the device and drain battery. Streaming ASR under 40% RTF is sustainable without visible performance impact.

Decision framework: when to switch, when to stay

Practical rules for the reader.

Stay with Whisper if:

Batch transcription of files (audio, wait, transcript). Live UX not required. Whisper accuracy is great, community is huge, plenty of ready tools.
Short one-shot voice notes (user hits record, speaks 10 seconds, releases, gets transcript). Streaming ASR initial buffer is not justified here.
You do not care about battery or CPU load (e.g., transcription runs only on-demand, not always-on).
You want maximum community support, example code, and integrations.

Switch to streaming ASR if:

Live voice agents (chatbot that responds while user speaks). Every 500-1500 ms delay equals broken conversation.
Real-time captions for accessibility (hearing-impaired users). Here latency directly impacts UX.
Live meeting transcription with on-screen text visible as people speak.
Always-on background listening (voice notes 24/7, wake-word context). CPU load matters.

Go with a hosted API (Deepgram, AssemblyAI, etc.) if:

Concurrency: thousands of parallel sessions. Self-hosted infrastructure becomes overhead.
Diarization required out of the box (who is speaking when). Native streaming ASR usually has no built-in diarization, you need a separate model (pyannote and similar).
Multi-language mixed audio. Streaming models are usually English-focused, hosted APIs have multilingual support.
Budget allows and privacy is not a hard requirement.

Go with self-hosted streaming ASR (native streaming architecture) if:

Privacy is required (data cannot leave device or your infrastructure).
Mobile deployment: apps must work offline.
Scale exists, but budget matters. Hosted APIs get expensive at high volume.
You want low sustained latency without a hosted API dependency.

Wrapping up

Streaming vs batch in ASR is not a runtime tweak, it is an architectural choice. Whisper for batch is a valid default. For live it is a mismatch. Native streaming ASR (NeMo FastConformer, streaming Conformer, Parakeet variants) is a different tool for a different job.

Three things worth remembering when picking a stack:

Latency pattern matters more than absolute latency. Persistent 500 ms per chunk (Whisper chunked) subjectively feels worse than 1 s initial buffer + continuous ~100 ms (native streaming).
Punctuation by default is critical for UX. Whisper and native streaming provide it. Vosk requires a separate 1.6 GB model.
Ready-made mobile SDK. The only project that ships symmetric SPM + Gradle packages for streaming ASR is us (VoxRT). Whisper.cpp has a community SPM (ggml-org/whisper.spm) but no Gradle. Vosk offers only bindings, wrap yourself.

We build a Rust runtime for on-device voice models. Our ASR repos: github.com/orgs/VoxRT/repositories?q=asr+sort:stars. If you are building a live voice app for iOS, Android, or Linux, this stack plus a native streaming model give you everything needed.

Our first post covered building a full on-device voice notes pipeline (VAD + ASR + local storage) with working Kotlin and Swift snippets: dev.to/voxrtio/building-private-voice-notes-for-ios-and-android-without-the-cloud-54a8.

Learn more about the runtime and product family: voxrt.com.

Attribution: the streaming ASR model is a derivative of NVIDIA NeMo stt_en_fastconformer_hybrid_medium_streaming_80ms_pc, released under CC-BY-4.0.

Building Private Voice Notes for iOS and Android Without the Cloud

VoxRT — Wed, 12 Aug 2026 09:01:00 +0000

The problem

Voice-notes apps became one of the main tools for lawyers, doctors, journalists, and anyone who needs to capture a thought by speaking and find it later as text. Otter, Rev, Descript, Notion AI, Fireflies, and dozens of others all work the same way: the user hits "record," audio goes to the company's servers, ASR (automatic speech recognition) runs there, transcript comes back.

The problem with this setup is simple but fundamental: your recordings leave the device. For a casual user, that is annoying. For a lawyer discussing case strategy with a client, it is an attorney-client privilege violation. For a doctor, it is a HIPAA problem. For a journalist, it can burn a source. And even without any incident on the provider's side, for many use cases the mere fact of audio being sent to a third-party server is already unacceptable.

There is an alternative, but it needs a different architecture: transcription has to happen on the device itself. Not a single byte of audio leaves the phone. No API keys. No network calls. No dependency on whether your provider's backend is up.

This post walks through how to do it on iOS and Android in roughly one evening of wiring. Preview: the code fits in about 50 lines per platform, the SDK is under 500 KB, and the transcription model is 60 MB.

What "on-device" actually means

Before jumping into code, let's break down what a voice-notes pipeline actually contains. Understanding the architecture makes it easier to pick the right SDK.

Three stages:

1. Microphone. Provides a continuous stream of PCM (uncompressed audio samples): 16 kHz mono, 16-bit signed integers. This is the standard format for speech recognition. The sampling rate covers the human voice range.

2. VAD (Voice Activity Detection). Determines whether the current audio chunk contains speech. A light model (~1 MB), but critical. Without VAD, you would run silence, air-conditioner hum, and chair squeaks through your ASR, which either produces garbage text or just burns CPU and battery for no reason. VAD cuts the input stream into speech segments and drops everything else.

3. ASR (Automatic Speech Recognition). The actual speech-to-text step. A heavier model (~60 MB) that takes an audio segment and returns text. Modern streaming models emit text incrementally as audio comes in, without waiting for the recording to end. That is important for UX: users see the transcript "typing" in real time.

Flow:

[Microphone] → [PCM: 16 kHz mono] → [VAD] → speech present?
    → yes → [segment buffer] → [ASR streaming] → [text]
    → no  → discard

All of this runs locally. The final text goes into a local database (SQLite, CoreData, whatever you use), optionally encrypted. Zero network traffic during transcription.

The key metric for on-device work is RTF (Real-Time Factor): the ratio of processing time to audio length. RTF 5% means the processor spends 50 milliseconds on one second of audio. Lower RTF, less CPU and battery burn. For always-on voice notes, RTF above 50% starts being a problem: the phone heats up and the battery drains. A decent solution keeps RTF under 40% for transcription and under 5% for VAD.

Comparing the alternatives

Before showing VoxRT code, let's look at the honest set of on-device options a mobile developer has in 2026. Four SDK ecosystems worth knowing:

whisper.cpp: a C++ port of OpenAI Whisper, MIT-licensed. Wide model zoo (tiny/base/small/medium/large). Community iOS SPM package. No official Android Gradle package.
Vosk: Kaldi-based, Apache-2.0. Broad language coverage. Bindings for many languages but no ready-made SPM or Gradle package for mobile.
sherpa-onnx: Apache-2.0, from the k2-fsa team. Fully open source. Ships iOS SPM and Android AAR packages. Runs several streaming architectures (Zipformer, Paraformer, and NVIDIA transducer variants). Punctuation is an optional separate model.
VoxRT: Apache-2.0 wrappers on top of a proprietary Rust runtime, running NeMo FastConformer streaming with in-model punctuation.

There are also the platform-native recognizers: SFSpeechRecognizer on iOS and Android's SpeechRecognizer. Both can run on-device on modern OS versions, but with caveats. Session lengths are capped (typically ~1 minute on iOS, shorter on Android), on-device mode has to be explicitly requested and depends on the user having downloaded the language pack, and vocabulary/accuracy are outside your control. Fine for short dictation, awkward for continuous voice notes.

The table below focuses on the four SDK options, because that is where you actually make an architectural choice.

	whisper.cpp (base.en)	Vosk (small-en-0.15)	sherpa-onnx (streaming Zipformer, en)	VoxRT ASR (streaming-medium-pc)
Model size	142 MB (whisper.cpp fp16 GGML `ggml-base.en.bin`)	40 MB (+ 1.6 GB for punctuation = 1.64 GB total)	~250 MB (fp32) / ~68 MB (int8) ³	60.4 MB
WER on LibriSpeech test-clean	4.27% ¹	9.85% ²	varies per model	6.96% ⁴
Streaming	✓ (chunked via `whisper-stream`, ~500 ms blocks)	✓	✓ (native)	✓ (native cache-aware, 80 ms lookahead)
Ready-made iOS SPM package	✓ (community `ggml-org/whisper.spm`)	✗	✓	✓ (from the main org)
Ready-made Android Gradle/AAR package	✗ (only `.android` example)	✗ (bindings only)	✓	✓
Punctuation by default	✓ (Whisper is trained with p&c)	✗ (needs +1.6 GB model)	✗ (separate post-processor)	✓ (in-model)
License	MIT (code + weights)	Apache-2.0 (code + weights)	Apache-2.0 (code + weights)	Apache-2.0 wrapper / proprietary runtime / CC-BY-4.0 model

¹ Source: HuggingFace model card openai/whisper-base.en, evaluation section.
² Source: alphacephei.com/vosk/models, evaluation table.
³ Source: sherpa-onnx pre-trained models page (sherpa-onnx-streaming-zipformer-en-2023-06-26).
⁴ Source: HuggingFace model card nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc, RNN-T on LibriSpeech test set.

Accuracy. No single option is a strict winner. Whisper base.en (4.27%) is currently the most accurate of the numbers we have side-by-side, at 2.4× the model size of VoxRT. sherpa-onnx running a recent streaming Zipformer is in the same accuracy band as Whisper base.en, at roughly 4× the model size of VoxRT (fp32) or comparable to VoxRT (int8, with the usual quantization tradeoffs). VoxRT (6.96%) trades some accuracy for a much smaller model with punctuation baked in. Vosk small (9.85%) is the least accurate, and its punctuation model triples the "small" story once you actually need punctuated output.

Size. VoxRT is the smallest option that ships with punctuation in the box. Vosk small looks smaller (40 MB) until you add the 1.6 GB vosk-recasepunc-en-0.22 model to get usable output, at which point Vosk becomes the heaviest in the table by a wide margin.

SDK integration. Both sherpa-onnx and VoxRT ship SPM (iOS) + Gradle/AAR (Android) packages, which is the meaningful gap versus whisper.cpp and Vosk. Between them: sherpa-onnx has a broader model zoo and a fully open-source runtime (heavier native binary, but no proprietary components). VoxRT has a smaller native runtime (~500 KB per platform on top of the model) and in-model punctuation, at the cost of a proprietary native library.

Licensing. whisper.cpp, Vosk, and sherpa-onnx are fully open source in code and weights, permissive licenses across the board. VoxRT is a hybrid stack. The Kotlin/Swift wrapper is Apache-2.0, safe to embed in closed-source products. The native Rust runtime is proprietary (LICENSE-BINARY, Elephant Enterprises LLC), shipped as part of the SDK. The model is under NVIDIA's CC-BY-4.0, which requires an attribution line in your app's credits section.

On decoders. VoxRT's .vxrt file ships both an RNN-T and a CTC decoder in the same weights, switchable via an API constant. RNN-T is what the table's WER reflects, and it's the more accurate of the two. CTC runs faster at inference and is useful when latency budget is tighter than accuracy budget.

For the rest of this post I'll use VoxRT as the example. The two-line dependency setup keeps the tutorial short, and the pipeline logic (VAD → ASR → text) is the same for any of the four SDK options.

Installing the SDK

Android. Add JitPack as a repository source in settings.gradle.kts:

dependencyResolutionManagement {
    repositories {
        maven { url = uri("https://jitpack.io") }
    }
}

Two dependencies in your app module's build.gradle.kts:

dependencies {
    implementation("com.github.VoxRT:voxrt-silero-android:v0.1.2")
    implementation("com.github.VoxRT:voxrt-asr-android:v0.1.1")
}

Drop the model files (.vxrt) into your app's assets/: silero_vad.vxrt (~1.2 MB) for VAD and streaming_medium_pc.vxrt (60.4 MB) for ASR. Download them from the releases of the corresponding repos at github.com/VoxRT.

One more thing for Android: tell Gradle not to compress the .vxrt assets, so the SDK can memory-map them directly from the APK. Add this to the android { ... } block in your app's build.gradle.kts:

android {
    androidResources {
        noCompress.add("vxrt")
    }
}

Skipping this leads to slower init and, on some setups, runtime errors when opening the model file descriptor.

Minimum supported Android is API 26 (Android 8.0). Architecture: arm64-v8a (which covers 99% of modern devices).

iOS. Add two packages to Package.swift:

dependencies: [
    .package(url: "https://github.com/VoxRT/voxrt-silero-ios.git", from: "0.1.3"),
    .package(url: "https://github.com/VoxRT/voxrt-asr-ios.git", from: "0.1.2"),
]

Or through the Xcode UI: File → Add Packages → paste URL. Model files go into the app bundle (Copy Bundle Resources).

Minimum iOS is 16.0, arm64 devices (iPhone 8 and newer, plus Apple Silicon simulators). iOS 16 dropped support for the older 64-bit iPhones (5s / 6 / 6 Plus / 6s / 6s Plus / 7 / 7 Plus / SE 1st gen), so those don't run this stack.

The basic workflow: code

Both platforms below. The logic is fully symmetric.

Kotlin (Android):

import com.voxrt.silero.VoxrtSileroVadEngine
import com.voxrt.silero.VadEvent
import com.voxrt.asr.VoxrtAsrStreamingEngine

class VoiceNotesRecorder(context: Context) {
    private val vad = context.assets.openFd("silero_vad.vxrt").use { fd ->
        VoxrtSileroVadEngine.fromAssetFd(fd)
    }
    private val asr = context.assets.openFd("streaming_medium_pc.vxrt").use { fd ->
        VoxrtAsrStreamingEngine.fromAssetFd(fd)
    }

    private var inSpeech = false

    fun processAudioFrame(pcm: ShortArray) {
        val events = vad.processPcm(pcm)

        for (event in events) {
            when (event) {
                is VadEvent.SpeechOnset -> {
                    asr.reset()  // fresh K/V cache + LSTM state for the new utterance
                    inSpeech = true
                }
                is VadEvent.SpeechOffset -> {
                    val tail = asr.stop()  // drain any final text the model was still holding
                    if (tail.isNotEmpty()) onTranscript(tail)
                    inSpeech = false
                }
            }
        }

        // Feed audio to the ASR only while VAD says we're inside speech.
        // processPcm returns the text emitted during this call — often empty
        // until ~1 s of audio has accumulated, then non-empty each chunk.
        if (inSpeech) {
            val floatPcm = FloatArray(pcm.size) { i -> pcm[i] / 32768f }
            val delta = asr.processPcm(floatPcm)
            if (delta.isNotEmpty()) onTranscript(delta)
        }
    }

    fun stop() {
        if (inSpeech) {
            val tail = asr.stop()
            if (tail.isNotEmpty()) onTranscript(tail)
        }
        asr.close()
        vad.close()
    }

    private fun onTranscript(text: String) {
        // Save to your local DB: SQLite, Room, whatever
        // Nothing goes over the network
    }
}

All you have to do from the outside: feed processAudioFrame with PCM frames from the microphone (16 kHz mono, Int16). The standard AudioRecord API on Android delivers exactly that format. onTranscript fires whenever the model emits a chunk of text: during speech as audio accumulates, and once more at the end of each utterance to flush the tail.

Swift (iOS):

import VoxrtSilero
import VoxrtAsr

class VoiceNotesRecorder {
    private let vad: VoxrtSileroVadEngine
    private let asr: VoxrtAsrStreamingEngine
    private var inSpeech = false

    init() throws {
        guard let vadURL = Bundle.main.url(forResource: "silero_vad", withExtension: "vxrt"),
              let asrURL = Bundle.main.url(forResource: "streaming_medium_pc", withExtension: "vxrt") else {
            fatalError(".vxrt files not found in bundle")
        }
        vad = try VoxrtSileroVadEngine(modelURL: vadURL)
        asr = try VoxrtAsrStreamingEngine(modelURL: asrURL)
    }

    func processAudioFrame(_ pcm: [Int16]) throws {
        let events = try vad.processPcm(pcm)

        for event in events {
            switch event {
            case .speechOnset(_):
                try asr.reset()  // fresh K/V cache + LSTM state for the new utterance
                inSpeech = true
            case .speechOffset(_):
                let tail = try asr.stop()  // drain any final text the model was still holding
                if !tail.isEmpty { onTranscript(tail) }
                inSpeech = false
            }
        }

        // Feed audio to the ASR only while VAD says we're inside speech.
        // processPcm returns the text emitted during this call — often empty
        // until ~1 s of audio has accumulated, then non-empty each chunk.
        if inSpeech {
            let floatPcm = pcm.map { Float($0) / 32768.0 }
            let delta = try asr.processPcm(floatPcm)
            if !delta.isEmpty { onTranscript(delta) }
        }
    }

    // iOS relies on ARC to release the ASR engine when this instance
    // deallocates — VoxrtAsrStreamingEngine has no explicit close() on iOS.
    // The VAD engine ships close() from an earlier API; call it here.
    func stop() throws {
        if inSpeech {
            let tail = try asr.stop()
            if !tail.isEmpty { onTranscript(tail) }
        }
        vad.close()
    }

    private func onTranscript(_ text: String) {
        // Save locally: CoreData / SwiftData / SQLite
    }
}

Audio on iOS comes from AVAudioEngine via installTap. Converting to 16 kHz Int16 mono is a standard task via AVAudioConverter, documented by Apple.

What this code does:

Audio from the microphone goes into VAD frame by frame.
When VAD signals speech started, we call asr.reset() to give the ASR fresh K/V cache and LSTM state for the new utterance.
During speech, each PCM frame is fed straight into asr.processPcm(). The ASR returns the delta of text emitted during that call. It's usually empty for the first ~1 s while the encoder accumulates audio, then non-empty every chunk boundary. The transcript grows on screen while the user is still talking.
When VAD signals speech ended, we call asr.stop() to drain any tail the model was still holding, then wait for the next onset. The same engine handles the next utterance after another reset().
On recorder stop(), we flush any in-flight utterance and release resources.

Skipped for brevity: error handling, microphone permissions (declare in AndroidManifest.xml and Info.plist), background execution, and converting mic audio to the right format. Standard mobile plumbing, not specific to voice recognition.

Real numbers: device performance

Any developer reading this and thinking about integration will ask: how heavy is this on the device? Numbers below come from real measurements, hardware specified for each.

VAD (Silero via the VoxRT runtime):

Snapdragon 662 (mid-range Android from 2020): RTF 3.05%, latency ~1 ms per 32-ms frame.
iPhone 13 Pro Max (Apple A15): RTF 1.85%, latency ~0.6 ms per 32-ms frame.

Both numbers mean the same thing: VAD is essentially free. Even on a mid-range Android from five years ago, it takes ~3% of one CPU core.

ASR (streaming FastConformer via the VoxRT runtime):

Snapdragon 662: RTF 0.302 (file replay) / 0.353 (live-mic). In practice, the processor handles one second of audio in about 300-350 milliseconds.
iPhone 13 Pro Max: RTF 0.08-0.10. On A15 it processes roughly 10× faster than real time.
First-output latency: ~1.12 s. This is the sum of two separate config values, not one. The encoder consumes speech in 1040 ms streaming chunks, and uses an 80 ms cache-aware lookahead to peek ahead within a chunk. Combined, that's the minimum audio you need before the model emits its first tokens. After that first output, new text flows every ~1 s as each chunk completes.

Accuracy (WER on LibriSpeech test):

RNN-T decoder (default): 6.96%, from the NVIDIA model card on the LibriSpeech test set. This is the number we run in production. We don't fine-tune the upstream weights.
The same .vxrt file also ships a CTC decoder, switchable via API constant. CTC runs faster than RNN-T at some accuracy cost. We publish only the RNN-T number.

Punctuation and capitalization come from the model directly, no post-processing. The output reads like normal English, not one long lowercase stream.

File sizes (what ends up in your APK/IPA):

VAD model: 1.2 MB
ASR model: 60.4 MB
VAD native binary (Android, stripped): ~424 KB
VAD framework (iOS, compressed): ~500 KB

Total impact on app size: ~62 MB (mostly the ASR model). Comparable to what an average App Store game takes, and smaller than the full whisper.cpp base.en (142 MB) or Vosk with punctuation (1.64 GB).

What else a production app needs

The tutorial above is the minimum working setup. A production app needs a few more things I won't walk through line by line but should mention.

Permissions. Android: RECORD_AUDIO in the manifest plus a runtime permission request. iOS: NSMicrophoneUsageDescription in Info.plist. Standard procedure, documented by both Apple and Google.

Background recording. iOS: background modes → audio. Android: foreground service with a notification. Without these, iOS kills your recorder when the app goes to background, and Android does the same after 5-10 minutes.

Local encryption for transcripts. Since we promised the user privacy, it is worth encrypting saved notes. iOS: Keychain for the key plus CryptoKit for AES-GCM. Android: EncryptedSharedPreferences, or Keystore plus Cipher. Simple plumbing, but must-have for privacy-first positioning.

Microphone interruption handling. An incoming call arrives, you have to stop the recording, release the audio session, then restore it. iOS: AVAudioSession.interruptionNotification. Android: AudioManager.OnAudioFocusChangeListener.

Model lifecycle. ASR models take ~150 MB RAM in the loaded state. If your app goes to background for a long time, call close() on the engine and reinitialize on return. VAD is light enough to keep loaded all the time.

Wrapping up

Private voice notes without the cloud are not an abstract idea or a heavy engineering effort. In one evening you can integrate a full pipeline (VAD → ASR → local storage) into an existing iOS/Android app. Result: the user gets an Otter-like experience, but not a single byte of audio leaves the device.

Three things worth remembering when picking a stack:

RTF is the target metric for on-device work. A good solution keeps transcription under 40% RTF on a mid-range Android and under 10% on iPhone.
Punctuation by default saves you 1.6 GB (Vosk's punctuation model) or the need to write custom post-processing.
Ready-made SPM (iOS) and Gradle (Android) packages from a single project save several days of boilerplate compared to wrapping a C++ library yourself, especially on Android where whisper.cpp and Vosk still require your own wrapper.

Tools used in this post:

Silero VAD wrapper: github.com/VoxRT/voxrt-silero-android / voxrt-silero-ios
Streaming ASR wrapper: github.com/VoxRT/voxrt-asr-android / voxrt-asr-ios
Models: releases in the corresponding -models repos in the github.com/VoxRT organization.
Main VoxRT site with a full stack overview: voxrt.com.

Full example code, including the wiring around AudioRecord / AVAudioEngine, will land as a small demo app in a follow-up post.

Attribution: the streaming ASR model is a derivative of NVIDIA NeMo stt_en_fastconformer_hybrid_medium_streaming_80ms_pc, released under CC-BY-4.0.