DEV Community: VoxRT

Add a wake-word to your web app in 5 minutes

VoxRT — Wed, 02 Sep 2026 08:00:03 +0000

Voice interfaces are getting easier to add to the web, but wake-words have been the hard part. Cloud APIs mean latency, per-request billing, and audio leaving the user's device. On-device wake-words in the browser were awkward until WebAssembly SIMD128 shipped in every major browser.

You will have a working "Hey Assistant" detector in about five minutes. Everything runs on-device, no cloud, no per-request cost.

What you will need

A modern browser (Chrome/Edge 91+, Firefox 89+, or Safari 16.4+). WASM SIMD128 is a hard requirement
A microphone
Any static file server on HTTPS or localhost. Browsers block getUserMedia on plain HTTP
Node.js if you want npm install. Otherwise CDN works fine

The full download is ~275 KB (170 KB WASM runtime + ~100 KB model). That is smaller than most icon fonts.

Step 1: install the SDK

Two options.

via npm:

npm install @voxrt/wake-word-browser

via CDN (no build step):

import init, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";

Use CDN to skip the bundler.

Step 2: HTML shell

Nothing fancy. A single button and a status line will do:

<!DOCTYPE html>
<html>
<head><meta charset="utf-8"><title>Wake-word demo</title></head>
<body>
  <button id="start">Start listening</button>
  <p id="status">Idle</p>
  <script type="module" src="./app.js"></script>
</body>
</html>

Serve this over HTTPS or localhost. Microphone access does not work on plain http:// except for localhost.

Step 3: initialize the engine and load the model

Create app.js:

import init, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";

await init();

const modelBytes = new Uint8Array(await (await fetch(
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt_wake_word.vxrt"
)).arrayBuffer());

const engine = WakeWordEngine.fromBytes(modelBytes);
engine.threshold = 0.9;

That is the whole "load the model" step. The .vxrt file is the pre-trained "Hey Assistant" model at ~100 KB. init() bootstraps the WebAssembly runtime.

The threshold is in sigmoid space [0, 1]. Default is 0.9. Lower it and you get more sensitive detection (more false accepts). Raise it and you get fewer false accepts but might miss quieter or accented pronunciations. We will tune this later.

Step 4: capture microphone audio

Wake-word models need 16 kHz mono audio. Modern browsers give you higher sample rates by default, so we ask AudioContext for 16 kHz explicitly. Not all browsers honor the request, so we also check:

document.getElementById("start").onclick = async () => {
  const stream = await navigator.mediaDevices.getUserMedia({audio: true});
  const audioContext = new AudioContext({sampleRate: 16000});
  if (audioContext.sampleRate !== 16000) {
    console.warn(`Got ${audioContext.sampleRate} Hz instead of 16000. Detection quality will drop. Resample or check the SDK docs.`);
  }
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(512, 1, 1);
  const mute = audioContext.createGain();
  mute.gain.value = 0;

  // detection loop goes here (Step 5)

  source.connect(processor);
  processor.connect(mute).connect(audioContext.destination);
  document.getElementById("status").textContent = "Listening";
};

Two notes on the code above:

ScriptProcessorNode is deprecated in favor of AudioWorkletNode for production. For a 5-minute demo, ScriptProcessor is simpler. Swap it out when you productionize
The muted GainNode between processor and destination is a Web Audio quirk. Without connecting the processor to something, audioprocess never fires. Setting gain to 0 avoids sending your microphone straight to the speakers (which would cause feedback)

Step 5: push audio frames and detect

The engine wants Int16 PCM samples in fixed-size chunks. Here is the detection loop:

const pcmBuffer = new Int16Array(512);

processor.addEventListener("audioprocess", (e) => {
  const inputFloat = e.inputBuffer.getChannelData(0);

  // convert Float32 [-1, 1] to Int16
  for (let i = 0; i < inputFloat.length; i++) {
    const s = Math.max(-1, Math.min(1, inputFloat[i]));
    pcmBuffer[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
  }

  // push to engine, iterate detections
  for (const detection of engine.pushPcmI16(pcmBuffer)) {
    console.log(`Wake detected at ${detection.timestampSec.toFixed(3)}s, score ${detection.score.toFixed(4)}`);
    document.getElementById("status").textContent = "Wake-word detected";
  }
});

That is it. Open DevTools (F12, then Console), click the button, say "Hey Assistant", and you should see a log line. The detection.score is the sigmoid probability at the moment of detection.

What just happened

You loaded a ~100 KB model into WebAssembly, streamed 16 kHz mono audio through it in 512-sample frames (32 ms each), and got detections when the score crossed threshold.

The engine handles its own cooldown so you do not get spam detections on the same utterance. Default cooldown is 100 frames (about 1 second at 10 ms hop). You can tune with engine.cooldownFrames.

Two quick customizations

Prefer Float32 over Int16? Skip the conversion:

processor.addEventListener("audioprocess", (e) => {
  const inputFloat = e.inputBuffer.getChannelData(0);
  for (const detection of engine.pushPcmF32(inputFloat)) {
    console.log(`Detected: ${detection.score.toFixed(4)}`);
  }
});

Tuning threshold: Watch engine.currentScore() to see live confidence. If your users often speak quietly, lower the threshold to 0.85. If you get false triggers from radio or TV, raise it to 0.93.

setInterval(() => {
  console.log(`Live score: ${engine.currentScore().toFixed(3)}`);
}, 100);

At the default threshold of 0.9, the model has precision 0.993 and recall 0.982 on the reference test split (11,656 utterances, ROC AUC 0.9966).

Full code (copy-paste starter)

Here is everything above, glued together:

import init, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";

await init();

const modelBytes = new Uint8Array(await (await fetch(
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt_wake_word.vxrt"
)).arrayBuffer());

const engine = WakeWordEngine.fromBytes(modelBytes);
engine.threshold = 0.9;

document.getElementById("start").onclick = async () => {
  const stream = await navigator.mediaDevices.getUserMedia({audio: true});
  const audioContext = new AudioContext({sampleRate: 16000});
  if (audioContext.sampleRate !== 16000) {
    console.warn(`Got ${audioContext.sampleRate} Hz instead of 16000. Detection quality will drop.`);
  }
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(512, 1, 1);
  const mute = audioContext.createGain();
  mute.gain.value = 0;
  const pcmBuffer = new Int16Array(512);

  processor.addEventListener("audioprocess", (e) => {
    const inputFloat = e.inputBuffer.getChannelData(0);
    for (let i = 0; i < inputFloat.length; i++) {
      const s = Math.max(-1, Math.min(1, inputFloat[i]));
      pcmBuffer[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
    }
    for (const detection of engine.pushPcmI16(pcmBuffer)) {
      console.log(`Wake at ${detection.timestampSec.toFixed(3)}s score ${detection.score.toFixed(4)}`);
      document.getElementById("status").textContent = "Wake-word detected";
    }
  });

  source.connect(processor);
  processor.connect(mute).connect(audioContext.destination);
  document.getElementById("status").textContent = "Listening";
};

Serve this alongside the HTML from Step 2 and you have a working wake-word demo.

Where to go from here

Custom phrase or another language: the free tier ships with "Hey Assistant" only. Custom phrases and additional languages are available on a paid tier. We train the model for you and swap the .vxrt file
Native mobile: same SDK exists for iOS (Swift) and Android (Kotlin). Wake-word runs at 1.5% RTF on iPhone A15, 2.1% on Snapdragon 662
Linux and edge: available for Linux aarch64 with bindings for Python, Node.js, Go, and C. Holds at 5.3% RTF sustained on a $15 Raspberry Pi Zero 2 W
Custom on-device pipeline: wake-word into ASR into your app logic works well for privacy-critical voice UIs

One note on licensing

The wrapper (Rust crate, wasm-bindgen bindings, examples) is Apache-2.0 so you can freely integrate it. The compiled WebAssembly runtime and the model weights are proprietary. Redistribution is allowed only as an unmodified part of the SDK package. In practical terms: install and use it in your product, no additional legal setup beyond reading the license.

Live demo: voxrt.com/wake-word-demo

Repo: github.com/VoxRT/voxrt-wake-word-browser

Wake-word on a $15 Raspberry Pi Zero 2 W: 5.3% RTF always-on

VoxRT — Fri, 21 Aug 2026 13:15:02 +0000

The problem: always-on wake-word is hard and expensive

A wake-word detector is the part of the voice stack that listens to the mic 24/7 and triggers the rest of the pipeline when it hears a specific phrase like "Alexa", "Hey Siri", or "Hey Google". Because it is always-on, wake-word dominates the energy and compute budget of any voice-first device.

Ready-made options for hobbyists are limited, and all have trade-offs:

Alexa/Google Nest/Apple HomePod work well but are locked to their vendor platforms, require cloud connectivity in some scenarios, and use closed hardware.
Picovoice / Porcupine is an on-device SDK that works offline after activation. The model is proprietary, and the free tier was deprecated as of 30 June 2026 (now a 7-day trial followed by paid).
openWakeWord is open source (Apache-2.0) with community models, but training a custom wake phrase is non-trivial. You need to gather data, train the model, and iterate.
Custom Python + TF Lite from scratch requires ML expertise and hours of work.

The problem: hobbyists want a simple recipe. Grab a $15 Pi, install a package with one line, hear a custom wake word, do not pay per activation.

This post covers our solution (proprietary, but with a free tier "Hey Assistant" by default). The headline number: 5.3% CPU on one A53 core of a $15 Raspberry Pi Zero 2 W, 24/7 sustained.

What a wake-word detector is: quick technical background

Three components usually connect:

VAD (Voice Activity Detection) is the lightest. It answers "is there any speech in this audio?" (regardless of what kind). Runs at ~1% CPU. Used as a pre-filter for heavier stages.
Wake-word detector answers "is there a specific phrase?". Heavier than VAD (needs actual pattern matching) but still light (typically 10-100 KB model). Runs 1-10% CPU in always-on mode.
ASR (Automatic Speech Recognition) does full transcription: "what exactly was said?". Heavy (~60 MB model, 30-40% CPU on mid-range Android). Only fires after wake-word success.

Wake-word sits in the sweet spot. Not as light as VAD (VAD does not distinguish "Hey Assistant" from "hello world"), not as heavy as ASR (no full vocabulary needed). Cache-friendly architecture, sub-100 ms decisions, runs on any ARM SoC from 2020+ without a DSP.

Why Pi Zero 2 W as target

Raspberry Pi Zero 2 W is the most popular "minimal" Pi (~$15). Specs:

ARM Cortex-A53 quad-core @ 1 GHz: same class as midrange smartphones from 2016-2018
512 MB RAM: enough for small ML models, not for heavy Whisper
No GPU: CPU-only inference
~$15 retail: cheap enough for DIY smart speakers, doorbell notifications, custom voice UI

The logic: if it works on Pi Zero 2 W, it works everywhere. More powerful Pi 4/5 and Jetson Nano are 4-10x faster, embedded ARM SoCs (Rockchip, Allwinner) are comparable class. So Pi Zero 2 W sets the lower bound: if the numbers hold there, they hold everywhere else.

Wake-word across the full Pi ladder:

Device	RTF (Real-Time Factor)	CPU budget
Pi Zero 2 W (Cortex-A53)	0.053 (5.3%)	one core
Pi 3 A+/B/B+ (Cortex-A53)	~0.038-0.044 (3.8-4.4%)	one core
Pi 4 B / 400 (Cortex-A72)	~0.018-0.024 (1.8-2.4%)	one core
Pi 5 (Cortex-A76)	~0.008-0.012 (0.8-1.2%)	one core

Pi Zero 2 W number is measured (60-second sustained live-mic push). Pi 3/4/5 numbers are estimates scaled by clock speed and architecture generation, not independently benchmarked.

RTF 0.053 means the processor spends 53 milliseconds per 1 second of audio. Wake-word takes ~5% of one core on Pi Zero 2 W, leaving 95% plus the other 3 cores for the application (UI, LEDs, LLM inference, storage). On Pi 5, wake-word is essentially free (<1%).

Model architecture

Key characteristics of our wake-word model:

~48 thousand parameters: very small neural network by modern standards (for comparison, Whisper base.en has ~74 million parameters, roughly 1500x larger).
File size: ~100 KB (encrypted .vxrt format).
Sampling rate: 16 kHz mono, 32 ms frames: standard for speech recognition.
Trained on 100% synthetic data with augmentation: no human voice samples in the training set, which addresses licensing concerns for commercial deployment.
Free tier phrase: "Hey Assistant": works out of the box.
Custom phrases: paid tier: training pipeline available on request (help@voxrt.com).

Accuracy metrics (measured on a test set of 5,240 positive + 6,416 negative utterances):

ROC AUC: 0.9966: very clean separation between positive and negative.
PR AUC (Average Precision): 0.9899.
Threshold 0.90: precision 0.993 / recall 0.982.

In practice: at threshold 0.90 the model produces ~1 false positive per 100+ triggers and misses ~2% of real wake-words. Acceptable for an always-on hot loop.

Benchmark: 5.3% RTF sustained on Pi Zero 2 W

RTF (Real-Time Factor) is CPU time divided by audio duration. RTF 5.3% means the processor spends 53 milliseconds of CPU time per 1 second of audio.

How we measured:

Pi Zero 2 W, Cortex-A53 @ 1 GHz, aarch64 Linux
60-second continuous loop with microphone input
Used one core out of four
Sustained RTF (not peak) because sustained better reflects real-world thermal throttling

Why sustained matters more than peak: Pi Zero 2 W without a heat sink can throttle CPU after 2-3 minutes of sustained load. Peak RTF (first 5 seconds) is typically 10-20% better than sustained. We publish the sustained number, 5.3%, because that is what a hobbyist actually gets in production.

What to do with the remaining 95% CPU plus 3 cores: UI (LEDs / display), Bluetooth / WiFi for sending triggers to home assistant, local logic (dimmer, thermostat), or on a mid-tier board like Pi 4, running a smaller LLM for command interpretation.

Why NEON matters: 8.7x speedup

The wake-word runtime is written in Rust. But "pure Rust" is not enough for these RTF numbers. The key ingredient is ARM NEON SIMD intrinsics in the inner loops (matrix multiply, activation functions).

Measured on Snapdragon 662, Cortex-A73 big cluster pinned via HIGH_PERF affinity (SD662 is a big.LITTLE SoC with 4x A73 + 4x A53, and the perf cluster is used for stable clock):

Implementation	RTF
Rust scalar (no SIMD)	0.182 (18.2%)
Rust + NEON SIMD	0.021 (2.1%)

Speedup: 8.7x.

What this means for the reader: even a small neural network (48K params) on CPU-only inference depends critically on SIMD acceleration. Writing your own solution in pure Python + NumPy gives you an RTF that is 10-20x higher than an optimized SIMD implementation.

On Pi Zero 2 W (A53, one generation older than A73), the NEON speedup ratio is similar, but A53 has fewer cycles per second per core. That is why we see 5.3% RTF on A53 vs 2.1% on A73 with the same code.

Practical takeaway: if you plan to write a custom wake-word, targeting ARM NEON is mandatory. Otherwise it will be too slow for always-on mode on any embedded ARM.

How to run: setup + code (4 languages)

The SDK is available for Python, Node.js, Go, and C with an identical API pattern: load a .vxrt file, set threshold and cooldown_frames, push PCM int16 chunks (16 kHz mono), receive detection events with frame_index, timestamp_sec, and score fields.

The voxrt_wake_word.vxrt model (~100 KB, free tier "Hey Assistant") for all examples below can be downloaded from the releases page or HuggingFace.

All examples below are simplified versions of the quickstarts in examples/{python,nodejs,go,c}/ in the repository. Full versions (with WAV parsing, ALSA live-mic, performance timing) live there too.

Python

pip install voxrt-wake-word

from voxrt_wake_word import WakeWordEngine

engine = WakeWordEngine.from_path("voxrt_wake_word.vxrt")
engine.threshold = 0.9
engine.cooldown_frames = 100

for chunk in mic_iter():  # int16 mono @ 16 kHz, 512-sample chunks
    for d in engine.push_pcm_i16(list(chunk)):
        print(f"wake! t={d.timestamp_sec:.3f}s score={d.score:.4f}")

For live mic, install sounddevice (pip install sounddevice) and feed sd.InputStream(samplerate=16000, channels=1, dtype='int16', blocksize=512, callback=...) chunks into engine.push_pcm_i16 inside the callback.

Node.js

npm install @voxrt/wake-word

const { WakeWordEngine } = require("@voxrt/wake-word");

const engine = WakeWordEngine.fromPath("voxrt_wake_word.vxrt");
engine.threshold = 0.9;
engine.cooldownFrames = 100;

for (const chunk of micIter()) {  // Int16Array, 512 samples, 16 kHz mono
  for (const d of engine.pushPcmI16(chunk)) {
    console.log(`wake! t=${d.timestampSec.toFixed(3)}s score=${d.score.toFixed(4)}`);
  }
}

For live mic, use the mic or node-record-lpcm16 npm packages. Both return Node Buffer objects, so convert to Int16Array before pushing: new Int16Array(buf.buffer, buf.byteOffset, buf.length / 2).

Go

go get github.com/VoxRT/voxrt-wake-word-linux/go

package main

import (
    "fmt"
    "log"
    wakeword "github.com/VoxRT/voxrt-wake-word-linux/go"
)

func main() {
    engine, err := wakeword.OpenFromPath("voxrt_wake_word.vxrt")
    if err != nil {
        log.Fatal(err)
    }
    defer engine.Close()
    engine.SetThreshold(0.9)
    engine.SetCooldownFrames(100)

    // micIter returns <-chan []int16 (512-sample chunks, 16 kHz mono)
    for chunk := range micIter() {
        for _, d := range engine.PushPcmI16(chunk) {
            fmt.Printf("wake! t=%.3fs score=%.4f\n", d.TimestampSec, d.Score)
        }
    }
}

For live mic on Linux, use github.com/gordonklaus/portaudio or pipe from arecord (simpler for a quickstart). Have your mic reader emit <-chan []int16 for the range loop above.

C

Install headers and shared library from the tarball release, then build with pkg-config for flags:

gcc main.c $(pkg-config --cflags --libs voxrt-wake-word) -lasound -o wake-word-app

#include <stdint.h>
#include <stdio.h>
#include <voxrt_wake_word.h>

int main(int argc, char **argv) {
    // Load model bytes via mmap. See full example in repo for the ~30-line
    // helper (open + fstat + mmap). Placeholders here for brevity:
    const uint8_t *model_bytes = NULL;   // TODO: mmap voxrt_wake_word.vxrt
    size_t model_len = 0;                // TODO: file size

    voxrt_wake_word_t *engine = NULL;
    voxrt_status_t rc = voxrt_wake_word_create(model_bytes, model_len, &engine);
    if (rc != VOXRT_OK) {
        fprintf(stderr, "voxrt_wake_word_create failed: %d\n", rc);
        return 1;
    }
    voxrt_wake_word_set_threshold(engine, 0.9f);
    voxrt_wake_word_set_cooldown_frames(engine, 100);

    int16_t buf[512];
    voxrt_wake_word_detection_t dets[8];
    for (;;) {
        // Read 512 int16 samples from ALSA / mic into buf.
        size_t written = 0;
        voxrt_wake_word_push_pcm_i16(engine, buf, 512, dets, 8, &written);
        for (size_t i = 0; i < written; ++i) {
            printf("wake! t=%.3fs score=%.4f\n",
                   dets[i].timestamp_sec, dets[i].score);
        }
    }
    voxrt_wake_word_destroy(engine);
    return 0;
}

The full C example with all ALSA setup (~150 lines) is at examples/c/alsa-mic-quickstart/main.c. There is also a C++ variant (alsa-mic-quickstart-cpp) and a CMake consumer example for integrating into existing projects.

Parameters and what is not shown

All four languages use identical parameters:

threshold (default 0.5): confidence threshold for firing. 0.9 is strict (fewer false positives). For always-on in a noisy environment, 0.85 to 0.90 is a reasonable baseline.
cooldown_frames (default 50, ~1.6 sec): how many silence frames after a detection before the next trigger. Prevents double-fire.

Not shown in the snippets (standard boilerplate): microphone init, error handling, action on detection (starting ASR, HTTP webhook to Home Assistant, turning on an LED).

Wake-word alternatives: honest comparison

Three main options for a hobbyist in 2026:

	Porcupine (Picovoice)	openWakeWord	VoxRT wake-word
Model size	~200 KB - 1 MB (varies by tier)	~50-400 KB per model + ~3 MB shared runtime	~100 KB
RTF on Pi Zero 2 W	~3.8% on Pi 3 per Picovoice docs, no Pi Zero 2 W numbers published (likely 4-8% scaled by clock)	reported CPU-heavy on Pi Zero 2 W, no public RTF, on Pi 4 <5 ms per 80 ms chunk	5.3% (measured)
Runtime	proprietary C/C++ SDK	Python + ONNX Runtime	proprietary Rust SDK
License	commercial (7-day trial then paid, no free tier since 30 June 2026)	Apache-2.0 (fully open)	proprietary (LICENSE-BINARY), free default phrase, paid custom
Custom wake phrase	commercial console (paid)	DIY training pipeline (Python, data collection, hours of work)	paid tier (email us)
Setup complexity (out-of-the-box "Hey X")	medium (auth flow)	high (need to train or download a community model)	low (pip install + download)

Each tool has its fit:

Porcupine: if you need a large catalog of pre-trained wake phrases and you are fine with commercial licensing.
openWakeWord: if 100% open source is critical and you have time for DIY training.
VoxRT wake-word: if you need the smallest footprint and simplest deployment ("Hey Assistant" out of the box, custom phrase via paid tier).

No tool is universally "better" than the others. It depends on requirements.

Licensing model

Important disclosure: VoxRT wake-word is a proprietary tool (unlike our Silero VAD packaging, which is MIT).

Runtime + model: LICENSE-BINARY (Elephant Enterprises LLC). Distributable as part of an unmodified SDK, no forking, no rebuilding.
Free tier phrase: "Hey Assistant" works out of the box without activation.
Custom wake phrases: paid tier. We train a custom model on your phrase (typical 2-3 weeks turnaround). Contact: help@voxrt.com.
No per-activation billing: unlike Porcupine, no ongoing fees. A one-time custom-phrase fee covers redistribution in your product.

This is a trade-off: you get the smallest footprint and simplest deployment, but you cannot modify the runtime or train your own model. For a 100% open-source requirement, use openWakeWord (Apache-2.0).

WASM bonus: 0.16% RTF in Chrome

The same ~100 KB .vxrt file runs via WASM SIMD128 in the browser:

Environment	RTF
Chrome / MacBook Pro M4 (WASM SIMD128)	0.16%
Safari / iPhone A15 (WASM)	0.23%

Essentially free. The full npm bundle @voxrt/wake-word-browser is ~275 KB (model + WASM runtime).

Not the primary story of this post, but worth mentioning because it is unusual. Voice AI in the browser without a backend, integrated into a web app via npm install. The same model runs on Pi, iPhone, Android, and in a user's Chrome tab.

Wrapping up

Wake-word detection on a $15 Raspberry Pi Zero 2 W, always on, 5.3% CPU is not an abstract idea. One-line install with pip install voxrt-wake-word, add a microphone reader loop, done.

Three things worth remembering:

RTF sustained (60+ sec) matters more than peak. Thermal throttling on a Pi Zero 2 W without a heat sink makes peak numbers misleading.
ARM NEON SIMD is critical for CPU inference. 8.7x speedup vs scalar Rust. Custom implementations without SIMD will be too slow for always-on.
Wake-word is not VAD, not ASR. Three tiers with different CPU budgets. Wake-word is the middle tier: heavier than VAD, light enough for always-on.

Tools from this post:

github.com/VoxRT/voxrt-wake-word-linux: Linux SDK (aarch64)
github.com/VoxRT/voxrt-wake-word-android: Android SDK
github.com/VoxRT/voxrt-wake-word-ios: iOS SDK
github.com/VoxRT/voxrt-wake-word-browser: WASM browser SDK
huggingface.co/VoxRT/wake-word-hey-assistant-vxrt: free tier model

Previous posts in this voice AI series:

Building Private Voice Notes for iOS and Android Without the Cloud: full tutorial for a VAD + ASR pipeline
Streaming ASR vs Whisper on mobile: when to switch: architectural comparison for live voice apps

Learn more about the runtime and product family: voxrt.com.

Custom wake phrase for your product: help@voxrt.com.

Streaming ASR vs Whisper on mobile: when to switch

VoxRT — Tue, 18 Aug 2026 08:45:10 +0000

The problem: Whisper feels laggy for live apps

Voice-first apps became mainstream in 2024-2026. Voice agents, real-time captions, live meeting transcription, voice notes with instant feedback. Developers building this new generation of apps typically look at Whisper first (whisper.cpp, faster-whisper). It is the de facto default for on-device speech-to-text.

And for batch transcription (upload audio file, wait, get transcript), Whisper is genuinely great. Interview transcription, meeting notes, lecture recordings on MacBook or in an Android app. Whisper handles all these accurately and reliably.

But as soon as you try Whisper for live scenarios (user speaks, expects instant response), something feels off. A 300-500 ms delay minimum shows up, often up to 1-2 seconds. Words get "sliced" at chunk boundaries. Transcript comes out in jerks. For a voice agent that should respond like a human in conversation, this is a frustrating UX.

This is not a Whisper bug. It is a design mismatch: Whisper is a batch model retrofitted for streaming, while a real-time scenario fundamentally needs a different architecture.

This post is about what native streaming ASR is, how it differs from Whisper's approach, what alternatives exist, and when you should switch (or stay).

What "streaming" really means in ASR

Before comparing options, terminology first.

Batch ASR processes audio as a whole: the model receives the entire file (or a fixed-length chunk), runs inference in one pass, returns the full transcript. Classic architectures (Whisper, original Wav2Vec2, seq2seq encoder-decoder) are batch by nature.

Streaming ASR processes audio continuously: the model takes a stream of samples, emits text incrementally as speech arrives. Key streaming parameters: lookahead (how many milliseconds of future audio the model "sees" before emitting text) and chunk latency (how many milliseconds the model buffers between emissions).

Classification by lookahead:

Infinite lookahead: batch (the whole file is known upfront). Whisper, original NeMo Conformer non-streaming.
Large lookahead (1000ms+): pseudo-streaming batch with large chunks. Whisper.cpp streaming mode, faster-whisper VAD-based.
Small lookahead (80-320 ms): native streaming. NeMo FastConformer streaming, streaming Conformer variants, Parakeet streaming, some Kaldi TDNN variants.
Zero lookahead: causal streaming. Fastest response but usually strictly worse accuracy.

The difference between "pseudo-streaming" and "native streaming" is fundamental but often confused. Both technically "stream." But native streaming is trained with a constraint to see only limited future audio, while pseudo-streaming is a hack on top of a batch model.

Whisper's approach: chunked batch retrofit

Whisper is trained on 30-second audio chunks. To get streaming behavior, developers use one of two workarounds.

Whisper.cpp streaming mode: rolling window of 500 ms to 3 s, each chunk is processed as a mini-batch, outputs are concatenated. It works, but has several side effects:

Chunk boundary artifacts: words at chunk borders can be cut, repeated, or misrecognized.
Latency equals chunk size plus inference time. On SD662 with Whisper base.en at 500 ms chunks, perceived latency is 700-1500 ms.
VAD and endpointing need separate tuning (Whisper has no native VAD).
Punctuation can be inconsistent between chunks (the model "loses context" at borders).

VAD-based streaming wrappers (e.g., Whisper-Streaming or WhisperLive built on faster-whisper): VAD triggers, the accumulated segment gets transcribed, output emits. Cleaner output (no chunk boundary artifacts), but latency is even higher (you have to wait for end of speech for VAD trigger).

Neither of these approaches delivers what native streaming does: continuous text stream with sub-100 ms lookahead. This is an architectural limitation, not a bug.

Whisper is not bad. Whisper is not for live. These are two different task categories.

Native streaming ASR: how it works

Native streaming architectures (NeMo FastConformer streaming, streaming Conformer, Parakeet streaming variants) work differently.

Cache-aware inference. The model is trained knowing it will see only limited future audio (typically 80-320 ms lookahead). Internal hidden states cache between chunks. No need to re-process previous audio for each new segment.

Continuous emission. The model emits tokens incrementally as speech arrives. No "chunk boundaries" in perceived output. Text simply prints in real time.

Trade-off exists, but minimal. Native streaming models are usually slightly less accurate than their batch counterparts (the limited future audio constraint reduces context understanding). But the gap is small, typically 0.2-0.5% WER on LibriSpeech test-clean.

Example architecture: NeMo FastConformer streaming (stt_en_fastconformer_hybrid_medium_streaming_80ms_pc by NVIDIA). Roughly 32 million parameters, hybrid decoder (RNN-T + CTC), 80 ms cache-aware lookahead, WER 3.267% on LibriSpeech test-clean with the RNN-T decoder (our WER-500 measurement, see footnote in the comparison table).

Similar variants:

Parakeet streaming: same NeMo family, different size variants.
Streaming Conformer: general architectural family, different implementations from Google, Microsoft, and NVIDIA.
Kaldi TDNN-F: old-school, but streaming, still used in Vosk.

If the task is live, these models deliver a qualitatively different experience than chunked Whisper.

Our implementation: VoxRT, a Rust runtime around NeMo streaming

We (VoxRT) built a Rust runtime that packages NeMo FastConformer streaming (the same cache-aware architecture described above) into ready-made SPM (iOS) and Gradle(Android) packages. The model is stt_en_fastconformer_hybrid_medium_streaming_80ms_pc from NVIDIA, ported into our .vxrt format without architectural changes (fp16 quantization). Our .vxrt format supports AES-256-GCM encryption for proprietary models, but the NeMo streaming ASR model here remains under CC-BY-4.0, not encrypted.

What this gives a developer compared to the "write your own wrapper" approach:

One-line integration in Gradle/SPM. No custom Python + ONNX bridge to write for JNI or Objective-C interop.
Symmetric API surface between iOS and Android. Kotlin and Swift code is nearly identical, same lifecycle pattern (init -> processPcm -> stop).
Native inference via Rust runtime. No PyTorch, no ONNX runtime dependency, minimal APK/IPA footprint (native binary ~424 KB stripped on Android, ~500 KB framework compressed on iOS).
Bundled Silero VAD for pre-filtering. No separate VAD integration to write, the 1.2 MB model works out of the box paired with ASR.

Honest trade-off: our runtime is proprietary (LICENSE-BINARY, Elephant Enterprises LLC), though the wrapper is Apache-2.0. The model is under CC-BY-4.0 (NVIDIA), which requires attribution in your app's credits. For commercial closed-source apps, no compliance issues. Pure GPL/AGPL projects will hit a license conflict with our proprietary runtime. In that case, writing your own wrapper around the open-source NeMo runtime is a better path.

From here on, all "streaming ASR" numbers in the post come from our implementation of this architecture. Whisper and Vosk numbers come from their upstream benchmarks (source cited).

Comparison table

Extended comparison versus the first post: added axes for streaming approach, chunk latency, and ready SDK availability. All numbers cited.

	whisper.cpp (base.en)	whisper.cpp (small.en)	Vosk (small-en-0.15)	VoxRT NeMo FastConformer streaming
Model type	Batch (encoder-decoder Transformer)	Batch	Streaming (Kaldi TDNN-F)	Native streaming (Conformer + cache)
Model size	~149 MB ¹	~489 MB ¹	40 MB (+ 1.6 GB for punctuation) ²	60.4 MB ³
WER (LibriSpeech test-clean)	4.27% ⁴	3.05% ⁴	9.85% ²	3.267% (RNN-T) / 4.895% (CTC) ⁵
Streaming approach	Chunked batch (~500 ms blocks)	Chunked batch	Native streaming	Native cache-aware (80 ms lookahead)
Latency pattern	500-1500 ms per chunk (persistent)	500-1500 ms per chunk	300-500 ms per chunk	1.12 s initial buffer, then continuous ~100-200 ms
Punctuation by default	✓ (in-model)	✓	✗ (needs +1.6 GB model)	✓ (in-model)
Ready iOS SPM package	✗ (only community `ggml-org/whisper.spm`)	✗	✗	✓
Ready Android Gradle package	✗	✗	✗	✓
License	MIT (code + weights)	MIT	Apache-2.0 (code + weights)	Apache-2.0 wrapper / proprietary runtime / CC-BY-4.0 model

¹ whisper.cpp README, ggml format disk size (source lists 142 MiB and 466 MiB, converted to MB).
² alphacephei.com/vosk/models, evaluation table.
³ voxrt-asr-models v0.1.2 downloads table.
⁴ HuggingFace model cards openai/whisper-base.en and openai/whisper-small.en, evaluation section.
⁵ Our WER-500 measurements on LibriSpeech test-clean.

Important latency nuance: whisper.cpp has persistent per-chunk latency (each chunk 500-1500 ms). VoxRT NeMo streaming has an initial buffer of 1.12 s (for cache warmup), then sustained latency of ~100-200 ms perceived. Meaning:

For short queries (under 3 s): whisper.cpp can feel faster (VoxRT's initial buffer is just overhead).
For longer live sessions (over 3 s): VoxRT streaming feels smoother (continuous), whisper.cpp remains "jerky."

Real numbers on mobile

Everything above is architectural comparison. But any dev reading this will ask one question: how heavy is this on the device?

VoxRT NeMo FastConformer streaming:

Snapdragon 662 (mid-range Android from 2020): RTF 0.302 (file replay) / 0.353 (live-mic). That is 300-350 milliseconds of CPU time per 1 second of audio.
iPhone 13 Pro Max (Apple A15): RTF 0.08-0.10. Processor is roughly 10× faster than real time.
Runtime memory: ~150 MB steady-state.
APK/IPA impact: 60.4 MB (model) + ~500 KB (native framework) = ~61 MB added to app size.

VoxRT Silero VAD (used as pre-filter):

SD662: RTF 3.05%, latency ~1 ms per 32-ms frame.
iPhone A15: RTF 1.85%, latency ~0.6 ms.
Model size: 1.2 MB.

Whisper.cpp benchmarks on mobile: no public RTF numbers for Snapdragon or iOS in the whisper.cpp README, only Apple Silicon benchmarks. Community measurements (posts in r/androiddev, r/iOSProgramming) usually put Whisper base.en RTF in the 40-70% range on mid-range Android SoCs from 2020-2022. That is roughly 2× heavier than VoxRT NeMo for comparable accuracy.

Practical conclusion: for always-on live transcription on mid-range Android, Whisper base.en often burns enough CPU to noticeably heat up the device and drain battery. Streaming ASR under 40% RTF is sustainable without visible performance impact.

Decision framework: when to switch, when to stay

Practical rules for the reader.

Stay with Whisper if:

Batch transcription of files (audio, wait, transcript). Live UX not required. Whisper accuracy is great, community is huge, plenty of ready tools.
Short one-shot voice notes (user hits record, speaks 10 seconds, releases, gets transcript). Streaming ASR initial buffer is not justified here.
You do not care about battery or CPU load (e.g., transcription runs only on-demand, not always-on).
You want maximum community support, example code, and integrations.

Switch to streaming ASR if:

Live voice agents (chatbot that responds while user speaks). Every 500-1500 ms delay equals broken conversation.
Real-time captions for accessibility (hearing-impaired users). Here latency directly impacts UX.
Live meeting transcription with on-screen text visible as people speak.
Always-on background listening (voice notes 24/7, wake-word context). CPU load matters.

Go with a hosted API (Deepgram, AssemblyAI, etc.) if:

Concurrency: thousands of parallel sessions. Self-hosted infrastructure becomes overhead.
Diarization required out of the box (who is speaking when). Native streaming ASR usually has no built-in diarization, you need a separate model (pyannote and similar).
Multi-language mixed audio. Streaming models are usually English-focused, hosted APIs have multilingual support.
Budget allows and privacy is not a hard requirement.

Go with self-hosted streaming ASR (native streaming architecture) if:

Privacy is required (data cannot leave device or your infrastructure).
Mobile deployment: apps must work offline.
Scale exists, but budget matters. Hosted APIs get expensive at high volume.
You want low sustained latency without a hosted API dependency.

Wrapping up

Streaming vs batch in ASR is not a runtime tweak, it is an architectural choice. Whisper for batch is a valid default. For live it is a mismatch. Native streaming ASR (NeMo FastConformer, streaming Conformer, Parakeet variants) is a different tool for a different job.

Three things worth remembering when picking a stack:

Latency pattern matters more than absolute latency. Persistent 500 ms per chunk (Whisper chunked) subjectively feels worse than 1 s initial buffer + continuous ~100 ms (native streaming).
Punctuation by default is critical for UX. Whisper and native streaming provide it. Vosk requires a separate 1.6 GB model.
Ready-made mobile SDK. The only project that ships symmetric SPM + Gradle packages for streaming ASR is us (VoxRT). Whisper.cpp has a community SPM (ggml-org/whisper.spm) but no Gradle. Vosk offers only bindings, wrap yourself.

We build a Rust runtime for on-device voice models. Our ASR repos: github.com/orgs/VoxRT/repositories?q=asr+sort:stars. If you are building a live voice app for iOS, Android, or Linux, this stack plus a native streaming model give you everything needed.

Our first post covered building a full on-device voice notes pipeline (VAD + ASR + local storage) with working Kotlin and Swift snippets: dev.to/voxrtio/building-private-voice-notes-for-ios-and-android-without-the-cloud-54a8.

Learn more about the runtime and product family: voxrt.com.

Attribution: the streaming ASR model is a derivative of NVIDIA NeMo stt_en_fastconformer_hybrid_medium_streaming_80ms_pc, released under CC-BY-4.0.

Building Private Voice Notes for iOS and Android Without the Cloud

VoxRT — Wed, 12 Aug 2026 09:01:00 +0000

The problem

Voice-notes apps became one of the main tools for lawyers, doctors, journalists, and anyone who needs to capture a thought by speaking and find it later as text. Otter, Rev, Descript, Notion AI, Fireflies, and dozens of others all work the same way: the user hits "record," audio goes to the company's servers, ASR (automatic speech recognition) runs there, transcript comes back.

The problem with this setup is simple but fundamental: your recordings leave the device. This is not a hypothetical concern. Over the past two years, several major transcription services had incidents where user recordings leaked. Some ended up in publicly accessible cloud buckets. Some got exposed through compromised employee accounts that had training-data access.

For a casual user, that is annoying. For a lawyer discussing case strategy with a client, it is an attorney-client privilege violation. For a doctor, it is a HIPAA problem. For a journalist, it can burn a source. And even without a leak, for many use cases the mere fact of audio being sent to a third-party server is already unacceptable.

There is an alternative, but it needs a different architecture: transcription has to happen on the device itself. Not a single byte of audio leaves the phone. No API keys. No network calls. No dependency on whether your provider's backend is up.

This post walks through how to do it on iOS and Android in roughly one evening of wiring. Preview: the code fits in about 50 lines per platform, the SDK is under 500 KB, and the transcription model is 60 MB.

What "on-device" actually means

Before jumping into code, let's break down what a voice-notes pipeline actually contains. Understanding the architecture makes it easier to pick the right SDK.

Three stages:

1. Microphone. Provides a continuous stream of PCM (uncompressed audio samples): 16 kHz mono, 16-bit signed integers. This is the standard format for speech recognition. The sampling rate covers the human voice range.

2. VAD (Voice Activity Detection). Determines whether the current audio chunk contains speech. A light model (~1 MB), but critical. Without VAD, you would run silence, air-conditioner hum, and chair squeaks through your ASR, which either produces garbage text or just burns CPU and battery for no reason. VAD cuts the input stream into speech segments and drops everything else.

3. ASR (Automatic Speech Recognition). The actual speech-to-text step. A heavier model (~60 MB) that takes an audio segment and returns text. Modern streaming models emit text incrementally as audio comes in, without waiting for the recording to end. That is important for UX: users see the transcript "typing" in real time.

Flow:

[Microphone] → [PCM: 16 kHz mono] → [VAD] → speech present?
    → yes → [segment buffer] → [ASR streaming] → [text]
    → no  → discard

All of this runs locally. The final text goes into a local database (SQLite, CoreData, whatever you use), optionally encrypted. Zero network traffic during transcription.

The key metric for on-device work is RTF (Real-Time Factor): the ratio of processing time to audio length. RTF 5% means the processor spends 50 milliseconds on one second of audio. Lower RTF, less CPU and battery burn. For always-on voice notes, RTF above 50% starts being a problem: the phone heats up and the battery drains. A decent solution keeps RTF under 40% for transcription and under 5% for VAD.

Comparing the alternatives

Before showing VoxRT code, let's take an honest look at the alternatives. A mobile developer in 2026 has three real options for on-device speech: whisper.cpp (a C++ port of OpenAI Whisper), Vosk (Kaldi-based, Apache-2.0), and VoxRT (our Rust runtime on top of Silero VAD and NeMo FastConformer).

	whisper.cpp (base.en)	Vosk (small-en-0.15)	VoxRT ASR (streaming-medium-pc)
Model size	142 MB	40 MB (+ 1.6 GB for punctuation = 1.64 GB total)	60.4 MB
WER on LibriSpeech test-clean	4.27% ¹	9.85% ²	3.267% (RNN-T) / 4.895% (CTC) ³
Streaming	✓ (chunked via `whisper-stream`, ~500 ms blocks)	✓	✓ (native cache-aware, 80 ms lookahead)
Ready-made iOS SPM package	✓ (community: `ggml-org/whisper.spm`)	✗	✓ (from the main org)
Ready-made Android Gradle package	✗ (only `.android` example)	✗ (bindings only)	✓
Punctuation by default	✓	✗ (needs +1.6 GB model)	✓ (in-model)
License	MIT (code + weights)	Apache-2.0 (code + weights)	Apache-2.0 wrapper / proprietary runtime / CC-BY-4.0 model

¹ Source: HuggingFace model card openai/whisper-base.en, evaluation section.
² Source: alphacephei.com/vosk/models, evaluation table.
³ Our WER-500 measurements on LibriSpeech test-clean.

What matters in the table:

Accuracy (WER on LibriSpeech test-clean). VoxRT (3.267% with RNN-T) beats Whisper base.en (4.27%) at a slightly smaller size (60.4 MB vs 142 MB). Vosk small (9.85%) loses to both. That is a mobile-friendly tradeoff. The "full" Vosk (vosk-model-en-us-0.22) reaches 5.69% WER but weighs 1.8 GB. Whisper also has a more accurate tier: whisper.cpp small.en gives 3.05% WER but weighs 466 MB, roughly 7.7× larger than ours for comparable accuracy.

Size. Vosk small is the smallest base package (40 MB), but to get punctuated transcripts you need an extra vosk-recasepunc-en-0.22 model at 1.6 GB. Full-featured Vosk ends up ~27× heavier than VoxRT.

SDK readiness. Only VoxRT ships both SPM (iOS) and Gradle (Android) packages from the same org with symmetric APIs. whisper.cpp has a community SPM package (ggml-org/whisper.spm) but no Gradle package, so on Android you still write your own wrapper around the C++ example. Vosk has neither. Bindings exist for many languages, but no ready-made SPM or Gradle package. If you want a stable, symmetric integration on both mobile platforms without writing your own native wrapper, VoxRT is the only option today.

Licensing. VoxRT SDK is ready for use in commercial iOS/Android apps. The Kotlin/Swift wrapper is Apache-2.0, a standard permissive license, safe to embed in closed-source products. The native Rust runtime is proprietary (LICENSE-BINARY, Elephant Enterprises LLC), shipped as part of the SDK. The model is under NVIDIA's CC-BY-4.0, which needs an attribution line in your app's credits/about section. For comparison: whisper.cpp is MIT, Vosk is Apache-2.0, both fully open-source (including native code).

On our WER. Both decoders live in the same model file, switched via an API constant. RNN-T is more accurate (3.267%), CTC is roughly 15% faster at inference (4.895%). Pick based on whether quality or speed matters more in your scenario.

For the rest of this post I'll use VoxRT as the example, because a one-line dependency setup lets us fit a working tutorial in a reasonable length. The pipeline logic (VAD → ASR → text) is the same for any of the three options.

Installing the SDK

Android. Add JitPack as a repository source in settings.gradle.kts:

dependencyResolutionManagement {
    repositories {
        maven { url = uri("https://jitpack.io") }
    }
}

Two dependencies in your app module's build.gradle.kts:

dependencies {
    implementation("com.github.VoxRT:voxrt-silero-android:v0.1.2")
    implementation("com.github.VoxRT:voxrt-asr-android:v0.1.1")
}

Drop the model files (.vxrt) into your app's assets/: silero_vad.vxrt (~1.2 MB) for VAD and streaming_medium_pc.vxrt (60.4 MB) for ASR. Download them from the releases of the corresponding repos at github.com/VoxRT.

One more thing for Android: tell Gradle not to compress the .vxrt assets, so the SDK can memory-map them directly from the APK. Add this to the android { ... } block in your app's build.gradle.kts:

android {
    androidResources {
        noCompress.add("vxrt")
    }
}

Skipping this leads to slower init and, on some setups, runtime errors when opening the model file descriptor.

Minimum supported Android is API 26 (Android 8.0). Architecture: arm64-v8a (which covers 99% of modern devices).

iOS. Add two packages to Package.swift:

dependencies: [
    .package(url: "https://github.com/VoxRT/voxrt-silero-ios.git", from: "0.1.3"),
    .package(url: "https://github.com/VoxRT/voxrt-asr-ios.git", from: "0.1.2"),
]

Or through the Xcode UI: File → Add Packages → paste URL. Model files go into the app bundle (Copy Bundle Resources).

Minimum iOS is 16.0, arm64 devices (iPhone 5s and newer, plus Apple Silicon simulators).

The basic workflow: code

Both platforms below. The logic is fully symmetric.

Kotlin (Android):

import com.voxrt.silero.VoxrtSileroVadEngine
import com.voxrt.silero.VadEvent
import com.voxrt.asr.VoxrtAsrStreamingEngine

class VoiceNotesRecorder(context: Context) {
    private val vad = context.assets.openFd("silero_vad.vxrt").use { fd ->
        VoxrtSileroVadEngine.fromAssetFd(fd)
    }
    private val asr = context.assets.openFd("streaming_medium_pc.vxrt").use { fd ->
        VoxrtAsrStreamingEngine.fromAssetFd(fd)
    }

    private val speechBuffer = mutableListOf<Float>()
    private var inSpeech = false

    fun processAudioFrame(pcm: ShortArray) {
        val events = vad.processPcm(pcm)

        for (event in events) {
            when (event) {
                is VadEvent.SpeechOnset -> inSpeech = true
                is VadEvent.SpeechOffset -> {
                    if (speechBuffer.isNotEmpty()) {
                        val delta = asr.processPcm(speechBuffer.toFloatArray())
                        if (delta.isNotEmpty()) onTranscript(delta)
                        speechBuffer.clear()
                    }
                    inSpeech = false
                }
            }
        }

        // Convert samples only when we are actually accumulating speech.
        if (inSpeech) {
            for (sample in pcm) speechBuffer.add(sample / 32768f)
        }
    }

    fun stop(): String {
        val tail = asr.stop()
        asr.close()
        vad.close()
        return tail
    }

    private fun onTranscript(text: String) {
        // Save to your local DB: SQLite, Room, whatever
        // Nothing goes over the network
    }
}

All you have to do from the outside: feed processAudioFrame with PCM frames from the microphone (16 kHz mono, Int16). The standard AudioRecord API on Android delivers exactly that format. onTranscript fires whenever a transcript chunk is ready. You can write it straight to the UI or to a database.

Swift (iOS):

import VoxrtSilero
import VoxrtAsr

class VoiceNotesRecorder {
    private let vad: VoxrtSileroVadEngine
    private let asr: VoxrtAsrStreamingEngine
    private var speechBuffer: [Float] = []
    private var inSpeech = false

    init() throws {
        guard let vadURL = Bundle.main.url(forResource: "silero_vad", withExtension: "vxrt"),
              let asrURL = Bundle.main.url(forResource: "streaming_medium_pc", withExtension: "vxrt") else {
            fatalError(".vxrt files not found in bundle")
        }
        vad = try VoxrtSileroVadEngine(modelURL: vadURL)
        asr = try VoxrtAsrStreamingEngine(modelURL: asrURL)
    }

    func processAudioFrame(_ pcm: [Int16]) throws {
        let events = try vad.processPcm(pcm)

        for event in events {
            switch event {
            case .speechOnset(_):
                inSpeech = true
            case .speechOffset(_):
                if !speechBuffer.isEmpty {
                    let delta = try asr.processPcm(speechBuffer)
                    if !delta.isEmpty { onTranscript(delta) }
                    speechBuffer.removeAll()
                }
                inSpeech = false
            }
        }

        // Convert samples only when we are actually accumulating speech.
        if inSpeech {
            speechBuffer.append(contentsOf: pcm.map { Float($0) / 32768.0 })
        }
    }

    func stop() throws -> String {
        let tail = try asr.stop()
        vad.close()
        return tail
    }

    private func onTranscript(_ text: String) {
        // Save locally: CoreData / SwiftData / SQLite
    }
}

Audio on iOS comes from AVAudioEngine via installTap. Converting to 16 kHz Int16 mono is a standard task via AVAudioConverter, documented by Apple.

What this code does:

Audio from the microphone goes into VAD frame by frame.
When VAD signals "speech started," we start accumulating samples in a buffer.
When VAD signals "speech ended," we send the accumulated buffer to ASR, get the text back, and clear the buffer.
On stop (stop()), we drain any remaining buffered text from ASR and release resources.

Skipped for brevity: error handling, microphone permissions (declare in AndroidManifest.xml and Info.plist), background execution, and converting mic audio to the right format. Standard mobile plumbing, not specific to voice recognition.

Real numbers: device performance

Any developer reading this and thinking about integration will ask: how heavy is this on the device? Numbers below come from real measurements, hardware specified for each.

VAD (Silero via the VoxRT runtime):

Snapdragon 662 (mid-range Android from 2020): RTF 3.05%, latency ~1 ms per 32-ms frame.
iPhone 13 Pro Max (Apple A15): RTF 1.85%, latency ~0.6 ms per 32-ms frame.

Both numbers mean the same thing: VAD is essentially free. Even on a mid-range Android from five years ago, it takes ~3% of one CPU core.

ASR (streaming FastConformer via the VoxRT runtime):

Snapdragon 662: RTF 0.302 (file replay) / 0.353 (live-mic). In practice, the processor handles one second of audio in about 300-350 milliseconds.
iPhone 13 Pro Max: RTF 0.08-0.10. On A15 it processes roughly 10× faster than real time.
Chunk latency: ~1.12 s of buffering before the first text output. That is an architectural property of the model: cache-aware streaming with 80 ms lookahead.

Accuracy (WER on LibriSpeech test-clean):

RNN-T decoder (default): 3.267%
CTC decoder (~15% faster inference): 4.895%

Punctuation and capitalization come from the model directly, no post-processing. The output reads like normal English, not one long lowercase stream.

File sizes (what ends up in your APK/IPA):

VAD model: 1.2 MB
ASR model: 60.4 MB
VAD native binary (Android, stripped): ~424 KB
VAD framework (iOS, compressed): ~500 KB

Total impact on app size: ~62 MB (mostly the ASR model). Comparable to what an average App Store game takes, and smaller than the full whisper.cpp base.en (142 MB) or Vosk with punctuation (1.64 GB).

What else a production app needs

The tutorial above is the minimum working setup. A production app needs a few more things I won't walk through line by line but should mention.

Permissions. Android: RECORD_AUDIO in the manifest plus a runtime permission request. iOS: NSMicrophoneUsageDescription in Info.plist. Standard procedure, documented by both Apple and Google.

Background recording. iOS: background modes → audio. Android: foreground service with a notification. Without these, iOS kills your recorder when the app goes to background, and Android does the same after 5-10 minutes.

Local encryption for transcripts. Since we promised the user privacy, it is worth encrypting saved notes. iOS: Keychain for the key plus CryptoKit for AES-GCM. Android: EncryptedSharedPreferences, or Keystore plus Cipher. Simple plumbing, but must-have for privacy-first positioning.

Microphone interruption handling. An incoming call arrives, you have to stop the recording, release the audio session, then restore it. iOS: AVAudioSession.interruptionNotification. Android: AudioManager.OnAudioFocusChangeListener.

Model lifecycle. ASR models take ~150 MB RAM in the loaded state. If your app goes to background for a long time, call close() on the engine and reinitialize on return. VAD is light enough to keep loaded all the time.

Wrapping up

Private voice notes without the cloud are not an abstract idea or a heavy engineering effort. In one evening you can integrate a full pipeline (VAD → ASR → local storage) into an existing iOS/Android app. Result: the user gets an Otter-like experience, but not a single byte of audio leaves the device.

Three things worth remembering when picking a stack:

RTF is the target metric for on-device work. A good solution keeps transcription under 40% RTF on a mid-range Android and under 10% on iPhone.
Punctuation by default saves you 1.6 GB (Vosk's punctuation model) or the need to write custom post-processing.
Ready-made SPM (iOS) and Gradle (Android) packages from a single project save several days of boilerplate compared to wrapping a C++ library yourself, especially on Android where whisper.cpp and Vosk still require your own wrapper.

Tools used in this post:

Silero VAD wrapper: github.com/VoxRT/voxrt-silero-android / voxrt-silero-ios
Streaming ASR wrapper: github.com/VoxRT/voxrt-asr-android / voxrt-asr-ios
Models: releases in the corresponding -models repos in the github.com/VoxRT organization.
Main VoxRT site with a full stack overview: voxrt.com.

Full example code, including the wiring around AudioRecord / AVAudioEngine, will land as a small demo app in a follow-up post.

Attribution: the streaming ASR model is a derivative of NVIDIA NeMo stt_en_fastconformer_hybrid_medium_streaming_80ms_pc, released under CC-BY-4.0.