DEV Community

VoxRT
VoxRT

Posted on

Add a wake-word to your web app in 5 minutes

#javascript #webdev #frontend #tutorial

Voice interfaces are getting easier to add to the web, but wake-words have been the hard part. Cloud APIs mean latency, per-request billing, and audio leaving the user's device. On-device wake-words in the browser were awkward until WebAssembly SIMD128 shipped in every major browser.

You will have a working "Hey Assistant" detector in about five minutes. Everything runs on-device, no cloud, no per-request cost.

What you will need

  • A modern browser (Chrome/Edge 91+, Firefox 89+, or Safari 16.4+). WASM SIMD128 is a hard requirement
  • A microphone
  • Any static file server on HTTPS or localhost. Browsers block getUserMedia on plain HTTP
  • Node.js if you want npm install. Otherwise CDN works fine

The full download is ~275 KB (170 KB WASM runtime + ~100 KB model). That is smaller than most icon fonts.

Step 1: install the SDK

Two options.

via npm:

npm install @voxrt/wake-word-browser
Enter fullscreen mode Exit fullscreen mode

via CDN (no build step):

import init, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";
Enter fullscreen mode Exit fullscreen mode

Use CDN to skip the bundler.

Step 2: HTML shell

Nothing fancy. A single button and a status line will do:

<!DOCTYPE html>
<html>
<head><meta charset="utf-8"><title>Wake-word demo</title></head>
<body>
  <button id="start">Start listening</button>
  <p id="status">Idle</p>
  <script type="module" src="./app.js"></script>
</body>
</html>
Enter fullscreen mode Exit fullscreen mode

Serve this over HTTPS or localhost. Microphone access does not work on plain http:// except for localhost.

Step 3: initialize the engine and load the model

Create app.js:

import init, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";

await init();

const modelBytes = new Uint8Array(await (await fetch(
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt_wake_word.vxrt"
)).arrayBuffer());

const engine = WakeWordEngine.fromBytes(modelBytes);
engine.threshold = 0.9;
Enter fullscreen mode Exit fullscreen mode

That is the whole "load the model" step. The .vxrt file is the pre-trained "Hey Assistant" model at ~100 KB. init() bootstraps the WebAssembly runtime.

The threshold is in sigmoid space [0, 1]. Default is 0.9. Lower it and you get more sensitive detection (more false accepts). Raise it and you get fewer false accepts but might miss quieter or accented pronunciations. We will tune this later.

Step 4: capture microphone audio

Wake-word models need 16 kHz mono audio. Modern browsers give you higher sample rates by default, so we ask AudioContext for 16 kHz explicitly. Not all browsers honor the request, so we also check:

document.getElementById("start").onclick = async () => {
  const stream = await navigator.mediaDevices.getUserMedia({audio: true});
  const audioContext = new AudioContext({sampleRate: 16000});
  if (audioContext.sampleRate !== 16000) {
    console.warn(`Got ${audioContext.sampleRate} Hz instead of 16000. Detection quality will drop. Resample or check the SDK docs.`);
  }
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(512, 1, 1);
  const mute = audioContext.createGain();
  mute.gain.value = 0;

  // detection loop goes here (Step 5)

  source.connect(processor);
  processor.connect(mute).connect(audioContext.destination);
  document.getElementById("status").textContent = "Listening";
};
Enter fullscreen mode Exit fullscreen mode

Two notes on the code above:

  • ScriptProcessorNode is deprecated in favor of AudioWorkletNode for production. For a 5-minute demo, ScriptProcessor is simpler. Swap it out when you productionize
  • The muted GainNode between processor and destination is a Web Audio quirk. Without connecting the processor to something, audioprocess never fires. Setting gain to 0 avoids sending your microphone straight to the speakers (which would cause feedback)

Step 5: push audio frames and detect

The engine wants Int16 PCM samples in fixed-size chunks. Here is the detection loop:

const pcmBuffer = new Int16Array(512);

processor.addEventListener("audioprocess", (e) => {
  const inputFloat = e.inputBuffer.getChannelData(0);

  // convert Float32 [-1, 1] to Int16
  for (let i = 0; i < inputFloat.length; i++) {
    const s = Math.max(-1, Math.min(1, inputFloat[i]));
    pcmBuffer[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
  }

  // push to engine, iterate detections
  for (const detection of engine.pushPcmI16(pcmBuffer)) {
    console.log(`Wake detected at ${detection.timestampSec.toFixed(3)}s, score ${detection.score.toFixed(4)}`);
    document.getElementById("status").textContent = "Wake-word detected";
  }
});
Enter fullscreen mode Exit fullscreen mode

That is it. Open DevTools (F12, then Console), click the button, say "Hey Assistant", and you should see a log line. The detection.score is the sigmoid probability at the moment of detection.

What just happened

You loaded a ~100 KB model into WebAssembly, streamed 16 kHz mono audio through it in 512-sample frames (32 ms each), and got detections when the score crossed threshold.

The engine handles its own cooldown so you do not get spam detections on the same utterance. Default cooldown is 100 frames (about 1 second at 10 ms hop). You can tune with engine.cooldownFrames.

Two quick customizations

Prefer Float32 over Int16? Skip the conversion:

processor.addEventListener("audioprocess", (e) => {
  const inputFloat = e.inputBuffer.getChannelData(0);
  for (const detection of engine.pushPcmF32(inputFloat)) {
    console.log(`Detected: ${detection.score.toFixed(4)}`);
  }
});
Enter fullscreen mode Exit fullscreen mode

Tuning threshold: Watch engine.currentScore() to see live confidence. If your users often speak quietly, lower the threshold to 0.85. If you get false triggers from radio or TV, raise it to 0.93.

setInterval(() => {
  console.log(`Live score: ${engine.currentScore().toFixed(3)}`);
}, 100);
Enter fullscreen mode Exit fullscreen mode

At the default threshold of 0.9, the model has precision 0.993 and recall 0.982 on the reference test split (11,656 utterances, ROC AUC 0.9966).

Full code (copy-paste starter)

Here is everything above, glued together:

import init, { WakeWordEngine } from
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt-wake-word-browser.js";

await init();

const modelBytes = new Uint8Array(await (await fetch(
  "https://unpkg.com/@voxrt/wake-word-browser@0.1.1/voxrt_wake_word.vxrt"
)).arrayBuffer());

const engine = WakeWordEngine.fromBytes(modelBytes);
engine.threshold = 0.9;

document.getElementById("start").onclick = async () => {
  const stream = await navigator.mediaDevices.getUserMedia({audio: true});
  const audioContext = new AudioContext({sampleRate: 16000});
  if (audioContext.sampleRate !== 16000) {
    console.warn(`Got ${audioContext.sampleRate} Hz instead of 16000. Detection quality will drop.`);
  }
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(512, 1, 1);
  const mute = audioContext.createGain();
  mute.gain.value = 0;
  const pcmBuffer = new Int16Array(512);

  processor.addEventListener("audioprocess", (e) => {
    const inputFloat = e.inputBuffer.getChannelData(0);
    for (let i = 0; i < inputFloat.length; i++) {
      const s = Math.max(-1, Math.min(1, inputFloat[i]));
      pcmBuffer[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
    }
    for (const detection of engine.pushPcmI16(pcmBuffer)) {
      console.log(`Wake at ${detection.timestampSec.toFixed(3)}s score ${detection.score.toFixed(4)}`);
      document.getElementById("status").textContent = "Wake-word detected";
    }
  });

  source.connect(processor);
  processor.connect(mute).connect(audioContext.destination);
  document.getElementById("status").textContent = "Listening";
};
Enter fullscreen mode Exit fullscreen mode

Serve this alongside the HTML from Step 2 and you have a working wake-word demo.

Where to go from here

  • Custom phrase or another language: the free tier ships with "Hey Assistant" only. Custom phrases and additional languages are available on a paid tier. We train the model for you and swap the .vxrt file
  • Native mobile: same SDK exists for iOS (Swift) and Android (Kotlin). Wake-word runs at 1.5% RTF on iPhone A15, 2.1% on Snapdragon 662
  • Linux and edge: available for Linux aarch64 with bindings for Python, Node.js, Go, and C. Holds at 5.3% RTF sustained on a $15 Raspberry Pi Zero 2 W
  • Custom on-device pipeline: wake-word into ASR into your app logic works well for privacy-critical voice UIs

One note on licensing

The wrapper (Rust crate, wasm-bindgen bindings, examples) is Apache-2.0 so you can freely integrate it. The compiled WebAssembly runtime and the model weights are proprietary. Redistribution is allowed only as an unmodified part of the SDK package. In practical terms: install and use it in your product, no additional legal setup beyond reading the license.

Live demo: voxrt.com/wake-word-demo

Repo: github.com/VoxRT/voxrt-wake-word-browser

Top comments (0)