Always-Listening Voice Commands for Vaadin web applications

This small tutorial takes 15 minutes from the start to a working demo. We use Picovoice Porcupine Wake Word Engine to enable a Vaadin-based Java web application.

(If you don't have that much time: clone the repo, create a accesskey and run it.)

Wake Word Detection is also known as Keyword Spotting, Hotword Detection, Always-Listening Voice Commands, Trigger Word Detection, and Voice Activation.

Setup the Project

The first step is to create a new Vaadin Java application project:

1. Download a new project from start.vaadin.com with an empty view.

2. Create new Java API for always-on keyword detection and import Picovoice libraries:

package com.example.application;

import com.vaadin.flow.component.UI;
import com.vaadin.flow.component.dependency.JsModule;
import com.vaadin.flow.component.dependency.NpmPackage;

@NpmPackage(value = "@picovoice/porcupine-web", version = "2.1.16")
@NpmPackage(value = "@picovoice/web-voice-processor", version = "4.0.5")
@JsModule("./porcupine-integration.js")
public class Porcupine { 

    boolean started = false;

    public Porcupine(String picovoiceAccesskey) {
        UI.getCurrent().getPage().executeJs("window.vaadinPorcupine.key=$0;"
                    , picovoiceAccesskey);
    }

    public void start() {
        this.started = true;
        UI.getCurrent().getPage().executeJs("window.vaadinPorcupine.start()");
    }

    public void stop() {
        this.started = false;
        UI.getCurrent().getPage().executeJs("window.vaadinPorcupine.stop()");
    }

    public boolean isStarted() {
        return this.started;
    }
}

This is the server API available for the rest of the Vaadin Java application.

3. Create porcupine-integration.js in project’s frontend folder.

import { WebVoiceProcessor } from "@picovoice/web-voice-processor";
import { PorcupineWorker, BuiltInKeyword } from "@picovoice/porcupine-web";
import modelParams from "./porcupine_params.js";

// a global 'vaadinPorcupine' integration instance is enough
window.vaadinPorcupine = window.vaadinPorcupine || {
    key: null,
    async start() {
        console.log('Starting wake word detection');
        window.vaadinPorcupine._worker = window.vaadinPorcupine._worker ||         
        await PorcupineWorker.create(
            window.vaadinPorcupine.key,
            [BuiltInKeyword.Computer],
            window.vaadinPorcupine.keywordDetectionCallback,
            {base64: modelParams }
        );
        await WebVoiceProcessor.subscribe(window.vaadinPorcupine._worker);
    },
    async stop() {
        console.log('Stopping wake word detection');
        await WebVoiceProcessor.unsubscribe(window.vaadinPorcupine._worker);
    },
    keywordDetectionCallback(detection) {
        console.log(`Detected keyword: ${detection.label}`);
        const e = new CustomEvent("voice-wakeword", 
                    { "detail": detection.label });
        document.body.dispatchEvent(e);
    }
}

This is the client side part of the API integrating the in-browser wake word detection library.

4. Download the Porcupine model (i.e. Deep Neural Network). From the project frontend folder, run the following to turn the binary .pv model into a base64 string model.

echo "const model_params='$( cat porcupine_params.pv | base64 )';\nexport default model_params;\n\n" > porcupine_params.js

5. Get your Picovoice AccessKey

Go to Picovoice Console's dashboard. Copy your AccessKey.

6. Add a test button to the EmptyView.java to turn the wake word detection on and off:

final Porcupine porcupine = new Porcupine(System.getenv("PICOVOICE_ACCESSKEY"));
add(new Button("Start/Stop wake word detection", e -> { 
    if (!porcupine.isStarted()) {porcupine.start(); }
    else { porcupine.stop();}
}));

Run the application using your own accessKey:

PICOVOICE_ACCESSKEY=your_accesskey_here ./mvnw

Source Code and more examples?

The source code for a fully-working Vaadin demo with Porcupine is available in my GitHub repository. I’ve added a custom event and handler example there and later I’ll show how to train and add your own custom wake word using Picovoice Porcupine… What should I do next?