DEV Community

Sami Ekblad
Sami Ekblad

Posted on • Edited on • Originally published at vaadin.com

6 1

Always-Listening Voice Commands for Vaadin web applications

This small tutorial takes 15 minutes from the start to a working demo. We use Picovoice Porcupine Wake Word Engine to enable a Vaadin-based Java web application.

(If you don't have that much time: clone the repo, create a accesskey and run it.)

Wake Word Detection is also known as Keyword Spotting, Hotword Detection, Always-Listening Voice Commands, Trigger Word Detection, and Voice Activation.

Setup the Project

The first step is to create a new Vaadin Java application project:

1. Download a new project from start.vaadin.com with an empty view.

2. Create new Java API for always-on keyword detection and import Picovoice libraries:

package com.example.application;

import com.vaadin.flow.component.UI;
import com.vaadin.flow.component.dependency.JsModule;
import com.vaadin.flow.component.dependency.NpmPackage;

@NpmPackage(value = "@picovoice/porcupine-web", version = "2.1.16")
@NpmPackage(value = "@picovoice/web-voice-processor", version = "4.0.5")
@JsModule("./porcupine-integration.js")
public class Porcupine { 

    boolean started = false;

    public Porcupine(String picovoiceAccesskey) {
        UI.getCurrent().getPage().executeJs("window.vaadinPorcupine.key=$0;"
                    , picovoiceAccesskey);
    }

    public void start() {
        this.started = true;
        UI.getCurrent().getPage().executeJs("window.vaadinPorcupine.start()");
    }

    public void stop() {
        this.started = false;
        UI.getCurrent().getPage().executeJs("window.vaadinPorcupine.stop()");
    }

    public boolean isStarted() {
        return this.started;
    }
}
Enter fullscreen mode Exit fullscreen mode

This is the server API available for the rest of the Vaadin Java application.

3. Create porcupine-integration.js in project’s frontend folder.

import { WebVoiceProcessor } from "@picovoice/web-voice-processor";
import { PorcupineWorker, BuiltInKeyword } from "@picovoice/porcupine-web";
import modelParams from "./porcupine_params.js";

// a global 'vaadinPorcupine' integration instance is enough
window.vaadinPorcupine = window.vaadinPorcupine || {
    key: null,
    async start() {
        console.log('Starting wake word detection');
        window.vaadinPorcupine._worker = window.vaadinPorcupine._worker ||         
        await PorcupineWorker.create(
            window.vaadinPorcupine.key,
            [BuiltInKeyword.Computer],
            window.vaadinPorcupine.keywordDetectionCallback,
            {base64: modelParams }
        );
        await WebVoiceProcessor.subscribe(window.vaadinPorcupine._worker);
    },
    async stop() {
        console.log('Stopping wake word detection');
        await WebVoiceProcessor.unsubscribe(window.vaadinPorcupine._worker);
    },
    keywordDetectionCallback(detection) {
        console.log(`Detected keyword: ${detection.label}`);
        const e = new CustomEvent("voice-wakeword", 
                    { "detail": detection.label });
        document.body.dispatchEvent(e);
    }
}
Enter fullscreen mode Exit fullscreen mode

This is the client side part of the API integrating the in-browser wake word detection library.

4. Download the Porcupine model (i.e. Deep Neural Network). From the project frontend folder, run the following to turn the binary .pv model into a base64 string model.

echo "const model_params='$( cat porcupine_params.pv | base64 )';\nexport default model_params;\n\n" > porcupine_params.js
Enter fullscreen mode Exit fullscreen mode

5. Get your Picovoice AccessKey

Go to Picovoice Console's dashboard. Copy your AccessKey.

Image description

6. Add a test button to the EmptyView.java to turn the wake word detection on and off:

final Porcupine porcupine = new Porcupine(System.getenv("PICOVOICE_ACCESSKEY"));
add(new Button("Start/Stop wake word detection", e -> { 
    if (!porcupine.isStarted()) {porcupine.start(); }
    else { porcupine.stop();}
}));
Enter fullscreen mode Exit fullscreen mode

Run the application using your own accessKey:

PICOVOICE_ACCESSKEY=your_accesskey_here ./mvnw
Enter fullscreen mode Exit fullscreen mode

Source Code and more examples?

The source code for a fully-working Vaadin demo with Porcupine is available in my GitHub repository. I’ve added a custom event and handler example there and later I’ll show how to train and add your own custom wake word using Picovoice Porcupine… What should I do next?

Top comments (0)

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay