DEV Community

Sami Ekblad
Sami Ekblad

Posted on • Edited on • Originally published at vaadin.com

6 1

Always-Listening Voice Commands for Vaadin web applications

This small tutorial takes 15 minutes from the start to a working demo. We use Picovoice Porcupine Wake Word Engine to enable a Vaadin-based Java web application.

(If you don't have that much time: clone the repo, create a accesskey and run it.)

Wake Word Detection is also known as Keyword Spotting, Hotword Detection, Always-Listening Voice Commands, Trigger Word Detection, and Voice Activation.

Setup the Project

The first step is to create a new Vaadin Java application project:

1. Download a new project from start.vaadin.com with an empty view.

2. Create new Java API for always-on keyword detection and import Picovoice libraries:

package com.example.application;

import com.vaadin.flow.component.UI;
import com.vaadin.flow.component.dependency.JsModule;
import com.vaadin.flow.component.dependency.NpmPackage;

@NpmPackage(value = "@picovoice/porcupine-web", version = "2.1.16")
@NpmPackage(value = "@picovoice/web-voice-processor", version = "4.0.5")
@JsModule("./porcupine-integration.js")
public class Porcupine { 

    boolean started = false;

    public Porcupine(String picovoiceAccesskey) {
        UI.getCurrent().getPage().executeJs("window.vaadinPorcupine.key=$0;"
                    , picovoiceAccesskey);
    }

    public void start() {
        this.started = true;
        UI.getCurrent().getPage().executeJs("window.vaadinPorcupine.start()");
    }

    public void stop() {
        this.started = false;
        UI.getCurrent().getPage().executeJs("window.vaadinPorcupine.stop()");
    }

    public boolean isStarted() {
        return this.started;
    }
}
Enter fullscreen mode Exit fullscreen mode

This is the server API available for the rest of the Vaadin Java application.

3. Create porcupine-integration.js in project’s frontend folder.

import { WebVoiceProcessor } from "@picovoice/web-voice-processor";
import { PorcupineWorker, BuiltInKeyword } from "@picovoice/porcupine-web";
import modelParams from "./porcupine_params.js";

// a global 'vaadinPorcupine' integration instance is enough
window.vaadinPorcupine = window.vaadinPorcupine || {
    key: null,
    async start() {
        console.log('Starting wake word detection');
        window.vaadinPorcupine._worker = window.vaadinPorcupine._worker ||         
        await PorcupineWorker.create(
            window.vaadinPorcupine.key,
            [BuiltInKeyword.Computer],
            window.vaadinPorcupine.keywordDetectionCallback,
            {base64: modelParams }
        );
        await WebVoiceProcessor.subscribe(window.vaadinPorcupine._worker);
    },
    async stop() {
        console.log('Stopping wake word detection');
        await WebVoiceProcessor.unsubscribe(window.vaadinPorcupine._worker);
    },
    keywordDetectionCallback(detection) {
        console.log(`Detected keyword: ${detection.label}`);
        const e = new CustomEvent("voice-wakeword", 
                    { "detail": detection.label });
        document.body.dispatchEvent(e);
    }
}
Enter fullscreen mode Exit fullscreen mode

This is the client side part of the API integrating the in-browser wake word detection library.

4. Download the Porcupine model (i.e. Deep Neural Network). From the project frontend folder, run the following to turn the binary .pv model into a base64 string model.

echo "const model_params='$( cat porcupine_params.pv | base64 )';\nexport default model_params;\n\n" > porcupine_params.js
Enter fullscreen mode Exit fullscreen mode

5. Get your Picovoice AccessKey

Go to Picovoice Console's dashboard. Copy your AccessKey.

Image description

6. Add a test button to the EmptyView.java to turn the wake word detection on and off:

final Porcupine porcupine = new Porcupine(System.getenv("PICOVOICE_ACCESSKEY"));
add(new Button("Start/Stop wake word detection", e -> { 
    if (!porcupine.isStarted()) {porcupine.start(); }
    else { porcupine.stop();}
}));
Enter fullscreen mode Exit fullscreen mode

Run the application using your own accessKey:

PICOVOICE_ACCESSKEY=your_accesskey_here ./mvnw
Enter fullscreen mode Exit fullscreen mode

Source Code and more examples?

The source code for a fully-working Vaadin demo with Porcupine is available in my GitHub repository. I’ve added a custom event and handler example there and later I’ll show how to train and add your own custom wake word using Picovoice Porcupine… What should I do next?

Heroku

Simplify your DevOps and maximize your time.

Since 2007, Heroku has been the go-to platform for developers as it monitors uptime, performance, and infrastructure concerns, allowing you to focus on writing code.

Learn More

Top comments (0)

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more

👋 Kindness is contagious

Engage with a sea of insights in this enlightening article, highly esteemed within the encouraging DEV Community. Programmers of every skill level are invited to participate and enrich our shared knowledge.

A simple "thank you" can uplift someone's spirits. Express your appreciation in the comments section!

On DEV, sharing knowledge smooths our journey and strengthens our community bonds. Found this useful? A brief thank you to the author can mean a lot.

Okay