DEV Community

Cover image for Playing With P5.js: Creating a Voice-Controlled Game
Kevin Lewis for Deepgram

Posted on • Originally published at


Playing With P5.js: Creating a Voice-Controlled Game

This is the final part in a series on P5.js (from here 'P5') - a creative coding library that makes working with the Canvas API much easier. In part one, we covered how to draw elements on the screen and react to keyboard and mouse input. We learned how to create common game features in part two - collision detection, entity management, and state management.

In today's tutorial, we'll bring together everything we know to create a voice-controlled game - try the game out now. A new enemy appears coming from one of four directions and begins moving towards you every few seconds. Each direction has a random word associated with it, and if said correctly, a bullet will fly in that direction. If an enemy reaches you, the game is over.

The final code for today's project can be found on GitHub.

Before We Start

You will need a Deepgram API Key - get one here.

Setting Up State

On your computer, create a new directory and open it in your code editor. Create an index.html file and add the following to it:

<!DOCTYPE html>
    <script src=""></script>
        // Global Variable Section Starts
        let playerSize = 50
        let score = 0
        let gameOver = false
        // Global Variable Section Ends

        function setup() {
            createCanvas(1000, 1000)

        function draw() {
            translate(width/2, height/2)

            text(`Score: ${score}`, width/2-20, height/2-20)

            if(!gameOver) {
                circle(0, 0, playerSize)

                // Game logic goes here

            } else {
                text(`Game over! Score: ${score}`, 0, 0)
Enter fullscreen mode Exit fullscreen mode

In the second post in this series, you learned how to keep score and show a game over screen - we are using both approaches here.

The only new thing here is translate(width/2, height/2), which moves the origin (0, 0) to the center of the canvas. This means the top-left is now (-500, -500), and the bottom-right is (500, 500). It makes sense to do this when entities often need to refer to the center position.

Create Enemies

At the bottom of your <script>, create a new Enemy class:

class Enemy {
    constructor(direction, distance) {
        this.direction = direction
        this.size = 25
        this.x = 0
        this.y = 0

        if(this.direction == 'UP') this.y = -Math.abs(distance)
        if(this.direction == 'RIGHT') this.x = distance
        if(this.direction == 'DOWN') this.y = distance
        if(this.direction == 'LEFT') this.x = -Math.abs(distance)

    move() {
        if(this.direction == 'UP') this.y++
        if(this.direction == 'RIGHT') this.x--
        if(this.direction == 'DOWN') this.y--
        if(this.direction == 'LEFT') this.x++

    touchedPlayer() {
        const d = dist(this.x, this.y, 0, 0)
        if(d < (playerSize/2) + (this.size/2)) gameOver = true

    display() {
        ellipse(this.x, this.y, this.size)
Enter fullscreen mode Exit fullscreen mode

When an instance is created, you must provide two arguments - direction - one of 'UP', 'DOWN', 'LEFT', or 'RIGHT', and distance - which dictates how far away from the center point the enemy should spawn.

In the constructor, the enemies are initially placed, and in move() they move one pixel closer to the center. touchedPlayer() uses collision detection -- we learned about that last week -- to set gameOver to true if an enemy touches the player in the center of the canvas. Finally, the enemy is drawn at its new (x, y) position.

In your global variable section, add these line:

let directions = ['UP', 'DOWN', 'LEFT', 'RIGHT']
let enemies = []
Enter fullscreen mode Exit fullscreen mode

At the bottom of your setup() function, begin spawning enemies randomly every 2-5 seconds:

setInterval(() => {
    enemies.push(new Enemy(random(directions), width/4, width/2))
}, random(2000, 5000))
Enter fullscreen mode Exit fullscreen mode

The first argument will be randomly chosen from the directions array you just created. The final step is to loop through all existing enemies and run their methods in draw(). In your game logic section, add this code:

for(let enemy of enemies) {
Enter fullscreen mode Exit fullscreen mode

Open index.html in your browser, and it should look like this:

A black square with a small white circle in the middle. The bottom-right reads 'Score: 0'. Small gray circles representing enemies appear either above, below, left, or right, and move towards the center. An enemy touches the center circle and the screens ays "Game over"

Create Bullets

Currently, there's no way to defend yourself. When a player presses their arrow keys, a new bullet will be created in that direction.

At the bottom of your <script>, create a new Bullet class. It should look familiar as it works largely the same as the Enemy class:

class Bullet {
    constructor(direction) {
        this.direction = direction
        this.size = 5
        this.speed = 6
        this.x = 0
        this.y = 0
        this.spent = false

    move() {
        if(this.direction == 'UP') this.y -= this.speed
        if(this.direction == 'RIGHT') this.x += this.speed
        if(this.direction == 'DOWN') this.y += this.speed
        if(this.direction == 'LEFT') this.x -= this.speed

    touchedEnemy() {
        for(let enemy of enemies) {
            const d = dist(enemy.x, enemy.y, this.x, this.y)
            if(d < (this.size/2) + (enemy.size/2)) {
                enemies = enemies.filter(e => e != enemy)
                this.spent = true

    display() {
        ellipse(this.x, this.y, this.size)
Enter fullscreen mode Exit fullscreen mode

If an enemy is hit, it is removed from the enemies array, and the bullet's this.spent value becomes true. In the global variable section, add a new array for bullets:

let bullets = []
Enter fullscreen mode Exit fullscreen mode

Underneath our enemies loop in draw(), add a loop for bullets:

for(let bullet of bullets) {
    if(!bullet.spent) {
Enter fullscreen mode Exit fullscreen mode

If the bullet has been spent, it won't be shown or run its collision detection logic. This means a bullet can only successfully hit an enemy once.

So far, you have used the P5 preload(), setup(), and draw() functions, but there are a host more that are triggered based on user input.

Unlike the keyIsPressed variable which is true every frame that a key is pressed, the built-in keyPressed() function is triggered only once when a user presses a key on their keyboard. In order to trigger the function twice, two distinct presses need to be made - much better for bullet firing. After you end the draw() function, add this:

function keyPressed() {
    if(key == 'ArrowLeft') bullets.push(new Bullet('LEFT'))
    if(key == 'ArrowRight') bullets.push(new Bullet('RIGHT'))
    if(key == 'ArrowUp') bullets.push(new Bullet('UP'))
    if(key == 'ArrowDown') bullets.push(new Bullet('DOWN'))
Enter fullscreen mode Exit fullscreen mode

That's the core game finished. Here's how it looks (recording is sped up):

As enemies appraoch the enemy, tiny red dots are fired out of the center player and towards enemies. When they hit an enemy, the bullet and the enemy disappear, and the score goes up by one.

Add Word Prompts

Create a new file called words.js, and copy and paste the content from this file on GitHub. This is a slight reformatting of the adamjgrant/Random-English-Word-Generator-42k-Words of over 42,000 English words.

As a note, this is a pretty long word list and includes some pretty long and complex words. You may want to experiment with the word selection you use to alter the difficulty.

Just before the <script> tag with our P5 logic, include the words.js file:

<script src="words.js"></script>
Enter fullscreen mode Exit fullscreen mode

Then, in your main <script> tag with our P5 logic, add the following:

function getRandomWord() {
    return words[Math.floor(Math.random() * 42812)]
Enter fullscreen mode Exit fullscreen mode

This function gets one word at random and returns the string. You can add it anywhere, but I tend to add these utility functions to the very bottom of my <script>.

In your global variable section, store four random words:

let currentWords = {
    UP: getRandomWord(),
    DOWN: getRandomWord(),
    LEFT: getRandomWord(),
    RIGHT: getRandomWord()
Enter fullscreen mode Exit fullscreen mode

Just after your bullet loop in the game logic section, draw the four random words to the canvas:

text(currentWords.UP, 0, -height/2+48)
text(currentWords.DOWN, 0, height/2-48)
text(currentWords.RIGHT, width/2-48, 0)
text(currentWords.LEFT, -width/2+48, 0)
Enter fullscreen mode Exit fullscreen mode

Finally, in the Bullet.touchedEnemy() function, where we increment the score, replace a word when an enemy is hit:

currentWords[enemy.direction] = getRandomWord()
Enter fullscreen mode Exit fullscreen mode

As enemies are hit, the word in their direction changes.

Shoot Bullets With Your Voice

It's time to create bullets with your voice! A persistent WebSocket connection will be made with Deepgram, allowing Deepgram to constantly listen to your mic to hear what you say.

This part of the tutorial will assume you know how to do live browser transcription with Deepgram. If not, we have a written and video tutorial available that explains every step in more detail.

In your global variable section, create one final value so we can display to the user what was heard:

let heard = ''
Enter fullscreen mode Exit fullscreen mode

At the very bottom of your <script>, add this:

navigator.mediaDevices.getUserMedia({ audio: true }).then(stream => {
    if (!MediaRecorder.isTypeSupported('audio/webm')) return alert('Browser not supported')
    const mediaRecorder = new MediaRecorder(stream, { mimeType: 'audio/webm' })
    const socket = new WebSocket('wss://', [ 'token', 'YOUR-DEEPGRAM-API-KEY' ])

    socket.onopen = () => {
        mediaRecorder.addEventListener('dataavailable', async (event) => {
            if ( > 0 && socket.readyState == 1) socket.send(

    socket.onmessage = (message) => {
        const received = JSON.parse(
        const transcript =[0].transcript
        if (transcript && received.is_final) {
            heard = transcript
            for(let direction in currentWords) {
                if(transcript.includes(currentWords[direction])) {
                    bullets.push(new Bullet(direction))
Enter fullscreen mode Exit fullscreen mode

Remember to provide your Deepgram API Key when creating the socket. At the bottom of this code, a check determines whether any of the directional words were heard and, if so, creates a bullet in that direction.

Finally, show the user what was heard just under all of the text() statements in draw():

if(`heard) text(`We heard "${heard}"`, -width/2+20, height/2-20)`
Enter fullscreen mode Exit fullscreen mode

In Summary

The fact it was so little code to integrate voice control into this game should be a testament to how easy Deepgram's Speech Recognition API is to use.

Once again, a live version of the game can be found here and the final codebase on GitHub.

If you want to deploy your own, I encourage you to also read how to protect your API Key when doing live transcription directly in your browser.

If you have any questions, please feel free to reach out to us on Twitter at @DeepgramDevs.

Top comments (3)

moose_said profile image
Mostafa Said

This is so cool! Games with AI speech are so cool, I have an idea that I'm working on which is a game with VueJS and Deepgram. I don't know if I will be able to finish up before the Hachathon is over but I will do it anyway.

Thanks for this article :)

_phzn profile image
Kevin Lewis

You should submit it regardless of how finished it is :)

moose_said profile image
Mostafa Said

Maybe I should. Will do that :)

11 Tips That Make You a Better Typescript Programmer


1 Think in {Set}

Type is an everyday concept to programmers, but it’s surprisingly difficult to define it succinctly. I find it helpful to use Set as a conceptual model instead.

#2 Understand declared type and narrowed type

One extremely powerful typescript feature is automatic type narrowing based on control flow. This means a variable has two types associated with it at any specific point of code location: a declaration type and a narrowed type.

#3 Use discriminated union instead of optional fields


Read the whole post now!