Breathless Reef: a diving game you play with your face and your breath

This is a submission for the Presage SmartSpectra SDK Challenge

What I Built

Breathless Reef is an underwater diving game. Your camera is the only controller.

You start on the deck of a ship. You smile to jump into the sea. Then:

• you turn your head to swim,
• you blink to fire a harpoon,
• you hold your breath to stay under water,
• and your face changes the sea: a smile makes it bright and calm, a sad face makes it dark, with big fish that bite.

There are two ways to play:

• Breath mode. At the surface you take a deep breath in, then look down to dive. Under water you may breathe out, but every breath in is a gasp: it costs life and points. To breathe again, you swim up to the surface. Five big treasure coins lie on the seabed around the place where you jumped in. A pointer shows the way to the nearest one. Find all five and you win.
• Free dive. Your breathing does not count. You stay under water, catch fish, collect coins and harpoons, and play for score.

A run lasts 3 minutes (or has no time limit, if you turn it off in the menu). Every finished run goes on a local winner ladder: wins first, the fastest on top, then the other runs by score.

The whole game works hands-free. The menu follows your head (look up or down to move, blink to choose), and an "eyes closed for 1.5 seconds" gesture leaves a run. The keyboard works too, with or without a camera.

Demo

The panel on the right (S key) shows the live SmartSpectra values the game reads: the chest trace (green while you breathe in), the breathing rate, blinks, eyes open or closed, and the expression scores with the threshold a held expression needs.

In the video:

1. Consent screen, then a 2-second head calibration.
2. Menu driven by head and blink.
3. Smile to jump from the ship.
4. Breath mode: breathe in at the surface, look down, dive. Breathe in under water: a gasp.
5. Smile and sad face: the sea turns bright or dark.
6. Held expressions: angry scares the fish away, surprise sends a sonar pulse, fear gives a speed boost.
7. All five treasure coins found, and the run on the winner ladder.

Code

valbarov / devto-presage-challenge

Breathless Reef

An underwater diving game, my entry for the DEV.to Presage SmartSpectra SDK Challenge.

• Demo video: https://www.youtube.com/watch?v=_E2XdJ1w5TM
• Article: https://dev.to/vadim_albarov/breathless-reef-a-diving-game-you-play-with-your-face-and-your-breath-3k3a

You are a free diver. Your camera is the only controller. You steer with your head fire a harpoon by blinking, hold your breath to stay under water, and change the world with your face. A run lasts 3 minutes, or until your life runs out. Catch fish collect old coins, and do not breathe in while you are under water.

Not a medical device. All signals are estimates.

Signals and controls

Signal (SmartSpectra)	Game use
Face landmarks (head position, eyes)	Steering: look left/right/up/down, and combinations. In the menu: look down or up to point at an item. Eyes closed for 1.5 s: leave the run for the menu
Blink	Fire the harpoon. In the menu: hold the head still and blink to choose. On the run summary: open

…

View on GitHub

Setup:

git clone https://github.com/valbarov/devto-presage-challenge.git
cd devto-presage-challenge
npm install            # also downloads the SDK native runtime
cp .env.example .env   # then paste your key from https://physiology.presagetech.com/
npm start

You need Node.js 20 or newer, a webcam and a SmartSpectra API key. The README has all controls and a troubleshooting list. npm test runs the Node self-tests.

Stack: Electron, the SmartSpectra Node SDK in the renderer, a Canvas 2D first-person renderer, and WebAudio sounds made at run time (no image or sound files).

How It Responds to Human Signals

SmartSpectra signal	What it does in the game
Face landmarks (head position)	Steering: look left, right, up, down. In the menu: one step per nod
Face landmarks (eye shape)	Eyes closed for 1.5 s: leave the run for the menu
Blink	Fire the harpoon. In the menu: choose
Breathing (chest trace)	Breath mode: a breath in at the surface lets you dive; a breath in under water is a gasp
Expression: happy	Bright, calm sea and a score multiplier. A held smile jumps from the ship and restarts a run
Expression: sad	Dark red sea with aggressive fish: more points, more danger. Held: the fish dance around you
Expression: angry (held)	Every fish nearby swims away
Expression: surprise (held)	Sonar pulse: nearby fish slow down
Expression: fear (held)	Speed boost

Everything reacts at once: the light of the sea follows your smile, the view follows your head, and the harpoon fires on the blink.

How I integrated the SDK

I started with a small test app (not in the repo) to measure how late each signal arrives and how often a blink is false. That told me what kind of game is possible: head and blink are fast enough for steering and firing; the breath trace is slower, so breath decides phases (dive, gasp, surface), not split-second moves.

The game uses @smartspectra/node-sdk in the Electron renderer. One module, src/signals.js, owns the SDK. It turns the raw metrics stream into a small controls object and a few events (blink, inhaleStart, expressionHold, eyesClosedHold). The game never reads the SDK directly, so the same game code runs with the keyboard.

const sdk = new SmartSpectraSDK({
    apiKey,
    requestedMetrics: defaultMetrics,
    enableAccumulatedOutput: false,
    enableTelemetry: telemetry,   // false unless the player ticks the box
});
sdk.on('metrics', (buf) => ingest(decodeMetrics(buf)));
sdk.on('validationStatus', (code, ts, hint) => showHint(code, hint));
sdk.on('error', (code, message, retryable) => { if (retryable) restartSession(); });

What each signal needed:

• Head steering. I take the head centre from the landmarks, subtract a calibrated "straight ahead" position, and add a small dead zone (about 1.5 cm of head sway). The centre slowly follows your posture while you look ahead, so you do not drift after a few minutes.
• Blink. The SDK marks the start of a blink. A blink lasts the same in the data whether the eyes close for 0.2 s or 1 s. So a blink is a short, debounced event (fire, choose), and "eyes closed for a while" comes from the eye shape in the landmarks instead (the eye aspect ratio).
• Breathing. I watch the slope of the chest trace over 0.4 s. "Breathing in" means the trace rises faster than a dead zone, and the dead zone scales with your own breathing range over the last 10 to 30 seconds. So it works for deep and shallow breathers. A key flips the trace direction for cameras where it comes out upside down.
• Expressions. Scores are smoothed, and the mood (bright, neutral, dark) has hysteresis, so the sea does not flicker. A "held" expression needs a score of 0.35 for 0.7 s, and must fall back before it can fire again.

What the camera taught me

• Light matters. In a dim room my laptop camera drops to about 20 frames per second, and the SDK then fails with "processing failed" every few seconds. The game now opens the camera itself and feeds the SDK a steady 30 frames per second through a small frame pacer (it repeats the last frame when no new one is ready).
• A gap stops the session. After a camera gap of about 2 seconds, the session stops accepting frames. A watchdog in signals.js restarts the session at once, with back-off. The camera preview stays on, because the game owns the stream.
• Validation hints pause the landmarks. When the SDK says, for example, "place more of the chest in view", head tracking pauses but blinks continue. The game shows the SDK hint at the bottom of the screen, so the player knows why steering stopped.

Prize Categories

Most Creative or Delightful Use.

Most camera games use your face as a pointer. I wanted the opposite: a game where your body state is part of the world. Holding your breath is the core of diving, so in Breathless Reef your real breath decides when you can go down and when you must come up. Your face sets the mood of the sea. A smile literally brings the light in; a sad face brings the sharks.

It is playful in small ways too: you smile to jump off the ship, you pull a scared face to swim faster, and you make an angry face to scare every fish away.

Consent and Data Handling

• Consent first. The game opens on a consent screen. It explains what the camera is used for, before the camera turns on. The camera starts only after you press Continue with camera. If you choose Play with keyboard only, the camera never turns on.
• The camera runs only while the game runs. It turns off when you close the game.
• No video is saved or uploaded by the game. Signal values stay in memory and are dropped when the game closes.
• SDK telemetry is off by default. A checkbox on the consent screen turns it on, for that session only. The screen links to Presage's telemetry and privacy page.
• Presage servers. The SDK talks to Presage servers, for example to check the API key.
• What is stored on your computer. The consent screen says this too. The winner ladder is saved in ladder.json in the app's user-data folder. Each entry has the date, the mode, the result, the score, the time and the treasure count. No face or body data. A few settings (breath mode, time limit, view, breath direction) are kept in the app's local storage, in the same folder. Nothing is shared. The data stays until you delete it: the ladder screen shows the file path, and deleting the file clears the ladder.
• The API key is read from a local .env file, which is not in the repository.
• Not a medical device. The game shows no health values and makes no health claims. Breathing only decides game events, and all signals are estimates. You can breathe at any time: a gasp only costs points. The consent screen and Help say: stop and breathe normally if you feel dizzy or unwell.

Thanks for reading! I'd love to hear your best treasure time.