DEV Community

Md Abid Hussain
Md Abid Hussain

Posted on Edited on AI-assisted

Hearth: a desk companion that looks after you kindly

Presage SmartSpectra SDK Challenge Submission

This is a submission for the Presage SmartSpectra SDK Challenge

What I Built

Most of us spend the better part of the day in front of a laptop or computer. We blink less, breathe quicker under pressure and forget to stand up, and because none of it is dramatic, it goes unnoticed. Wearables can track it, but only if you wear one. Yet almost every laptop already has a camera facing us all day.

Hearth is a small companion that lives at the edge of your screen. It reads your pulse, breathing, heart rate variability and blinks from that webcam, talks with you by voice, and speaks up only when it's worth it: a break after long sitting, a slower breath when your pulse climbs, a reminder you asked for. Connect your apps and it can also read your calendar, mail or notes, but only when you ask, never on its own.

It's a small widget that floats above your windows (hold Right Ctrl or click it to talk), plus a dashboard with your live signals, Presage's insights, reminders and privacy controls.

Built with Presage SmartSpectra for the camera signals and insights, Electron and React for the app, AssemblyAI for voice, the AI SDK with OpenAI for the conversation, and MCP and Composio for read-only access to apps like Calendar, Gmail and Notion.

Hearth is a wellness companion, not a medical device. Its numbers are estimates, and it never diagnoses.

Demo

Code

GitHub logo md-abid-hussain / hearth

A desk companion that reads your pulse, breathing and blinks through the webcam with Presage SmartSpectra, and talks it over with you by voice.

Hearth: reads your pulse and breath, looks after you kindly

Hearth is a small companion that lives on your desktop. It reads your pulse, breathing and blinks from the webcam with Presage SmartSpectra, talks with you by voice, and speaks up only when it matters: a break after long sitting, a slower breath when your pulse climbs, a reminder you asked for. Connect your apps and it can also read your calendar, mail or notes, but only when you ask, never on its own.

Built for the DEV.to Presage SmartSpectra challenge. A wellness companion, not a medical device.

The Hearth dashboard

What it does

  • Reads live signals, on your computer. Pulse, breathing, HRV and blinks come from the SmartSpectra SDK, and you choose which ones it reads. Camera frames never leave the device.
  • Talks by voice. Hold Right Ctrl (Right Option on a Mac), ask, let go. Hearth answers out loud.
  • Knows when to stay quiet. Breaks, breathing nudges and reminders (once…

Try it in five minutes

  1. Get a free API key at smartspectra.presagetech.com.
  2. Clone and install:
   git clone https://github.com/md-abid-hussain/hearth.git
   cd hearth
   npm install
Enter fullscreen mode Exit fullscreen mode
  1. Copy .env.example to .env and add SMARTSPECTRA_API_KEY, ASSEMBLYAI_API_KEY and OPENAI_API_KEY.
  2. Run npm run dev, agree to the consent screen, and click Start sensing.
  3. Hold Right Ctrl and try "how am I doing?", "remind me to drink water every hour" or "keep an eye on my eyes every ten minutes".

Where the SmartSpectra code lives:

How It Responds to Human Signals

SmartSpectra SDK turns the webcam into pulse, breathing, HRV, blink and talking signals, computed on the computer itself, with no wearable. Hearth keeps the live values, five-minute averages and a session baseline, and acts only on readings SmartSpectra marks stable, so a wobbly first few seconds never trigger anything.

How it works Camera → SmartSpectra (on this computer) → stable readings and baselines → Hearth speaks, notifies or stays quiet, with Presage Insights on a "numbers only" branch.

Signal What Hearth does with it
Pulse Compared with your own session baseline. A sustained rise of 15 bpm or more brings a "three slow breaths?" nudge.
Breathing Twenty breaths a minute or more brings a slower-breath nudge. Ignored while you talk and for 20 s after, since speech moves the chest.
HRV Part of the answer when you ask how you're doing, and part of Presage's insights.
Blinks Under 8 a minute during an eye check-in means your eyes are locked on the screen.
Talking Hearth never interrupts. Nudges and reminders wait until you've stopped.
Presence 10 s without a face pauses sensing. Three minutes away counts as a break.
Expressions (opt-in) Smile and Hearth smiles back. Never sent to the language model.

Signal page the whole Signals page from the live run in one image: each reading with its confidence and the pulse and breathing chart. The camera preview is an illustration in place of my face. HRV shows "--" there: it needs a minute of steady signal.

1. You choose what the camera reads

The Signals page has a switch for each reading. Hearth asks the SDK only for the ones that are on, and at least one always stays on.

// src/renderer/src/lib/sensing.ts
function requestedMetrics(settings: Settings): number[] {
  const metrics: number[] = []
  if (settings.measureBreathing) metrics.push(...breathingMetrics)
  if (settings.measurePulse) metrics.push(MetricType.PULSE_RATE)
  if (settings.measureHrv) metrics.push(MetricType.HRV)
  if (settings.measureBlinks) metrics.push(MetricType.BLINKING, MetricType.TALKING)
  if (settings.measureExpressions) metrics.push(MetricType.EXPRESSIONS)
  return metrics
}
Enter fullscreen mode Exit fullscreen mode

2. Check-ins, reminders and restraint

Ask Hearth to "keep an eye on my breathing every ten minutes" and the model calls a watch_signal tool; from then on, plain code re-reads the signal, stays silent while it's fine and speaks up when it isn't. Reminders work the same way, once or up to ten times. Its own nudges never come in your first five minutes or more than three times an hour, nothing it says interrupts you while you talk, and the Timeline records every time it held back.

Reminders Hearth's judgement from the live run: "held back … user appears to be talking".

3. Presage Insights

About once a minute, Presage's model writes a short note from the session's numbers, which Hearth keeps on the computer. When you ask "how am I doing?", the agent can also ask Presage directly.

Insight a real Presage insight from the live run.

4. A face with personality

Hearth has moods (idle, listening, thinking, speaking, asleep), each with its own motion, and its breathing follows your measured breathing rate. Shake the widget while dragging it and it gets dizzy: "Whoa! Slow down, buddy."

Hearth mood Idle, listening, thinking, speaking, asleep and dizzy.

Prize Categories

I'm entering Best Thoughtful Real-World Use and Most Creative or Delightful Use.

  • Thoughtful, real-world: it's for anyone who spends the day at a screen, and needs nothing but the webcam they already have. Consent comes first, video stays on the device, you choose what it reads, and it holds back more than it speaks.
  • Creative, delightful: most vitals apps are dashboards; Hearth is a character. It breathes with you, smiles back, dozes off when ignored, and gets dizzy when you shake it.

Consent and Data Handling

Hearth shows a consent screen before the camera or microphone can start, and the camera only reads what you switch on. Until you agree, sensing, voice and chat stay off, and the main process refuses camera and microphone access. You can review or withdraw consent any time under Privacy.

Consent

What Where it goes Where to check
Camera video Stays on this computer. SmartSpectra processes frames locally; they're never stored or sent. sensing.ts
Pulse, breathing and HRV values To Presage about once a minute, for insights. Your question goes along if you ask one. insights.ts
Your voice To AssemblyAI, only while you hold the talk key (or until you send in hands-free mode). voice/stt.ts
Your words, with a short summary of signals, memory and reminders To the language model you configure, OpenAI or Anthropic. Each Presage insight also goes there once, to be laid out as a headline and suggestions. Expressions are never included. agent.ts
Calendar, mail and other apps Read only, and only the apps you sign into. Composio holds Google sign-ins. mcp.ts, composio.ts
Settings, memory, reminders, insights and the log Stay on this computer. App tokens are encrypted. Privacy page
  • SDK telemetry is off by default. If you turn it on, it sends session summaries and device diagnostics, never video or vital signs (Presage: Telemetry and privacy).
  • Retention and deletion. Hearth keeps local data until you delete it with Privacy > Delete all local data, which also signs you out of every app. Data sent to Presage, AssemblyAI or your model provider is kept under their own policies.
  • Presage's policy. Presage's general privacy policy is broader than its docs; read it before registering for a key.

Limitations

  • Wellness estimates, not medical readings. Hearth never diagnoses, and the numbers can be off.
  • The camera matters. SmartSpectra needs steady light, a fairly still face and at least 25 frames a second. A dim room or a wireless camera can stop sensing.
  • HRV is the hardest signal. It needs a still, quiet minute and may never stabilise. Hearth then shows nothing rather than a guess.

How I Built It

The app is Electron 44 with React 19 and Mantine 9, in TypeScript, on @smartspectra/node-sdk 3.5, built half in Cursor and half in Claude Code. Two things I learned:

  • Frame rate decides everything. Capturing at 640×480 instead of 720p kept SmartSpectra above its 25 fps floor.
  • Restraint is a feature. The hard part wasn't making Hearth speak. It was deciding when not to.

Agent Session

You can access agent session here: hearth-build

Top comments (2)

Collapse
 
rohan_sharma profile image
Rohan Sharma •

Another great project by Abid!

Some comments may only be visible to logged-in visitors. Sign in to view all comments.