This is a submission for the Presage SmartSpectra SDK Challenge
What I Built
Idlewild is a desktop performance studio for animators and game makers. You breathe, move and perform in front of a webcam; SmartSpectra measures your chest breathing and, when your chest and face are well framed, breathing rate and pulse rate; on-device tracking follows your face and body, and a character performs with you. Record once, replay the same performance on any character, then export video, procedural sound, a baked GLB with real breathing curves and CSV curves for your animation tools. Idle loops and cutscenes get breathing captured from a real person instead of a sine wave.
Demo
Code
Idlewild
A desktop performance studio for animators and game makers, built on the Presage SmartSpectra SDK with Electron, TypeScript and Three.js. Breathe, move and perform in front of your camera: SmartSpectra measures your chest breathing, breathing rate and pulse rate while on-device tracking follows your face and body. The character breathes with you. Record the performance once, replay it on any character or scene, and export video with a live signal panel, procedural sound, a baked GLB with breathing and pulse curves, and CSV curves for your animation tools.
How SmartSpectra drives the studio:
| SmartSpectra signal | What it does |
|---|---|
| Chest breathing trace | Characters' chests, shoulders and heads breathe; bubbles rise on inhale and release on exhale; the star glows; the sound's breath layer swells. |
| Breathing rate | Slow posture and scene pacing within comfortable bounds. Faster breathing is never rewarded. |
| Pulse rate | Slow color warmth and glow; shown with its trend. |
How It Responds to Human Signals
- Chest breathing trace drives chest, shoulder and head motion on Rex, Sprite and the Rain runner; bubbles rise on inhale and release on exhale; the star glows; the sound's breath layer swells. The trace has no declared units, so Idlewild centers and scales it against its own recent range.
- Breathing rate sets slow, bounded scene pacing. Faster breathing is never rewarded, and the runner's pace is chosen, not breath-driven.
- Pulse rate, when SmartSpectra reports it, warms colors and glow and is shown with its trend. It is never turned into fake heartbeats.
- Pulse waveform (uncalibrated arterial trace), when reported, adds a fine shimmer and a subtle tremolo.
- Blink and talking move eyelids and jaw when no face is tracked locally. HRV and expression estimates are opt-in.
- Every signal carries its quality and confidence onto an on-stage panel that is also composited into exported video. Missing or stale input fades out and is never invented. SmartSpectra frames and local MediaPipe tracking from the same camera are merged onto one timeline.
Prize Categories
Most Creative or Delightful Use; Best Thoughtful Real-World Use.
Consent and Data Handling
- Nothing starts at launch. The camera opens only after in-app consent; SmartSpectra needs your own key and a separate metered-measurement consent (18+).
- Keys stay in main-process memory for the session and are never written to disk. Camera images are never saved, uploaded by Idlewild or exported. No microphone is used; all audio is synthesized.
- SmartSpectra sends camera-derived data to Presage for processing. Idlewild disables SDK telemetry and never requests insights, but the SDK may generate automatic vitals insights that upload data and Idlewild cannot disable them; vendor retention is governed by Presage's terms and is not verified by Idlewild.
- Takes live in memory until you export them. Exports that include measured breathing and pulse are personal data, and you choose where they go. Face and body tracking runs entirely on the device.
- Creative animation, not medical measurement: no diagnosis, no health claims, no covert monitoring.
Top comments (0)