title: Ad Pulse Lab: Webcam Pulse and Breathing Next to What Viewers Say

This is a submission for the Presage SmartSpectra SDK Challenge

What I Built

Ad Pulse Lab is an Electron desktop app for a consented ad test.

• A viewer watches 3 coffee ads in random order.
• Presage SmartSpectra measures pulse and breathing from the webcam, on the viewer's own PC.
• After each ad, the viewer answers two questions: Would you buy it? and Did you like the ad?
• The app shows the body numbers next to those answers, ad by ad, and refuses to compare any number whose signal was too weak.

It does not guess feelings and does not use facial expressions.

Why the body, not the face. NielsenIQ stopped using facial coding in its ad testing in 2022: across 100+ ads, positive expressions barely tracked sales (r = 0.18). Lab work (Venkatraman et al., 2015, Journal of Marketing Research) linked a slowing heart during an ad to attention, but it needed lab sensors. Ad Pulse Lab tries the body side with only a webcam.

Session flow (about 9 minutes):
consent → blurred positioning preview → camera check → 30 s warm-up + 90 s rest (the reference) → 3 ads, each followed by Buy/Like and 45 s rest → per-ad results → Approve and save or Delete.
A Compare view lines up result files from several viewers.

Demo

Ads are covered in the video for copyright. Narration is an AI voice (ElevenLabs).

Code

gracelee087 / ad-pulse-lab

Ad Pulse Lab

A consented ad-reaction test. A viewer watches 3 coffee ads in random order while the webcam measures breathing and pulse on their own PC with the Presage SmartSpectra SDK. After each ad the viewer answers two questions themselves: Would you buy it? and Did you like the ad?

The app shows body measurements next to the viewer's own answers. It does not guess feelings, does not use facial expressions, and does not judge the viewer or the ads.

Built for the DEV × Presage SmartSpectra challenge (2026).

What it does

1. Consent screen: two required checkboxes (measurement, reviewing and sending the result yourself) and one optional checkbox (anonymous, combined use in a write-up and demo video).
2. Positioning: a blurred camera preview with face and chest boxes. The preview is not saved.
3. Camera check: waits until the SDK reports the face and lighting are OK.
4. …

View on GitHub

Setup instructions are in the README. Tested on Windows 11, Node.js 22, Electron 43, @smartspectra/node-sdk 3.4.0.

How It Responds to Human Signals

1. Signals used

SmartSpectra output	Used for
cardio.pulseRate (value, stable, timestamp)	live pulse; per-ad pulse vs. rest
breathing.rate (value, stable)	live breathing rate; per-ad breathing vs. rest
breathing.upperTrace	breathing wave in each ad's chart
validationStatus events	camera check, and alerts during the test

2. Live response

• A status bar shows live pulse (BPM) and breathing rate (per minute).
• The test starts only after validationStatus reports a good face and light.
• An on-screen alert, with what to do, appears when the face/light check stays bad for 2 s, no new pulse arrives for 15 s, or no stable breathing value arrives for 45 s. → renderer.js#L165-L198

3. Architecture

Electron main process                          Renderer (page)
─────────────────────                          ───────────────
SmartSpectraSDK ── 'metrics' (buffer) ───IPC──▶ decodeMetrics(buf)
  useCamera(1280×720 @ 30 fps)                   → pulse / rate / trace series
                ── 'validationStatus' ───IPC──▶  → alerts, status bar
                                                 → windowStats → result file

// main.js
sdk = new SmartSpectraSDK({ ..., enableAccumulatedOutput: false, enableTelemetry: false });
sdk.on('metrics', (buf, ts) => send({ kind: 'metrics', buf: new Uint8Array(buf), ts }));
sdk.on('validationStatus', (code, ts, hint) => send({ kind: 'validationStatus', code, ts, hint }));
sdk.useCamera(CameraSelection.default, { width: 1280, height: 720, fps: 30 });
sdk.start();

→ main.js#L160-L185

// renderer.js
const { decodeMetrics } = require('@smartspectra/node-sdk/messages');
function onMetrics(buf, ts) {
  const d = decodeMetrics(buf);
  const row = { ts, br: pick(d.breathing?.rate), bt: pick(d.breathing?.upperTrace), pr: pick(d.cardio?.pulseRate) };
  store.offsets.push(Date.now() - ts / 1000);              // SDK µs clock → wall clock
  for (const m of row.pr || []) store.pulse.set(m.ts, m);  // keyed by timestamp: repeats collapse
  ...
}

→ renderer.js#L342-L360 · bridge: preload.js#L6-L18

• Why the main process. In my first version the SDK ran in the page. While a YouTube ad was loading, the camera stream showed a 2.7 s frame gap and the app crashed. In the main process, a busy page can no longer stall the camera frames.
• Clock mapping. SDK values carry microsecond timestamps; ads and rest periods are timed with the wall clock. The smallest Date.now() − ts/1000 offset (the least-delayed message) maps every value onto the wall clock, so it lands in the right ad. → renderer.js#L209-L213

4. From signal to result

const MIN_USABLE_PCT = 70;
function windowStats(ser, from, to, lagMs) {
  const all = ser.filter(([t]) => t >= from && t < to);
  const settled = all.filter(([t, , stable]) => stable && t >= from + lagMs).map(([, v]) => v);
  const usable = all.length ? Math.round(100 * all.filter(([, , st]) => st).length / all.length) : null;
  return { mean: r1(mean(settled)), usable, reliable: usable >= MIN_USABLE_PCT && settled.length > 0 };
}
const basePulse = windowStats(P, rest.start, rest.end, 12000);
const adPulse   = windowStats(P, ad.started, ad.ended, 12000);   // breathing: 30000

→ renderer.js#L214-L220 · used in renderer.js#L243-L252

• Lag. Pulse is a 12-second average and breathing rate a 30-second average, so a value at the start of an ad still holds the seconds before it. Means use only values from 12 s (pulse) or 30 s (breathing) into the window.
• Stable only. Only values SmartSpectra marks stable go into a mean.
• 70% gate. The share of stable values is the window's "usable %". If the ad or the rest window is below 70%, no change vs. rest is computed and the result says not compared.
• Charts. Breathing wave in 250 ms bins (unstable parts drawn faint), pulse in 1 s bins, with scene marks (brand, product, key message, emotional peak, tagline). → renderer.js#L231-L240

5. What it showed

One person (me), two sessions with a different ad order. Pulse change vs. rest:

Ad	Session 1 (Starbucks shown 1st)	Session 2 (Starbucks shown 3rd)
Starbucks	−3.0 BPM	−3.2 BPM
Dolce Gusto	−0.9	−0.7
Taylors	−0.8	+0.5

• In session 2, pulse was usable for 97–100% of each ad, so all three were compared.
• Breathing was usable for only 50–69%, so the app compared none of them.
• Starbucks dropped the most in both orders. With one person, this is a reason to run a panel, not a finding.

6. Other engineering notes

• Non-ASCII paths on Windows. The SDK's native runtime could not load its model files from a folder with Korean characters. main.js copies the runtime once to an ASCII-only folder and points SMARTSPECTRA_CAPI_PATH there before loading the SDK. → main.js#L21-L48
• Locked-down page. The page is served from 127.0.0.1 with a fixed file list. Camera permission is granted only for video-only requests from that origin, every IPC handler checks the sender frame, and the page gets a small contextBridge API, not Node. → main.js#L62-L78, main.js#L96-L105
• Ad timing. The YouTube IFrame API plays the ads; an ad starts at its first PLAYING state and ends at ENDED. → renderer.js#L97-L114

Prize Categories

Entering Best Thoughtful Real-World Use and Most Creative or Delightful Use.

Best Thoughtful Real-World Use

• The need: before an ad goes on air, teams pretest it. Asking viewers is not enough, and reading faces barely tracked sales in NielsenIQ's review. Body signals have needed lab sensors.
• Who it's for: ad and market researchers running a consented viewer panel, and the viewers themselves.
• Designed to be usable:
  • One guided session of about 9 minutes, no account, no extra hardware: a webcam and a PC.
  • The camera check and the alerts say what to do in plain words (face the camera, sit still, keep the chest visible, do not talk).
  • The viewer sees their own results first, then chooses Approve and save or Delete. Nothing is uploaded.
  • Weak signals are marked not compared instead of being turned into numbers that look certain.
  • The Compare view lines up result files from a whole panel.

Most Creative or Delightful Use

• Inspiration: NielsenIQ dropping facial coding, and lab studies where a slowing heart during an ad went with attention.
• What's different: instead of reading faces, the app reads the body with just a webcam and puts it next to what the viewer says, scene by scene, with the brand, product and tagline moments marked on each chart.

Consent and Data Handling

• Camera consent: measuring starts only after two required boxes (measurement; reviewing and sending the result yourself). A third box is optional (anonymous, combined use in a write-up and demo video). The viewer can stop at any time by closing the window.
• On the viewer's PC: SmartSpectra runs locally. Camera video and face data are never saved and never sent by the app. The positioning preview is blurred and not saved.
• Network: the SDK connects to Presage's servers. The ads are YouTube embeds, so YouTube receives the usual player requests.
• SDK telemetry: off (enableTelemetry: false). Per Presage's Telemetry and Privacy doc, when disabled "the SDK starts no telemetry session and transmits nothing".
• AI services: none are used by the app.
• Storage: during a session, measurement values only (no video) go to a session log in the app folder. Approve and save writes one result file to Documents\AdPulseLab\ and deletes the log. Delete deletes the log and writes nothing. A log left by closing the window mid-session is deleted at the next start.
• Result file: random ID (no name), consent choices, ad order, Buy/Like answers, pulse and breathing numbers per ad and for rest, and (if the viewer leaves the box ticked) one value per second. No video, no images, no face landmarks.
• Sharing: the app never uploads anything. The viewer sends the file to the organizer themselves.
• Retention: the viewer can delete their files at any time. Received files are deleted on request, and all of them after the contest results (December 2026).
• Not medical: not a medical test, not health advice, not used for any medical decision.

Credits

• Presage SmartSpectra (@smartspectra/node-sdk), used under Presage's Terms of Service.
• Ads: Starbucks UK, NESCAFÉ Dolce Gusto, Taylors of Harrogate (public YouTube videos, played embedded, not copied).
• Sources: NielsenIQ (2022) "About face"; Venkatraman et al. (2015) Journal of Marketing Research.