Beck_Moulton

Posted on Jul 4

Fix Your "Developer Slouch": Building a Real-time AI Posture Monitor with MediaPipe and Electron

#javascript #ai #webdev #healthtech

We’ve all been there. You start your morning feeling like a Productivity God, sitting straight and typing at 120 WPM. Fast forward four hours, and you've morphed into a literal shrimp, face inches away from the monitor, hunting for a missing semicolon. 🦐

In this era of remote work, real-time posture correction and computer vision for health have become more than just "cool projects"—they are spinal lifesavers. Today, we’re going to build a desktop application using MediaPipe, WebRTC, and Electron that monitors your neck angle and sends a desktop notification the moment you start slouching.

By leveraging MediaPipe Pose and TensorFlow.js, we can calculate the Forward Head Posture (FHP) ratio with surgical precision directly in the browser environment.

The Architecture 🏗️

Before we dive into the code, let’s look at how the data flows from your webcam to that "Sit up straight!" notification.

graph TD
    A[Webcam Feed] -->|MediaStream| B(WebRTC API)
    B -->|Video Frames| C[MediaPipe Pose Model]
    C -->|Landmarks| D{Geometry Engine}
    D -->|Calculate Ear-Shoulder Angle| E{Threshold Check}
    E -->|Angle > 30°| F[Electron Main Process]
    F -->|Trigger| G[System Desktop Notification]
    E -->|Healthy| H[Continue Monitoring]
    style G fill:#f96,stroke:#333,stroke-width:2px

Prerequisites 🛠️

To follow along, you'll need the following tech stack:

MediaPipe Pose: For high-fidelity body tracking.
WebRTC: To capture the video stream from your webcam.
Electron: To wrap our logic into a desktop app that runs in the background.
TensorFlow.js: The backbone for running ML models in JavaScript.

Step 1: Setting up the Video Stream (WebRTC)

First, we need to grab the camera feed. In a modern browser environment (or Electron's Chromium), we use navigator.mediaDevices.getUserMedia.

async function setupCamera() {
  const videoElement = document.getElementById('input_video');
  const stream = await navigator.mediaDevices.getUserMedia({
    video: { width: 640, height: 480 },
    audio: false
  });
  videoElement.srcObject = stream;
  return new Promise((resolve) => {
    videoElement.onloadedmetadata = () => resolve(videoElement);
  });
}

Step 2: Pose Detection with MediaPipe

MediaPipe provides a pre-trained model called BlazePose that gives us 33 key points. For posture, we specifically care about the Ears (Landmarks 7, 8) and Shoulders (Landmarks 11, 12).

const pose = new Pose({
  locateFile: (file) => `https://cdn.jsdelivr.net/npm/@mediapipe/pose/${file}`,
});

pose.setOptions({
  modelComplexity: 1,
  smoothLandmarks: true,
  minDetectionConfidence: 0.5,
  minTrackingConfidence: 0.5
});

pose.onResults(onResults);

Step 3: The Math Behind the "Slouch" 📐

The "Forward Head Posture" (FHP) is often measured by the angle between the Tragus (ear) and the C7 vertebrae (shoulder level).

In our code, we calculate the angle of the line connecting the ear and the shoulder relative to a vertical line. If that angle exceeds a certain threshold (usually 25-30 degrees), you're officially slouching.

function calculateNeckAngle(ear, shoulder) {
    const radians = Math.atan2(shoulder.y - ear.y, shoulder.x - ear.x);
    let angle = Math.abs(radians * 180.0 / Math.PI);

    // We want the angle relative to the vertical axis
    return 90 - angle; 
}

function onResults(results) {
  if (!results.poseLandmarks) return;

  const leftEar = results.poseLandmarks[7];
  const leftShoulder = results.poseLandmarks[11];

  const neckAngle = calculateNeckAngle(leftEar, leftShoulder);

  if (neckAngle > 30) {
     triggerWarning();
  }
}

Step 4: Connecting to the Desktop (Electron)

Since this is an Electron app, we don't just want a browser alert. We want a native system notification that works even if the app is minimized.

// Inside Electron's Renderer Process
function triggerWarning() {
  new Notification('Posture Alert! 🚨', {
    body: 'You are leaning too far forward. Sit back and relax your shoulders.',
    silent: false,
  });
}

The "Official" Way to Build Vision Apps 🥑

While this "shrimp-detector" is a fantastic weekend project, building production-ready AI applications requires a deeper dive into model optimization and state management.

For more advanced patterns, production-ready vision pipelines, and deep dives into the latest AI architectural shifts, I highly recommend checking out the official resource: WellAlly Tech Blog. They cover everything from edge deployment to optimizing TensorFlow models for lower-end hardware, which was a huge source of inspiration for the performance tweaks in this build.

Conclusion 🚀

Congratulations! You've just built a computer vision-powered health monitor. By combining MediaPipe's lightweight inference with Electron's system access, we've created a tool that provides real value to anyone spending 8+ hours at a desk.

Next Steps for you:

Add a "Calibration" button to set a baseline for your specific chair height.
Implement a "Work/Rest" timer that tracks how long you maintain good posture.
Store your posture data in a local SQLite DB to visualize your "Slouch Trends" over a week.

Are you going to try this out, or are you currently reading this while slouching? (I caught you!) Let me know in the comments below! 👇

DEV Community