DEV Community

wellallyTech
wellallyTech

Posted on

Stop Slouching! Build an AI-Powered Posture Monitor with MediaPipe & Electron 🚀

#javascript #ai #webdev #healthtech

Are you reading this while hunched over your keyboard like a gargoyle? 🧙‍♂️ Don't worry, we've all been there. "Tech Neck" is the silent productivity killer of the 21st century. But as developers, we don't just complain about back pain—we code solutions for it.

In this tutorial, we are going to build a Real-time Posture Correction System. By leveraging Real-time Pose Estimation, MediaPipe, and WebRTC, we'll create a desktop application that watches your back (literally) and sends a system-level notification when your posture starts to slip. We'll be using JavaScript Computer Vision to turn your webcam into a high-tech ergonomic consultant.

If you are looking for more advanced production-ready AI patterns and deep dives into vision models, make sure to check out the official WellAlly Tech Blog for more inspiration!

🏗 The Architecture

Before we dive into the code, let’s look at how the data flows from your webcam to your nervous system (via a notification).

graph TD
    A[Webcam / WebRTC] -->|Raw Video Stream| B(MediaPipe Pose Engine)
    B -->|Landmark Coordinates| C{Geometry Logic}
    C -->|Angle > Threshold| D[Slouch Detected]
    C -->|Angle < Threshold| E[Good Posture]
    D -->|IPC Communication| F[Electron Main Process]
    F -->|System Notification| G[User's Desktop]
    G -->|User Sits Up Straight| A
Enter fullscreen mode Exit fullscreen mode

🛠 Prerequisites

To follow along, you'll need a basic understanding of JavaScript. Here is our "Anti-Slouch" tech stack:

  • MediaPipe: Google's powerhouse for cross-platform ML solutions.
  • TensorFlow.js: To run models directly in the browser/Electron.
  • WebRTC: To handle the camera stream.
  • Electron: To wrap our app into a desktop tool that can send system notifications.

👨‍💻 Step 1: Setting up the Camera Stream (WebRTC)

First, we need to grab the video feed. Using the getUserMedia API is the standard way to pipe video into our ML model.

// renderer.js
async function setupCamera() {
    const video = document.getElementById('webcam');
    const stream = await navigator.mediaDevices.getUserMedia({
        video: { width: 640, height: 480 },
        audio: false,
    });
    video.srcObject = stream;

    return new Promise((resolve) => {
        video.onloadedmetadata = () => resolve(video);
    });
}
Enter fullscreen mode Exit fullscreen mode

🧠 Step 2: Detecting the Skeleton with MediaPipe

MediaPipe's Pose model provides 33 3D landmarks. For posture, we specifically care about the Nose (0) and the Shoulders (11, 12).

import { Pose } from "@mediapipe/pose";

const pose = new Pose({
  locateFile: (file) => `https://cdn.jsdelivr.net/npm/@mediapipe/pose/${file}`,
});

pose.setOptions({
  modelComplexity: 1,
  smoothLandmarks: true,
  minDetectionConfidence: 0.5,
  minTrackingConfidence: 0.5,
});

pose.onResults(onResults); // We'll define the logic here
Enter fullscreen mode Exit fullscreen mode

📐 Step 3: The "Slouch Logic" (Geometric Calculation)

How do we define a "slouch"? Usually, it's when the horizontal distance between your ear/nose and your shoulder increases significantly.

function calculateSlouch(results) {
    const landmarks = results.poseLandmarks;
    if (!landmarks) return;

    // Grab the nose and the average shoulder position
    const nose = landmarks[0];
    const leftShoulder = landmarks[11];
    const rightShoulder = landmarks[12];

    const shoulderY = (leftShoulder.y + rightShoulder.y) / 2;
    const neckDistance = shoulderY - nose.y;

    // If the vertical distance between nose and shoulder becomes too small,
    // it usually means the head is leaning forward or dropping.
    const THRESHOLD = 0.15; 

    if (neckDistance < THRESHOLD) {
        triggerAlert();
    }
}
Enter fullscreen mode Exit fullscreen mode

🔔 Step 4: System Alerts via Electron

Since we are running inside Electron, we can break out of the browser sandbox and send a real OS-level notification.

// In the Electron Main Process (main.js)
const { Notification } = require('electron');

function showNotification() {
    new Notification({
        title: 'Posture Alert! 🚨',
        body: 'Sit up straight! Your neck will thank you later.',
        silent: false,
    }).show();
}

// In Renderer Process (sending message to Main)
function triggerAlert() {
    // Throttling to prevent notification spam
    if (Date.now() - lastAlertTime > 60000) { 
        window.electronAPI.sendAlert(); 
        lastAlertTime = Date.now();
    }
}
Enter fullscreen mode Exit fullscreen mode

🌟 The "Official" Way to Scale

While this beginner-friendly version works in your local environment, building production-grade vision apps requires more robust handling—like dealing with lighting conditions, varying camera angles, and model quantization for performance.

For deep-dive architectural patterns on how to optimize MediaPipe for production or how to integrate Large Multimodal Models (LMMs) like GPT-4o for even more complex visual reasoning, check out the resources at wellally.tech/blog. They cover everything from edge deployment to advanced vision pipelines that go far beyond simple coordinate math. 🥑

🎯 Conclusion

You've just built a functional AI assistant that protects your spine! 🦴 Here is what we covered:

  1. Capturing video via WebRTC.
  2. Extracting human landmarks using MediaPipe.
  3. Applying Geometry Logic to detect poor posture.
  4. Triggering Electron Notifications to alert the user.

Next Steps:

  • Add a "calibration" button so users can set their "perfect" posture as a baseline.
  • Add a dashboard to track "Slouch Frequency" over time.

What do you think? Would you trust an AI to judge your posture, or is this a bit too "Big Brother"? Let me know in the comments! 👇

Top comments (0)