wellallyTech

Posted on May 30

Save Your Spine: Building a Real-Time AI Posture Guard with MediaPipe and Electron 🧘‍♂️💻

#webdev #computervision #ai #mediapipe

We've all been there. You’re deep in a coding session, your head slowly drifts closer to the monitor, and before you know it, you’re shaped like a shrimp 🦐. "Tech neck" is a real productivity killer, and while ergonomic chairs help, they don't fix bad habits.

In this tutorial, we are going to build a Real-Time AI Posture Guard. We'll leverage Edge AI and Computer Vision to monitor your neck angle locally on your machine—no video data ever leaves your device! By combining MediaPipe for pose estimation and Electron for the desktop overlay, we can create a lightweight, privacy-focused solution to save your cervical spine.

If you’re interested in more production-ready patterns or advanced Edge AI optimizations, definitely check out the deep dives over at the WellAlly Tech Blog.

The Architecture 🏗️

The logic is simple: we capture the video stream, pass it through a pre-trained model to find specific body "landmarks," calculate the angle of your head relative to your shoulders, and trigger an alert if you're slouching.

graph TD
    A[Webcam Stream] --> B[OpenCV / HTML5 Video]
    B --> C[MediaPipe Pose Engine]
    C --> D{Landmark Extraction}
    D --> |Coordinates| E[Geometric Angle Logic]
    E --> F{Is Angle < Threshold?}
    F -- "Yes (Slouching)" --> G[Electron Alert/Overlay]
    F -- "No (Good Posture)" --> H[Status: Healthy]
    G --> A
    H --> A

Prerequisites 🛠️

To get started, make sure you have the following in your tech_stack:

MediaPipe: Google’s high-fidelity body tracking framework.
Electron: To run our app as a desktop overlay.
TensorFlow.js: The engine powering the model in the browser environment.
OpenCV.js: Useful for image preprocessing (optional but recommended).

Step 1: Setting up the Pose Engine

MediaPipe makes Pose Estimation incredibly accessible. We don't need to train a model from scratch; we can use their "BlazePose" model which runs efficiently on the CPU or GPU via WebGL.

import { Pose } from "@mediapipe/pose";

const pose = new Pose({
  locateFile: (file) => `https://cdn.jsdelivr.net/npm/@mediapipe/pose/${file}`,
});

pose.setOptions({
  modelComplexity: 1, // 0: Fast, 1: Balanced, 2: Heavy
  smoothLandmarks: true,
  minDetectionConfidence: 0.5,
  minTrackingConfidence: 0.5,
});

// Listener for model results
pose.onResults(onResults);

Step 2: The Geometry of a "Slouch" 📐

To detect "Forward Head Posture," we track the relationship between the Ear (Landmark 7/8) and the Shoulder (Landmark 11/12).

A healthy posture usually maintains a vertical alignment. We can calculate the angle using the arctangent of the vertical and horizontal distance between these points.

function calculateNeckAngle(ear, shoulder) {
    const radians = Math.atan2(shoulder.y - ear.y, shoulder.x - ear.x);
    let angle = Math.abs(radians * 180.0 / Math.PI);

    // We want the angle relative to a vertical line
    return 90 - angle; 
}

function onResults(results) {
    if (!results.poseLandmarks) return;

    const landmarks = results.poseLandmarks;
    const leftEar = landmarks[7];
    const leftShoulder = landmarks[11];

    const neckAngle = calculateNeckAngle(leftEar, leftShoulder);

    if (neckAngle > 25) {
        triggerAlert("Sit up straight! 🚨");
    }
}

Step 3: Electron Integration for Desktop Alerts

Because this is a Vision / Edge AI project, we want it to run in the background. Electron allows us to create a "Transparent Overlay" or a Tray notification.

In your main.js:

const { app, BrowserWindow } = require('electron');

function createWindow() {
  const win = new BrowserWindow({
    width: 800,
    height: 600,
    webPreferences: {
      nodeIntegration: true,
      contextIsolation: false
    },
    alwaysOnTop: true, // Keep it visible to remind you!
    transparent: true,
    frame: false
  });

  win.loadFile('index.html');
}

app.whenReady().then(createWindow);

Advanced Optimization: The "Official" Way 🥑

While the code above works for a prototype, running AI models in a desktop environment requires careful resource management. You don't want your posture detector to consume 50% of your CPU while you're trying to compile code!

For advanced patterns on optimizing TensorFlow.js memory leaks and using Web Workers to offload the vision processing, I highly recommend checking out the technical guides at WellAlly Tech Blog. They have fantastic resources on making Edge AI apps production-ready.

Conclusion 🚀

Building a "Posture Guard" is a perfect weekend project to get your hands dirty with Real-time Computer Vision. Not only do you learn how to manipulate coordinate data from MediaPipe, but you also build something that actually benefits your health.

Summary of what we achieved:

Setup a MediaPipe pipeline in a JS environment.
Implemented geometric logic to detect "Tech Neck."
Wrapped it in Electron for a native desktop experience.

Next Steps:

Add a "Focus Mode" that blurs your screen when your posture is bad.
Log your "Posture Score" to a local database to track improvement over time.

What do you think? Would you trust an AI to tell you to sit up straight? Let me know in the comments! 👇

DEV Community