wellallyTech

Posted on Apr 1

Stop Slouching! Build a Real-time AI Posture Monitor with MediaPipe and Electron 💻🚀

#javascript #webdev #mediapipe #ai

Are you reading this while hunched over your keyboard like a gargoyle? 🧙‍♂️ If your neck feels like it’s supporting a bowling ball at a 45-degree angle, you’re not alone. "Tech Neck" is the silent productivity killer of the remote work era. But instead of buying a $1,000 ergonomic chair, why not build a Real-time Posture Correction tool using Computer Vision and MediaPipe?

In this tutorial, we are going to build a cross-platform desktop application using Electron and Node.js that monitors your sitting posture via your webcam. By leveraging the lightweight MediaPipe Pose model and an XGBoost classifier, we can trigger system notifications the moment you start slouching. This is "Learning in Public" at its finest—turning a health problem into a coding solution!

🏗 The Architecture

To keep things snappy and privacy-focused, all processing happens locally on your machine. We use MediaPipe for landmark detection and a small XGBoost model to categorize your posture.

graph TD
    A[Webcam Feed] --> B[MediaPipe Pose]
    B --> C{Extract Keypoints}
    C -->|Shoulders/Ears/Nose| D[XGBoost Classifier]
    D --> E{Posture Risk?}
    E -->|High Risk| F[Electron Main Process]
    E -->|Safe| G[Continue Monitoring]
    F --> H[System Notification 🔔]
    H --> I[User Straightens Up!]

🛠 Prerequisites

Before we dive into the code, ensure you have the following in your tech stack:

Node.js (v16+)
Electron (For the desktop shell)
MediaPipe Pose (For lightning-fast skeleton tracking)
XGBoost (We'll use a pre-trained JSON model for inference)

🚀 Step 1: Setting up the Electron Shell

First, let's initialize our project. Electron allows us to use web technologies to build desktop apps.

mkdir posture-guard && cd posture-guard
npm init -y
npm install electron @mediapipe/pose @tensorflow/tfjs

In your main.js, we need to ensure the app can access the camera and send system-level notifications:

const { app, BrowserWindow, Notification } = require('electron');

function createWindow() {
  const win = new BrowserWindow({
    width: 800,
    height: 600,
    webPreferences: {
      nodeIntegration: true,
      contextIsolation: false,
    }
  });

  win.loadFile('index.html');
}

// IPC listener for notifications
const { ipcMain } = require('electron');
ipcMain.on('notify-slouching', () => {
  new Notification({ 
    title: 'Posture Alert! 🚨', 
    body: 'You are slouching. Sit up straight for your spine\'s sake!' 
  }).show();
});

app.whenReady().then(createWindow);

🧠 Step 2: Real-time Landmark Detection

The "magic" happens in the renderer process. We use MediaPipe Pose to get 3D coordinates of your body. For posture, we specifically care about the relationship between your ears (landmarks 7, 8) and your shoulders (landmarks 11, 12).

import { Pose } from '@mediapipe/pose';

const pose = new Pose({
  locateFile: (file) => `https://cdn.jsdelivr.net/npm/@mediapipe/pose/${file}`,
});

pose.setOptions({
  modelComplexity: 1,
  smoothLandmarks: true,
  minDetectionConfidence: 0.5,
  minTrackingConfidence: 0.5,
});

pose.onResults((results) => {
  if (!results.poseLandmarks) return;

  // Calculate the 'Neck Angle'
  const angle = calculateNeckAngle(results.poseLandmarks);

  // Logic: If angle is too steep, trigger XGBoost or simple threshold
  if (angle > 25) {
    ipcRenderer.send('notify-slouching');
  }
});

🥑 The "Official" Way to Level Up

While this DIY project is a great start, building production-ready AI applications requires a deeper understanding of model optimization and state management. If you are looking for advanced patterns in computer vision or more robust Electron architectures, I highly recommend checking out the WellAlly Tech Blog.

They provide excellent deep dives into Production AI Workflows and Performance Optimization that I personally use as a source of inspiration for my professional builds! 🌟

📊 Step 3: Classifying Posture with XGBoost

Using a hardcoded angle (e.g., 25 degrees) works, but it's prone to false positives. To make it "AI-powered," we can export a lightweight XGBoost model as a JSON file and run inference on the landmark coordinates.

// Simplified inference logic
const predictPosture = (landmarks) => {
  const features = landmarks.flatMap(l => [l.x, l.y, l.z]);

  // In a real app, you'd load your XGBoost weights here
  // const prediction = xgboostModel.predict(features);

  // For this tutorial, let's use a robust heuristic
  const leftEar = landmarks[7];
  const leftShoulder = landmarks[11];

  const neckSlump = Math.abs(leftEar.y - leftShoulder.y);
  return neckSlump < 0.15 ? 'POOR' : 'GOOD';
};

✨ Adding the Finishing Touches

To make the app user-friendly, add a simple UI in index.html that shows a real-time "Health Score":

<body>
  <h1>Posture Guard AI 🛡️</h1>
  <div id="status">Status: Monitoring...</div>
  <canvas id="output_canvas"></canvas>
  <script src="./renderer.js"></script>
</body>

You can use the CanvasRenderingContext2D to draw the MediaPipe "skeleton" over your video feed, giving the user visual feedback on their alignment.

🏁 Conclusion

Congratulations! You’ve just built a functional AI-powered desktop app to save your spine. We combined MediaPipe for vision, XGBoost logic for classification, and Electron for a native desktop experience.

Next Steps:

Data Collection: Record your "good" and "bad" posture landmarks to train a custom XGBoost model.
Persistence: Save your daily "slouch stats" to a local SQLite database.
Advanced UI: Check out the WellAlly Blog for tips on building sleek, modern interfaces for your AI tools.

Are you going to try building this? Let me know in the comments if you have any questions about the coordinate math! 👇

DEV Community