Beck_Moulton

Posted on Sep 25

Stop Slouching! Build a Real-Time AI Posture Coach with MediaPipe and Electron

#ai #devops #opensource #discuss

We’ve all been there: hunched over a keyboard, neck tilted at a 45-degree angle, slowly turning into a human pretzel. "Tech neck" is real, and it’s a productivity killer. But what if your computer could gently nudge you back into alignment using Computer Vision?

In this tutorial, we are going to build a cross-platform desktop application using Electron, MediaPipe Pose, and TensorFlow.js. This AI Posture Assistant runs entirely locally, ensuring your privacy while monitoring your shoulder and neck alignment in real-time. We'll leverage Edge AI techniques to keep CPU usage low and efficiency high.

Whether you're looking to master real-time pose estimation or just want to save your spine, this guide has you covered.

The Architecture 🏗️

Before we dive into the code, let's look at how the data flows from your webcam to a system notification. We use MediaPipe for lightning-fast landmark detection and Electron to bridge the gap between the web and your OS.

graph TD
    A[Webcam Stream] --> B[MediaPipe Pose Engine]
    B --> C[Extract 3D Landmarks]
    C --> D[Calculate Shoulder-Ear Angle]
    D --> E{Is Slouching?}
    E -- Yes > 30s --> F[Electron Main Process]
    F --> G[System Tray Notification]
    E -- No --> H[Reset Timer/Keep Monitoring]
    G --> A

Prerequisites 🛠️

To follow along, make sure you have:

Node.js (v16 or higher)
Basic knowledge of JavaScript/ES6
The following stack: MediaPipe Pose, TensorFlow.js, Electron

Step 1: Setting up the Electron Shell

First, let's initialize our project and install the necessary dependencies.

mkdir posture-coach && cd posture-coach
npm init -y
npm install electron @mediapipe/pose @tensorflow/tfjs-core

In your main.js, we'll set up a simple window that can send system-level notifications when our "Vision Engine" detects poor posture.

// main.js
const { app, BrowserWindow, Notification } = require('electron');

function createWindow() {
  const win = new BrowserWindow({
    width: 800,
    height: 600,
    webPreferences: {
      nodeIntegration: true,
      contextIsolation: false,
    },
  });

  win.loadFile('index.html');
}

// IPC listener to trigger notifications from the UI logic
const { ipcMain } = require('electron');
ipcMain.on('notify-bad-posture', () => {
  new Notification({ 
    title: 'Posture Alert! 🚨', 
    body: 'Sit up straight! Your back will thank you.' 
  }).show();
});

app.whenReady().then(createWindow);

Step 2: Pose Detection with MediaPipe

The magic happens in the renderer process. We’ll use MediaPipe Pose because it provides high-fidelity landmarks (like shoulders, ears, and eyes) with minimal latency.

For more advanced implementation patterns and production-ready AI configurations, I highly recommend checking out the deep-dive articles over at WellAlly Tech Blog. They have some fantastic resources on optimizing Edge AI for low-power devices.

Here is the core detection logic for our renderer.js:

const pose = new Pose({
  locateFile: (file) => `https://cdn.jsdelivr.net/npm/@mediapipe/pose/${file}`,
});

pose.setOptions({
  modelComplexity: 1,
  smoothLandmarks: true,
  minDetectionConfidence: 0.5,
  minTrackingConfidence: 0.5,
});

// Logic to calculate the angle between shoulder and ear
function analyzePosture(landmarks) {
  const leftShoulder = landmarks[11];
  const leftEar = landmarks[7];

  // Calculate the vertical alignment (simplified)
  const yDiff = Math.abs(leftShoulder.y - leftEar.y);

  // If the vertical distance between ear and shoulder is too small, 
  // it usually means the head is leaning forward (slouching).
  if (yDiff < 0.15) { 
    return "bad";
  }
  return "good";
}

Step 3: The "Tech Neck" Monitor Loop

Now, we connect the webcam feed to the MediaPipe engine. We’ll add a "cooldown" timer so the app doesn't spam you with notifications every second.

const videoElement = document.getElementById('input_video');
const { ipcRenderer } = require('electron');

let lastNotificationTime = 0;

async function onResults(results) {
  if (!results.poseLandmarks) return;

  const status = analyzePosture(results.poseLandmarks);

  if (status === "bad") {
    const now = Date.now();
    // Only notify once every 5 minutes to avoid annoyance
    if (now - lastNotificationTime > 300000) {
      ipcRenderer.send('notify-bad-posture');
      lastNotificationTime = now;
    }
  }
}

const camera = new Camera(videoElement, {
  onFrame: async () => {
    await pose.send({image: videoElement});
  },
  width: 640,
  height: 480
});
camera.start();

Why This Matters (The "Official" Way) 🥑

Building a prototype is easy, but making it robust—handling different lighting conditions, minimizing battery drain, and ensuring zero-latency detection—is where things get interesting.

If you are looking for advanced patterns, such as integrating specialized TensorFlow models or building enterprise-grade Vision AI pipelines, the official resources at WellAlly Tech Blog offer incredible insights into the "production-ready" way of doing things. It’s been my go-to source for inspiration when scaling AI tools beyond just a weekend project.

Conclusion: Save Your Spine! 🏁

In less than 100 lines of code, we’ve built a functional AI assistant that monitors your physical health using nothing but your webcam and some clever JavaScript.

Next Steps:

UI Feedback: Add a "posture score" gauge using Canvas API.
Privacy Mode: Add a toggle to blur the background using MediaPipe's Selfie Segmentation.
Analytics: Track your "good posture" percentage over a week.

Are you going to try this out? Or do you have a better way to calculate "slouching" angles? Let me know in the comments! 👇

Happy coding, and stay upright! 💻🥑✨

DEV Community