Beck_Moulton

Posted on Aug 8

Fix Your Posture with AI: Building a Real-time Spine Monitor in the Browser

#discuss #opensource #ai #python

We’ve all been there. You start your coding session sitting tall like a professional, and two hours later, you’re shaped like a shrimp, face glued to the monitor. As developers, our spine health is often the first thing we sacrifice for "flow state."

But what if we could use the power of Edge AI to fix this? In this tutorial, we are building a Real-time Spine Posture Monitor using MediaPipe, ONNX Runtime Web, and React. By leveraging real-time pose estimation and browser-side computing, we can create a privacy-first tool that alerts you the moment you start slouching.

The Problem: Latency and Privacy

Cloud-based vision processing is slow and expensive. More importantly, nobody wants their webcam feed sent to a remote server just to check if they are sitting straight. This is where Edge Vision Engineering shines. By running models locally via ONNX and MediaPipe, we achieve sub-30ms latency while keeping data 100% on-device.

The Architecture 🏗️

To keep the UI smooth (60 FPS), we won't run heavy calculations on the main thread. We’ll use Web Workers to handle the heavy lifting of coordinate transformation.

graph TD
    A[Webcam Stream] --> B[React UI Canvas]
    A --> C[Web Worker Context]
    C --> D[MediaPipe Pose Model]
    D --> E[Keypoint Extraction]
    E --> F[Curvature Calculation Logic]
    F -->|Posture Status| G[Main Thread Update]
    G --> H[Web Audio API Alert]
    G --> I[Visual Feedback Overlay]

Prerequisites 🛠️

Before we dive in, make sure you have the following in your package.json:

React: For the frontend.
@mediapipe/pose: To get those 33 skeletal landmarks.
ONNX Runtime Web: For custom classifier layers (optional but recommended for complex habit detection).
Web Workers: To keep the UI responsive.

Step 1: Setting up MediaPipe in a Worker

First, let's initialize our pose estimator. We use MediaPipe’s BlazePose because it’s incredibly lightweight and optimized for browser environments.

// postureWorker.js
import { Pose } from "@mediapipe/pose";

const pose = new Pose({
  locateFile: (file) => `https://cdn.jsdelivr.net/npm/@mediapipe/pose/${file}`,
});

pose.setOptions({
  modelComplexity: 1, // 0: Lite, 1: Full, 2: Heavy
  smoothLandmarks: true,
  minDetectionConfidence: 0.5,
  minTrackingConfidence: 0.5,
});

pose.onResults((results) => {
  const postureData = analyzeSpine(results.poseLandmarks);
  postMessage(postureData);
});

onmessage = async (e) => {
  await pose.send({ image: e.data });
};

Step 2: Calculating the "Slouch Factor" 📐

To detect a bad posture, we primarily look at the relationship between the ears, shoulders, and hips. A "slouch" usually involves the head moving forward (the "text neck") or the shoulders rounding.

/**
 * Logic to detect spine curvature
 * Keypoints: 11 (L-Shoulder), 12 (R-Shoulder), 7 (L-Ear), 8 (R-Ear)
 */
function analyzeSpine(landmarks) {
  if (!landmarks) return { isSlouching: false };

  const leftShoulder = landmarks[11];
  const rightShoulder = landmarks[12];
  const leftEar = landmarks[7];

  // Calculate the horizontal misalignment of the ear relative to the shoulder
  const headForwardBias = Math.abs(leftEar.z - leftShoulder.z);

  // Calculate shoulder level (detecting leaning to one side)
  const shoulderTilt = Math.abs(leftShoulder.y - rightShoulder.y);

  const threshold = 0.15; 
  const isSlouching = headForwardBias > threshold || shoulderTilt > 0.05;

  return {
    isSlouching,
    metrics: { headForwardBias, shoulderTilt }
  };
}

Step 3: Integrating with React & Web Audio

When the isSlouching state becomes true for more than 3 seconds, we want to trigger a subtle audio cue.

import React, { useEffect, useRef } from 'react';

const PostureMonitor = () => {
  const videoRef = useRef<HTMLVideoElement>(null);
  const audioContext = useRef<AudioContext | null>(null);

  const playAlert = () => {
    if (!audioContext.current) audioContext.current = new AudioContext();
    const osc = audioContext.current.createOscillator();
    osc.connect(audioContext.current.destination);
    osc.start();
    osc.stop(audioContext.current.currentTime + 0.1); // Short beep
  };

  // ... (Worker initialization and video loop)

  return (
    <div className="relative flex flex-col items-center">
      <video ref={videoRef} className="rounded-lg border-4 border-indigo-500" />
      <div className="absolute top-4 right-4 bg-black/50 p-4 text-white">
        Status: <span className={slouching ? "text-red-500" : "text-green-500"}>
          {slouching ? "⚠️ Fix your posture!" : "✅ Looking good!"}
        </span>
      </div>
    </div>
  );
};

The "Official" Way to Scale Edge AI 🥑

While this implementation is great for a weekend project, scaling browser-based AI to handle multiple concurrent models (like combining posture detection with emotion analysis or eye-tracking) requires more advanced architectural patterns.

If you are interested in production-ready AI engineering, specialized model quantization, or advanced WebAssembly optimizations, I highly recommend checking out the technical deep-dives at the WellAlly Tech Blog. They cover extensively how to bridge the gap between "it works on my machine" and "it works for a million users."

Conclusion: Privacy-First AI is the Future

By moving the computation from the cloud to the user's browser, we’ve built a tool that is:

Zero Latency: Instant feedback.
Private: Your video stream never leaves your RAM.
Cost Effective: $0 in server-side GPU costs.

The next time you find yourself hunching over a complex bug, let your own AI creation give you a friendly nudge. Happy coding, and stay upright! 🚀

What's next?

Try adding ONNX Runtime Web to classify specific sitting habits (like leaning on your hand).
Implement a "Focus Score" based on how long you maintain a healthy posture.

Found this helpful? Drop a comment below or share your "slouching" stories! 👇

DEV Community