Fix Your Squat Form in the Browser! Building a Real-time AI Coach with MediaPipe and Vue.js 🏋️‍♂️🤖

Have you ever wondered if your "deep squats" are actually just "shallow knee bends"? We’ve all been there. Getting the form right is the difference between building tree-trunk legs and ending up with a nagging back injury.

In this tutorial, we are diving into the world of Computer Vision and Real-time Pose Estimation to build a web-based AI coach. Using the power of MediaPipe, TensorFlow.js, and Vue.js, we will transform your browser into a movement scientist that analyzes your skeletal geometry in real-time. No expensive sensors required—just a webcam and some clever vector math!

Why Real-Time Motion Analysis?

The fitness tech landscape is exploding. From digital mirrors to AI personal trainers, the demand for web-based AI that respects privacy (processing data locally) is at an all-time high. By the end of this guide, you’ll understand how to extract 33 skeletal landmarks from a video stream and calculate joint angles to provide instant feedback.

The Architecture 🏗️

Before we write a single line of code, let’s visualize how the data flows from your webcam to the feedback UI.

graph TD
    A[Webcam Stream] --> B[Vue.js Lifecycle Hook]
    B --> C[MediaPipe Pose Landmarker]
    C --> D{Extract 33 Landmarks}
    D --> E[Geometric Vector Calculation]
    E --> F[Squat Depth & Knee Alignment Check]
    F --> G[Real-time Feedback Overlay]
    G --> H[Render to HTML5 Canvas/Three.js]
    H --> B

Prerequisites

To follow along, you'll need a basic grasp of JavaScript and Vue.js. Our tech stack includes:

• MediaPipe: Google’s high-fidelity body tracking framework.
• TensorFlow.js: The engine that runs our models in the browser.
• Vue.js (Composition API): For a reactive and clean UI.
• Three.js: (Optional) For 3D visualization of the skeleton.

---

Step 1: Setting Up the Pose Landmarker

First, we need to initialize the MediaPipe Pose model. MediaPipe provides a pre-trained model that is incredibly fast and works directly in the browser via WebAssembly (WASM).

import { PoseLandmarker, FilesetResolver } from "@mediapipe/tasks-vision";

let poseLandmarker;
const createPoseLandmarker = async () => {
  const vision = await FilesetResolver.forVisionTasks(
    "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@latest/wasm"
  );

  poseLandmarker = await PoseLandmarker.createFromOptions(vision, {
    baseOptions: {
      modelAssetPath: `https://storage.googleapis.com/mediapipe-models/pose_landmarker/pose_landmarker_heavy/float16/1/pose_landmarker_heavy.task`,
      delegate: "GPU"
    },
    runningMode: "VIDEO",
    numPoses: 1
  });
};

Step 2: The Secret Sauce – Vector Math 📐

To tell if a squat is "good," we need to calculate the angle of the knee. In geometry, the angle between three points (Hip, Knee, Ankle) can be calculated using the atan2 function.

const calculateAngle = (p1, p2, p3) => {
  // p1: Hip, p2: Knee, p3: Ankle
  const radians = Math.atan2(p3.y - p2.y, p3.x - p2.x) - 
                  Math.atan2(p1.y - p2.y, p1.x - p2.x);
  let angle = Math.abs((radians * 180.0) / Math.PI);

  if (angle > 180.0) angle = 360 - angle;
  return angle;
};

Step 3: Real-time Feedback Loop

In our Vue component, we use requestAnimationFrame to process the video frames continuously.

const detectPose = async () => {
  const startTimeMs = performance.now();
  const results = await poseLandmarker.detectForVideo(videoElement, startTimeMs);

  if (results.landmarks.length > 0) {
    const landmarks = results.landmarks[0];

    // MediaPipe Landmarks: 24 (Left Hip), 26 (Left Knee), 28 (Left Ankle)
    const kneeAngle = calculateAngle(landmarks[24], landmarks[26], landmarks[28]);

    // Simple Logic for Squat Correction
    if (kneeAngle < 90) {
      status.value = "Great Depth! ✅";
    } else if (kneeAngle > 160) {
      status.value = "Stand tall! ⬆️";
    } else {
      status.value = "Go lower! 📉";
    }

    drawSkeleton(landmarks); // Custom function to draw on Canvas
  }
  window.requestAnimationFrame(detectPose);
};

Advanced Patterns & Production Ready Tips 🥑

While this demo gets you 80% of the way there, building a production-grade rehabilitation app requires handling edge cases like occlusion (when a limb is hidden), jittery coordinates, and varying lighting conditions.

If you are interested in deep-diving into more advanced motion tracking algorithms, smoothing techniques like One Euro Filter, or building HIPAA-compliant health interfaces, I highly recommend checking out the WellAlly Blog. They have some fantastic resources on how to take these "learning in public" experiments and turn them into scalable healthcare solutions.

Step 4: Visualizing with Three.js (Optional)

To give the app a "premium" feel, you can map the 3D coordinates from MediaPipe (landmarks[i].x, .y, .z) to a Three.js rigged model. This allows the user to see their "digital twin" from different angles while they exercise!

// Quick snippet for Three.js mapping
const updateAvatar = (landmarks) => {
  const kneeBone = myThreeJsModel.getObjectByName("Knee_L");
  const angle = calculateAngle(landmarks[24], landmarks[26], landmarks[28]);
  kneeBone.rotation.x = angle * (Math.PI / 180);
};

Conclusion

We’ve just scratched the surface of what’s possible with MediaPipe and Web-based AI. In less than 100 lines of code, we created a tool that understands human movement! 🚀

Next Steps for You:

1. Try adding a "Rep Counter" using a simple state machine.
2. Add sound feedback (e.g., a "ding" when you reach full depth).
3. Check out wellally.tech/blog for more inspiration on combining AI with physical therapy.

Did you find this helpful? Drop a comment below with the sport or movement you want to analyze next! 👇