RepCheck: An AI Helper That Watches Workout Videos for Online Coaches (MediaPipe + Gemma)

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

RepCheck is a helper for online personal trainers.

Online coaches get workout videos from their clients every day. Watching each one closely takes hours, so feedback often ends up as just "looks good 👍". The client doesn't improve, and the coach burns out.

RepCheck fixes that in four simple steps:

1. Film it. The client films one set on their phone and uploads it.
2. We measure it. MediaPipe finds the body in every frame. Python code counts each rep and measures depth, speed, tempo and left/right balance.
3. We check it. Clear, fixed rules turn those numbers into a 0–100 form score and one or two simple tips. The rules use the client's profile: a beginner gets fewer cues, and pain, age 65+ or an injury switches on a careful mode with no load increases.
4. The AI explains it. An open model (built for Gemma, running locally with Ollama) reads only the numbers, never the video, and drafts friendly feedback, a progress note and next week's plan.

Then the most important part: the coach decides. Nothing reaches the client until the coach reads the draft, edits it and clicks Approve. The client can also get it as a voice note.

Clients also get a GitHub-style training page: a contribution heatmap where every workout adds a square, plus streaks and badges.

Who pays? Online coaches with 20–100 remote clients. They already spend their evenings watching videos. My price idea is about $10 per month per coach for up to 20 clients. That only works because the AI runs locally and costs almost nothing per video (more on that below).

Demo

Code

Bin4yi / repCoach

RepCheck

▶️ Watch the 2-minute demo on YouTube

AI helper for online personal trainers. Clients film a set; MediaPipe finds body points, Python code measures and judges form, and Gemma (open-weight, run locally with Ollama) turns only the numbers into feedback, a progress summary and a next-week plan. The coach reviews, edits and approves everything. The AI only drafts.

Run it

Code: https://github.com/Bin4yi/repCoach

git clone https://github.com/Bin4yi/repCoach.git && cd repCoach
cp .env.example .env                         # set COACH_PASSWORD
docker compose up --build                    # backend, frontend, mongo, ollama (pulls Gemma)
docker compose exec backend python -m scripts.seed_demo_data   # 3 demo clients, ~1 year of data
# open http://localhost:3000/login  (coach)   and   http://localhost:3000/c/demo-kasun  (client)

No Docker? Quick local demo with an in-memory DB (no Ollama needed, template messages are used):

cd backend && python -m venv .venv && .venv/Scripts/pip install -r requirements-dev.txt   # Linux/Mac: .venv/bin/pip
python scripts/download_model.py
MONGODB_URI=mongomock:// DEMO_SEED_ON_START=true COACH_PASSWORD=demo uvicorn app.main:app --port 8000
cd

…

View on GitHub

• Backend: Python, FastAPI, MediaPipe Tasks (PoseLandmarker), OpenCV, NumPy/SciPy, MongoDB (Motor).
• Frontend: Next.js, TypeScript, Tailwind, Recharts.
• Tests: 128 backend tests (pose maths, rep counting, judgement rules, the LLM layer with a mocked Ollama, API tests, heatmap and streak logic), plus a frontend type-check, build and tests.
• Run it: docker compose up starts the backend, frontend, MongoDB and Ollama, then pulls Gemma.

How I Built It

The big idea: MediaPipe measures. Code judges. Gemma explains.

I split the work so that each tool does only what it is good at.

1. Measuring (open-source vision). MediaPipe PoseLandmarker finds 33 body points in every frame. My code smooths the jitter, fills short gaps, and checks the video quality. Is the whole body in view? Is the camera angle the one the client chose? Is there only one person in the shot? If not, the client gets a friendly "please film again from the side" message.

2. Counting and scoring (plain code, no AI). Reps are found as valleys in the knee angle (squat) or elbow angle (push-up, curl). Each rep gets its depth, its down time and up time, and its speed. The form score comes from simple weighted rules that I documented, so the same video always gets the same score.

I tested it on real, openly licensed footage, not just synthetic data, and real footage found real bugs:

• A tracking glitch at the bottom of a squat split one rep into two. Fixed: two valleys now count as separate reps only if the signal climbs back up between them.
• Setting up and re-racking the weight looked like extra reps. Fixed: shallow dips at the start or end of a set are dropped.
• My "knees caving in" check gave false alarms in deep squats. Fixed: it now uses the knee-width to ankle-width ratio.

After these fixes, a real side-view squat set counts 6 of 6 reps and a real push-up set counts 6 of 6, with no false knee-cave flag on a front-view goblet squat.

3. Explaining (open-weight LLM). Gemma runs locally through Ollama and uses structured JSON output. It never sees the video, the images, or even the client's name, only the numbers. Then my code checks every answer:

• No invented numbers. Every number in the AI's text must exist in the input data. If not, it regenerates once, then falls back to a plain template.
• No diet or calorie talk, and client messages stay under 120 words.
• Pain means pain. If the client reported pain, the message must tell them to stop and talk to their coach, and it may not suggest more weight.
• Weight increases are capped in code at +5% per week (0% in careful mode), even if the AI suggests more.

4. Storing and sharing.

• MongoDB stores only numbers and text. The raw video is deleted right after processing, and the skeleton overlay video is deleted when the coach approves.
• ElevenLabs turns the approved message into an optional voice note.
• render.yaml deploys the backend and frontend to Render. Ollama runs on the coach's own machine or a GPU box.

Why Does Open Innovation Matter?

For this product, open models aren't just a nice extra. They're what makes it possible.

• Privacy of body videos. People don't want videos of their bodies sent to a big AI company. With MediaPipe and a local Gemma, the video never leaves the server, and it's deleted after measuring. Only numbers and text are kept.
• Near-zero cost per video. A closed vision API would have to read dozens of video frames for every upload. RepCheck sends the AI a tiny JSON of numbers instead, and the model runs locally, so the cost per video is basically electricity. That is what makes a low monthly price for coaches realistic.
• Accuracy by design. An LLM looking at a video can "see" a knee angle that isn't there. Here, measuring is done by an open pose model plus deterministic code, and the LLM only explains numbers it was given, with a check that it didn't invent any.
• No lock-in. Switching to a newer Gemma means changing one setting (OLLAMA_MODEL). A coach can run everything on their own hardware.
• I could fix it. When real footage exposed rep-counting bugs, I could dig into the landmark signals myself and fix them. With a closed "send video, get feedback" API, I would have been stuck.

Prize Categories

• Gemma. The writing layer (session feedback, progress summary, weekly plan) is built for Gemma through Ollama, with structured output and code-side safety checks.
• MongoDB Atlas. Clients, sessions, weight logs and plans are stored in MongoDB (MONGODB_URI points at Atlas).
• ElevenLabs. The coach can send the approved message as a voice note. The promo video's narration was also made with ElevenLabs.