I Built a Focus-Scoring Browser Using Camera AI, Claude, and MediaPipe

empsoft — Tue, 31 Mar 2026 13:29:43 +0000

Social media and content were already saturated. Then AI arrived and multiplied the volume of information and parallel tasks by orders of magnitude. In this world, I wanted to bring the value of focus I learned from meditation practice into a tool anyone could use — a browser that watches how you browse the web and shows you, in real time, how focused you are.

That's Gaze Browser — a mobile browser that scores your concentration from 0 to 100 using your front camera and AI face analysis.

The Problem I Wanted to Solve

We check our phones dozens of times a day. We open a browser to look something up, get distracted by a notification, click a link, and 20 minutes later we've forgotten what we originally searched for.

I wanted to build something that gently reminds you: "Hey, are you still focused?" — not by blocking content, but by reflecting your own behavior back at you.

How It Works

Gaze Browser uses Google MediaPipe FaceLandmarker to detect 478 facial landmark points in real time via your front camera. It then analyzes:

Gaze direction — Are you looking at the screen?
Blink rate — Are you alert or drowsy?
Head pose — Are you tilting or turning away?
Facial muscle tension — Are you engaged or relaxed?

These signals are combined using principles from cognitive science and behavioral analysis to produce a single focus score from 0 to 100, displayed as a real-time floating widget while you browse.

Key Technical Decisions

Native AI execution: Swift on iOS, Kotlin on Android — no web-based ML overhead
Flutter for UI: Cross-platform UI with platform channels for loose coupling between AI and interface
Personal calibration: The app learns your baseline focus posture during an initial calibration phase
Fully offline: All AI processing happens on-device. No data leaves your phone.

The Tech Stack

Layer	Technology
AI/ML	Google MediaPipe FaceLandmarker
iOS Native	Swift + Platform Channels
Android Native	Kotlin + Platform Channels
UI Framework	Flutter
Testing	Flutter integration_test + Fastlane
Dev Partner	Anthropic Claude (Claude Code)

Building with Claude as My Dev Partner

I built Gaze Browser solo, but not alone. Claude (via Claude Code) was my constant development partner throughout the project. From architecture decisions to debugging platform channel issues between Flutter and native code, Claude helped me move fast without sacrificing quality.

Some specific ways Claude helped:

Designing the scoring algorithm based on cognitive science literature
Debugging iOS-specific MediaPipe integration issues
Writing and reviewing Flutter widget tests
Optimizing the real-time rendering pipeline to maintain 60fps while running face analysis

What I Learned

Face analysis is surprisingly expressive — even small changes in blink rate or micro-expressions correlate with attention shifts
Gamification works — showing a score triggers positive reinforcement and self-awareness
Privacy-first AI is possible — running everything on-device means zero data compromise
Solo development with AI assistance is a new paradigm — Claude Code made a solo project feel like a team effort

Try It Out

Gaze Browser is available now on the App Store:

👉 Download Gaze Browser

I'd love to hear your feedback, feature requests, or questions about the technical implementation. Drop a comment below!

Fine-Tuning YOLO with Colab MCP Claude Code — No Local GPU Required

empsoft — Mon, 30 Mar 2026 03:57:56 +0000

Fine-Tuning YOLO with Colab MCP × Claude Code — No Local GPU Required

TL;DR

Used Google's official Colab MCP Server to give Claude Code direct access to Colab's GPU for YOLO fine-tuning
Ran the entire ML pipeline — data preprocessing, training config, training execution, evaluation, and model conversion — without leaving the terminal
Built a custom on-device model for a mobile traffic counting app using nothing but a Mac with no GPU

Introduction

In March 2026, Google officially released the Colab MCP Server.

It's an open-source bridge that lets MCP-compatible AI agents like Claude Code and Gemini CLI programmatically control Google Colab's GPU runtimes. In practice, this means you can issue commands from your local terminal, and Claude Code will create cells in a Colab notebook, write code, execute it on a GPU, and return the results — all without touching a browser.

I used this to fine-tune a YOLO model for a traffic counting app I'm building.

On a Mac with no GPU.

The Problem with the Old Workflow

Before Colab MCP, the typical indie developer's Colab workflow looked like this:

1. Prepare data locally
2. Upload to Google Drive
3. Open Colab in browser
4. Copy-paste code into cells
5. Run → Error → Fix → Re-run (manual loop)
6. Download trained model from Drive
7. Convert and evaluate locally

The problem is constant context switching. Terminal → Browser → Drive → Browser → Terminal. Your focus breaks every time, and the manual error-fix-rerun loop quietly eats hours.

Fine-tuning is fundamentally about iteration — adjust parameters → train → evaluate → adjust again. The more friction in this loop, the fewer iterations you run, and the worse your model ends up.

How Colab MCP Changes Everything

With Colab MCP Server, the workflow becomes:

1. Give Claude Code an instruction
2. Claude Code creates a cell in Colab → writes code → executes it
3. Results come back to your terminal
4. Claude Code interprets the results and proposes the next action
5. Approve, and it immediately runs the next step

No browser needed. You can run the entire fine-tuning pipeline from your terminal, on Colab's GPU.

The Fine-Tuning Pipeline

Step 1: Environment Setup

First, set up the training environment on Colab.

Prompt to Claude Code:

Create a new cell in the Colab notebook and install:
- ultralytics (YOLOv8)
- albumentations (for data augmentation)
Verify GPU availability and show nvidia-smi output.

Claude Code creates the cell via Colab MCP, writes the code, and executes it. Once it confirms a Tesla T4 GPU is available, we move on.

Step 2: Data Preprocessing

Have Claude Code handle annotation format conversion.

There's a COCO-format annotation JSON in /content/drive/MyDrive/dataset/.
Convert it to YOLOv8 format and run the script on Colab.
Class mapping:
- car → 0, truck → 1, bus → 2, motorcycle → 3, bicycle → 4, person → 5
Split into train/val at 80:20.

Claude Code writes the conversion script into a Colab cell and executes it — including bounding box clipping and format validation.

Step 3: Data Augmentation

Real-world traffic counting happens in all conditions — clear sky, overcast, backlit, nighttime.

Run an albumentations augmentation pipeline on Colab:
- Random brightness adjustment (±30%)
- Contrast adjustment
- Light motion blur
- Horizontal flip
Sync annotation coordinates with transformations.
Original:augmented ratio = 1:3.

Step 4: Fine-Tuning

The main event.

Fine-tune a YOLOv8s model for 6 classes.
Config:
- Input size: 640
- Epochs: 100
- Batch size: 16 (to fit in T4 VRAM)
- Learning rate: 0.01 with cosine scheduler
- Early stopping at 10 epochs
- Save training curve plots

Claude Code generates the YAML config and runs model.train() in Colab cells.

Training progress is relayed back through Claude Code as it fetches cell outputs. A 100-epoch run on a T4 takes a few hours, but you can check progress along the way.

Step 5: Evaluation

After training, evaluate the model.

Run validation on the trained model.
Output mAP@0.5, per-class Precision/Recall, and Confusion Matrix.
Pay special attention to truck vs car confusion rate.

Claude Code fetches the results and analyzes weak points: "Truck Recall is low — likely insufficient truck samples in the training data."

Based on this, you decide whether to add more data and retrain or adjust parameters. Thanks to Colab MCP, this entire iteration loop stays in your terminal.

Step 6: Mobile Model Conversion

Finally, convert the model for on-device inference.

Convert best.pt to TFLite and CoreML formats.
Apply int8 quantization.
Report model size and mAP difference before/after conversion.

What I Liked About Colab MCP

1. Zero Context Switching

Never leaving the terminal is the biggest win. Without the friction of switching between browser and terminal, your train of thought stays intact and iteration speed goes way up.

2. Automated Error Handling

When an error occurs in Colab, Claude Code fetches the cell output → analyzes the cause → proposes a fix → re-executes on approval. No more manually reading stack traces and Googling.

3. Claude Code Knows ML Best Practices

Fine-tuning has a lot of "conventions" — directory structure, config file formats, data splitting strategies. Claude Code handles these automatically, following best practices without you having to look them up.

4. No Local GPU Needed

Whether you're on an M1 Mac or a Windows laptop, you can access Colab's T4/A100 GPUs remotely. For indie developers, not having to buy expensive GPU hardware is a major advantage.

Gotchas

Colab Session Limits

Free Colab disconnects after 90 minutes idle or 12 hours max. For long training runs, consider Colab Pro. If your session drops mid-training, you can ask Claude Code to resume from the last checkpoint.

File Transfer

As of March 2026, direct file upload via Colab MCP Server is still limited. The reliable approach is to use Google Drive for data transfer.

Data Quality Is Still on You

Claude Code writes scripts perfectly, but whether your annotations are accurate and your training data is representative — that's your responsibility. Garbage in, garbage out applies to AI-driven development too.

What I Built

The fine-tuned model powers AI Foot Traffic Survey, a mobile app that counts vehicles and pedestrians in real time using your smartphone camera.

It detects 6 vehicle types (car, truck, bus, motorcycle, bicycle) plus pedestrians, runs all AI inference on-device, and never saves any images — privacy-first by design.

Free for up to 10 minutes per session.

iOS: App Store
Android: Google Play

Q&A

Q: Does Colab MCP cost anything?
A: The Colab MCP Server itself is free and open source. You can use Colab's free tier for T4 GPU access, but for longer training sessions, Colab Pro is recommended due to session limits. Claude Code requires an Anthropic subscription.

Q: What do I need besides Colab MCP?
A: Just uv (Python package manager) and git installed locally. No GPU required on your machine.

Q: How long does fine-tuning take?
A: Depends on dataset size and epochs. With ~5,000 images and 100 epochs, expect a few hours on a Colab T4 GPU.

Q: Can I use Colab MCP without Claude Code?
A: Yes — it works with Gemini CLI and any other MCP-compatible agent. That said, the experience of building an ML pipeline entirely through natural language instructions was smoothest with Claude Code.

Q: How much does fine-tuning improve accuracy over a pretrained model?
A: The improvement was most noticeable for domain-specific cases — distinguishing small trucks from vans, handling backlit conditions, and reducing bicycle/motorcycle confusion. Specific mAP numbers vary by dataset and app version, but real-world counting accuracy improved significantly.

Q: How did you collect training data?
A: Started with public datasets (COCO, etc.) and supplemented with annotated footage from real traffic environments. Data augmentation (brightness, blur, flipping) helped the model generalize across varied conditions.

Q: What happens if the Colab session disconnects?
A: Training is interrupted, but checkpoints are saved to Google Drive. After reconnecting, you can tell Claude Code to "resume from the last checkpoint" and training picks up where it left off.

Q: Where can I download the app?
A: Available on both iOS (App Store) and Android (Google Play). Search for "AI Foot Traffic Survey". Free for up to 10 minutes per session.

Takeaways

Colab MCP × Claude Code let me run the full YOLO fine-tuning pipeline from a Mac with no GPU
No browser needed — everything from data preprocessing to model conversion happened in the terminal
Colab MCP was just released in March 2026 — it's applicable far beyond ML, to any workflow involving Colab
Fine-tuning ML models is no longer exclusive to ML engineers

DEV Community: empsoft