When Open Source Meets Vision Therapy: A Builder's Reflection on Meta's Muse and Amblyotube
I still remember the night I cloned the Amblyotube repo onto a spare Raspberry Pi, hoping to turn a dusty hallway into a low‑latency gaze‑tracking lab. The README promised sub‑30 ms frame‑to‑photon latency if you paired it with Meta’s Muse model, a lightweight transformer that runs on‑device thanks to quantization to INT8. I spent an afternoon wiring a cheap IR camera, tweaking the camera‑intrinsics JSON, and watching the first heat‑map appear on the terminal — jittery at first, then steady as the Kalman filter settled.
The real breakthrough came when I swapped the stock loss for a contrastive objective that the Muse paper suggested for few‑shot adaptation. Training on a handful of labeled sessions from my niece’s amblyopia therapy cut the calibration error from 2.4° to 0.7° in under 200 epochs. The model size stayed under 4 MB, so the Pi could infer at 60 fps without a fan.
What struck me most wasn’t the numbers — it was the moment my niece laughed when the on‑screen butterfly followed her eyes without lag. That instant turned a spreadsheet of latency budgets into a human story. Open source gave me the scaffolding; Muse gave me the brain; Amblyotube gave me the plumbing. Together they turned a research prototype into a living room therapy station.
If you’ve ever wondered whether a side project can become a real aid, try wiring the pieces yourself — you might be surprised by what a few lines of config can unlock. The code is on GitHub, waiting for your pull request.
Top comments (0)