Amblyotube: engineering a dichoptic vision gym
The Binance $6.3 billion heist dominates headlines — state‑level actors, compromised infrastructure, billions vanishing in minutes. It is a stark reminder that even decentralized systems have weak points. Millions of adults with amblyopia (lazy eye) face a quieter vulnerability: they are told the ‘critical period’ for treatment closed in childhood, left with compromised depth perception, eye strain, and reduced coordination that subtly sabotages daily tasks — driving, sports, reading, even pouring coffee.
Amblyotube, developed by Seven Sports for the Meta Quest platform, challenges that dogma. The app exploits the Quest’s dual‑eyepiece architecture to deliver two independent visual streams simultaneously, turning any YouTube clip into a structured dichoptic training session. Below is a breakdown of the engineering decisions that make this possible, the trade‑offs we accepted, and the measurable constraints we operate within.
Core rendering pipeline
The Quest presents two physically separate displays (one per eye) driven by a single GPU. Amblyotube creates a stereo‑aware render pass that samples the same video texture twice, each with its own post‑process chain. In Unity’s URP this looks roughly like:
// Pseudo‑code for the per‑eye blur/contrast shader
Shader "Amblyotube/DominantEyeShader" {
Properties {
_MainTex ("Video", 2D) = "white" {}
_BlurRadius ("Blur", Range(0, 8)) = 0
_Contrast ("Contrast", Range(0, 2)) = 1
_Gamma ("Gamma", Range(0.1, 3)) = 1
_Opacity ("Opacity", Range(0, 1)) = 1
}
SubShader {
Pass {
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#include "UnityCG.cginc"
sampler2D _MainTex;
float _BlurRadius, _Contrast, _Gamma, _Opacity;
struct v2f { float4 pos : SV_POSITION; float2 uv : TEXCOORD0; };
v2f vert (appdata_base v) {
v2f o; o.pos = UnityObjectToClipPos(v.vertex); o.uv = v.texcoord; return o;
}
fixed4 frag (v2f i) : SV_Target {
fixed4 col = tex2D(_MainTex, i.uv);
// Gaussian blur approximation
col.rgb = blur(_MainTex, i.uv, _BlurRadius);
// Contrast & gamma
col.rgb = pow((col.rgb - 0.5) * _Contrast + 0.5, 1.0 / _Gamma);
col.a *= _Opacity;
return col;
}
ENDCG
}
}
}
The dominant‑eye shader exposes four tunable uniforms — blur radius, contrast, gamma, opacity — that the user adjusts at runtime via the floating UI. At 100 % opacity the shader behaves like a digital patch; the therapeutic sweet spot is usually a partial transparency (≈0.3‑0.5) so both eyes receive usable signal.
Lazy‑eye sharpening (MFBF mode)
The “Lazy Eye Sharpener” runs a lightweight object‑detector on the video frame before it reaches the GPU. The current build bundles a quantized MobileNet‑SSD (≈2 MB) that runs on the Quest’s Snapdragon XR2 NPU at ~30 fps. Detected human silhouettes trigger a per‑pixel sharpening kernel applied only to the lazy‑eye view:
- Edge‑enhancement – unsharp mask with a 3×3 kernel.
- Color coding – red outline (R = 1, G = 0, B = 0) + yellow‑green highlight (R = 1, G = 1, B = 0).
- Temporal pulsing – a 2 Hz sinusoidal alpha modulation (0.6 → 1.0) that creates a subtle “flicker” without exceeding the 60 Hz display refresh.
Because the detector runs on the NPU, the GPU budget for the stereoscopic video decode stays under 12 ms/frame, leaving headroom for the AR pass‑through mode.
Magenta Focus Cue
Binocular fusion is encouraged by a moving circular marker rendered twice:
- Lazy eye: high‑visibility magenta (R = 1, G = 0, B = 1) at full opacity.
- Dominant eye: low‑contrast grey (R = G = B = 0.3) at 30 % opacity.
Magenta was chosen because it sits outside the dominant skin‑tone and foliage gamut, maximizing saliency for the suppressed eye while remaining unobtrusive for the fellow eye. The marker follows a slow Lissajous trajectory (≈0.2 Hz) to avoid predictability.
Ancillary tools
| Feature | Implementation note |
|---|---|
| Wedge Occlusion | Two black triangular meshes parented to each eye’s camera; vertex shader scales them to cover the peripheral 15° where rivalry is strongest. |
| Room View (AR pass‑through) | Uses Quest’s OVRPassthroughLayer with a custom shader that composites the dichoptic video on top of the real‑world feed. Latency ≈ 12 ms end‑to‑end. |
| LogMAR vision test | Generates Landolt‑C optotypes at a virtual 3 m distance. Staircase algorithm (1‑up/1‑down) runs up to 30 trials per eye; results stored locally and fed back into the therapy profile (blur/contrast defaults). |
| Floating UI | Built with UnityEngine.UI + XR Interaction Toolkit. Panel is a Canvas in World Space, draggable via controller ray or hand‑tracking pinch. All controls (play/pause, seek, volume, language, Reset All) bind to the same VideoPlayer instance. |
| Offline catalog | ~150 MB of pre‑downloaded MP4s (cartoons, educational clips) stored in Application.persistentDataPath. No network call required after first launch. |
Performance envelope
| Metric | Measured (Quest 2) | Target |
|---|---|---|
| Frame time (stereo video + shaders) | 11.3 ms | ≤ 13 ms (72 fps) |
| NPU inference (MobileNet‑SSD) | 4.8 ms | ≤ 6 ms |
| AR pass‑through compositing | 2.1 ms | ≤ 3 ms |
| Total CPU (UI, logging) | 3.5 ms | ≤ 5 ms |
| Battery drain (30 min session) | ~7 % | ≤ 10 % |
All numbers come from internal profiling on a Quest 2 (Snapdragon XR2, 6 GB RAM). The 1‑hour daily ceiling is a hard guard‑rail enforced by the app: a background timer disables the render loop after 60 minutes and prompts a cooldown.
Engineering trade‑offs
- Shader complexity vs. NPU load – We kept the dominant‑eye shader intentionally simple (single pass, no texture‑lookups beyond the video) to guarantee headroom for the object detector. A more sophisticated frequency‑domain filter would improve suppression granularity but would push GPU time > 15 ms.
- Model size vs. detection recall – The quantized MobileNet‑SSD misses small faces at > 2 m distance. We accept this because the therapeutic goal is to highlight human figures in typical living‑room viewing distances (1‑3 m). A larger YOLO‑v5‑s model would raise recall ~8 % but adds 6 ms NPU latency.
- Offline vs. streaming – YouTube streaming adds network jitter and DRM constraints. The offline catalog guarantees deterministic frame timing, essential for the flicker stimulus (2 Hz ± 0.05 Hz). Users can still paste a YouTube URL; the app downloads the first 5 minutes locally before training starts.
Deployment & updates
The package is shipped as a single .apk via the Meta Quest Store. OTA updates are delivered through Meta’s App Lab channel, allowing us to push shader uniform defaults or new video packs without a full store review. Versioning follows SemVer; the LogMAR staircase parameters are version‑locked so that a profile generated in v1.2 remains valid in v1.3.
Closing thoughts
Amblyotube demonstrates that a consumer‑grade XR headset can be repurposed as a programmable vision‑therapy platform without custom optics. By treating each eye as an independent compute target, we gain a software‑defined “patch” that is adjustable, measurable, and — crucially — engaging enough for daily adherence. The system is deliberately supplementary: it does not claim to cure amblyopia, nor does it replace a clinician’s prescription. It simply gives adults a structured, data‑backed way to keep the visual cortex plastic.
If you’re building on Quest and need a reference for stereoscopic post‑processing, NPU‑accelerated detection, or AR‑overlay compositing, the code paths above are a good starting point.
https://www.meta.com/en-gb/experiences/amblyotube/25906906972338493/
Top comments (0)