DEV Community

Ashish
Ashish

Posted on

Early deafness may reshape the brain to devote more resources to peripheral vision

Dichoptic VR for Amblyopia: Building AmblyoPunch on Meta Quest

A 2022 PNAS study showed that early deafness drives enhanced peripheral vision through cross-modal neuroplasticity. The same principle — targeted, intensive practice reshaping cortical maps — underpins modern amblyopia therapy. AmblyoPunch applies it in VR: a Unity 2022 LTS project on Meta Quest that replaces the eye patch with a four-stage dichoptic protocol wrapped in a Beat Saber–style punch-and-dodge loop.

Why Dichoptic, Why VR

Traditional patching forces the weak eye to work alone. Dichoptic training presents different images to each eye simultaneously so the brain must combine them. A headset gives per-pixel, per-eye control at 72 fps — impossible on a monitor — and the 3D environment supplies natural disparity cues for fusion practice.

AmblyoPunch ships on the Meta Quest store (https://www.meta.com/en-gb/experiences/amblyopunch/1239507485902689/) and runs entirely on-device. No streaming, no PC tether.

Architecture Overview

Layer Stack Choice Rationale
Engine Unity 2022 LTS Long-term support, mature XR plug-in ecosystem
XR SDK Meta XR SDK (OpenXR backend) Direct access to Quest compositor, layer submission, hand-tracking
Rendering Dual-camera rig + custom Gabor shader One camera per eye; shader driven by per-eye uniforms
Protocol JSON-driven 4-stage state machine Designer-tweakable without code rebuilds
Input Hand tracking + Touch controllers Accessibility; controller fallback for precision
Telemetry Opt-in, local-first JSONL Privacy by default; exportable for clinicians

Dual-Camera Dichoptic Rig

Two cameras share a single XRRig but target separate RenderTexture eyes. The left/right eye matrices come straight from XRDisplaySubsystem. A command buffer blits each texture to the corresponding compositor layer, guaranteeing zero interocular crosstalk on Quest 2/3 panels.

// Simplified setup
var leftCam  = rig.camera.leftEye;
var rightCam = rig.camera.rightEye;
leftCam.targetTexture  = leftRT;
rightCam.targetTexture = rightRT;
Enter fullscreen mode Exit fullscreen mode

Gabor Patch Shader

Coins render a single full-screen quad per eye with a fragment shader evaluating a Gabor function:

float gabor(vec2 uv, float freq, float sigma, float theta, float phase) {
    vec2 rot = vec2(cos(theta), -sin(theta); sin(theta), cos(theta)) * uv;
    float env = exp(-dot(rot, rot) / (2.0 * sigma * sigma));
    return env * cos(2.0 * PI * freq * rot.x + phase);
}
Enter fullscreen mode Exit fullscreen mode

Uniforms (_Frequency, _Sigma, _Theta, _Phase) are set per eye each frame from the JSON protocol. Stage 1 sends non-zero params only to the amblyopic eye; Stage 4 introduces a controlled disparity offset inside Panum's fusion area.

The Four-Stage Protocol (Automatic Mode)

The JSON schema encodes every stage, sub-stage, and transition rule. Example fragment:

{
  "stage": 2,
  "name": "Breaking Suppression",
  "dominantEyeDimFactor": 0.15,
  "advanceThreshold": 0.70,
  "demoteThreshold": 0.40,
  "minTrials": 10
}
Enter fullscreen mode Exit fullscreen mode
  1. Monocular Warm-up — Targets visible only to the lazy eye; dominant eye sees blank background.
  2. Breaking Suppression — Both eyes see targets; dominant eye luminance dropped to ~15 %.
  3. Rebalancing — Dominant eye brightness ramps over 5–10 sub-steps.
  4. Fusion Training — Subtle horizontal disparity (±15 arc-min) forces vergence; success requires ≥ 80 % for strabismus profiles.

Progression uses a rolling 10-trial window. The state machine lives in a ScriptableObject so designers can add worlds (village → city → space → moon → Mars → Venus) without touching C#.

Performance Budget

Metric Target Technique
Frame time ≤ 13.8 ms (72 fps) Single-pass instanced rendering; shader LOD by distance
GPU ≤ 8 ms Baked lighting, no real-time shadows, 1 draw call per coin pool
CPU ≤ 5 ms ECS-lite job system for spawn/pool; Burst-compiled Gabor math
Memory ≤ 1.2 GB Addressables for world assets; streaming load between stages

Hand tracking adds ~0.8 ms; controller path is cheaper. We profile with Unity.PerformanceTesting and Meta's OVRMetricsTool on device.

Comfort & Safety Guardrails

  • Session cap: 15 min default (configurable in wrist menu).
  • IPD check: Startup prompts user to verify headset IPD setting.
  • Suppression monitor: If dominant-eye dimming < 10 % for > 3 stages, log warning for clinician review.
  • Disclaimer screen: "Training / assistive game — not a medical device, treatment, or cure. Complements professional eye care."

What We Learned

  1. Shader simplicity wins. A single Gabor quad per eye outperforms particle systems and avoids fill-rate spikes when coins cluster.
  2. JSON protocol = fast iteration. Clinicians adjusted stage durations over a weekend without engineering involvement.
  3. Hand tracking is viable for gross motor (punch/dodge) but controllers remain preferred for fine orientation tasks.
  4. Opt-in analytics build trust. Zero PII; only stage transitions, success rates, and frame-time histograms.

Try It / Extend It

The experience is live on Quest: https://www.meta.com/en-gb/experiences/amblyopunch/1239507485902689/

If you're building health-focused XR, the dichoptic rendering pattern and JSON-driven protocol are reusable primitives. Fork the concept, swap the Gabor target for your stimulus, and you have a neuroplasticity engine in a headset.

Top comments (0)