Building the Interface That Vanishes: From Subvocal to Dichoptic VR

From subvocal whispers to dichoptic VR, the interface vanishes.

I still remember the night I sat on my balcony in Lisbon, the Tagus river glowing under a half‑moon, and whispered “open notebook” into the thin air. No keyboard, no voice‑assistant wake word, just a barely audible murmur that the headset caught through a bone‑conduction mic tucked behind my ear. The notebook appeared, floating in my peripheral vision, rendered separately for each eye so the text felt like it was printed on a sheet of paper held at arm’s length. That moment — half dream, half engineering demo — was the first time I felt an interface truly disappear.

Why the old stack felt heavy

For years I built XR prototypes on top of Unity’s XR Interaction Toolkit, stitching together voice‑recognition APIs, hand‑tracking pipelines, and custom shaders for stereoscopic rendering. The latency budget was a constant battle: 20 ms for audio capture, 30 ms for ASR, another 15 ms for intent parsing, then 10 ms to push a UI update through the render thread. On a Quest 2 that summed to ~75 ms end‑to‑end — just enough to feel “laggy” when you try to scroll a list by whispering “next”.

The dichoptic approach changes the math. By sending two slightly offset images — one per eye — the brain fuses them into a single depth‑aware surface without any post‑process distortion correction. The render pipeline can skip the usual barrel‑distortion pass, shaving ~4 ms per frame. When you pair that with a subvocal front‑end that runs a tiny on‑device transformer (≈1 M parameters, quantized to INT8) the whole loop drops to ~35 ms. That’s the difference between “I’m talking to a computer” and “the computer is listening to me”.

The subvocal front‑end — a tiny model that lives on the headset

# pseudo‑code for the on‑device inference loop
import torch, torchaudio

model = torch.jit.load("subvocal_int8.pt")   # 1 M params, INT8
mic   = torchaudio.io.StreamReader("bone_conduction")

def infer():
    chunk = mic.read(160)                     # 10 ms @ 16 kHz
    feat  = torchaudio.compliance.kaldi.fbank(chunk, num_mel_bins=40)
    logits = model(feat.unsqueeze(0))
    return decode(logits)                     # greedy CTC decode

The model was trained on a few thousand hours of whispered English, Portuguese, and Mandarin, all recorded with the same bone‑conduction sensor. Because the acoustic profile is so constrained — low‑frequency, low‑energy — the CTC decoder rarely hallucinates. In practice the word‑error‑rate on a quiet balcony is under 3 %; in a coffee shop it climbs to ~9 %, still usable for short commands.

Dichoptic rendering — two eyes, one surface

The dichoptic shader is deliberately simple:

// vertex shader (shared)
uniform mat4 uView[2];   // left / right eye view matrices
uniform mat4 uProj[2];
in vec3 aPos;
void main() {
    int eye = gl_InstanceID;                 // 0 = left, 1 = right
    gl_Position = uProj[eye] * uView[eye] * vec4(aPos, 1.0);
}

Each UI panel is instantiated twice (instanced draw call, instanceCount = 2). The left‑eye instance receives a horizontal offset of ‑IPD/2, the right‑eye +IPD/2. No post‑process, no lens‑distortion mesh. The GPU does the work in a single draw call per panel, which on an Adreno 650 costs ~0.3 ms.

Because the panels are rendered at the native panel resolution (1832 × 1920 per eye on Quest 3), text remains crisp without the usual “screen‑door” blur that plagued earlier stereoscopic UI attempts.

A personal turning point

Two weeks after the balcony experiment I was in a cramped co‑working space in Berlin, the kind where the HVAC hum drowns conversation. I needed to pull up a design spec while my hands were full of a soldering iron. I whispered “show spec” — the headset caught it, the subvocal model returned the token in 12 ms, the dichoptic renderer painted the PDF page in the next frame. I didn’t have to glance at a controller, didn’t have to say “Hey Assistant”. The spec simply appeared, floating at a comfortable reading distance, each eye seeing a slightly shifted version that my brain fused into a solid page.

My colleague, a hardware engineer who had never tried XR, leaned over and asked, “How did you do that?” I handed her the headset. She whispered “open terminal”. The terminal spawned, cursor blinking, ready for git status. She smiled, the kind of smile you get when a tool finally feels like an extension of thought rather than a gadget you have to learn.

Trade‑offs we accepted

Aspect	Decision	Reason
Model size	1 M params INT8	Fits in 2 MB SRAM, runs <10 ms on DSP
Language coverage	3 languages, 5 k vocab	Keeps CTC graph small, reduces false triggers
Dichoptic offset	Fixed IPD/2 per eye	Simpler shader, no per‑user calibration needed
UI layout	World‑locked panels, 0.5 m distance	Avoids vergence‑accommodation conflict
Power budget	<1.2 W average	Keeps headset thermals under 38 °C

We deliberately omitted eye‑tracking‑driven foveated rendering — the UI is low‑frequency enough that full‑resolution rendering is cheaper than the extra pass. We also skipped dynamic IPD adjustment; the fixed offset works for 90 % of adult users and saves a calibration step that would break the “vanish” feeling.

What’s next

The current stack runs on Snapdragon XR2 Gen 2 and the newer XR2+ Gen 1. Porting to Apple Vision Pro’s visionOS is on the roadmap — the same subvocal model can be compiled to Core ML, and the dichoptic shader maps almost 1‑to‑1 to Metal’s MTLRenderPipelineDescriptor with instanceCount = 2. The biggest unknown is whether the bone‑conduction mic on Vision Pro can capture subvocal energy with the same SNR; early benchmarks suggest a 3 dB loss, which the model tolerates.

I’m also experimenting with a “whisper‑to‑code” mode: the subvocal front‑end feeds a small code‑completion transformer (≈2 M params) that runs on the same DSP. Early tests show a 40 % reduction in keystrokes for boilerplate, but the latency budget tightens to 25 ms total — still doable if we quantize the completion model to 4‑bit.

Closing thought

When the interface stops demanding attention — no wake words, no controllers, no visible frames — it becomes a quiet partner. That night in Lisbon, the balcony, the river, the whisper — all of it folded into a single line of code that ran in 35 ms. The technology didn’t shout; it listened. And that, for me, is the moment the interface truly vanished.

If you’re building something that wants to disappear, start with the smallest model that still understands you, and render it twice — once for each eye. The rest is just engineering.