Soniox STT v6 Adds Agent Text Input — Dual-Track WASAPI Capture Still Sets the Latency Floor

On 3 November 2026 Soniox shipped STT v6. The headline features are agent text input and agent voice echo cancellation. You stream the agent’s outgoing text (the words it just sent to TTS) over the same WebSocket while the model listens to the caller. The model uses that context to improve short-reply recognition and to drop residual agent audio that leaked past acoustic echo cancellation. It keeps listening during playback so barge-in still works. Pricing is roughly $0.12/hour streaming. Coval’s latest numbers still put the previous RT v5 at ~48 ms time-to-final.

That is useful for voice-agent stacks where TTS playback leaks into the mic. It does not change the capture budget on a Windows desktop or laptop when you are building a local or hybrid assistant for calls and interviews.

The Windows capture path still dominates

A practical end-to-end timeline on modern Windows hardware looks like this:

1. Endpoint (USB headset, Bluetooth HFP, built-in mic, or WASAPI loopback of the render device).
2. Audio engine (audiodg.exe + audioeng.dll) in shared mode: mixing, communications enhancements, vendor APOs. Adds 10–40+ ms and is fragile under load or with Bluetooth.
3. Your client buffer period. Exclusive mode can hold 10 ms (or lower if the driver allows); shared mode is usually larger.
4. Resample + format conversion (commonly 48 kHz float32 → 16 kHz int16) + lightweight energy/VAD gate. 1–3 ms if you stay off the audio callback thread.
5. Streaming STT first partial / final. Even a fast model is tens of milliseconds once the audio leaves your process.
6. Overlay or UI paint (<16 ms on a dedicated thread).

Agent text input in v6 improves the model’s understanding of the content once the audio arrives. It does not shrink steps 1–4. If your capture path is shared-mode with Bluetooth and a 40 ms period, the model never sees clean, low-latency audio in the first place.

Core Audio path. Shared mode goes through the engine mixer; exclusive mode bypasses it. Loopback is an input stream on a render device. Diagram from Microsoft Learn.

Dual-track capture is still the correct architecture

For any call or interview assistant you need two independent streams:

• Microphone (your voice)
• Loopback of the render endpoint (the other side, or the agent’s TTS if it is playing locally)

Mixing them early destroys speaker labels and forces the recognizer to handle two languages, two accents, or agent playback in one beam. The correct layout is two lock-free SPSC ring buffers, two consumer threads that resample once and run a cheap VAD, and only then open or reuse the streaming WebSocket (or local model) with 20–40 ms chunks.

WASAPI exclusive mode on the capture side (and on the render side if you are doing local TTS) is the only reliable way to keep the buffer period under ~10 ms and avoid the engine’s enhancements. Shared mode is easier and more compatible; it also adds engine latency and any “communications” processing the OEM enabled.

A minimal exclusive-mode capture skeleton (C++, simplified):

WAVEFORMATEX wfx = { WAVE_FORMAT_IEEE_FLOAT, 1, 48000, 48000 * 4, 4, 32, 0 };
// AUDCLNT_SHAREMODE_EXCLUSIVE | AUDCLNT_STREAMFLAGS_EVENTCALLBACK
// IAudioCaptureClient::GetBuffer in the event thread
// copy into SPSC ring, never process on the callback

On the loopback side you open the render endpoint as a capture client with AUDCLNT_STREAMFLAGS_LOOPBACK. The two rings stay time-aligned by using the same clock (QueryPerformanceCounter or the audio engine’s clock) and dropping or duplicating frames only at the consumer.

Where agent text input actually helps

Once you have clean dual-track audio, v6’s agent text input is a useful optional signal:

• You already know the exact words your local TTS or the remote agent just spoke.
• You can stream those words as context so short replies (“yes”, “the second one”, a name) are less likely to be misheard.
• Residual echo that survives your own AEC or headphone leakage can be filtered at the model instead of requiring perfect acoustic isolation.

It does not replace the dual-track design. If you mix mic + loopback before the model, you lose the ability to label speakers and you still pay the full capture latency. If you keep them separate, you can send the remote-side track to STT with the agent text as context, and the mic track to a different stream or the same model with different biasing.

Local models (faster-whisper, Whisper.cpp, or the smaller Nemotron/Granite speech checkpoints) avoid the network hop entirely. They trade higher first-token latency and CPU/GPU load for offline use and privacy. Hybrid is common: local for the hot path and partials, cloud for higher-accuracy finals on the remote track.

Practical trade-offs

• Exclusive mode requires careful device handling (device change notifications, format negotiation, recovery from AUDCLNT_E_DEVICE_INVALIDATED). Shared mode is more resilient but slower.
• Bluetooth HFP is still a latency and quality tax. Prefer USB or the built-in mic + headphones for interviews.
• Format conversion must happen off the audio thread. A single extra memcpy or a blocking resample on the callback will glitch.
• Overlay latency is cheap if you keep the UI thread free and only push finalized or high-confidence partials.

The Soniox v6 numbers are real. The agent-text context is a genuine improvement for cascaded voice agents. On Windows the latency floor is still set by how you open the WASAPI clients, how you separate the two tracks, and how little work you do on the capture callback. Everything after that is downstream.

I'm building Talkaside, a Windows app for job interviews and calls in another language.
Drafted with AI assistance from this project's code and logs.