Soniox STT RT v5 Hits 45 ms Time-to-Final — Capture Budget Still Dominates on Windows

Coval’s public STT benchmark, last run on 3 November 2026, puts Soniox’s STT RT v5 at a mean 45 ms time-to-final-segment (TTFS) — second of 28 measured models — with a 4.7 % word error rate. Qwen3 ASR 1.7B edges it at 35 ms. These numbers are useful. They are also almost irrelevant until the audio has left the Windows audio engine and reached the network (or the local inference process).

The 45 ms number is downstream of capture

TTFS is measured from the end of speech (detected by SileroVAD in the Coval setup) until the final transcript segment arrives. On a real Windows call the timeline looks different:

• WASAPI shared-mode period: typically 10 ms default, or 5–8 ms if the driver exposes a smaller buffer and you request it
• Ring-buffer copy + format conversion (float32 → int16, 48 kHz → 16 kHz): 1–3 ms on a modern CPU if you stay off the audio callback
• VAD or simple energy gate: 2–5 ms
• Network RTT + model first/final: the 45 ms (or whatever the provider returns)
• Overlay or UI update: <16 ms if you paint on a dedicated thread

Add them up and a “45 ms model” becomes a 80–120 ms end-to-end partial on a good day, longer when the shared-mode engine is busy or the device is Bluetooth. Exclusive mode can drop the capture side under 5 ms but steals the endpoint from every other app — usually unacceptable for an interview or support call.

The model latency is real. It is not the dominant term once you leave the benchmark harness.

Dual-track still requires two independent streams

A call has two sources that must stay separate until after recognition:

• Microphone (your side)
• Loopback of the render endpoint (the other side)

Mixing them early forces the recognizer to handle two speakers, two languages, or two acoustic conditions in one beam. Speaker labels and later translation both suffer.

On Windows the loopback path is still “open a render device as an input stream” through WASAPI. In practice that means:

// Minimal sketch — production code also handles device change notifications
// and format negotiation. Callback must not allocate or block.
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};

fn open_loopback(host: &cpal::Host, render_name: &str) -> anyhow::Result<cpal::Stream> {
    let device = host
        .output_devices()?
        .find(|d| d.name().ok().as_deref() == Some(render_name))
        .ok_or_else(|| anyhow::anyhow!("render device not found"))?;

    let supported = device.default_output_config()?;
    let stream = device.build_input_stream(
        &supported.into(),
        |data: &[f32], _: &cpal::InputCallbackInfo| {
            // push into lock-free ring buffer
            // resample / convert on the consumer side
        },
        |err| eprintln!("loopback: {err}"),
        None,
    )?;
    stream.play()?;
    Ok(stream)
}

The callback is real-time. Any allocation, mutex that the UI or network thread might hold, or system call turns into glitches. The 45 ms model does not change that constraint.

Shared mode keeps other apps alive; exclusive mode buys latency at the cost of the device. Most production call tools stay in shared mode and accept the 10 ms quantum, then compensate with aggressive partials and a small look-ahead buffer.

Partial vs final and the overlay budget

Streaming STT APIs return non-final tokens while the speaker is still talking and finalize when they pause. The useful number for an overlay is the time from the end of a phrase until the final text is stable enough to show (or to feed a translator).

A practical pipeline on Windows:

1. Capture both tracks into separate lock-free rings (SPSC, power-of-two size, no allocation on the writer).
2. Consumer thread pulls, resamples to 16 kHz mono int16, runs a lightweight VAD.
3. On speech start, open (or reuse) the streaming WebSocket and start sending 20–40 ms chunks.
4. On each partial, update the overlay only if the text changed and the previous paint is done.
5. On final, optionally run a local or remote translation pass and commit the turn.

The 45 ms TTFS means that once the last audio chunk has left your process, the final text can appear quickly. It does not remove the need for the ring buffer, the VAD, or the careful separation of the two tracks.

Trade-offs that still matter

• Cloud vs local: A 45 ms cloud final is excellent when the network is clean. Local models (Whisper.cpp, faster-whisper, or the new offline checkpoints) trade higher first-token latency and GPU/CPU load for privacy and offline use. Hybrid is common: local for the hot path, cloud for higher accuracy on finalized turns.
• Endpointing: Aggressive endpoint detection reduces TTFS but can cut speakers off. Most production systems expose a configurable silence threshold (300–700 ms) rather than trusting the model’s internal detector alone.
• Rate limits and BYOK: Streaming STT providers still rate-limit concurrent sessions. A dual-track tool that opens two sockets per call hits that limit faster. Pooling, session reuse, and careful back-off remain necessary.
• Device changes: Bluetooth headsets, USB DACs, and docking stations disappear mid-call. The WASAPI notification path (IMMNotificationClient) must restart both streams without dropping the session state.

None of these problems disappear because one benchmark number got smaller.

What the number is actually good for

The 45 ms result is useful as a ceiling. If your end-to-end partial is 400 ms, the model is not the problem — look at capture buffering, resampling, or the fact that you are waiting for a full turn before sending. If you are already under 100 ms end-to-end and the transcript is still late, then the provider’s TTFS (and its p95) becomes the next thing to measure.

For tools that sit on top of live calls — interview assistants, support overlays, real-time translation — the durable work is still the audio graph, the dual-track separation, and the decision of what stays on the hot path versus what can be deferred. Model improvements help. They do not rewrite the Windows audio stack.

I'm building Talkaside, a Windows app for job interviews and calls in another language.
Drafted with AI assistance from this project's code and logs.