Recall.ai WASAPI Deep Dive Lands — Dual-Track Loopback Still Sets the Real Overlay Latency Floor

#windows #wasapi #audio #stt

On 4–5 November 2026 Recall.ai published a deep dive on building a desktop meeting recorder with Windows Graphics Capture and WASAPI. The post correctly notes that WASAPI is the only native API that can grab system audio (loopback on a render endpoint) without a virtual cable or third-party driver, while a separate capture client handles the microphone.

That is the right starting point. For anyone shipping a live interview or call overlay on Windows the same two clients appear, but the constraints are tighter: the streams must stay separate all the way to distinct STT sessions, the callback path cannot allocate, and the end-to-end partial latency budget is measured in tens of milliseconds rather than “whatever the recorder can tolerate.”

Shared-mode period is still the dominant term

WASAPI shared mode is the practical default. The audio engine mixes every client, runs the APO chain (communications enhancements, spatial, vendor noise suppression), and delivers buffers on a fixed period. On most modern hardware that period is 10 ms. With IAudioClient3 you can ask for the engine’s minimum period; capable drivers often return 5–8 ms. Exclusive mode bypasses the mix and most APOs but seizes the endpoint, kills system sounds, and breaks the call app itself.

The Recall.ai sketch (initialize device enumerator → activate IAudioClient on the render endpoint with AUDCLNT_STREAMFLAGS_LOOPBACK → start the stream) is correct. What the post does not emphasize is that every buffer that leaves the engine already carries that period. Downstream resampling, VAD, network RTT, and model first-partial simply add to it.

Typical additive costs once the samples leave the callback:

Stage	Typical cost
WASAPI shared period	10 ms (5–8 ms if requested)
Callback → SPSC ring copy + format convert	1–3 ms
Resample to 16 kHz mono + lightweight VAD	2–5 ms
Network RTT + model first partial	35–100 ms
Overlay paint	<16 ms

A model that claims 45 ms time-to-final still produces 80–150 ms end-to-end partials once the Windows path is included. Bluetooth headsets, communications enhancements left on, or a busy audio engine push the number higher. No amount of model progress removes the period.

Minimal dual-track shape that actually ships

Two independent clients:

Mic: input stream on the capture endpoint.
Loopback: input stream opened on the render endpoint (cpal and the wasapi crate both expose this; the flag is AUDCLNT_STREAMFLAGS_LOOPBACK under the hood).

Each owns a lock-free SPSC ring (power-of-two capacity). The audio callback only copies samples and advances the write index. A consumer thread resamples, runs VAD, and feeds a dedicated streaming STT session. Cross-track gating (energy or simple correlation) stops the mic from re-transcribing the speakers.

In Rust with cpal the loopback side looks like:

let device = find_output_device(name)?;
let default_cfg = device.default_output_config()?;
let config = StreamConfig {
    channels: default_cfg.channels(),
    sample_rate: default_cfg.sample_rate(),
    buffer_size: BufferSize::Default, // or Fixed for tighter periods
};
// build_input_stream on the render device → loopback
let stream = device.build_input_stream(
    &config,
    move |data: &[f32], _: &InputCallbackInfo| {
        // push to SPSC ring; never allocate, never block
        ring.push_slice(data);
    },
    |err| log::error!("loopback error: {err}"),
    None,
)?;
stream.play()?;

The wasapi crate (recently at 0.27) gives lower-level control if you need IAudioClient3::GetSharedModeEnginePeriod or explicit event-driven buffering. Either path works; the period remains.

Device change and mute-state handling

Recall.ai notes that WASAPI does not automatically surface mute-state or device-change events in a way that is convenient for privacy. You still have to register for IMMNotificationClient callbacks and poll or listen for volume/mute changes on the endpoint. For an overlay that must stay silent when the user mutes, that extra bookkeeping is mandatory. Dropping the stream and reopening on the new default device is the usual recovery path; the rings and STT sessions survive the swap if you design for it.

Why the overlay still cares

Meeting recorders can buffer a few hundred milliseconds and clean up later. A live interview overlay cannot. The other side’s words need to appear while the sentence is still being spoken so the user can respond in the correct language. That is why the capture path, the ring discipline, and the choice of shared-mode period dominate the measured latency long after the model vendor has advertised a new “45 ms” number.

The same dual-track stack also lets you run different models or different languages on each side—useful when the interviewer and the candidate are not speaking the same language.

Trade-offs that remain

Shared mode + period request is the only path that coexists with the call app.
Exclusive mode is faster but unusable in practice for this workload.
Local GGUF/ONNX models remove the network term but still sit behind the WASAPI period.
Cloud models with agent-text or echo-cancellation features (useful for bot-to-human) do not help a pure human-to-human call where both sides must be transcribed cleanly.

The architecture has been stable for years. The model layer keeps getting faster and cheaper. The Windows audio engine has not moved the period.

I'm building Talkaside, a Windows app for job interviews and calls in another language.
Drafted with AI assistance from this project's code and logs.

DEV Community