When you want a desktop app that listens to a Zoom/Teams/Meet call and your own microphone without injecting a bot into the meeting, the only reliable path on Windows is OS-level audio capture.
Why two tracks
A single mixed stream loses who is speaking. For live translation or interview assistance you need:
- Mic track — your voice
- Loopback track — everything the other person says (system audio)
That separation lets the STT engine produce clean transcripts for each side and the UI show them side-by-side.
The capture stack
On Windows the practical combination is:
- cpal (Rust) or the equivalent in your language for device enumeration and stream opening.
- WASAPI loopback for the system audio (the call).
- Separate buffer management so the two streams stay time-aligned.
Key constraints
- Exclusive mode vs shared mode changes latency and whether other apps can still play sound.
- Sample rate conversion is almost always required (calls are often 16 kHz or 48 kHz, your pipeline may want 16 kHz for Whisper-class models).
- You must handle device hot-plug and default-device changes mid-call.
Practical lessons
- Buffer size is a latency vs stability trade-off. Too small and you get underruns; too large and the translation lags.
- Always timestamp both tracks from the same clock so you can later align transcripts.
- Test with headphones. Loopback that includes your own mic creates feedback loops that are painful to debug.
The result is a private, local pipeline: no meeting bot joins the call, both sides are heard, and the rest of the app (STT, translation, overlay) can work on clean streams.
I'm building Talkaside, a Windows app for job interviews and calls in another language.
Drafted with AI assistance from this project's code and logs.
Top comments (0)