Most "voice AI" demos feel unnatural for one simple reason: they are still turn-based walkie-talkies under the hood.
When a user interrupts the AI mid-sentence, a typical wrapper waits for endpointing (300ms+), waits for STT finalization (200ms+), and then cancels the server-side TTS stream while WebSocket frames are still draining. The result is 650ms–900ms of awkward cross-talk.
While building HardCall — an open-source full-duplex AI voice interview & high-stakes drill simulator live at hardcall.statementflow.win — we replaced turn-based waiting with a 52ms dual-loop barge-in architecture.
1. Never Wait for STT End-of-Turn
If you wait for a transcript before cutting audio, you have already lost the human conversation feel. Humans notice interruption lag above ~200ms.
HardCall splits interruption into two asynchronous loops:
- 50ms Audio Fast-Loop (Deterministic): Local Silero VAD detects speech onset (~18ms) and immediately invokes to flush the outbound audio ring buffer in 0.16ms. Zero server round-trip required to stop talking.
- 200ms Semantic Slow-Loop (Intent Gate): While audio is already silent, a lightweight intent gate checks whether the user's utterance is a real rebuttal ("Wait—our p99 dropped 42%...") or meaningless backchannel/noise ("uh-huh", cough).
2. Why Dual-Loop Beats Raw Energy Thresholds
Pure energy VAD cuts out on keyboard clicks or throat clearing. Pure LLM turn-detection feels sluggish.
By combining:
- Instant local buffer flush on speech onset
- State reset to drop stale TTS frames
- Fast semantic classification to either pivot the rebuttal prompt (<200ms) or smoothly resume
you get crisp 52.16ms perceived barge-in without false-positive interruptions derailing the session.
3. Try It Live or Inspect the Source
- Live Full-Duplex Simulator: https://hardcall.statementflow.win
- Open-Source Repo (macOS + Web): https://github.com/wangguode/hardcall
If you are building real-time voice agents over WebRTC or raw 8kHz RTP, I'd love to hear how you handle outbound buffer flushing and false-trigger recovery!

Top comments (0)