DEV Community

Cover image for How to measure barge-in on your voice agent's phone calls
Piyush Sahoo
Piyush Sahoo

Posted on

How to measure barge-in on your voice agent's phone calls

Why now

For two years most voice agents have been polite in a very specific way: they talk, then they listen. While the agent speaks, a lot of pipelines simply stop hearing the caller, or hear them and ignore it until the agent's sentence is done.

In the last two weeks four launches bet on the opposite: the agent should keep listening while it talks, say "mm-hm" when that fits, and stop when you cut in.

  • Phonic-1 (Oct 29) is a speech-to-speech model that calls tools while the caller is still talking and answers in under 600 ms speech-in to speech-out, per Phonic's docs. (Phonic blog)
  • GPT-Live-1 on Retell (Nov 2) is a full-duplex voice model that talks to the caller directly while a backend text LLM keeps running your prompt and tools. Retell notes it works for single-prompt agents only, not for transfer agents or conversation flows. (Retell changelog)
  • Soniox STT v6 (Nov 3) feeds the agent's own text into recognition so it can filter the agent's voice out of the transcript and keep hearing the user when they interrupt. (Soniox blog)
  • Higgs Realtime Duplex from Boson AI (Nov 5) listens, watches and talks at the same time in 160 ms chunks, with an API waitlist for now. (Boson AI blog)

Four launches in two weeks, one bet. Phonic-1 (Oct 29): speech-to-speech model that calls tools while the caller is still talking, under 600 ms speech-in to speech-out. GPT-Live-1 on Retell (Nov 2): full-duplex model talks to the caller while a backend text LLM runs the prompt and tools, single-prompt agents only for now. Soniox STT v6 (Nov 3): takes the agent's own text to filter its voice out and keep hearing interruptions. Higgs Realtime Duplex from Boson AI (Nov 5): listens, watches and talks at the same time in 160 ms chunks, API waitlist for now.

Here's the funny part. The phone network has worked this way for over a century. A phone call is two audio streams running at the same time, one each way. Nobody on a call waits for a turn token.

So when a full-duplex model meets a phone line, the line isn't the obstacle. The obstacle is everything we bolted between them while models were half-duplex.

Half-duplex versus full-duplex. In a half-duplex pipeline the agent talks, then listens, and the caller is muted while the agent speaks, so an interruption never reaches the model. On a full-duplex phone call both streams are always on: the model hears the caller's "mm-hm", keeps going, and stops shortly after the caller cuts in.

What you need

One two-channel call recording: caller on one side, agent on the other. Most SIP recordings and agent frameworks can give you this. You don't need any vendor tools, and you don't need to change your stack to start.

The method

Vendor demos show the model stopping instantly. What your caller hears is the model plus your bridge plus the network. The only honest number is the one in your own call recordings. Measuring it takes three steps:

  1. Split each channel into 20 ms frames and mark which side is talking in each one.
  2. Find every moment the caller starts speaking while the agent is mid-sentence.
  3. Separate short bursts (under 200 ms, the "mm-hm"s) from real barge-ins, and time how long the agent keeps talking after each barge-in.

Short bursts are backchannels: the caller saying "mm-hm" or "yeah" while the agent talks. A good duplex agent should keep going through those. Real barge-ins are the ones where it should stop, and how long it keeps talking after one is the agent's stop time.

Read the result and set a stop budget

That last number, the agent's stop time, is the one to watch. On a synthetic test call (six seconds of generated tones, not a real call), this method found two real barge-ins and one backchannel, with 1.3 seconds of overlapping speech in total. After the first barge-in the agent kept talking for 700 ms, and after the second for 500 ms.

Timeline of a synthetic test call, six seconds of generated tones, not a real call. Agent and caller speech are drawn as two lanes. One short caller "mm-hm" of 120 ms counts as a backchannel. Two real barge-ins follow, after which the agent keeps talking for 700 ms and then 500 ms. Overlapping speech totals 1.3 seconds.

I use a 300 ms stop budget as my own starting point, not a standard. Pick yours, then measure a batch of calls from your current stack and again after you try a duplex model. If the stop time doesn't drop, the model isn't your bottleneck. Your bridge is.

Two honest limits: deciding who is talking from loudness alone means a noisy line will look like speech, so tune the loudness threshold to your lines, and you can only judge what's in the recording, so record as close to the caller as you can.

Four half-duplex habits to check in your bridge

If the stop time doesn't move when you switch models, look for these:

Checklist of four half-duplex habits to check in your call path: inbound audio gated while the agent speaks, echo coming back up the line, extra buffering between the phone line and the model, and the handoff to a human.

1. Inbound audio gated while the agent speaks. A common trick in cascaded stacks was to mute or drop caller audio during TTS playback, so the agent wouldn't transcribe itself. A duplex model needs that audio. If your bridge still drops frames during playback, the model never hears the interruption it was trained to handle.

2. Echo coming back up the line. On a phone call, some of the agent's own voice can come back on the caller's stream. A half-duplex pipeline hid this by not listening. A duplex one hears it, and without handling it can "interrupt itself". This is the exact problem Soniox v6 is aiming at with agent-side text input. (I wrote about the self-interrupt bug last week; duplex models make it matter more, not less.)

3. Frame pacing. Phone audio travels in small frames, typically 20 ms of 8 kHz audio. Duplex models work on their own chunk size (Boson quotes 160 ms). Your bridge buffers between the two. Every extra buffer you add is time the agent keeps talking after the caller has started.

4. The handoff. Retell's own note is telling: GPT-Live-1 isn't available for transfer agents. Transfers, holds and warm handoffs happen at the call layer, not in the model. Plan where a duplex call goes when it needs a human.

Wrapping up

Full-duplex models are good news for phone agents. Phone calls were always the place where turn-taking broke worst: people interrupt, say "yeah yeah", talk over hold music. Models that expect that fit the phone better than the ones that came before.

But they move the hard part. Once the model can listen while it talks, the quality of the audio path decides how it behaves: whether frames arrive on time, whether echo is handled, whether the caller's interruption reaches the model or gets dropped in a buffer.


I work on the phone-line side at Vobiz, telephony for voice agents. Whatever model you pick this month, measure your call path first.

Top comments (0)