OpenAI published engineering detail on GPT-Live showing that its voice system removes the separate end-of-turn detector from the audio path entirely and lets the voice model decide for itself when to listen, speak, or overlap. Alongside it, OpenAI describes cutting session startup from six network round trips to one, so a client can begin a conversation with a single UDP packet. The reframing is the point: voice is being engineered as a live media system, not as a model with a microphone attached.
Key facts
- OpenAI's transport work reduces media and data startup "from six network round trips to just one."
- GPT-Live "removes the turn detector from the audio path" and its voice model "is full-duplex, which means it can listen and speak at the same time."
- The posts went up on 3 August 2026 from OpenAI's engineering team.
- Primary sources: Continuous voice interaction with GPT-Live and Delivering low-latency voice AI at scale.
The problem with taking turns
OpenAI states the old failure mode crisply: previous systems "relied on tiny models known as turn detectors, which faced an unenviable task: guess too soon, and the user gets cut off; guess too late, and the response feels sluggish. Only after the detector made its decision could the much larger LLM get to work."
That is a walkie-talkie. One party transmits, the other waits, and a referee decides when to flip the switch. Human conversation is not a walkie-talkie; it is a phone call, where both lines are open the whole time and people interrupt, hum agreement, and trail off. Full duplex means both directions carry audio at once. Our new lesson on full-duplex speech models walks through how that is built.
Where the latency actually was
The instructive part of OpenAI's account is that the fix was not a faster model. It was six or seven unglamorous systems problems.
Transport. A protocol effort OpenAI calls WARP piggybacks the security handshake onto connection setup, collapsing six round trips to one. A second piece, Instant Connect, negotiates session parameters in advance so the usual signaling exchange is off the critical path. Together, OpenAI writes, "the client can now start a session with a single UDP packet."
Frame delivery. The media path moved off Python's asyncio and onto Go. OpenAI reports the new system's 95th-percentile frame smoothness matches the old system's median. A late audio frame is not a slow request; it is an audible click.
Geography. Routing a session to distant capacity adds delay at several points, so model rollouts are now validated together with regional capacity and traffic steering, and latency is broken out by where the user is.
State. Long conversations eventually need context compaction, which normally destroys the key-value cache and forces an expensive refill. OpenAI instead warms and prefills a replacement instance alongside the live one and cuts over while the old instance is still talking. A long call becomes a state-migration problem, not a context-window problem.
The governing principle, in OpenAI's phrase: "the voice must flow."
Thinking beside the conversation, not inside it
The second architectural move is that the voice model owns timing and presence while heavier reasoning, tools, and search run asynchronously behind a boundary. OpenAI pre-creates and prefills the frontier-model session and keeps session affinity and prompt caching warm, so a slow search cannot freeze the microphone loop. It also concedes the limit: the voice model can hold the floor, but it cannot hide an arbitrarily slow delegate.
There is a quieter detail that says a lot about running this in a real product. The live model is continuous, but ChatGPT still needs discrete messages for its interface, its analytics, and its safety systems. So the application server reconstructs turns after the fact, maintaining "a speculative view of the current state and an authoritative record of what was said," finalizing speaker attribution only once it is reliable. Every segmentation policy, OpenAI notes, "trades freshness for certainty." A large part of the product is that reconciliation layer, invisible to the user.
Why it matters
The capacity question changes shape. It stops being requests per GPU and becomes concurrent sessions in which every audio frame arrives on time. That is closer to running a telephone network than to serving a chat endpoint, and it is a different kind of moat than model quality. Our earlier coverage of the GPT-Live launch covered the product; this is the plumbing underneath it.
The honest caveat: full duplex creates a new failure mode rather than removing all of them. The r/singularity discussion that followed the launch mixed enthusiasm with complaints that the backchannels, the interruptions, and the occasional clipped speech were distracting. A builder on r/LocalLLaMA went further and kept a turn-based cascade for a household assistant, because in a room with several people the hard question is not whether the model can listen while speaking. It is whether the person speaking is talking to it at all. In cars, kitchens, and call centers, push-to-talk may keep winning.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)