My 2026 voice agent hit a 280ms voice-to-first-byte in the demo. Alexa+ was slower on the same phrase. I felt good about that number for exactly one day.
Then I ran the same phrase past my wife, who has been shouting at Google Assistant for eight years, and she cut me off mid-response the way people cut off Google. My agent kept talking. She said "no, wait, stop." My agent kept talking. Alexa would have stopped. The 280ms was a lie.
The next week I read the 2020 NN/g study of Alexa, Siri, and Google Assistant end to end. Then I re-ran the parts that still apply in 2026, because Amazon, Apple, and Google have all rebuilt these agents on LLMs, and every one of them slipped their 2025 launch dates. I stole three UX tricks from the losers. This is what they are and where each one went into my stack.
Where the three legacy stacks are in 2026
Before the tricks, one paragraph on the current state so we are all looking at the same map. Amazon launched Alexa+ nationwide in the US on October 13, 2025, months later than promised, after insiders reported that the AI was "too slow to respond to questions". Apple pushed the LLM-powered Siri from 2025 into 2026, citing security and integration issues. Google extended its Assistant-to-Gemini migration into March 2026 after users called the new Home experience "painfully slow", and developers reported that gemini-2.5-flash-native-audio-preview lands at 1400-1800ms in real deployments.
All three shipped late. All three shipped slow. And all three still do three things my faster agent got wrong.
Trick 1: Alexa's pause detection is the barge-in you cannot skip
Alexa Plus's most talked-about single feature is that it can tell the difference between a user pausing to think and a user having finished. That is not a small feature. It is the difference between an assistant that lets you talk and an assistant that talks over you.
My agent had barge-in on paper. What it did not have was pause-vs-finish classification. If the user paused for 400ms mid-sentence, my VAD flipped to "user done" and the LLM started responding. If they then resumed, we had two audio streams fighting.
Alexa+ solves this with a longer pause window plus a partial-intent check: is what the user said so far a plausibly-complete request? "Set a timer for" is not complete. "Set a timer for five minutes" is. The pause tolerance stretches or shrinks based on that check.
I copied this in the crudest possible way. My VAD's end-of-utterance timeout is now a function of two signals: the raw silence duration, and a fast partial-transcript classifier that answers "does this look complete?" in ~15ms. If the transcript ends in a preposition, a conjunction, or a number-that-usually-precedes-a-unit, the timeout stretches from 400ms to 900ms. If it ends in a period-like intonation, the timeout stays at 400ms.
The 300ms voice-to-first-byte number did not change. What changed is that the agent no longer starts talking over people mid-sentence, which was making the 300ms feel like 3 seconds of arguing.
Trick 2: Siri's short answers are not a limitation
ACM CHI 2022 published a study on voice assistant response length and user satisfaction. The finding was flat: shorter answers score higher. Not by a little. Reliably, across tasks, across users.
Siri has always been the shortest of the three. That is often held against it (Siri "can't do anything"), but the CHI data says it is the most survivable design choice of the group. Voice is a one-way medium. You cannot skim it. A long correct answer is a long wait for the correct answer.
My agent, on the other hand, was returning LLM prose. "The weather today in Tokyo is a mostly sunny day with a high of 28 degrees Celsius and a low of 21, with light winds from the northeast." Correct. Also 22 words. Also, by second 4, the user has stopped listening.
I now cap the first spoken response at 12 words. The full answer is still generated; the model just knows to structure the first sentence as the punchy version and put the details in a second sentence the TTS only speaks if the user does not barge in within 800ms. "Sunny, 28. Details?" is one Alexa-length answer plus one optional Google-length follow-up. The completion rate went from wherever it was to somewhere I actually want to look at every week.
The trick is not "make answers short." The trick is: the first sentence is the whole answer, and the second sentence is an offer to continue. Users treat that as respectful. They treat 22-word paragraphs as an assistant that did not listen.
Trick 3: Google Assistant's stability was a UX moat, and Gemini forgot it
Here is the pattern nobody names: for a decade, Google Assistant did the same thing the same way. Ask for a timer, get a timer. Ask twice, get the same behavior twice. It was boring. It was also predictable, and predictable is what makes a voice interface usable at all, because you cannot see what the machine is going to do.
Google Home users on the Gemini rollout are complaining about slowness, but the deeper complaint underneath is variance. Sometimes Gemini answers in one way. Sometimes another. Sometimes the same request routes to a completely different capability. This is inherent to LLMs and it is not free.
My agent had this problem in miniature. I was letting temperature drift at 0.7 for "natural feel," which meant the same command produced different lengths, different phrasings, and once, memorably, a different language. I dropped temperature to 0.2 for the "assistant-shaped" commands (timer, list, control, status) and left it at 0.7 for the "conversation-shaped" ones (chat, question-answer). The variance dropped to something a user could learn.
The trick, restated: LLMs make everything conversational-feeling by default. Users want conversation-feel from the parts that are conversation and command-feel from the parts that are commands. Route accordingly.
The full comparison, in one table
| Signal | Alexa+ (2025) | Siri (delayed to 2026) | Google Assistant → Gemini | My 2026 agent (before → after) |
|---|---|---|---|---|
| Voice-to-first-byte | Slow enough that launch was held | On-device target; not shipped | 1400-1800ms on Live API | 280ms |
| Barge-in / pause detection | Best-in-class pause vs. finish | N/A (legacy) | Standard cut-off VAD | Fixed 400ms VAD → adaptive 400-900ms |
| Default answer length | ~1 sentence | Shortest of the three | Multi-sentence, drifting | 22 words → 12-word first sentence + optional follow-up |
| Command variance | Low (rule-augmented) | Very low (rule-based) | Rising after Gemini switch | 0.7 temperature → 0.2 for command-shaped |
| The user-visible bug my agent had | Interrupting mid-sentence | Talking too long | Different phrasing every time | All three |
The row that hurts to look at is the last one. My "faster" agent was making all three of the mistakes the legacy assistants had already fixed, and my speed advantage was buying me exactly zero of them back.
Where the 2025 delays came from, and why they matter to a small stack
Amazon, Apple, and Google all missed. Amazon shipped Alexa+ but 18 months after announcement and only after adding custom silicon (AZ3 Pro) to Echo devices to close the latency gap. Apple pushed the personalized Siri work into 2026, reportedly because the LLM-based agent architecture opens prompt-injection attack surface that the legacy command architecture did not. Google is running two Assistant stacks in parallel through March 2026 rather than force the swap.
The lesson for a small voice agent stack is not "you can beat these companies." You cannot. The lesson is: the parts of the UX these companies were about to change are the parts that took them 18 more months than they planned. Barge-in nuance, answer-length calibration, and variance control are the tail. You will not get them right by accident either.
The 500-word summary I owed the reader from paragraph one
Three tricks, restated flat:
- Barge-in is not a boolean. It is a stretch/shrink of the end-of-utterance timeout based on whether the partial transcript looks complete. Steal this from Alexa.
- First-sentence-is-the-whole-answer. The rest is an opt-in follow-up the TTS only speaks if the user does not interrupt. Steal this from Siri.
- Route command-shaped requests through low-temperature deterministic prompts and only let the chat-shaped ones drift. Steal this from Google Assistant's pre-Gemini stability.
None of them cost latency. All three are things my 280ms agent needed to stop bragging about the 280ms and start acting like it earned the number.
I wrote the longer version of the 300ms latency budget in this book, covering the barge-in state machine, the length-cap prompt patterns, and the Pipecat vs LiveKit trade-offs for each: Voice AI: The Sub-300ms UX Playbook. The Alexa+ / Siri / Gemini section is Chapter 10, expanded from the version I ran here.
Top comments (0)