Hacker News is buzzing today about a builder who achieved sub-500ms latency in a voice agent (Show HN: I built a sub-500ms latency voice agent from scratch). 409 points and climbing. Why does this matter so much to builders?
Because latency is the difference between a voice agent that feels like a person and one that feels like a broken phone tree.
The Amazon Alexa engineer insight
One of the top comments came from someone who worked on Alexa (and holds patents in this space):
"The median delay between human speakers during a conversation is 0ms (zero). In many cases, the listener starts speaking before the speaker is done."
Our brains predict what the other person will say and start forming a response in parallel. It is why we "finish each other sentences." When that prediction breaks — like on a lagging call — you get the awkward "no, you go ahead" dance.
Voice assistants have trained us to expect delay. But that expectation is eroding fast.
What this means for AI receptionists
If you are using an AI receptionist to handle inbound calls for a dental clinic, law firm, or real estate office, latency is not a technical metric — it is the thing that determines whether callers hang up.
Here is what real-world deployments show:
- < 500ms — Natural conversation, caller does not notice the AI
- 500ms – 1.5s — Slight hesitation, still acceptable
- 1.5s – 3s — "Is this broken?" caller attention drops
- > 3s — Hang up
Most legacy IVR systems (press 1 for billing, press 2 for...) run at 2-4 second response times. Callers hate them. They have been trained to expect bad experiences from automated phone systems.
A sub-500ms AI voice agent breaks that expectation in the best way. Callers stop thinking "I am talking to a robot" and start just... talking.
The technical unlock: semantic end-of-turn detection
The Alexa engineer also dropped this:
"Semantic end-of-turn is the key here. It is something we were working on years ago, but did not have the compute power to do it. So at least back then, end-of-turn was just 300ms of silence."
Legacy voice systems detect end-of-turn by silence. 300ms of quiet = your turn. This is why they constantly interrupt or wait too long.
Modern systems use semantic detection — understanding that "I need to schedule an appointment for... next Tuesday" is one thought, even with pauses. This is what makes AI receptionists feel genuinely conversational.
What to actually look for
If you are evaluating AI receptionist vendors or building your own, here is the latency checklist:
- STT (Speech-to-Text): Must be streaming, not batch. Deepgram Nova-2 is the current benchmark.
- LLM inference: Fast model (GPT-4o mini, Claude Haiku) with low time-to-first-token.
- TTS (Text-to-Speech): Must stream audio output. ElevenLabs Turbo v2 hits ~200ms.
- Semantic end-of-turn: Do not use pure silence detection. You will interrupt callers constantly.
- Regional deployment: Match your compute region to your callers.
Stack those right and sub-500ms is achievable today, without custom hardware.
The business case
For a dental clinic that misses 30% of inbound calls (industry average):
- 100 calls/month missed → 40 new appointments booked (40% conversion)
- Average appointment value: $150
- Monthly revenue recovered: $6,000
- Cost of AI receptionist: $299/month
The ROI math is not subtle.
The technology is mature. The latency problem is solved. The only question is whether you implement it before your competitor does.
We build AI receptionists for local businesses at RooxAI. Live demo on that page — hear what sub-500ms actually sounds like on a real call.
Top comments (0)