Why "time to first byte" lies about voice latency (and how to get a lower number)

Voice API pages love one number: how fast the first audio arrives. The trouble is that "first audio" can mean at least four different things, and they differ by hundreds of milliseconds. Here is what we found when we measured our own speech API properly, including one speed claim we had to take back.

Disclosure: I run Speakvora, a speech API. Every number below is from our own tests, with dates, and I say where they are not comparable to anyone else's.

1. The first byte is not the first sound

A streaming text-to-speech response can begin with audio that is silent. The first chunk arrives quickly, your timer stops, and the caller still hears nothing for another 100 to 400 ms. What a person experiences is the arrival of the first chunk plus the silence before the first audible sample.

That is also how Coval's published TTS benchmark defines it: first chunk arrival plus leading silence, where "audible" is the first 10 ms window with RMS above 0.01 on a -1 to 1 scale (their method). We implemented that description ourselves; we did not run Coval's own harness. On 30 October 2026 we measured our Phonon voices that way, on a test server of ours over plain HTTP, next to Amazon Polly from the same client in the same run:

Voice	Time to first sound (median)	Of which: chunk arrival + leading silence
Phonon, before we trimmed the opening silence	148 ms	25 ms + 123 ms
Phonon, after the trim	70 ms	57 ms + 13 ms
Amazon Polly Neural	143 ms	37 ms + 107 ms
Amazon Polly Standard	42 ms	31 ms + 9 ms

Our voice produced its first chunk in 25 ms and then spent 123 ms saying nothing. The early "fast" figure was true as a first-byte time and misleading as a description of what a caller hears. Other engines show the same gap. A stock Pocket TTS voice (Kyutai's model, on a CPU) delivered its first chunk in 125 to 173 ms, then added 210 to 240 ms of leading silence. A local Qwen3-TTS model on my Mac sent its first chunk at about 100 ms and became audible at about 210 ms.

When you compare vendors, ask which of the two numbers they are quoting.

2. Where you measure changes the answer

The same Phonon voice, on a test server of ours, measured in October 2026 (these are first-byte times, taken before we started counting the silence):

• On the server itself (loopback, no network): 24 to 28 ms for a short line.
• From my laptop over the internet, with a 16.5 ms round trip: 57 to 62 ms.
• The same laptop with a kept-alive connection and the text split at the first clause: 42 to 45 ms.
• From a cloud VM in the same region: 25 to 27 ms.

A number measured "on the box" tells you about the model. Your app lives somewhere else, so measure there.

3. Setting up a connection is part of your latency

On 5 November 2026 we measured time to first audio bytes for a Phonon voice through our API, five runs, medians, same voice and server:

• 43 ms when the connection and the stream session were already open.
• 101 ms when a new session was requested for every line over a reused connection.
• 207 ms (range 152 to 310) for a brand-new visitor with new connections.

That is nearly a five-fold spread from setup alone, with no change to the model.

4. Fast and wrong is not fast

One more lesson. We once compiled a speech model for speed and its first-byte time dropped into the 40 to 47 ms range. Then we ran an accuracy check: Whisper transcribed the compiled model's output with a word error rate of 0.27, against 0.023 for the uncompiled model. It was fast and it was wrong. We withdrew every speed figure from that path and corrected our pages. Always listen to (or transcribe) the audio, not just read the clock.

Measure it yourself

This function returns milliseconds from just before your request until the first audible sound. It takes any iterator of raw 16-bit mono PCM chunks:

import time
import numpy as np


def first_audible_ms(chunks, started, sr=24000, thresh=0.01):
    """ms from `started` (time.perf_counter() just before the request) to the first audible sound.
    Audible = a 10 ms window (1 ms step) with RMS above `thresh` on a -1..1 scale."""
    win, step = sr // 100, sr // 1000
    pcm, rest, pos, t_first = np.zeros(0, dtype=np.float32), b"", 0, None
    for chunk in chunks:
        if t_first is None:
            t_first = time.perf_counter()                       # arrival of the first chunk
        data = rest + chunk
        rest = data[len(data) // 2 * 2:]                        # keep a split sample for next time
        pcm = np.concatenate([pcm, np.frombuffer(data[:len(data) // 2 * 2], dtype="<i2").astype(np.float32) / 32768.0])
        while pos + win <= len(pcm):
            w = pcm[pos:pos + win]
            if np.sqrt(np.mean((w - w.mean()) ** 2)) > thresh:
                return (t_first - started) * 1000 + pos / sr * 1000     # arrival + the silence before the sound
            pos += step
    return None

Use it with any streaming HTTP client, for example:

import requests
t0 = time.perf_counter()
r = requests.post(STREAM_URL, headers={"authorization": f"Bearer {TOKEN}"}, json=BODY, stream=True)
print(first_audible_ms(r.iter_content(4096), t0))

Run it 20 times, from where your app runs, and look at the median and the 95th percentile, not the best run. (I tested the function against synthetic audio with a known 150 ms of silence; it reported 176 ms for a 30 ms arrival, within a few milliseconds of the expected 180, because the window sees the sound a little before its midpoint.)

How to get a lower number in your app

1. Measure first audible sound, from your own machine or region. Use the function above, not a vendor's headline.
2. Run your calling code near the API. Our servers are in AWS us-east-1. A 16.5 ms round trip is paid more than once on a cold request: for the TCP connection, the TLS handshake and the request itself.
3. Reuse connections. Keep one pooled HTTP client alive instead of creating a client per request. In our numbers above, the case with brand-new connections was the slowest.
4. Ask for the stream ahead of time. Our streaming flow is two calls: ask for a stream (you get a one-use URL and token), then open it. The first call adds roughly 100 to 150 ms, once, so make it before you need the audio: when a field gets focus, on hover, or when the user starts typing.
5. Take out hops. Every proxy between the audio and the listener adds its own connection and its own delay. Where you can, give the browser the one-use stream URL and token so the audio goes straight to it, instead of relaying audio through your server. Keep your API key on your server; never put it in a page.
6. Start playing at the first chunk. A player that waits for a full second of audio, or a whole sentence, throws away everything above. Check the buffer settings in your audio element or worklet.
7. Look at the leading silence. Two voices with the same first-byte time can differ by 100 ms or more of dead air before the sound. Compare voices by first audible sound.
8. Pre-generate lines that never change. Greetings, confirmations and menu prompts can be generated once and played from your own storage, with no generation delay at all.
9. Warm up. The first request after a quiet period can be slow. On one path we saw a 95th-percentile cold tail of 4.8 seconds, so send a tiny request when your app starts rather than letting your first real caller pay for it.

What these numbers are not

They are our own tests, in our own setup, on specific days. They are not a third-party benchmark and not a ranking, and your network will give you different numbers, which is exactly the point. If you try the function and get a result that disagrees with ours, I would like to hear about it.

More detail, with the method: Speakvora's latency page and the streaming docs.

Originally published at https://speakvora.hashnode.dev/why-time-to-first-byte-lies-about-voice-latency-and-how-to-get-a-lower-number