If you have ever compared "fastest AI API" benchmarks and found they disagree with each other, there is a good chance they were not measuring the same thing. Two numbers get used interchangeably, and they answer different questions:
- TTFB — time to first byte. DNS resolution, TCP connect, TLS handshake, first response byte. The network path and the provider's front door. No model is involved at all.
- TTFT — time to first token. A real streamed completion, timed until the first token appears. That is the same network time, plus queueing at the provider, plus the model's prefill over your prompt.
TTFB is the floor you pay on every single request no matter what you call. TTFT is what a human actually stares at before words show up on screen.
I run llmlatency.dev, which measures both continuously for ~45 inference providers from four regions (Germany, US Central, Tokyo, São Paulo). Here is what the data says about why these two numbers should never be merged into one leaderboard.
The same region, the same probes, two different winners
Measured from São Paulo over the last 24 hours:
| Metric | Winner | Number |
|---|---|---|
| Lowest TTFB (edge latency) | openrouter | 58 ms |
| Lowest TTFT (first token) | cerebras | 757 ms |
openrouter's front door answers 3.2× faster than cerebras's (58 ms vs 184 ms). And cerebras still streams its first token 264 ms sooner (757 ms vs 1021 ms).
Same probes, same schedule, same region, opposite conclusions. "Which API is fastest?" is not a question with one answer until you say which of the two you are paying for.
Most of TTFT is not the API you are benchmarking
Here is the split for every provider where I have both numbers side by side:
| Requested from | Provider | TTFT p50 | TTFB p50 | Network share of TTFT |
|---|---|---|---|---|
| São Paulo | groq | 764 ms | 233 ms | 31% |
| Germany | groq | 900 ms | 296 ms | 33% |
| São Paulo | cerebras | 757 ms | 184 ms | 24% |
| Germany | cerebras | 803 ms | 201 ms | 25% |
| Tokyo | cerebras | 1312 ms | 196 ms | 15% |
| São Paulo | 1143 ms | 159 ms | 14% | |
| Tokyo | groq | 1256 ms | 147 ms | 12% |
| US Central | groq | 1355 ms | 108 ms | 8% |
| Germany | openrouter | 1300 ms | 100 ms | 8% |
| Germany | 1361 ms | 101 ms | 7% | |
| São Paulo | openrouter | 1021 ms | 58 ms | 6% |
| US Central | cerebras | 1485 ms | 68 ms | 5% |
| Tokyo | 1483 ms | 57 ms | 4% | |
| Tokyo | openrouter | 1292 ms | 55 ms | 4% |
| US Central | openrouter | 1567 ms | 59 ms | 4% |
| US Central | 1524 ms | 42 ms | 3% |
Between 3% and 33% of time to first token is network time. The other 67–97% is the provider queueing your request and the model doing prefill.
That has an uncomfortable consequence for benchmark tables: if a TTFT comparison does not name the model behind every number, most of what it measured was the model, not the API. A 30B model will beat a 120B model on first-token time regardless of whose GPUs it runs on. Here are the models behind the numbers above, which is exactly why I do not publish them as a provider ranking:
| Provider | Model answering |
|---|---|
| cerebras | gpt-oss-120b |
| groq | llama-3.3-70b-versatile |
| openrouter | nvidia/nemotron-3-nano-30b-a3b:free |
gemini-flash-lite-latest |
Four different models. Ranking them against each other and calling it a provider ranking would be a category error wearing a table.
"Fast" is meaningless without "from where"
The regional spread is larger than the gap between most providers:
| Provider | Fastest region | Slowest region | Spread |
|---|---|---|---|
| sambanova | 21 ms (Tokyo) | 398 ms (Germany) | 18.9× |
| fireworks | 16 ms (Tokyo) | 251 ms (São Paulo) | 15.3× |
| upstage | 58 ms (Tokyo) | 596 ms (São Paulo) | 10.4× |
| aleph-alpha | 99 ms (Germany) | 568 ms (Tokyo) | 5.7× |
Identical API, identical request, an 18.9× difference depending on which continent you dialled from. A single-machine benchmark run from one US datacenter is not wrong so much as it is answering a question about that datacenter.
A checklist for reading anyone's latency benchmark
Including mine. If a benchmark cannot answer these, treat the number as a vibe:
- TTFB or TTFT? If it does not say, it is usually TTFB dressed up in TTFT language.
- Which model? No model name next to a TTFT figure means the figure is about the model.
- From where? One region is one data point, not a ranking.
- p50 or p95? Tail latency is where user-visible pain lives, and p95 orderings often differ from p50.
- When, and how often? A benchmark run once in March describes March.
- How many samples? My tables above sit on 57–58 samples per provider per region per day.
The honest limits of my own data
TTFT is measured for 4 of the 45 providers I track. Not because the other 41 are uninteresting, but because a streamed completion requires a paid API key for each provider and every probe spends tokens. Edge latency needs no key, so TTFB is published for all 45.
I would rather publish 4 measured TTFT figures than 45 estimated ones. An estimate presented as a measurement is the one thing a latency tracker must never do.
Everything is free, and machine-readable
The data is CC-BY-4.0. No key, no signup, CORS open:
# Rankings for every region, updated continuously
curl https://llmlatency.dev/api/rankings.json
# Any page as markdown, for agents and scripts
curl https://llmlatency.dev/time-to-first-token.md
There is also a remote MCP server if you want your assistant to look this up mid-conversation:
curl -X POST https://llmlatency.dev/mcp \
-H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' \
-d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}'
Full method, including what the probes do and do not prove: llmlatency.dev/methodology. The TTFT breakdown lives at llmlatency.dev/time-to-first-token, and the prober itself is open source at github.com/mazamaka/llm-latency-tracker.
If you have a provider you want added, or you think one of these numbers is wrong, tell me — measurements that survive being checked are the only kind worth publishing.
Top comments (1)
The TTFB versus TTFT split is the useful part. For agent UIs, I mostly care about route variance because a 200 ms regional penalty gets paid again on every model call in the chain. Publishing the raw probe endpoint is also the bit that makes the numbers worth arguing with.