DEV Community: Max Bob

TTFT is not TTFB: what 45 AI APIs measured from 4 regions actually show

Max Bob — Sun, 16 Aug 2026 22:04:40 +0000

If you have ever compared "fastest AI API" benchmarks and found they disagree with each other, there is a good chance they were not measuring the same thing. Two numbers get used interchangeably, and they answer different questions:

TTFB — time to first byte. DNS resolution, TCP connect, TLS handshake, first response byte. The network path and the provider's front door. No model is involved at all.
TTFT — time to first token. A real streamed completion, timed until the first token appears. That is the same network time, plus queueing at the provider, plus the model's prefill over your prompt.

TTFB is the floor you pay on every single request no matter what you call. TTFT is what a human actually stares at before words show up on screen.

I run llmlatency.dev, which measures both continuously for ~45 inference providers from four regions (Germany, US Central, Tokyo, São Paulo). Here is what the data says about why these two numbers should never be merged into one leaderboard.

The same region, the same probes, two different winners

Measured from São Paulo over the last 24 hours:

Metric	Winner	Number
Lowest TTFB (edge latency)	openrouter	58 ms
Lowest TTFT (first token)	cerebras	757 ms

openrouter's front door answers 3.2× faster than cerebras's (58 ms vs 184 ms). And cerebras still streams its first token 264 ms sooner (757 ms vs 1021 ms).

Same probes, same schedule, same region, opposite conclusions. "Which API is fastest?" is not a question with one answer until you say which of the two you are paying for.

Most of TTFT is not the API you are benchmarking

Here is the split for every provider where I have both numbers side by side:

Requested from	Provider	TTFT p50	TTFB p50	Network share of TTFT
São Paulo	groq	764 ms	233 ms	31%
Germany	groq	900 ms	296 ms	33%
São Paulo	cerebras	757 ms	184 ms	24%
Germany	cerebras	803 ms	201 ms	25%
Tokyo	cerebras	1312 ms	196 ms	15%
São Paulo	google	1143 ms	159 ms	14%
Tokyo	groq	1256 ms	147 ms	12%
US Central	groq	1355 ms	108 ms	8%
Germany	openrouter	1300 ms	100 ms	8%
Germany	google	1361 ms	101 ms	7%
São Paulo	openrouter	1021 ms	58 ms	6%
US Central	cerebras	1485 ms	68 ms	5%
Tokyo	google	1483 ms	57 ms	4%
Tokyo	openrouter	1292 ms	55 ms	4%
US Central	openrouter	1567 ms	59 ms	4%
US Central	google	1524 ms	42 ms	3%

Between 3% and 33% of time to first token is network time. The other 67–97% is the provider queueing your request and the model doing prefill.

That has an uncomfortable consequence for benchmark tables: if a TTFT comparison does not name the model behind every number, most of what it measured was the model, not the API. A 30B model will beat a 120B model on first-token time regardless of whose GPUs it runs on. Here are the models behind the numbers above, which is exactly why I do not publish them as a provider ranking:

Provider	Model answering
cerebras	`gpt-oss-120b`
groq	`llama-3.3-70b-versatile`
openrouter	`nvidia/nemotron-3-nano-30b-a3b:free`
google	`gemini-flash-lite-latest`

Four different models. Ranking them against each other and calling it a provider ranking would be a category error wearing a table.

"Fast" is meaningless without "from where"

The regional spread is larger than the gap between most providers:

Provider	Fastest region	Slowest region	Spread
sambanova	21 ms (Tokyo)	398 ms (Germany)	18.9×
fireworks	16 ms (Tokyo)	251 ms (São Paulo)	15.3×
upstage	58 ms (Tokyo)	596 ms (São Paulo)	10.4×
aleph-alpha	99 ms (Germany)	568 ms (Tokyo)	5.7×

Identical API, identical request, an 18.9× difference depending on which continent you dialled from. A single-machine benchmark run from one US datacenter is not wrong so much as it is answering a question about that datacenter.

A checklist for reading anyone's latency benchmark

Including mine. If a benchmark cannot answer these, treat the number as a vibe:

TTFB or TTFT? If it does not say, it is usually TTFB dressed up in TTFT language.
Which model? No model name next to a TTFT figure means the figure is about the model.
From where? One region is one data point, not a ranking.
p50 or p95? Tail latency is where user-visible pain lives, and p95 orderings often differ from p50.
When, and how often? A benchmark run once in March describes March.
How many samples? My tables above sit on 57–58 samples per provider per region per day.

The honest limits of my own data

TTFT is measured for 4 of the 45 providers I track. Not because the other 41 are uninteresting, but because a streamed completion requires a paid API key for each provider and every probe spends tokens. Edge latency needs no key, so TTFB is published for all 45.

I would rather publish 4 measured TTFT figures than 45 estimated ones. An estimate presented as a measurement is the one thing a latency tracker must never do.

Everything is free, and machine-readable

The data is CC-BY-4.0. No key, no signup, CORS open:

# Rankings for every region, updated continuously
curl https://llmlatency.dev/api/rankings.json

# Any page as markdown, for agents and scripts
curl https://llmlatency.dev/time-to-first-token.md

There is also a remote MCP server if you want your assistant to look this up mid-conversation:

curl -X POST https://llmlatency.dev/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}'

Full method, including what the probes do and do not prove: llmlatency.dev/methodology. The TTFT breakdown lives at llmlatency.dev/time-to-first-token, and the prober itself is open source at github.com/mazamaka/llm-latency-tracker.

If you have a provider you want added, or you think one of these numbers is wrong, tell me — measurements that survive being checked are the only kind worth publishing.

There was no independent, measured view of AI-API latency by region — so I built one

Max Bob — Thu, 23 Jul 2026 18:19:48 +0000

If you build anything on top of hosted AI APIs, latency isn't a detail you get to ignore — it's a feature. A sluggish time-to-first-token is the difference between an assistant that feels alive and one that feels broken.

Yet when I went looking for an honest answer to a simple question — how fast is provider X from where my users actually are? — I couldn't find one. The numbers people quote tend to come from a single machine in a single region (usually somewhere in the US), from vendor-reported status pages, or from a benchmark that got run once and never refreshed. There was no independent, measured, regional view.

So I built one: LLM Latency Tracker, a provider-neutral tracker of latency and uptime for AI inference APIs.

How it works

The core idea is boring on purpose: actually measure, don't scrape.

A small Python prober — standard library only, no API key needed for the edge probes — opens real connections to each provider's endpoint and times every phase of the handshake: DNS resolution → TCP connect → TLS negotiation → time-to-first-byte (TTFB). That's the edge view: how long the network path itself takes before a single byte comes back.

Separately, where possible, it measures inference time-to-first-token (TTFT) — the thing your users actually feel, i.e. how long after you hit "send" the model starts streaming.

Those two numbers answer different questions, and keeping them apart matters. Edge latency is about the network and the front door; TTFT is about the model and the queue behind it.

The probes run from four regions — Europe (Germany), US Central, Asia (Tokyo), and South America (São Paulo) — because "fast" is meaningless without "from where." Results land in a SQLite time-series; a static-site generator turns that into the pages you see, hosted on Cloudflare Pages, and the whole thing is self-updating on a schedule. There's no always-on backend to rot or page me at 3am.

It currently covers ~45 providers — the usual Western labs (OpenAI, Anthropic, Google, Mistral, Groq, Together, Fireworks, Cerebras, OpenRouter, Perplexity, xAI, DeepSeek…) plus Chinese ones (GLM/Zhipu, Kimi/Moonshot, Qwen, MiniMax) that rarely show up in Western benchmarks. All of it is free and open under CC-BY-4.0, with a JSON API and an OpenAPI spec.

The interesting part: building it agent-native

Here's what makes this more than another dashboard. I built it to be read by machines, not just humans.

The way people ask questions is shifting from search boxes to answer engines and agents. If your data only exists as HTML wrapped in JavaScript, an agent can't reliably consume or cite it — which means your measurements never reach the place where someone actually asks "which provider has the lowest TTFT in Europe right now?"

So the site speaks several agent dialects:

A real MCP server (Streamable HTTP) at /mcp, exposing a get_ai_api_latency tool. An agent can call it directly and get structured latency data back — no scraping, no guessing.
An MCP Server Card at /.well-known/mcp/server-card.json, plus WebMCP, so agents can discover the tool.
llms.txt and llms-full.txt, giving models a clean map of the site.
Markdown content negotiation: ask a page for text/markdown and you get tidy Markdown instead of a DOM you have to reverse-engineer.

The nice side effect is that building for agents forces you to build clean for everyone. The same discipline that makes a page legible to an LLM — structured, fast, no junk — is what earned a Lighthouse 100 and Cloudflare's "Agent-Native" (Level 5) rating on their "Is Your Site Agent-Ready?" check. I won't pretend that means much on its own; it's a checklist, not a trophy. But it's a decent proxy for "this data is actually reachable."

There's also a small companion: a model deprecation & migration calendar at /deprecations, hand-verified from official provider docs, so you can spot which models are being sunset before they break your app.

Honest limitations

A tracker that oversells itself is worse than none, so — plainly:

The dataset is young. It's only days old as I write this. Treat trends as provisional; don't over-read a single snapshot.
Edge latency is not a full round-trip. TTFB to an endpoint measures the network path, not the time to a finished generation. TTFT is closer to lived experience, but still isn't total completion time.
Keyless network probes and keyed inference probes see different things. Inference coverage is narrower than edge coverage.
Four regions is a sample, not the planet. If your users sit somewhere I don't probe, your mileage will differ.

It's a baseline and a starting point, not gospel.

Try it / break it

It's free, open-source, and the data is CC-BY-4.0 — use it, chart it, cite it, build on it.

Site: https://llmlatency.dev
JSON API: https://llmlatency.dev/api/rankings.json
MCP endpoint: https://llmlatency.dev/mcp
Code: https://github.com/mazamaka/llm-latency-tracker

If a provider or region you care about is missing, open an issue or a PR — that's exactly the kind of contribution that makes this more useful for everyone. And if you're building agents, point one at the MCP endpoint and tell me what breaks.

Update, August 2026. The single most common confusion in the replies to this post was TTFB vs TTFT, so I wrote up the measured difference — including a region where the TTFB winner and the TTFT winner are different providers: TTFT is not TTFB: what 45 AI APIs measured from 4 regions actually show.