Short answer. No single AI inference API is fastest everywhere. Over 30 days (27 September – 26 October 2026), Fireworks AI had the closest API front door in US Central (28 ms median) and Tokyo (19 ms), with SambaNova a close second in Tokyo (25 ms). In São Paulo, Fireworks was 252 ms away and OpenRouter led (61 ms). For time to first token, Groq (openai/gpt-oss-120b) delivered its first token in a median of 0.8 s from São Paulo, 1.4 s from US Central and 1.5 s from Tokyo. Google's gemini-flash-lite-latest took 1.2–1.7 s, with a p95 of 5.6–6.1 s.
The useful part is not the winner. It's how much the answer changes with where your client runs. SambaNova's API front door is 25 ms away from Tokyo and 371 ms away from São Paulo, a 15× difference for the same API.
What was measured
- Network latency, all 46 providers. Every 5 minutes from US Central, Tokyo and São Paulo, a probe opens an HTTPS connection to each provider's official API host and times one request: from sending it to receiving the first byte of the reply (TTFB). That is one round trip to the provider's front door plus its edge processing. No API key is used, so the reply is usually a 401. That still measures the path, but it says nothing about the GPUs behind the door. 1,190,063 successful probes went into this post.
-
Time to first token (TTFT), where we hold a working key. The same probe nodes stream a real completion: a one-word prompt with
max_tokens: 1. TTFT is the time from opening a fresh HTTPS connection to the first non-empty chunk of the streamed reply. In this window that ran continuously for Groq (openai/gpt-oss-120b) and Google (gemini-flash-lite-latest): 12,992 requests. - Not measured here: TTFT for Cerebras and SambaNova. We don't have a working paid key for either in this window, so you only get their network numbers below. Throughput (tokens per second) isn't measured either. Vendor tokens-per-second claims are a different quantity and can't be compared with these figures.
- Germany is excluded. We also probe from Germany, but while preparing this post I found that container was CPU-throttled. That pushed its timings up in steps of 100 ms: Google's API showed 99 ms median in the stored data against 29 ms when measured from the same host without the limit. It's fixed as of 27 October. German numbers come back once there are 30 clean days.
Every request goes straight to the provider's own API, never through a gateway.
Time to first token: Groq vs Gemini Flash-Lite
| Requested from | Provider · model | TTFT p50 | TTFT p95 | Successful | Network TTFB p50 |
|---|---|---|---|---|---|
| São Paulo | Groq · gpt-oss-120b | 794 ms | 1,338 ms | 100.0% | 234 ms |
| US Central | Groq · gpt-oss-120b | 1,379 ms | 1,906 ms | 99.9% | 121 ms |
| Tokyo | Groq · gpt-oss-120b | 1,497 ms | 2,040 ms | 99.8% | 164 ms |
| São Paulo | Google · gemini-flash-lite-latest | 1,204 ms | 5,602 ms | 89.7% | 155 ms |
| US Central | Google · gemini-flash-lite-latest | 1,579 ms | 5,656 ms | 91.1% | 47 ms |
| Tokyo | Google · gemini-flash-lite-latest | 1,682 ms | 6,103 ms | 91.6% | 64 ms |
What this shows:
- The network is the small part. Groq's front door is 121–234 ms away. Its first token arrives 560–1,330 ms after that. The gap is connection setup plus time spent inside the provider (queueing and prefill), and it is the larger part in all three regions.
- The closest region is not the fastest one. Groq's front door is farthest from São Paulo (234 ms), yet São Paulo got the fastest first token, about 600 ms faster than US Central. The pattern held in both halves of the month. We can see that it happens, not why. A likely reason is that requests from different regions are served by different Groq capacity, but we can't confirm that from outside. The practical point: measure from where your users are, because network distance doesn't predict TTFT.
- The tail is where Flash-Lite loses. Gemini Flash-Lite's median is within a few hundred milliseconds of Groq's, but its p95 is three to four times higher (5.6–6.1 s). 8–10% of requests timed out or returned a 5xx. Our Gemini key is on the free tier, so part of that may be free-tier capacity rather than what a paid account sees.
- These are two different models. Groq serves a 120B open-weights model here and Google serves its smallest Gemini. Read this as "what these two endpoints did", not as a ranking of the providers.
Network latency to 20 inference APIs, by region (median TTFB)
| Provider | US Central | Tokyo | São Paulo | Slowest ÷ fastest region |
|---|---|---|---|---|
| Fireworks AI | 28 ms | 19 ms | 252 ms | 13.2× |
| OpenRouter ¹ | 63 ms | 56 ms | 61 ms | 1.1× |
| Google (Gemini) | 47 ms | 64 ms | 155 ms | 3.3× |
| Baseten | 47 ms | 157 ms | 152 ms | 3.3× |
| Novita | 83 ms | 153 ms | 247 ms | 3.0× |
| Groq | 121 ms | 164 ms | 234 ms | 1.9× |
| SambaNova | 167 ms | 25 ms | 371 ms | 14.9× |
| xAI | 111 ms | 171 ms | 229 ms | 2.1× |
| Together AI | 120 ms | 180 ms | 268 ms | 2.2× |
| Cerebras | 71 ms | 201 ms | 189 ms | 2.8× |
| Anthropic | 94 ms | 234 ms | 199 ms | 2.5× |
| OpenAI | 139 ms | 214 ms | 257 ms | 1.9× |
| Inference.net | 122 ms | 229 ms | 225 ms | 1.9× |
| Mistral | 189 ms | 317 ms | 254 ms | 1.7× |
| Friendli | 181 ms | 284 ms | 285 ms | 1.6× |
| DeepSeek | 315 ms | 181 ms | 437 ms | 2.4× |
| Nebius | 178 ms | 477 ms | 366 ms | 2.7× |
| DeepInfra | 162 ms | 449 ms | 536 ms | 3.3× |
| Hyperbolic | 149 ms | 477 ms | 837 ms | 5.6× |
| Featherless | 354 ms | 488 ms | 737 ms | 2.1× |
¹ OpenRouter is a gateway: its front door is close in every region, and then it forwards your request to an upstream provider. A low number here doesn't make the model behind it fast.
Full table for all 46 providers: llmlatency.dev/fastest-ai-api (rebuilt every 30 minutes from the last 24 hours).
What stands out:
- Cerebras is close to US Central (71 ms) but 189–201 ms away from Tokyo and São Paulo.
- Groq was the most even of the three fast-inference specialists: 121–234 ms, a 1.9× spread.
- SambaNova is the extreme case: 25 ms from Tokyo, 167 ms from US Central, 371 ms from São Paulo.
- Fireworks AI is closest in two regions (19 ms Tokyo, 28 ms US Central) and 252 ms away from São Paulo.
If you're choosing for latency
- Measure from the region where your servers run, not from your laptop. In this data, provider order changed in every region.
- Reuse connections. TTFT above includes opening a fresh HTTPS connection every time, and a warm connection skips that.
- Look at p95 as well as p50. A provider that is 300 ms slower at the median but has a tight tail can feel faster in a product.
- Re-check monthly. Front doors move: providers add regions and change CDNs.
Data
- Live, last 24 h: fastest AI API by region · lowest time to first token · per provider: Groq, Cerebras, SambaNova
- How probes work: methodology
- Daily aggregates under CC BY 4.0, with a DOI: doi.org/10.5281/zenodo.21954788 · from Python:
pip install llmlatency - Related: AI API uptime comparison, last 30 days
Disclosure: I run llmlatency.dev, the independent tracker these numbers come from. No provider pays for placement. If you measure something different from your region, post it in the comments with the UTC time and I'll check it against the raw probes.
Top comments (2)
Comparing Groq at 120B with the smallest Gemini hides a model size gap, so the Flash-Lite p95 of 5.6 to 6.1 seconds is not a clean provider comparison. A same size run would tell more about the front door itself.
Agents wait on each call, so one slow response stalls the whole run, and a median hides that. Did you look at p95 per call, or just the median?