DEV Community: Max Bob

There is no fastest AI API: 100 days of probing 44 providers from 4 regions

Max Bob — Sat, 07 Nov 2026 21:13:13 +0000

Since July I've been running probes in four places (Germany, US Central, Tokyo and São Paulo) that connect to the official API host of every AI inference provider I track, every five minutes. That's 44 providers today and 4.8 million measurements in the current 90-day window. Here is what the data says as of November 7, 2026.

Which AI API is the fastest?

It depends on where you call it from. Fastest endpoint per region, median time to first byte over the last 24 hours:

Region	Fastest API	p50 TTFB
Germany	Fireworks	16 ms
Tokyo	Fireworks	19 ms
US Central	Fireworks	30 ms
São Paulo	OpenRouter	62 ms

The same endpoint can be ten times slower from another continent:

Provider	Fastest from	Slowest from	Spread
Fireworks	16 ms, Germany	254 ms, São Paulo	16.1×
Nebius	33 ms, Germany	466 ms, Tokyo	14.2×
SambaNova	30 ms, Tokyo	371 ms, São Paulo	12.3×
Upstage	58 ms, Tokyo	612 ms, São Paulo	10.5×
GLM	94 ms, Tokyo	542 ms, São Paulo	5.7×

TTFB here is the network floor: a request over an already open TLS connection until the first byte comes back. It says nothing about model speed or quality.

How much of time to first token is just the network?

Between 2% and 78%, depending on the model and the region. That comes from real streamed completions for the providers where I hold a key. Groq serving gpt-oss-120b has a median TTFT of 354 ms from Germany and 1,371 ms from Tokyo. Same model, same provider, the difference is distance. For a small fast model, picking a host on the right continent matters more than picking the model.

Which AI providers are the most reliable?

Over the last 24 hours, 36 of 44 providers answered every probe (50,006 probes). Widen the window and the picture changes: 26 of 44 dropped at least one probe in the last 7 days. Availability quoted without a window length doesn't compare to anything.

Observed availability over the 90-day window, rounded to 0.1% (what an outside client saw, not an SLA):

Provider	Observed, 90 days
Anthropic	100%
Google	100%
Groq	100%
Fireworks	100%
OpenRouter	100%
Cerebras	99.9%
SambaNova	99.4%
Doubao	99.3%
OpenAI	99.2%

An outage counts against a provider only if it failed from two or more regions in the same minute while no other provider was failing. A failure seen from one region only is listed but never counted.

Does availability depend on where you call from?

Yes. Identical requests to identical endpoints over the last 7 days:

Region	Failure rate
Germany	0.089%
US Central	0.119%
São Paulo	0.126%
Tokyo	0.174%

Tokyo failed 2× more often than Germany. A monitor that probes from one location can't see that. Also worth knowing before you plan a fallback around a status page: only 11 of the 44 providers publish a status feed a program can read.

How much notice do providers give before retiring a model?

Computed from 233 records that carry both an announcement and a shutdown date on the provider's own deprecation page:

Provider	Median notice	Shortest notice
OpenAI	183 days	3 days
Mistral	92 days	0 days
Anthropic	63 days	60 days
Cohere	0 days	0 days

Coming up next: claude-opus-4-5 retires on Nov 24 and claude-sonnet-4-5 on Nov 30 (replacement: claude-sonnet-5-5). On Dec 11 OpenAI retires gpt-5, gpt-5-mini, gpt-5-nano and o3 (replacements: the gpt-5.6 family).

Caveats

Probes run from data centres, so absolute numbers are lower than what a home connection sees.
TTFB is reachability and network distance, not throughput or output quality.
TTFT is measured only where I pay for a key, so coverage is thin.

Full method: https://llmlatency.dev/methodology

Use the data

Everything is free under CC BY 4.0:

JSON: https://llmlatency.dev/api/rankings.json, https://llmlatency.dev/api/incidents.json, https://llmlatency.dev/api/deprecations.json
MCP endpoint for agents: https://llmlatency.dev/mcp
pip install llmlatency
Dataset on Hugging Face (https://huggingface.co/datasets/llmlatency/llm-latency-tracker) and Zenodo (https://doi.org/10.5281/zenodo.23388429)

I built and run https://llmlatency.dev, self-funded, no provider pays for anything. Which providers or regions would you add, and where is the method wrong?

Fastest AI inference API in 2026: 30 days of latency data for Groq, Cerebras, SambaNova and 43 more, from 3 regions

Max Bob — Tue, 27 Oct 2026 23:56:01 +0000

Short answer. No single AI inference API is fastest everywhere. Over 30 days (27 September – 26 October 2026), Fireworks AI had the closest API front door in US Central (28 ms median) and Tokyo (19 ms), with SambaNova a close second in Tokyo (25 ms). In São Paulo, Fireworks was 252 ms away and OpenRouter led (61 ms). For time to first token, Groq (openai/gpt-oss-120b) delivered its first token in a median of 0.8 s from São Paulo, 1.4 s from US Central and 1.5 s from Tokyo. Google's gemini-flash-lite-latest took 1.2–1.7 s, with a p95 of 5.6–6.1 s.

The useful part is not the winner. It's how much the answer changes with where your client runs. SambaNova's API front door is 25 ms away from Tokyo and 371 ms away from São Paulo, a 15× difference for the same API.

What was measured

Network latency, all 46 providers. Every 5 minutes from US Central, Tokyo and São Paulo, a probe opens an HTTPS connection to each provider's official API host and times one request: from sending it to receiving the first byte of the reply (TTFB). That is one round trip to the provider's front door plus its edge processing. No API key is used, so the reply is usually a 401. That still measures the path, but it says nothing about the GPUs behind the door. 1,190,063 successful probes went into this post.
Time to first token (TTFT), where we hold a working key. The same probe nodes stream a real completion: a one-word prompt with max_tokens: 1. TTFT is the time from opening a fresh HTTPS connection to the first non-empty chunk of the streamed reply. In this window that ran continuously for Groq (openai/gpt-oss-120b) and Google (gemini-flash-lite-latest): 12,992 requests.
Not measured here: TTFT for Cerebras and SambaNova. We don't have a working paid key for either in this window, so you only get their network numbers below. Throughput (tokens per second) isn't measured either. Vendor tokens-per-second claims are a different quantity and can't be compared with these figures.
Germany is excluded. We also probe from Germany, but while preparing this post I found that container was CPU-throttled. That pushed its timings up in steps of 100 ms: Google's API showed 99 ms median in the stored data against 29 ms when measured from the same host without the limit. It's fixed as of 27 October. German numbers come back once there are 30 clean days.

Every request goes straight to the provider's own API, never through a gateway.

Time to first token: Groq vs Gemini Flash-Lite

Requested from	Provider · model	TTFT p50	TTFT p95	Successful	Network TTFB p50
São Paulo	Groq · gpt-oss-120b	794 ms	1,338 ms	100.0%	234 ms
US Central	Groq · gpt-oss-120b	1,379 ms	1,906 ms	99.9%	121 ms
Tokyo	Groq · gpt-oss-120b	1,497 ms	2,040 ms	99.8%	164 ms
São Paulo	Google · gemini-flash-lite-latest	1,204 ms	5,602 ms	89.7%	155 ms
US Central	Google · gemini-flash-lite-latest	1,579 ms	5,656 ms	91.1%	47 ms
Tokyo	Google · gemini-flash-lite-latest	1,682 ms	6,103 ms	91.6%	64 ms

What this shows:

The network is the small part. Groq's front door is 121–234 ms away. Its first token arrives 560–1,330 ms after that. The gap is connection setup plus time spent inside the provider (queueing and prefill), and it is the larger part in all three regions.
The closest region is not the fastest one. Groq's front door is farthest from São Paulo (234 ms), yet São Paulo got the fastest first token, about 600 ms faster than US Central. The pattern held in both halves of the month. We can see that it happens, not why. A likely reason is that requests from different regions are served by different Groq capacity, but we can't confirm that from outside. The practical point: measure from where your users are, because network distance doesn't predict TTFT.
The tail is where Flash-Lite loses. Gemini Flash-Lite's median is within a few hundred milliseconds of Groq's, but its p95 is three to four times higher (5.6–6.1 s). 8–10% of requests timed out or returned a 5xx. Our Gemini key is on the free tier, so part of that may be free-tier capacity rather than what a paid account sees.
These are two different models. Groq serves a 120B open-weights model here and Google serves its smallest Gemini. Read this as "what these two endpoints did", not as a ranking of the providers.

Network latency to 20 inference APIs, by region (median TTFB)

Provider	US Central	Tokyo	São Paulo	Slowest ÷ fastest region
Fireworks AI	28 ms	19 ms	252 ms	13.2×
OpenRouter ¹	63 ms	56 ms	61 ms	1.1×
Google (Gemini)	47 ms	64 ms	155 ms	3.3×
Baseten	47 ms	157 ms	152 ms	3.3×
Novita	83 ms	153 ms	247 ms	3.0×
Groq	121 ms	164 ms	234 ms	1.9×
SambaNova	167 ms	25 ms	371 ms	14.9×
xAI	111 ms	171 ms	229 ms	2.1×
Together AI	120 ms	180 ms	268 ms	2.2×
Cerebras	71 ms	201 ms	189 ms	2.8×
Anthropic	94 ms	234 ms	199 ms	2.5×
OpenAI	139 ms	214 ms	257 ms	1.9×
Inference.net	122 ms	229 ms	225 ms	1.9×
Mistral	189 ms	317 ms	254 ms	1.7×
Friendli	181 ms	284 ms	285 ms	1.6×
DeepSeek	315 ms	181 ms	437 ms	2.4×
Nebius	178 ms	477 ms	366 ms	2.7×
DeepInfra	162 ms	449 ms	536 ms	3.3×
Hyperbolic	149 ms	477 ms	837 ms	5.6×
Featherless	354 ms	488 ms	737 ms	2.1×

¹ OpenRouter is a gateway: its front door is close in every region, and then it forwards your request to an upstream provider. A low number here doesn't make the model behind it fast.

Full table for all 46 providers: llmlatency.dev/fastest-ai-api (rebuilt every 30 minutes from the last 24 hours).

What stands out:

Cerebras is close to US Central (71 ms) but 189–201 ms away from Tokyo and São Paulo.
Groq was the most even of the three fast-inference specialists: 121–234 ms, a 1.9× spread.
SambaNova is the extreme case: 25 ms from Tokyo, 167 ms from US Central, 371 ms from São Paulo.
Fireworks AI is closest in two regions (19 ms Tokyo, 28 ms US Central) and 252 ms away from São Paulo.

If you're choosing for latency

Measure from the region where your servers run, not from your laptop. In this data, provider order changed in every region.
Reuse connections. TTFT above includes opening a fresh HTTPS connection every time, and a warm connection skips that.
Look at p95 as well as p50. A provider that is 300 ms slower at the median but has a tight tail can feel faster in a product.
Re-check monthly. Front doors move: providers add regions and change CDNs.

Data

Live, last 24 h: fastest AI API by region · lowest time to first token · per provider: Groq, Cerebras, SambaNova
How probes work: methodology
Daily aggregates under CC BY 4.0, with a DOI: doi.org/10.5281/zenodo.21954788 · from Python: pip install llmlatency
Related: AI API uptime comparison, last 30 days

Disclosure: I run llmlatency.dev, the independent tracker these numbers come from. No provider pays for placement. If you measure something different from your region, post it in the comments with the UTC time and I'll check it against the raw probes.

OpenAI API deprecations 2026–2027: every shutdown date and recommended replacement (GPT-4o, GPT-4.1, GPT-5, o3, Whisper)

Max Bob — Tue, 27 Oct 2026 23:54:10 +0000

Short answer. GPT-4o, GPT-4o mini, GPT-4.1 and GPT-4.1 mini leave the OpenAI API on 14 January 2027. OpenAI's recommended replacement for all four is gpt-6-luna. Before that, GPT-5, GPT-5 mini, GPT-5 nano, GPT-5 pro, o3 and o3-pro shut down on 11 December 2026. After that come the legacy audio and realtime models (20 January), Whisper and the GPT-4o transcription models (26 February), and GPT-5.1 (1 April 2027).

Every date and replacement below is copied from OpenAI's deprecations page. I also compared them with the shutdown_date field that GET /v1/models returned on 27 October 2026 (UTC): for all 90 models that carry a date there, it matched the page exactly. "Days left" is counted from 28 October 2026.

Upcoming OpenAI API shutdowns

Shutdown date	What shuts down	Recommended replacement	Notice given	Days left
31 Oct 2026	Evals platform: existing evals become read-only	Promptfoo migration guide	150 days	3
30 Nov 2026	Evals dashboard and API	Promptfoo (same guide)	180 days	33
30 Nov 2026	Agent Builder (ChatKit stays)	Agents SDK or ChatGPT Workspace Agents	180 days	33
30 Nov 2026	`v1/prompts` API and reusable prompt objects	Move prompt text into your own code	180 days	33
1 Dec 2026	`gpt-image-1-mini`, `gpt-image-1.5` (and `-2025-12-16`), `chatgpt-image-latest`	`gpt-image-2.5-sunburst` or `gpt-image-2.5-flare`	182 days	34
11 Dec 2026	`gpt-5`, `o3` (and their snapshots)	`gpt-5.6-sol`	183 days	44
11 Dec 2026	`gpt-5-mini`	`gpt-5.6-terra`	183 days	44
11 Dec 2026	`gpt-5-nano`	`gpt-5.6-luna`	183 days	44
11 Dec 2026	`gpt-5-pro`, `o3-pro`	`gpt-5.6-sol` with `reasoning.mode: pro`	183 days	44
6 Jan 2027	`tts-1`, `tts-1-hd`, `gpt-4o-mini-tts` (both snapshots)	`gpt-realtime-2.1-mini`	97 days	70
6 Jan 2027	Creating new fine-tuning jobs (existing fine-tuned models keep working until their base model retires)	—	244 days	70
14 Jan 2027	GPT-4o (`gpt-4o-2024-08-06`, `gpt-4o-2024-11-20`), GPT-4o mini (`gpt-4o-mini-2024-07-18`), GPT-4.1 (`gpt-4.1-2025-04-14`), GPT-4.1 mini (`gpt-4.1-mini-2025-04-14`)	`gpt-6-luna`	92 days	78
20 Jan 2027	`gpt-realtime`, `gpt-4o-realtime`	`gpt-realtime-2.1`	184 days	84
20 Jan 2027	`gpt-realtime-mini`, `gpt-4o-mini-realtime`	`gpt-realtime-2.1-mini`	184 days	84
20 Jan 2027	`gpt-audio`, `gpt-audio-mini`, `gpt-4o-audio`, `gpt-4o-mini-audio`	`gpt-audio-1.5`	184 days	84
20 Jan 2027	`gpt-4o-mini-transcribe-2025-03-20`	`gpt-4o-mini-transcribe-2025-12-15` (see the catch below)	184 days	84
26 Feb 2027	`whisper-1`, `gpt-4o-transcribe`, `gpt-4o-mini-transcribe`, `gpt-4o-transcribe-diarize`	`gpt-live-transcribe` or `gpt-transcribe`	184 days	121
26 Feb 2027	`/v1/audio/translations` endpoint	`v1/realtime/translations`	184 days	121
1 Apr 2027	`gpt-5.1`, `gpt-5.3-codex`	`gpt-6-sol`	182 days	155
1 Apr 2027	`gpt-5.4-nano`	`gpt-6-luna`	182 days	155

Fine-tuned versions of GPT-4o, GPT-4o mini, GPT-4.1 and GPT-4.1 mini follow the same 14 January schedule as their base models.

Three things the table doesn't make obvious

GPT-4o got 92 days of notice, not six months. OpenAI's page says generally available models get "at least 6 months" of notice, "unless safety or compliance concerns require a faster timeline". The GPT-4 series notice went out on 14 October 2026 for a 14 January 2027 shutdown. Most other upcoming model retirements on the page got 182–184 days. If you planned around six months, you now have 78 days.
One recommended replacement retires 37 days after it takes over. gpt-4o-mini-transcribe-2025-03-20 should move to gpt-4o-mini-transcribe-2025-12-15 by 20 January 2027. That model is itself shut down on 26 February 2027. Go straight to gpt-transcribe or gpt-live-transcribe and migrate once.
Four model families collapse into one. GPT-4o, GPT-4o mini, GPT-4.1 and GPT-4.1 mini all map to gpt-6-luna. If you picked a mini model for cost or latency, re-run your evals and check the bill before switching. One recommendation for all four does not mean the same price or speed as each of them.

What already shut down in 2026

Shut down	Models / systems	Recommended replacement
23 Oct 2026	`gpt-4`, `gpt-4-turbo`, `gpt-4-1106-preview`, `gpt-4o-2024-05-13`, `o1`, `o3-mini`	`gpt-5.6-sol`
23 Oct 2026	`o1-pro`	`gpt-5.6-sol` with `reasoning.mode: pro`
23 Oct 2026	`gpt-3.5-turbo`, `o4-mini`	`gpt-5.6-terra`
23 Oct 2026	`gpt-4.1-nano`	`gpt-5.6-luna`
23 Oct 2026	`gpt-image-1`	`gpt-image-2.5-sunburst` or `gpt-image-2.5-flare`
1 Oct 2026	`gpt-5.4-cyber`	"The most capable cyber model available to you"
28 Sep 2026	`gpt-3.5-turbo-instruct`, `gpt-3.5-turbo-1106`, `babbage-002`, `davinci-002`	`gpt-5.6-terra`
24 Sep 2026	Videos API and all Sora 2 models	none listed
26 Aug 2026	Assistants API	Responses API + Conversations API
10 Aug 2026	`gpt-5.2-chat-latest`, `gpt-5.3-chat-latest`	`gpt-5.6-sol`
23 Jul 2026	`gpt-5-chat-latest`, `gpt-5.1-chat-latest`, `gpt-5-codex`, `gpt-5.1-codex`, `gpt-5.1-codex-max`, `gpt-5.2-codex`, `o3-deep-research`, `o4-mini-deep-research`	`gpt-5.6-sol`
23 Jul 2026	`computer-use-preview`, `gpt-5.1-codex-mini`, `gpt-4o-search-preview-2025-03-11`, `gpt-4o-mini-search-preview-2025-03-11`	`gpt-5.6-terra`

Check your own account in 30 seconds

OpenAI's page says every model in GET /v1/models carries an optional shutdown_date (YYYY-MM-DD, or null if none is announced). This prints every model your key can see that has a date, and marks the ones already past it:

import datetime, json, os, urllib.request

req = urllib.request.Request(
    "https://api.openai.com/v1/models",
    headers={"Authorization": f"Bearer {os.environ['OPENAI_API_KEY']}"},
)
models = json.load(urllib.request.urlopen(req, timeout=30))["data"]
today = datetime.date.today().isoformat()

for m in sorted(models, key=lambda m: m.get("shutdown_date") or "9999"):
    d = m.get("shutdown_date")
    if d:
        print("PAST " if d < today else "     ", d, m["id"])

Or with curl and jq:

curl -s https://api.openai.com/v1/models -H "Authorization: Bearer $OPENAI_API_KEY" \
  | jq -r '.data[] | select(.shutdown_date != null) | "\(.shutdown_date) \(.id)"' | sort

Here is what I saw when I ran it on 27 October 2026 (UTC). The list depends on your organisation, so yours may differ.

My key could see 142 models. 90 had a shutdown_date, and 42 of those dates were already in the past. That included gpt-4, gpt-3.5-turbo, o1, o3-mini and o4-mini (23 October) and computer-use-preview (23 July). A model showing up in the list does not mean it still works. Compare the date with today.
Some snapshots have no date even though a sibling does. tts-1-1106 and tts-1-hd-1106 came back with shutdown_date: null, while tts-1 and tts-1-hd retire on 6 January 2027. The same was true of gpt-4o-search-preview (the -2025-03-11 snapshot shut down in July) and gpt-3.5-turbo-instruct-0914. The deprecations page doesn't list these IDs at all. Treat null as "unknown", not "safe".

Where to track it

The source of truth: OpenAI API deprecations
One calendar for OpenAI, Anthropic, Mistral, Cohere and Azure OpenAI retirements, re-checked against each provider's own page every week: llmlatency.dev/deprecations. There is also an OpenAI-only view and an .ics feed you can add to Google Calendar or Outlook.
A page for each model, e.g. migrating off GPT-4o

Disclosure: I run llmlatency.dev, an independent tracker of AI API latency, uptime and model retirements. I have no affiliation with OpenAI. Every row in this post links back to OpenAI's own page; if you spot a mismatch, tell me in the comments and I'll fix it.

AI API uptime comparison, last 30 days: OpenAI, Anthropic, Google and 43 more, measured from 4 regions

Max Bob — Sun, 18 Oct 2026 10:04:53 +0000

Short answer. Between 18 September and 17 October 2026, the four most available AI APIs we probe were Meta's Llama API (99.986%), Anthropic (99.985%), OpenRouter (99.985%) and Google Gemini (99.983%). OpenAI was at 99.930%. The spread among the big providers is a few hundredths of a percent; the difference that matters shows up in the worst region, not in the average.

These are observed-availability figures, not SLAs: what an outside client saw, from four data centres, every five minutes.

What was measured

46 AI inference APIs, probed every 5 minutes from 4 regions: Germany, US Central, Tokyo and São Paulo.
1,580,667 probes over 30 full days (18 Sep – 17 Oct 2026, UTC).
A probe is the network path to the provider's API host: DNS → TCP → TLS → first byte. It fails on a timeout, a connection error or an HTTP 5xx. A 4xx counts as "the API answered" — the probe carries no key.
Attributed outage = the provider failed from 2+ regions at once and no other provider was failing in the same minutes. A failure seen from one region can't be separated from that network path; a failure shared with other providers has a cause above all of them.
Probes run directly against each provider, never through a gateway or an aggregator.

Results

#	Provider	30-day availability	Worst region	Worst-region availability	Attributed outages
1	Meta Llama API	99.986%	São Paulo	99.954%	0
2	Anthropic (Claude)	99.985%	São Paulo	99.965%	0
3	OpenRouter	99.985%	São Paulo	99.977%	0
4	Google (Gemini)	99.983%	São Paulo	99.977%	0
5	AI21	99.965%	Tokyo	99.942%	0
6	Replicate	99.965%	Germany	99.942%	0
7	Baseten	99.959%	Tokyo	99.942%	0
8	Cohere	99.959%	Tokyo	99.919%	0
9	Together AI	99.959%	Germany	99.942%	0
10	Groq	99.954%	Tokyo	99.907%	0
11	Hyperbolic	99.951%	Tokyo	99.885%	0
12	Novita	99.945%	Germany	99.872%	0
13	Fireworks AI	99.939%	Germany	99.907%	0
14	Perplexity	99.939%	Tokyo	99.861%	0
15	Inference.net	99.934%	Tokyo	99.885%	1 (11 min)
16	DeepSeek	99.931%	São Paulo	99.850%	0
17	Mistral	99.931%	Tokyo	99.827%	0
18	OpenAI	99.930%	Tokyo	99.814%	0
19	Baidu (ERNIE)	99.928%	São Paulo	99.792%	0
20	xAI (Grok)	99.922%	Tokyo	99.815%	0
21	Reka	99.919%	Tokyo	99.815%	0
22	Cerebras	99.907%	Tokyo	99.804%	2 (6 min)
23	Writer	99.902%	Tokyo	99.826%	1 (7 min)
24	Sarvam	99.901%	Germany	99.849%	1 (3 min)
25	SiliconFlow	99.893%	São Paulo	99.746%	0
26	Nscale	99.881%	Tokyo	99.676%	0
27	Z.ai (GLM)	99.879%	São Paulo	99.665%	0
28	Friendli	99.867%	Tokyo	99.723%	3 (5 min)
29	Typesafe ¹	99.852%	Tokyo	99.712%	1 (3 min)
30	Moonshot (Kimi)	99.850%	Tokyo	99.792%	1 (36 min)
31	Alibaba (Qwen)	99.826%	Tokyo	99.676%	0
32	Nebius	99.812%	São Paulo	99.607%	0
33	MiniMax	99.806%	Tokyo	99.561%	0
34	Targon	99.803%	Tokyo	99.561%	0
35	DeepInfra	99.780%	São Paulo	99.607%	0
36	Upstage	99.780%	São Paulo	99.515%	0
37	Tencent (Hunyuan)	99.749%	São Paulo	99.457%	0
38	SenseNova	99.667%	São Paulo	99.249%	1 (2 min)
39	StepFun	99.647%	São Paulo	99.203%	2 (1 min)
40	Aleph Alpha	99.645%	São Paulo	99.238%	4 (22 min)
41	Featherless	99.642%	São Paulo	99.434%	2 (4 min)
42	SambaNova	99.535%	Tokyo	99.260%	9 (71 min)
43	iFlytek	99.266%	Tokyo	98.406%	2 (11 min)
44	ByteDance (Doubao)	99.260%	São Paulo	98.891%	11 (91 min)
45	Baichuan	99.243%	Tokyo	98.394%	0
46	01.AI (Yi)	97.275%	Tokyo	90.650%	10 (118 min)

¹ Typesafe was added to the probes on 27 September: 21 days of data instead of 30.

What stands out

Uptime barely separates the large providers. 24 of 46 APIs stayed at or above 99.9% over the month; the median was 99.90%. Between Anthropic and OpenAI the gap is 0.055 percentage points — about 24 minutes a month.
The worst region is where the differences are. For 24 providers the worst region was Tokyo, for 17 it was São Paulo, for 5 Germany. US Central was never the worst region for anyone. If your users are in Asia or South America, look at the regional column, not the average.
Attributed outages concentrate in a few providers. 51 outages in 30 days, across 15 providers; ByteDance Doubao (11), 01.AI Yi (10) and SambaNova (9) account for more than half. The four most available APIs had none.

Limitations, stated up front

The vantage points are cloud data centres, not consumer networks, so absolute numbers are better than an end user on mobile would see. Comparison between providers is the meaningful part.
Network-level availability is not the same as a successful completion. A host that answers while the model queue is stuck still counts as up here; inference probes (real time-to-first-token requests) are reported separately.
30 days is one window. The live page is rebuilt every 30 minutes from the last 24 hours.

Live data and the dataset

Current uptime ranking (last 24 h): https://llmlatency.dev/best-uptime
Per-provider status, e.g. "Is the OpenAI API down?": https://llmlatency.dev/provider/openai
Incident log with the attribution rule above: https://llmlatency.dev/incidents
Full daily aggregates, CC-BY-4.0, with a DOI: https://doi.org/10.5281/zenodo.21954788
From Python: pip install llmlatency

Disclosure: I run llmlatency.dev, the independent tracker these numbers come from. No provider pays for placement, and nothing on the ranking is affiliate-sorted.

No one measured AI API latency and uptime independently across regions — so I built it. 38 days and 2M probes in, here's what the data shows

Max Bob — Mon, 31 Aug 2026 10:34:00 +0000

If you build on hosted AI APIs, two questions quietly decide your architecture: how fast is provider X from where your users actually are, and how long will the model you pinned actually live. When I went looking for measured answers, I found single-region benchmarks, vendor status pages, and numbers measured through gateways that attribute the gateway's problems to the provider. So I built an independent tracker.

How it works (boring on purpose)

A small prober opens a real connection to each provider's official API endpoint every 5 minutes and times every phase of the handshake: DNS → TCP → TLS → time-to-first-byte. Separately, where possible, it measures inference time-to-first-token.
It runs from 4 regions: Germany, US Central, Tokyo, São Paulo. Requests go directly to providers — no gateway, no aggregator, so nothing gets attributed to the wrong party.
45 providers: the usual Western labs (OpenAI, Anthropic, Google, Mistral, Groq, Together, Fireworks, Cerebras, DeepSeek…) plus Chinese ones (GLM/Zhipu, Kimi/Moonshot, Qwen, MiniMax) that rarely show up in Western benchmarks.
Results land in SQLite; a static site regenerates every 30 minutes. 38 days in: just over 2,000,000 probes.

What the data shows so far

1. Region beats provider. Over the same 7 days, on identical requests to identical endpoints, probes from Tokyo failed 0.315% of the time vs 0.089% from Germany — a 3.5× gap. A monitor that probes from one location cannot see this, by construction.

2. Uptime numbers lie by omission of the window. Over any 24-hour window almost every provider reads ~100%. Over 7 days, 40 of 45 providers dropped at least one probe. Same fleet, different window, opposite conclusion — availability quoted without a window length is not comparable to anything.

3. Availability doesn't differentiate providers; latency does. Right now the fastest p50 TTFB in the fleet is ~99 ms and the slowest is ~1424 ms — a 14× spread between endpoints whose availability is statistically indistinguishable.

4. How much warning you get before a model is killed. Computed from the providers' own announcement and shutdown dates across 203 dated retirements — a number nobody publishes, including the providers themselves:

Provider	Retirements tracked	Median notice	Shortest
OpenAI	133	182 days	3 days
Mistral	40	92 days	0 days
Anthropic	19	63 days	60 days
Cohere	11	0 days	0 days

Cohere's announcements land the day the model dies. A pinned model version can stop working the day you read the notice — that's a planning constraint, not a judgement.

The part I care most about: agent-native by design

The way people ask questions is shifting from search boxes to answer engines and agents. If your data only exists as JavaScript-rendered HTML, an agent can't reliably consume or cite it. So every page has a markdown twin, there's llms.txt and llms-full.txt, a JSON API, and an MCP server. Two observations after five weeks:

~23% of all crawler fetches take the markdown layer, not the HTML.
"A human asked their AI assistant to open the page" is now a bigger channel than browser clicks from AI answers — I can tell them apart server-side, and the agent-initiated reads outnumber referral clicks by an order of magnitude.

Honest limitations

This measures endpoint reachability and TTFB/TTFT — not model quality, not rate-limit behaviour, not incidents visible only inside a provider. Four vantage points is not the whole internet. The dataset is 38 days young, and the incident log deliberately keeps a short window because two coincidences don't make a trend.

Everything is free, no accounts, no ads: llmlatency.dev — the deprecation calendar is at /deprecations, measured uptime at /best-uptime, method and limits at /methodology. Data is CC BY 4.0.

What's missing? What would you measure next?

Update, October 2026. A month-long follow-up with the full table is out: AI API uptime comparison, last 30 days — 46 APIs, 1.58 million probes from 4 regions, worst region and attributed outages for each provider.

TTFT is not TTFB: what 45 AI APIs measured from 4 regions actually show

Max Bob — Sun, 16 Aug 2026 22:04:40 +0000

If you have ever compared "fastest AI API" benchmarks and found they disagree with each other, there is a good chance they were not measuring the same thing. Two numbers get used interchangeably, and they answer different questions:

TTFB — time to first byte. DNS resolution, TCP connect, TLS handshake, first response byte. The network path and the provider's front door. No model is involved at all.
TTFT — time to first token. A real streamed completion, timed until the first token appears. That is the same network time, plus queueing at the provider, plus the model's prefill over your prompt.

TTFB is the floor you pay on every single request no matter what you call. TTFT is what a human actually stares at before words show up on screen.

I run llmlatency.dev, which measures both continuously for ~45 inference providers from four regions (Germany, US Central, Tokyo, São Paulo). Here is what the data says about why these two numbers should never be merged into one leaderboard.

The same region, the same probes, two different winners

Measured from São Paulo over the last 24 hours:

Metric	Winner	Number
Lowest TTFB (edge latency)	openrouter	58 ms
Lowest TTFT (first token)	cerebras	757 ms

openrouter's front door answers 3.2× faster than cerebras's (58 ms vs 184 ms). And cerebras still streams its first token 264 ms sooner (757 ms vs 1021 ms).

Same probes, same schedule, same region, opposite conclusions. "Which API is fastest?" is not a question with one answer until you say which of the two you are paying for.

Most of TTFT is not the API you are benchmarking

Here is the split for every provider where I have both numbers side by side:

Requested from	Provider	TTFT p50	TTFB p50	Network share of TTFT
São Paulo	groq	764 ms	233 ms	31%
Germany	groq	900 ms	296 ms	33%
São Paulo	cerebras	757 ms	184 ms	24%
Germany	cerebras	803 ms	201 ms	25%
Tokyo	cerebras	1312 ms	196 ms	15%
São Paulo	google	1143 ms	159 ms	14%
Tokyo	groq	1256 ms	147 ms	12%
US Central	groq	1355 ms	108 ms	8%
Germany	openrouter	1300 ms	100 ms	8%
Germany	google	1361 ms	101 ms	7%
São Paulo	openrouter	1021 ms	58 ms	6%
US Central	cerebras	1485 ms	68 ms	5%
Tokyo	google	1483 ms	57 ms	4%
Tokyo	openrouter	1292 ms	55 ms	4%
US Central	openrouter	1567 ms	59 ms	4%
US Central	google	1524 ms	42 ms	3%

Between 3% and 33% of time to first token is network time. The other 67–97% is the provider queueing your request and the model doing prefill.

That has an uncomfortable consequence for benchmark tables: if a TTFT comparison does not name the model behind every number, most of what it measured was the model, not the API. A 30B model will beat a 120B model on first-token time regardless of whose GPUs it runs on. Here are the models behind the numbers above, which is exactly why I do not publish them as a provider ranking:

Provider	Model answering
cerebras	`gpt-oss-120b`
groq	`llama-3.3-70b-versatile`
openrouter	`nvidia/nemotron-3-nano-30b-a3b:free`
google	`gemini-flash-lite-latest`

Four different models. Ranking them against each other and calling it a provider ranking would be a category error wearing a table.

"Fast" is meaningless without "from where"

The regional spread is larger than the gap between most providers:

Provider	Fastest region	Slowest region	Spread
sambanova	21 ms (Tokyo)	398 ms (Germany)	18.9×
fireworks	16 ms (Tokyo)	251 ms (São Paulo)	15.3×
upstage	58 ms (Tokyo)	596 ms (São Paulo)	10.4×
aleph-alpha	99 ms (Germany)	568 ms (Tokyo)	5.7×

Identical API, identical request, an 18.9× difference depending on which continent you dialled from. A single-machine benchmark run from one US datacenter is not wrong so much as it is answering a question about that datacenter.

A checklist for reading anyone's latency benchmark

Including mine. If a benchmark cannot answer these, treat the number as a vibe:

TTFB or TTFT? If it does not say, it is usually TTFB dressed up in TTFT language.
Which model? No model name next to a TTFT figure means the figure is about the model.
From where? One region is one data point, not a ranking.
p50 or p95? Tail latency is where user-visible pain lives, and p95 orderings often differ from p50.
When, and how often? A benchmark run once in March describes March.
How many samples? My tables above sit on 57–58 samples per provider per region per day.

The honest limits of my own data

TTFT is measured for 4 of the 45 providers I track. Not because the other 41 are uninteresting, but because a streamed completion requires a paid API key for each provider and every probe spends tokens. Edge latency needs no key, so TTFB is published for all 45.

I would rather publish 4 measured TTFT figures than 45 estimated ones. An estimate presented as a measurement is the one thing a latency tracker must never do.

Everything is free, and machine-readable

The data is CC-BY-4.0. No key, no signup, CORS open:

# Rankings for every region, updated continuously
curl https://llmlatency.dev/api/rankings.json

# Any page as markdown, for agents and scripts
curl https://llmlatency.dev/time-to-first-token.md

There is also a remote MCP server if you want your assistant to look this up mid-conversation:

curl -X POST https://llmlatency.dev/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}'

Full method, including what the probes do and do not prove: llmlatency.dev/methodology. The TTFT breakdown lives at llmlatency.dev/time-to-first-token, and the prober itself is open source at github.com/mazamaka/llm-latency-tracker.

If you have a provider you want added, or you think one of these numbers is wrong, tell me — measurements that survive being checked are the only kind worth publishing.

There was no independent, measured view of AI-API latency by region — so I built one

Max Bob — Thu, 23 Jul 2026 18:19:48 +0000

If you build anything on top of hosted AI APIs, latency isn't a detail you get to ignore — it's a feature. A sluggish time-to-first-token is the difference between an assistant that feels alive and one that feels broken.

Yet when I went looking for an honest answer to a simple question — how fast is provider X from where my users actually are? — I couldn't find one. The numbers people quote tend to come from a single machine in a single region (usually somewhere in the US), from vendor-reported status pages, or from a benchmark that got run once and never refreshed. There was no independent, measured, regional view.

So I built one: LLM Latency Tracker, a provider-neutral tracker of latency and uptime for AI inference APIs.

How it works

The core idea is boring on purpose: actually measure, don't scrape.

A small Python prober — standard library only, no API key needed for the edge probes — opens real connections to each provider's endpoint and times every phase of the handshake: DNS resolution → TCP connect → TLS negotiation → time-to-first-byte (TTFB). That's the edge view: how long the network path itself takes before a single byte comes back.

Separately, where possible, it measures inference time-to-first-token (TTFT) — the thing your users actually feel, i.e. how long after you hit "send" the model starts streaming.

Those two numbers answer different questions, and keeping them apart matters. Edge latency is about the network and the front door; TTFT is about the model and the queue behind it.

The probes run from four regions — Europe (Germany), US Central, Asia (Tokyo), and South America (São Paulo) — because "fast" is meaningless without "from where." Results land in a SQLite time-series; a static-site generator turns that into the pages you see, hosted on Cloudflare Pages, and the whole thing is self-updating on a schedule. There's no always-on backend to rot or page me at 3am.

It currently covers ~45 providers — the usual Western labs (OpenAI, Anthropic, Google, Mistral, Groq, Together, Fireworks, Cerebras, OpenRouter, Perplexity, xAI, DeepSeek…) plus Chinese ones (GLM/Zhipu, Kimi/Moonshot, Qwen, MiniMax) that rarely show up in Western benchmarks. All of it is free and open under CC-BY-4.0, with a JSON API and an OpenAPI spec.

The interesting part: building it agent-native

Here's what makes this more than another dashboard. I built it to be read by machines, not just humans.

The way people ask questions is shifting from search boxes to answer engines and agents. If your data only exists as HTML wrapped in JavaScript, an agent can't reliably consume or cite it — which means your measurements never reach the place where someone actually asks "which provider has the lowest TTFT in Europe right now?"

So the site speaks several agent dialects:

A real MCP server (Streamable HTTP) at /mcp, exposing a get_ai_api_latency tool. An agent can call it directly and get structured latency data back — no scraping, no guessing.
An MCP Server Card at /.well-known/mcp/server-card.json, plus WebMCP, so agents can discover the tool.
llms.txt and llms-full.txt, giving models a clean map of the site.
Markdown content negotiation: ask a page for text/markdown and you get tidy Markdown instead of a DOM you have to reverse-engineer.

The nice side effect is that building for agents forces you to build clean for everyone. The same discipline that makes a page legible to an LLM — structured, fast, no junk — is what earned a Lighthouse 100 and Cloudflare's "Agent-Native" (Level 5) rating on their "Is Your Site Agent-Ready?" check. I won't pretend that means much on its own; it's a checklist, not a trophy. But it's a decent proxy for "this data is actually reachable."

There's also a small companion: a model deprecation & migration calendar at /deprecations, hand-verified from official provider docs, so you can spot which models are being sunset before they break your app.

Honest limitations

A tracker that oversells itself is worse than none, so — plainly:

The dataset is young. It's only days old as I write this. Treat trends as provisional; don't over-read a single snapshot.
Edge latency is not a full round-trip. TTFB to an endpoint measures the network path, not the time to a finished generation. TTFT is closer to lived experience, but still isn't total completion time.
Keyless network probes and keyed inference probes see different things. Inference coverage is narrower than edge coverage.
Four regions is a sample, not the planet. If your users sit somewhere I don't probe, your mileage will differ.

It's a baseline and a starting point, not gospel.

Try it / break it

It's free, open-source, and the data is CC-BY-4.0 — use it, chart it, cite it, build on it.

Site: https://llmlatency.dev
JSON API: https://llmlatency.dev/api/rankings.json
MCP endpoint: https://llmlatency.dev/mcp
Code: https://github.com/mazamaka/llm-latency-tracker

If a provider or region you care about is missing, open an issue or a PR — that's exactly the kind of contribution that makes this more useful for everyone. And if you're building agents, point one at the MCP endpoint and tell me what breaks.

Update, August 2026. The single most common confusion in the replies to this post was TTFB vs TTFT, so I wrote up the measured difference — including a region where the TTFB winner and the TTFT winner are different providers: TTFT is not TTFB: what 45 AI APIs measured from 4 regions actually show.