I Spent Weeks Benchmarking AI APIs So You Don't Have To

Check this out: i Spent Weeks Benchmarking AI APIs So You Don't Have To

honestly, when I first started building AI-powered features into my SaaS, I had NO idea how much speed mattered. Like, I thought a 2-second response was fine. People can wait two seconds, right?

WRONG.

Every extra 100ms I shaved off my response time, my conversion rate went up. Not by a little either. Pretty much every study out there confirms what I learned the hard way — latency is the silent killer of good UX.

So I did what any obsessive indie hacker would do. I spent weeks running benchmarks. Tested 15 different models. Measured TTFT (time to first token), tokens per second, all of it. Hit Global API from two different regions, ran the same prompt 10 times each, averaged the numbers.

Heres what I found.

Why I Even Bothered With This

Look, theres a million AI API comparison posts out there. Most of them are just regurgitating marketing copy. I wanted REAL data, from a builders perspective, on what actually performs when users are staring at a loading spinner.

My setup:

• Test date: May 20, 2026
• Two regions: US East (Ohio) and Asia (Singapore)
• Prompt: "Explain recursion in 200 words"
• Output: ~150 tokens per test
• Ran each test 10 times, took the average
• Streaming via SSE, always
• All calls went through Global API at https://global-apis.com/v1

I figured if I was gonna burn that much time, I should share it.

The Speed Winners (And The Surprises)

okay so the rankings kind of shocked me. I was expecting the big names to dominate. They didnt. Here are all 15 models ranked from fastest to slowest, with TTFT, tokens per second, and price:

Rank	Model	TTFT	Tok/s	Price ($/M out)
1	Step-3.5-Flash	120ms	80	$0.15
2	DeepSeek V4 Flash	180ms	60	$0.25
3	Hunyuan-TurboS	200ms	55	$0.28
4	Qwen3-8B	150ms	70	$0.01
5	Qwen3-32B	250ms	45	$0.28
6	Doubao-Seed-Lite	220ms	50	$0.40
7	Hunyuan-Turbo	280ms	42	$0.57
8	GLM-4-32B	300ms	38	$0.56
9	Qwen3.5-27B	350ms	35	$0.19
10	DeepSeek V4 Pro	400ms	30	$0.78
11	MiniMax M2.5	450ms	28	$1.15
12	GLM-5	500ms	25	$1.92
13	Kimi K2.5	600ms	20	$3.00
14	DeepSeek-R1	800ms	15	$2.50
15	Qwen3.5-397B	1200ms	10	$2.34

The big takeaway? Step-3.5-Flash is the absolute speed king at 80 tokens/second. But Qwen3-8B is RIGHT there at 70 tok/s, and its literally ONE CENT per million output tokens. ONE CENT. I had to look at that number three times.

A Quick Note On The Slow Models

I gotta say, before you go dismissing the bottom of the list — those slow models arent bad. Some of them are reasoning models (DeepSeek-R1, Kimi K2.5) which means they spend time "thinking" internally before spitting out the first token. Qwen3.5-397B is a massive 397 billion parameter model, of course its slow.

But for a chat app or any kind of interactive experience? These would feel painful.

Grouping By Price (This Is Where It Gets Interesting)

raw speed rankings are fine, but as an indie hacker, I care way more about VALUE. So I sorted everything into price tiers and found some really useful patterns.

The "Pennies Per Million" Tier (under $0.15/M output)

Model	Tok/s	Price
Qwen3-8B	70	$0.01
Step-3.5-Flash	80	$0.15

Qwen3-8B is honestly absurd. 70 tokens per second for one cent per million. You could run a million tokens through it and not even buy a coffee. For stuff like classification, simple Q&A, summarization where raw speed matters more than brilliance — its unbeatable.

I built a tagging feature with it and the thing flies.

The Sweet Spot ($0.15-$0.30/M)

Model	Tok/s	Price
DeepSeek V4 Flash	60	$0.25
Hunyuan-TurboS	55	$0.28
Qwen3-32B	45	$0.28

This is where I live now. Pretty much every new feature I build uses something in this tier.

DeepSeek V4 Flash is my goto. 60 tok/s with quality that punches way above its weight class. Honestly it gives GPT-4o a run for its money on most tasks and costs a fraction. $0.25/M is just a phenomenal price for what you get.

If you want a single recommendation from this whole post, its DeepSeek V4 Flash. Done.

The Middle Ground ($0.30-$0.80/M)

Model	Tok/s	Price
Doubao-Seed-Lite	50	$0.40
GLM-4-32B	38	$0.56
Hunyuan-Turbo	42	$0.57
DeepSeek V4 Pro	30	$0.78

Youll notice the speeds start dropping here. These are bigger models, more parameters, more thinking per token. V4 Pro at 30 tok/s is noticeably slower than V4 Flash, but the output quality is meaningfully better.

I use this tier when the user is waiting for something thats harder to regenerate — like an email draft or a blog post. Faster isnt always better.

Premium Tier ($0.80+/M)

Model	Tok/s	Price
MiniMax M2.5	28	$1.15
GLM-5	25	$1.92
Kimi K2.5	20	$3.00

These are the "correctness is everything" models. Kimi K2.5 at $3.00/M is expensive, but if you need the model to do complex reasoning and not hallucinate, this is where you go.

I use these for backend stuff where latency doesnt matter — like overnight batch processing of customer support tickets. No user is staring at a spinner, so the slow speed is fine.

The Geography Thing Nobody Talks About

heres something I didnt think about until I started testing: where your servers are matters MORE than I expected.

I ran the same tests from US East and from Asia. Heres what I found:

Model	US East TTFT	Asia TTFT	Difference
DeepSeek V4 Flash	180ms	150ms	-30ms
Qwen3-32B	250ms	210ms	-40ms
GLM-5	500ms	420ms	-80ms
Kimi K2.5	600ms	480ms	-120ms

Asian models (Qwen, GLM, Kimi) get a 16-20% latency boost from Asia, which makes sense — their servers are physically closer. DeepSeek is well-distributed though, so it performs well everywhere.

For me, this confirmed: if your users are mostly in Asia, you should be reaching for Qwen or GLM first. If your users are global, DeepSeek V4 Flash is your best bet because it doesnt have a regional weakness.

How Users Actually Perceive Speed

This part changed how I think about API speed entirely. I used to think "well its under a second, thats fast enough." Then I made this table for myself based on actual user feedback I collected:

TTFT	What Users Say
Under 200ms	"Instant" — they dont even notice the wait
200-400ms	"Fast" — feels snappy, totally fine
400-800ms	"Noticeable delay" — some users get antsy
800ms+	"Slow" — people start dropping off

The hard line is 400ms for me. Anything above that and my support tickets go up. Anything below 200ms and users literally think the app is broken sometimes ("did it even process?").

DeepSeek V4 Flash at 180ms? That hits the sweet spot where it feels instant AND users know something happened.

The Code I Actually Use

Let me show you what my streaming setup looks like. Heres a simple Python example that benchmarks a model through Global API:

import requests
import time

API_KEY = "your-global-api-key"
BASE_URL = "https://global-apis.com/v1"

def stream_test(model_name, prompt):
    start = time.time()
    first_token_time = None
    token_count = 0

    response = requests.post(
        f"{BASE_URL}/chat/completions",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={
            "model": model_name,
            "messages": [{"role": "user", "content": prompt}],
            "stream": True
        },
        stream=True
    )

    for line in response.iter_lines():
        if line:
            if first_token_time is None:
                first_token_time = time.time() - start
            token_count += 1

    total_time = time.time() - start
    tokens_per_sec = token_count / total_time if total_time > 0 else 0

    return {
        "model": model_name,
        "ttft_ms": first_token_time * 1000,
        "tokens_per_sec": tokens_per_sec
    }

# Run the test
result = stream_test("deepseek-v4-flash", "Explain recursion in 200 words")
print(f"TTFT: {result['ttft_ms']:.0f}ms")
print(f"Speed: {result['tokens_per_sec']:.1f} tok/s")

And heres a production-ready function I use in my actual app for non-streaming calls (good for batch jobs):

import requests
import time

API_KEY = "your-global-api-key"
BASE_URL = "https://global-apis.com/v1"

def chat_complete(model, messages, max_tokens=500):
    start = time.time()

    response = requests.post(
        f"{BASE_URL}/chat/completions",
        headers={
            "Authorization": f"Bearer {API_KEY}",
            "Content-Type": "application/json"
        },
        json={
            "model": model,
            "messages": messages,
            "max_tokens": max_tokens,
            "temperature": 0.7
        }
    )

    elapsed = time.time() - start
    data = response.json()

    return {
        "content": data["choices"][0]["message"]["content"],
        "elapsed_ms": elapsed * 1000,
        "usage": data.get("usage", {})
    }

result = chat_complete(
    "deepseek-v4-flash",
    [{"role": "user", "content": "Write a product description for a smart water bottle."}]
)
print(f"Took {result['elapsed_ms']:.0f}ms")
print(f"Cost estimate: ${result['usage'].get('completion_tokens', 0) * 0.25 / 1_000_000:.6f}")

Pretty simple stuff, but the BASE_URL = "https://global-apis.com/v1" trick is what lets me swap providers without changing code. I literally just change the model name and it works.

My Personal Stack As An Indie Hacker

After all this testing, heres what I ended up with:

• Default chat / interactive features: DeepSeek V4 Flash ($0.25/M, 60 tok/s, 180ms TTFT)
• Bulk classification / tagging: Qwen