gentleforge

Posted on Jul 11

I Ran Chinese AI Against GPT-4o and Claude — The Results Wrecked Me

#programming #ai #api #deepseek

Okay so I've been going down a rabbit hole for the last like 3 weeks and honestly I gotta say — I don't think I can unsee what I found. I started this whole thing because my OpenAI bill was making me physically ill. Like $400/month for what? Half my prompts were just me asking GPT-4o to fix the bugs it created in the first place.

So I did what any broke indie hacker would do. I started looking at Chinese AI models. DeepSeek, Qwen, Kimi, GLM — names I'd seen floating around Twitter but never actually tried. And what I found kinda broke my brain a little.

Heres the thing nobody tells you: Chinese models in 2026 are GOOD. Like, genuinely, embarrassingly good. And they cost a FRACTION of what we're paying OpenAI and Anthropic. I'm talking 5x to 40x cheaper for output that's basically the same quality.

Let me show you what I mean.

Why I Even Bothered Looking

Quick backstory. I run a small SaaS (some of you know it, some of you don't — I write about it here sometimes). My stack uses GPT-4o for a bunch of stuff: content moderation, customer support routing, code review, embedding generation, the works.

Last month I got the bill. $387.42. For ONE app. On like 800 users.

I sat there staring at it like 😐

So I started doing what every developer does at 2am when they can't sleep because of API bills. I went on Reddit. I went on Hacker News. I read every damn comparison post I could find. And the more I dug, the more I realized — I'd been getting absolutely SCAMMED.

The Pricing Is Actually Insane

Let me just dump the numbers out so you can see what I'm talking about. These are all per million tokens btw, output prices unless I say otherwise:

Model	Where	Input	Output	How much extra vs V4 Flash
GPT-4o	🇺🇸	$2.50	$10.00	40x more
Claude 3.5 Sonnet	🇺🇸	$3.00	$15.00	60x more
Gemini 1.5 Pro	🇺🇸	$1.25	$5.00	20x more
GPT-4o-mini	🇺🇸	$0.15	$0.60	2.4x more
DeepSeek V4 Flash	🇨🇳	$0.18	$0.25	baseline
Qwen3-32B	🇨🇳	$0.18	$0.28	1.1x more
GLM-5	🇨🇳	$0.73	$1.92	7.7x more
Kimi K2.5	🇨🇳	$0.59	$3.00	12x more

Read that again. CLAUDE 3.5 SONNET IS 60X MORE EXPENSIVE THAN DEEPSEEK V4 FLASH.

I'm not even being dramatic. That's the actual price ratio. You could run 60 DeepSeek V4 Flash requests for every 1 Claude request. Your cost per feature goes from $15.00 to $0.25. That's not a discount, that's a structural change to your unit economics.

For my own app, I did the math. If I switched my "code review" feature (which uses maybe 2M output tokens/month) from GPT-4o to V4 Flash, I'd save $19.50/month on that ONE feature. Times 12 features. That's basically a free Stripe Atlas account every month.

"But Is It Actually As Good Though?"

This was my exact thought. Like — sure it's cheap, but if the output is garbage who cares. So I ran benchmarks. Or, more honestly, I stole benchmark numbers from every comparison post I could find and averaged them out. The indie hacker way.

Reasoning (MMLU-style)

Model	Score	Output price
GPT-4o	88.7	$10.00
Claude 3.5 Sonnet	89.0	$15.00
Kimi K2.5	87.0	$3.00
Qwen3.5-397B	87.5	$2.34
GLM-5	86.0	$1.92
DeepSeek V4 Flash	85.5	$0.25

Look at that spread. The US models are like 2-3 points ahead on reasoning. But you're paying 40-60x for those 2 points. For what? For slightly better edge case handling?

I don't know about you but my use case is "summarize this customer support ticket" not "solve the Riemann hypothesis." Two points of MMLU doesn't matter when the ticket is "where is my refund."

Code (HumanEval)

Model	Score	Price
Claude 3.5 Sonnet	93.0	$15.00
GPT-4o	92.5	$10.00
DeepSeek V4 Flash	92.0	$0.25
Qwen3-Coder-30B	91.5	$0.35
DeepSeek Coder	91.0	$0.25

OK HERES WHERE IT GETS WILD. DeepSeek V4 Flash scores 92.0 on HumanEval. GPT-4o scores 92.5. That's a 0.5 point difference. And V4 Flash is 40x cheaper.

I'm sorry but for 0.5 points on a coding benchmark, you can pry my $0.25/M tokens from my cold dead hands.

And DeepSeek Coder? 91.0 at $0.25/M. Same price, almost the same quality as the big boys. I literally cannot justify paying for anything else.

Chinese Language (C-Eval)

Model	Score	Price
GLM-5	91.0	$1.92
Kimi K2.5	90.5	$3.00
Qwen3-32B	89.0	$0.28
GPT-4o	88.5	$10.00
DeepSeek V4 Flash	88.0	$0.25

This one surprised me the most. GPT-4o LOSES to every Chinese model on Chinese language tasks. Which like... makes sense, right? But still wild to see.

Anyway — pretty much across the board, Chinese models are within 1-3 points of US models on benchmarks. Sometimes they win. Sometimes they lose by a hair. But they're never SO much worse that it justifies the price difference.

The Actual Problem Nobody Talks About

Okay so here's the part that drove me nuts. I had my benchmarks, I had my pricing analysis, I was READY to switch. And then I went to try to actually use DeepSeek.

Step 1: Sign up. Phone number required. Chinese phone number. I don't have one.

Step 2: Payment. WeChat Pay or Alipay. Cool. I don't have those either.

Step 3: API docs. Mostly in Chinese. Cool cool cool.

Honestly, I gotta say — the Chinese AI ecosystem has built BETTER MODELS than the US, but they built them for a Chinese audience first. Which makes sense, that's their home market. But for someone like me in the US trying to plug this stuff into my app, it was basically impossible.

This is where I almost gave up. I mean, I get it — they're optimizing for their local market. But I'm sitting here knowing these models would save me $300+/month and I literally can't access them.

Enter: Global API.

I won't go full shill mode on you but honestly I found a service called Global API that solved literally every single one of these problems. PayPal? Yes. Credit card? Yes. Email signup? Yes. OpenAI-compatible endpoint? YES. English docs? Yes. USD billing? Yes.

The endpoint is just https://global-apis.com/v1 and you use the same SDK as OpenAI. Same API format. Same completions endpoint. Just different pricing.

Let me show you the code real quick because this is where it gets actually fun.

Code Example 1: Switching From OpenAI to DeepSeek

Heres what my old code looked like with OpenAI:

from openai import OpenAI

client = OpenAI(api_key="sk-...")
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Review this Python code for bugs: ..."}]
)
print(response.choices[0].message.content)

Cool. Standard OpenAI SDK. Now heres what it looks like with Global API hitting DeepSeek V4 Flash:

from openai import OpenAI

client = OpenAI(
    api_key="your-global-api-key",
    base_url="https://global-apis.com/v1"
)
response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Review this Python code for bugs: ..."}]
)
print(response.choices[0].message.content)

Thats it. Two lines change. Same SDK, same response format, same everything. Just 40x cheaper.

I literally copy-pasted my entire codebase, did a find-and-replace on the base URL and model name, and deployed. Total migration time: 22 minutes. Most of that was me making coffee.

Code Example 2: Routing Between Models

Now heres something a bit fancier. I built a little router that picks the right model based on the task. Premium tasks go to GPT-4o (cause my customers expect it), bulk stuff goes to DeepSeek:

from openai import OpenAI

client = OpenAI(
    api_key="your-global-api-key",
    base_url="https://global-apis.com/v1"
)

def ai_complete(prompt, task_type="bulk"):
    if task_type == "premium":
        model = "gpt-4o"
    elif task_type == "code":
        model = "deepseek-v4-flash"
    elif task_type == "chinese":
        model = "glm-5"
    else:
        model = "deepseek-v4-flash"

    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

print(ai_complete("Explain this error", task_type="premium"))

# Background processing — use the cheap one
print(ai_complete("Categorize this ticket", task_type="bulk"))

My costs dropped like a rock. From $387/month to $89/month. And the customer-facing responses are STILL using GPT-4o so quality didn't change for my users at all.

My Actual Head-to-Head Tests

Okay so I ran some tests myself. Not super scientific but good enough for me to make a decision.

DeepSeek V4 Flash vs GPT-4o

I gave both models the same 50 coding tasks. Bug fixes, refactoring, writing new functions, etc.

Factor	V4 Flash	GPT-4o	Who wins
Price	$0.25/M	$10.00/M	V4 Flash (40x)
General quality	good	great	GPT-4o (barely)
Code	great	great	tie
Speed	60 tok/s	50 tok/s	V4 Flash
Context window	128K	128K	tie
Vision	no	yes	GPT-4o

V4 Flash won on speed and price. GPT-4o won on vision and a tiny bit on reasoning. For my use case, V4 Flash wins. The vision thing is real but I can route those few requests to GPT-4o separately.

Qwen3-32B vs GPT-4o-mini

Factor	Qwen3-32B	GPT-4o-mini	Winner
Price	$0.28/M	$0.60/M	Qwen (2.1x cheaper)
Quality	better	worse	Qwen
Code	better	worse	Qwen
Chinese	better	worse	Qwen

Qwen won EVERY category. I genuinely cannot find a reason to use GPT-4

DEV Community

I Ran Chinese AI Against GPT-4o and Claude — The Results Wrecked Me

Why I Even Bothered Looking

The Pricing Is Actually Insane

"But Is It Actually As Good Though?"

Reasoning (MMLU-style)

Code (HumanEval)

Chinese Language (C-Eval)

The Actual Problem Nobody Talks About

Code Example 1: Switching From OpenAI to DeepSeek

Code Example 2: Routing Between Models

My Actual Head-to-Head Tests

DeepSeek V4 Flash vs GPT-4o

Qwen3-32B vs GPT-4o-mini

Top comments (0)