I Ran 10 Coding LLMs Through 5 Tasks — Here's What the Data Says

I Ran 10 Coding LLMs Through 5 Tasks — Here's What the Data Says

Let me be upfront with you: I'm a data person. When someone tells me "Model X is the best for code," my first instinct is to ask for the sample size, the test conditions, and how they scored it. So when I saw a flood of "best AI coding model" articles online — most of them with zero methodology, no scoring rubric, and suspiciously confident rankings — I decided to run my own benchmark.

Over the course of two weekends (and way too much coffee), I put 10 different LLMs through a structured coding test. I'm going to walk you through exactly what I found, what the numbers say, and where I think the real winners hide. If you stick around, I'll also show you how to call any of these models through a single endpoint, which honestly saved me from juggling ten different API accounts.

The Lineup

Before I get into the methodology, here's what I was testing. I tried to pick a mix of dedicated code models, reasoning-heavy models, and general-purpose workhorses. All prices below are output costs per million tokens (the number that actually hits your wallet when you generate code):

#	Model	Provider	Output $/M	Category
1	DeepSeek V4 Flash	DeepSeek	$0.25	General (strong code)
2	DeepSeek Coder	DeepSeek	$0.25	Code-specialized
3	Qwen3-Coder-30B	Qwen	$0.35	Code-specialized
4	DeepSeek V4 Pro	DeepSeek	$0.78	Premium general
5	DeepSeek-R1	DeepSeek	$2.50	Reasoning (code thinking)
6	Kimi K2.5	Moonshot	$3.00	Premium general
7	GLM-5	Zhipu	$1.92	Premium general
8	Qwen3-32B	Qwen	$0.28	General purpose
9	Hunyuan-Turbo	Tencent	$0.57	General purpose
10	Ga-Standard	GA Routing	$0.20	Smart routing

The price spread here is wild — we're looking at a 15x difference between the cheapest and most expensive option. That alone is reason enough to test rigorously. Most of you reading this probably don't have unlimited budget, and even if you did, you'd want to know if the premium models are actually worth 15x more.

How I Ran the Tests

I'm a stickler for methodology, so let me be transparent about exactly what I did. Each model received the same five tasks, scored blindly by me on a 1-10 scale based on four criteria: correctness, code quality, documentation, and edge-case handling. I scored without knowing which model produced which output until the end (mostly — I'm human, some signatures are obvious).

The five tasks were:

1. Function Implementation — flatten a nested list recursively in Python
2. Bug Fix — debug a JavaScript async/await race condition
3. Algorithm — implement Dijkstra's shortest path in TypeScript
4. Code Review — security and performance review on a Go snippet
5. Full Feature — build a paginated, filtered REST API in Express.js

Sample size is admittedly modest — one run per task per model — so treat individual differences of less than 0.5 points as noise. The overall rankings, but, showed strong correlations across tasks, which gives me more confidence there.

The Overall Numbers

Here are the aggregated scores. The "Value" column is score divided by output price — basically, points per dollar. It's the metric I personally care about most:

Rank	Model	Avg Score	Price	Value Score
🥇	Qwen3-Coder-30B	8.8	$0.35	25.1
🥈	DeepSeek V4 Flash	8.7	$0.25	34.8
🥉	DeepSeek Coder	8.6	$0.25	34.4
4	DeepSeek V4 Pro	9.1	$0.78	11.7
5	DeepSeek-R1	9.4	$2.50	3.8
6	Kimi K2.5	9.0	$3.00	3.0
7	Qwen3-32B	8.3	$0.28	29.6
8	GLM-5	8.0	$1.92	4.2
9	Hunyuan-Turbo	7.5	$0.57	13.2
10	Ga-Standard	8.5*	$0.20	42.5*

*Ga-Standard is a routing layer, so its score varies depending on which underlying model it dispatches to. Treat that asterisk accordingly.

The correlation between raw score and price? Surprisingly weak — about 0.31 across the sample. The correlation between value score and price? Obviously strongly negative, by construction. But the interesting statistical story is this: the top three by raw score (DeepSeek-R1 at 9.4, DeepSeek V4 Pro at 9.1, Kimi K2.5 at 9.0) cost between $0.78 and $3.00 per million output tokens, while the best-value models (Ga-Standard, DeepSeek V4 Flash, DeepSeek Coder) all sit at or below $0.25. You're paying roughly 10x more for about 0.7 points of quality. Whether that's worth it depends entirely on your use case — but for the vast majority of everyday coding tasks, I'd argue it isn't.

Task 1: Flattening a Nested List

This was the warm-up — Python, recursive, classic. Almost every model nailed it, which is honestly what I'd expect. Here's what stood out:

Model	Score	What I Noticed
DeepSeek V4 Flash	9.0	Clean recursive solution with type hints
Qwen3-Coder-30B	9.0	Added an iterative alternative + edge cases
DeepSeek Coder	8.5	Correct, but more verbose than necessary
Kimi K2.5	9.0	Most readable output, included docstring
DeepSeek-R1	9.5	Included Big-O analysis and explained tradeoffs

DeepSeek-R1's 9.5 here was the first hint that reasoning models earn their keep on educational tasks — the chain-of-thought output genuinely explained why you'd choose recursion vs. iteration. For a junior dev learning the language, that context is gold. For shipping a one-liner to production? Probably overkill.

Task 2: The Async/Await Race Condition

This one's fun because the bug is subtle and the fix matters. The buggy code:

let data = null;
fetch('/api/data').then(r => r.json()).then(d => data = d);
console.log(data); // Always logs null — classic race condition

Every model I tested correctly identified the issue. The differentiator was how they explained it and what fix they offered:

Model	Score	What I Noticed
DeepSeek V4 Flash	9.0	Clear explanation + three fix options
Qwen3-Coder-30B	9.0	Added proper error handling
DeepSeek Coder	8.5	Correct fix, minimal explanation
Qwen3-32B	8.5	Good fix, slightly verbose

I called this one a tie between DeepSeek V4 Flash and Qwen3-Coder-30B because both delivered production-ready solutions — the kind you'd actually merge without rewriting. Here's roughly what Qwen3-Coder-30B generated:

async function fetchData() {
  try {
    const response = await fetch('/api/data');
    if (!response.ok) throw new Error(`HTTP ${response.status}`);
    const data = await response.json();
    console.log(data);
    return data;
  } catch (error) {
    console.error('Fetch failed:', error);
    throw error;
  }
}
fetchData();

Clean, correct, handles errors. What more do you want?

Calling These Models in Practice

Here's where things get practical. One of the annoyances of this whole experiment was juggling credentials and SDKs across providers. I started routing everything through Global API at global-apis.com/v1 — same OpenAI-compatible interface, so my existing Python code worked unchanged. Here's what my test harness looked like:

import openai

client = openai.OpenAI(
    api_key="your-global-api-key",
    base_url="https://global-apis.com/v1"
)

def score_task(model_name, prompt):
    response = client.chat.completions.create(
        model=model_name,
        messages=[
            {"role": "system", "content": "You are a precise, senior software engineer."},
            {"role": "user", "content": prompt}
        ],
        temperature=0.2,
        max_tokens=1500
    )
    return response.choices[0].message.content

models = [
    "deepseek-v4-flash",
    "qwen3-coder-30b",
    "deepseek-r1",
    "kimi-k2.5",
    "glm-5",
]

dijkstra_prompt = """
Implement Dijkstra's shortest path algorithm in TypeScript.
Requirements:
- Use a priority queue
- Include proper types (no 'any')
- Handle disconnected graphs gracefully
- Include unit tests with 3 cases
"""

for model in models:
    output = score_task(model, dijkstra_prompt)
    print(f"\n{'='*60}\n{model}\n{'='*60}\n{output[:2000]}")

The OpenAI SDK format is a beautiful thing — once you've written one integration, you've written them all. Being able to swap deepseek-v4-flash for deepseek-r1 by changing one string saved me hours during this benchmark.

The Value Math Nobody Else Shows You

Let me put this in concrete terms. Suppose you're a small team generating roughly 5 million output tokens of code per month (totally plausible for a startup with active development):

Model	Monthly Cost	Annual Cost	Quality vs. Top
DeepSeek V4 Flash	$1.25	$15	92.6%
DeepSeek Coder	$1.25	$15	91.5%
Ga-Standard	$1.00	$12	90.4%
Qwen3-Coder-30B	$1.75	$21	93.6%
Qwen3-32B	$1.40	$17	88.3%
DeepSeek V4 Pro	$3.90	$47	96.8%
GLM-5	$9.60	$115	85.1%
DeepSeek-R1	$12.50	$150	100%
Kimi K2.5	$15.00	$180	95.7%

Read that table again. DeepSeek-R1