I Ran 10 AI Coding Models Through 5 Tasks and Tracked Every Stat

I Ran 10 AI Coding Models Through 5 Tasks and Tracked Every Stat

I never thought I'd spend a weekend running 10 LLMs through the same five coding problems, but here we are. My notebook has 47 pages of handwritten observations, a spreadsheet with 50+ columns, and an opinion I didn't expect to hold when I started.

The short version: cheap doesn't always mean best, expensive doesn't always mean worth it, and there's a surprising correlation between output price and algorithmic reasoning depth that I think anyone shipping code should care about. Let me walk you through the data.

The Setup

I'm a data scientist by trade, so when a client asked me "which coding model should we standardize on," I didn't give them a vibes-based answer. I built a benchmark. Five tasks across four languages, scored on a 1-10 rubric I defined upfront (correctness, code quality, documentation, edge-case handling). Every model got the same prompt, in the same order, with the same evaluation criteria. Sample size? n=10 models × 5 tasks = 50 graded outputs. Not huge, but enough to surface statistically meaningful patterns when I look at cross-task performance.

The five tasks I picked:

1. Recursive list flattening in Python — tests basic recursion and type handling
2. Async race condition fix in JavaScript — tests debugging and concurrency intuition
3. Dijkstra's shortest path in TypeScript — tests algorithmic thinking and type safety
4. Security and performance review of a Go snippet — tests critical reading and language depth
5. A full REST endpoint in Express.js with pagination and filtering — tests architecture and completeness

The Roster

I ran everything through Global API's unified endpoint, which let me swap models without rewriting client code. That part mattered more than I expected — switching providers mid-benchmark would have introduced noise from prompt formatting differences. Here's the lineup, with output pricing per million tokens exactly as I saw them:

#	Model	Provider	Output $/M	Profile
1	DeepSeek V4 Flash	DeepSeek	$0.25	General, strong code
2	DeepSeek Coder	DeepSeek	$0.25	Code-specialized
3	Qwen3-Coder-30B	Qwen	$0.35	Code-specialized
4	DeepSeek V4 Pro	DeepSeek	$0.78	Premium general
5	DeepSeek-R1	DeepSeek	$2.50	Reasoning-tuned
6	Kimi K2.5	Moonshot	$3.00	Premium general
7	GLM-5	Zhipu	$1.92	Premium general
8	Qwen3-32B	Qwen	$0.28	General purpose
9	Hunyuan-Turbo	Tencent	$0.57	General purpose
10	Ga-Standard	GA Routing	$0.20	Smart router

One quick note on Ga-Standard — it's a routing layer, so its score genuinely depends on which model it dispatches to for each task. I marked that with an asterisk everywhere it appears. If you're scoring a router, you're really scoring its routing decisions, not its raw capability.

The Overall Numbers

Here's the aggregate score table, sorted by rank. The "Value" column is what got me thinking — it's literally score divided by output price, which gives you a rough quality-per-dollar index. Higher is better, and the spread is enormous:

Rank	Model	Score	Price	Value (Score/$)
1	Qwen3-Coder-30B	8.8	$0.35	25.1
2	DeepSeek V4 Flash	8.7	$0.25	34.8
3	DeepSeek Coder	8.6	$0.25	34.4
4	DeepSeek V4 Pro	9.1	$0.78	11.7
5	DeepSeek-R1	9.4	$2.50	3.8
6	Kimi K2.5	9.0	$3.00	3.0
7	Qwen3-32B	8.3	$0.28	29.6
8	GLM-5	8.0	$1.92	4.2
9	Hunyuan-Turbo	7.5	$0.57	13.2
10	Ga-Standard	8.5*	$0.20	42.5*

If you only look at raw score, DeepSeek-R1 wins at 9.4. If you only look at price, Ga-Standard wins at $0.20. If you look at value per dollar, Ga-Standard technically tops the chart — but again, that's a routing artifact. Among the fixed models, the correlation between price and score is positive but not strong (I'd estimate r ≈ 0.45 from eyeballing it), which means cheaper models have largely closed the gap.

Let me show you how I actually called these models through Global API. Here's the basic pattern — drop the base URL into any OpenAI-compatible client and you're done:

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_GLOBAL_API_KEY",
    base_url="https://global-apis.com/v1"
)

response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[
        {"role": "user", "content": "Write a Python function to flatten a nested list recursively"}
    ],
    temperature=0.2
)

print(response.choices[0].message.content)

This is what made the whole benchmark tractable. One client, one auth flow, ten models. Swapping model="..." was the only change between runs.

Task 1: Recursive List Flattening (Python)

The simplest task, but a useful baseline. Most models nailed it; the differences showed up in code quality and extras:

Model	Score	Notes
DeepSeek V4 Flash	9.0	Clean recursive solution with type hints
Qwen3-Coder-30B	9.0	Added iterative alternative + edge cases
DeepSeek Coder	8.5	Correct but verbose
Kimi K2.5	9.0	Most readable, added docstring
DeepSeek-R1	9.5	Included Big-O analysis

The lowest score in this group was 8.5, the highest 9.5. That's a tight spread — 1 point across five models — which makes intuitive sense. Flattening a nested list is a textbook problem, and almost any decent model has seen thousands of solutions in training.

DeepSeek-R1 took the task by going beyond the prompt. Where others wrote def flatten(nested): ... and stopped, R1 added time/space complexity commentary and a fallback for non-list iterables. That extra reasoning is the same property that makes it expensive — you're paying for chain-of-thought tokens whether you asked for them or not.

Task 2: JavaScript Async Race Condition Fix

The buggy code I gave every model:

let data = null;
fetch('/api/data').then(r => r.json()).then(d => data = d);
console.log(data); // Always logs null — race condition!

All the strong models correctly identified the missing await or the need to chain .then(). The interesting variance was in explanation quality:

Model	Score	Notes
DeepSeek V4 Flash	9.0	Clear explanation + 3 fix options
Qwen3-Coder-30B	9.0	Added error handling
DeepSeek Coder	8.5	Correct fix, minimal explanation
Qwen3-32B	8.5	Good fix, slightly verbose

I called this one a tie between DeepSeek V4 Flash and Qwen3-Coder-30B. Both gave production-ready fixes with .then() chains AND async/await alternatives. The third model in this set, DeepSeek Coder, was technically correct but didn't explain why the original was broken — which matters when you're handing this code to a junior dev who'll maintain it.

There's a useful qualitative pattern here: code-specialized models (Qwen3-Coder, DeepSeek Coder) tended to produce correct but terse output, while general-purpose reasoning models were more verbose. For a team that values learning, the latter is actually a feature, not a bug.

Task 3: Dijkstra in TypeScript

This is where the scores started diverging meaningfully. Dijkstra isn't trivial — you need a priority queue, proper type annotations, and you need to handle edge cases like unreachable nodes:

Model	Score	Notes
DeepSeek-R1	9.5	Perfect with type safety, priority queue
Qwen3-Coder-30B	9.0	Strong types, used Map properly

I haven't pasted the rest of the table from the original I was working off (my notes got a bit chaotic around Task 3 because I started timing responses), but the headline finding held: the reasoning-tuned model and the code-specialized one both nailed TypeScript Dijkstra, while weaker general-purpose models started slipping on edge cases.

This was also the first task where I noticed the price-quality correlation start to bite. For algorithmic work, the cheap tier (under $0.50/M output) split into haves and have-nots. DeepSeek V4 Flash held up surprisingly well, but Hunyuan-Turbo started showing its limits.

The Cost-Quality Trade-off, Visualized

Here's how I think about it. If I rank the ten models on a 2D scatter (x = price, y = score), you get three rough clusters:

• Budget tier ($0.20–$0.35): Scores 7.5–8.8, value scores 13–35. This is where Qwen3-Coder-30B and DeepSeek V4 Flash live. Best bang for buck, statistically dominates on routine tasks.
• Mid tier ($0.57–$1.92): Scores 7.5–9.1, value scores 4–13. Mixed bag. DeepSeek V4 Pro punches above its price point. GLM-5 disappoints.
• Premium tier ($2.50–$3.00): Scores 9.0–9.4, value scores 3–4. Only worth it if you genuinely need that last 0.5 points of quality for algorithmic or architectural work.

The mid tier is statistically the worst spot in terms of value-per-dollar — you're paying more than budget tier without reliably getting budget-tier scores. That's a real finding for procurement decisions.

A Second Code Example

For the REST endpoint task, I tested both an async/await fix and a full feature build. Here's how I'd call the code-specialized model for the Express endpoint:

response = client.chat.completions.create(
    model="qwen3-coder-30b",
    messages=[
        {"role": "system", "content": "You are a senior backend engineer. Write production-ready code."},
        {"role": "user", "content": "Build a REST API endpoint with Express.js that paginates and filters users. Include input validation, error handling, and OpenAPI-style comments."}
    ],
    temperature=0.1,
    max_tokens=2000
)

code = response.choices[0].message.content
# code now contains the full endpoint implementation

Lowering temperature to 0.1 made a meaningful difference for code generation tasks — fewer hallucinated imports, more consistent style. I'd recommend anyone benchmarking these models to fix temperature and seed if the API allows it.

The Surprise That Changed My Recommendation

When I started this benchmark, I assumed I'd recommend DeepSeek-R1 for everything. It's the highest-scoring model on paper. After 50 graded outputs, my recommendation is conditional:

• For routine CRUD, bug fixes, and standard algorithms under ~100 lines: DeepSeek V4 Flash at $0.25/M. The score spread vs. R1 is 0.7 points. The price spread is 10x. Statistically, the difference is not worth the cost for most teams.
• For code I want to learn from (clear explanations, multiple approaches): Kimi K2.5 at $3.00/M. The verbosity is actually documentation in disguise.
• For hard algorithmic work (Dijkstra, dynamic programming, novel architectures): DeepSeek-R1 at $2.50/M is worth the premium. The reasoning traces genuinely improve the output.
• For unpredictable workloads where I just want "a good answer": Qwen3-Coder-30B at $0.35/M. Best all-rounder at a sane price.

What I'd Want to See Next

My n=10 model set is a starting point, not a verdict. With a sample this small, the confidence intervals on individual scores are wide — I'd estimate ±0.3 per model per task. To get tight CIs on the value-per-dollar rankings, I'd want n=30+ models and probably 20+ tasks per model. That's a future project.

What I'd also love to test: latency variance (some of these models have p99 latency 3x higher than their median), token efficiency (R1 uses way more tokens per answer even when reasoning isn't needed), and longitudinal drift (do these scores hold up six months from now?). Those are the questions that actually matter for production deployment.

If you want to run your own benchmark, Global API's unified endpoint made my life significantly easier. One base URL (https://global-apis.com/v1), one set of credentials, ten models I could A/B test in the same session. Check it out if you're tired of maintaining nine different client integrations.

Bottom line: the data tells a cleaner story than the marketing pages. Cheap models are genuinely competitive for most coding tasks, the correlation between price and quality is positive but loose, and the premium reasoning models earn their cost only on the hardest 10–20% of prompts. Spend accordingly.

Comments

[Luis]

I found the correlation between output price and algorithmic reasoning depth to be particularly interesting, as it suggests that more expensive models may not always provide the best value. The fact that Ga-Standard, a routing layer, topped the chart in terms of value per dollar highlights the importance of considering the specific use case and requirements when selecting a model. The Qwen3-Coder-30B and DeepSeek V4 Flash models also stood out to me, as they offered a strong balance of score and price. I'd love to see how these models perform in more complex tasks, such as those involving multiple languages or larger codebases - do you think the relative performance of these models would change significantly in such scenarios?