Alibaba dropped Qwen3.8-Max this week with a blog post titled "A New Bar for Coding and Cowork." Bold claim. 2.4 trillion parameters. Second only to Fable 5 on internal evals. Beats GPT-5.6 Sol across seven evaluations.
Cool story. Where's the benchmark table?
There isn't one. Go read the launch post yourself. Every number being quoted in comparison posts right now — "beats GPT-5.5 by 7-10 Elo," "ahead of GPT-5.6, slightly behind Fable 5" — is a vendor claim, not a published, reproducible score. No SWE-bench Verified number. No SWE-bench Pro number. Nothing you can independently check.
For context, the previous generation, Qwen3.7-Max, actually did publish: 80.4 on SWE-bench Verified, 60.6% on SWE-bench Pro. Those numbers are out there, other people have tried to reproduce them, you can argue about methodology. Qwen3.8-Max skipped that step entirely and went straight to marketing copy.
The actual spec sheet
Strip out the benchmark theater and here's what's real:
- 2.4T total parameters, 95B activated — mixture-of-experts, so you're not paying full freight on every token
- 1M token context window, one flat pricing tier for the whole thing
- $2 / $6 per million tokens (input/output) at general availability
- Reasoning defaults to "xhigh" effort, and thinking tokens bill as output — so that $6 number is quietly higher in practice than it looks on the tin
- Open weights landing on Hugging Face and ModelScope within the week
The pricing is the actual headline here, not the Elo score nobody can verify. GPT-5.6-class output pricing has been sitting well above $6/M on the frontier tier. If Qwen3.8-Max holds up at even 85% of frontier coding quality at a third of the cost, that's the story — not "beats GPT on a benchmark Alibaba made up."
What happens when you actually run it
This is where it gets interesting, and where the vendor claims start to wobble.
One dev did a head-to-head on a non-trivial HTML/CSS generation task — the kind of "build me this layout" prompt that separates models that pattern-match from models that reason about layout constraints. Claude Opus finished in 16 minutes. Qwen3.8-Max took over 2 hours of shepherding to get to something usable.
That's not a benchmark. That's one task. But it's the kind of gap that a 7-10 Elo point win on an internal eval conveniently doesn't capture. Elo deltas that small are noise-level on any given real task — they tell you about aggregate tendencies across thousands of prompts, not about whether the model can one-shot your actual layout.
Meanwhile, on the local-deployment side, people are genuinely happy. Multiple reports of 45 tok/s on an M1 Mac Studio, 20+ tok/s quantized on more modest hardware, and at least one dev who canceled their Claude subscription and now runs Qwen3.6-35B as a daily driver for routine coding work. That's a real, defensible use case: not "beats the frontier," but "good enough, private, and free after the hardware cost."
The pattern you should recognize by now
Every few weeks a Chinese lab ships a model with:
- A blog post claiming near-frontier performance
- Zero published benchmark tables backing the headline number
- Genuinely aggressive pricing
- A promise of open weights "next week"
- A comment section split between "I canceled my Claude subscription" and "this fell over on my actual codebase"
Qwen3.7-Max did this. Qwen3.6-Max-Preview did this. Now Qwen3.8-Max is doing it again. The open-weights part is real and it matters — when the 27B/95B-active variants land on Hugging Face, you'll get actual reproducible numbers from people who don't have a stake in the marketing. That's when this gets worth your time.
Until then, here's what I'd actually do with it:
# Sane way to evaluate this yourself instead of trusting either
# the vendor post or the hype thread: run YOUR test suite against it.
import openai
client = openai.OpenAI(
base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
api_key="YOUR_KEY",
)
response = client.chat.completions.create(
model="qwen3.8-max",
messages=[{"role": "user", "content": "Your actual failing test case here"}],
reasoning_effort="medium", # don't default to xhigh, you're paying for those tokens
)
print(response.choices[0].message.content)
Set reasoning_effort yourself. Don't let it default to xhigh and bill you for thinking tokens on tasks that don't need them — that's the single most concrete piece of advice in this whole post, and cheaper than any Elo argument.
Bottom line
Qwen3.8-Max is a genuinely interesting pricing and open-weights play wrapped in a benchmark claim that nobody can verify yet. The MoE efficiency is real. The context window is real. The $2/$6 pricing is real and worth testing against your own workload. "A new bar for coding" is copy, not a citation.
Wait for the open weights, run it against your own repo, and ignore every comparison post — including the ones linked above — until someone posts a benchmark methodology you can actually reproduce.
Sources:
Top comments (0)