DEV Community

Aditi Gupta
Aditi Gupta

Posted on • Originally published at honestaireview.hashnode.dev

Qwen3.8-Max Beat Claude on Three Benchmarks. The Full Story Is Less Convenient

Alibaba has released Qwen3.8-Max, its largest and most capable AI model to date.

The headline specifications are absurd: 2.4 trillion total parameters, 95 billion activated per token, multimodal input, and a one-million-token context window.

But the specification sheet is not the most interesting part.

Alibaba says Qwen3.8-Max operated autonomously for roughly 16 days, starting with an empty repository and building a working software project through issues, code changes, testing, pull requests, and self-correction.

That sounds impressive. It also sounds suspiciously like the sort of claim that deserves more inspection than a celebratory repost.

So I examined Alibaba's announcement, the public repository, and its benchmark results to answer a more useful question:

Does Qwen3.8-Max actually beat Claude, or did the benchmark department simply have an excellent week?

Prefer the two-minute version?

TL;DR

  • Qwen3.8-Max is a mixture-of-experts model with 2.4 trillion total parameters and 95 billion active per token.
  • Alibaba reports a one-million-token context window and support for text, images, and agentic workflows.
  • It beats the Claude models in Alibaba's table on Terminal Bench 2.1, PaperBench, and OSWorld-Verified.
  • It trails Claude Fable 5 on demanding repository-level engineering benchmarks such as SWE-bench Pro and FrontierSWE.
  • Alibaba says the model weights will be released next week. At the time of writing, the API is available, but the weights are not yet downloadable.
  • The 16-day coding trace is public, but the results still come from Alibaba's own evaluation setup and need independent replication.

What is Qwen3.8-Max?

Qwen3.8-Max is Alibaba's new flagship mixture-of-experts model.

It contains 2.4 trillion parameters in total, with approximately 95 billion active during each forward pass. This architecture allows Alibaba to scale the model's capacity without paying the full inference cost of a dense 2.4-trillion-parameter model on every token.

The model also supports a one-million-token context window, making it suitable for large repositories, long documents, persistent agent sessions, and other tasks where context compression usually arrives carrying a shovel.

Alibaba calls it the first Qwen model at Max scale that will receive an open-weight release. However, there is an important distinction:

The API is available now. The model weights are scheduled for release next week.

So calling it an "open-weight model" is reasonable when discussing Alibaba's release plan, but saying the weights are already available would be inaccurate.

You can find the specifications and release details in the official Qwen3.8-Max announcement.

The 16-day autonomous coding run

Alibaba asked Qwen3.8-Max to create a project called oh-my-cli from an empty repository.

Instead of responding to a single prompt and stopping, the model worked through a continuous engineering loop:

  1. Convert feedback and requirements into GitHub issues.
  2. Claim and execute individual tasks.
  3. Write and modify code.
  4. Run builds, unit tests, end-to-end tests, and lifecycle checks.
  5. Route failures back into the issue workflow.
  6. Fix the problems and verify the result.
  7. Merge completed pull requests.

According to Alibaba, the repository had accumulated 265 commits, 127 pull requests, and 151 issues after approximately 16 days of autonomous operation.

The complete project history is available in the public oh-my-cli GitHub repository.

That transparency matters. Most autonomous-agent demonstrations give us a polished video and ask us to believe that nothing caught fire outside the frame. Here, developers can inspect the issues, commits, pull requests, tests, and failures.

Still, this does not prove that the model can autonomously build any production system for 16 days. It proves that Qwen3.8-Max performed this particular task inside a structured environment with automated testing and feedback loops.

That is still meaningful, just narrower than the marketing headline.

Where Qwen3.8-Max beats Claude

Alibaba published a large benchmark table comparing Qwen3.8-Max with Claude Opus 4.8, Claude Fable 5, GPT-5.6 Sol, and Qwen3.7-Max.

Here are the most relevant results:

Benchmark Qwen3.8-Max Claude Opus 4.8 Claude Fable 5 Result
Terminal Bench 2.1 86.6 84.6 84.6 Qwen leads
PaperBench 93.0 80.3 88.8 Qwen leads
OSWorld-Verified 86.1 83.4 85.0 Qwen leads
SWE-bench Pro 67.7 69.2 80.0 Claude leads
FrontierSWE 73.5 70.0 88.8 Mixed; Fable 5 leads

These results suggest three areas where Qwen3.8-Max looks particularly strong.

1. Terminal-based agent work

Its 86.6 score on Terminal Bench 2.1 puts it ahead of both Claude models in Alibaba's comparison.

That makes Qwen especially interesting for command-line agents, environment setup, testing, deployment workflows, and tasks that require repeated tool use rather than a single code-generation response.

2. Research reproduction

Qwen3.8-Max scored 93.0 on PaperBench, ahead of Claude Fable 5's 88.8 and Opus 4.8's 80.3.

Alibaba also demonstrated a five-day research task in which the model reproduced a paper's experimental pipeline, ran 33 rounds of GPU training, and then searched for improvements to the original method.

This is potentially more useful than another model becoming marginally better at generating React components nobody requested.

3. Computer and visual interaction

On OSWorld-Verified, which evaluates an agent's ability to operate computer environments, Qwen3.8-Max scored 86.1.

The model uses visual output as part of its feedback loop. It can inspect an interface, identify errors, revise its plan, and try again. That matters for browser agents, desktop automation, document workflows, UI testing, and multimodal development.

Where Claude still wins

The "Qwen kills Claude" headline falls apart once we examine harder repository-level engineering tasks.

On SWE-bench Pro, Qwen3.8-Max scored 67.7. Claude Fable 5 scored 80.0.

On FrontierSWE, Qwen scored 73.5 while Fable 5 reached 88.8.

That is not a rounding error. It suggests Claude remains stronger when a task requires deep repository understanding, architectural judgment, and reliable changes across a complicated codebase.

The more honest conclusion is:

  • Qwen looks excellent at long-running, tool-heavy work.
  • It performs strongly on terminal, research, multimodal, and computer-use tasks.
  • Claude Fable 5 remains ahead on some of the hardest software-engineering benchmarks.
  • Neither model "wins" every category, because reality rudely refuses to fit inside one thumbnail.

There is another caveat: these scores come from Alibaba's evaluation table. Different benchmarks used different harnesses, time limits, context settings, and judging methods. The numbers are useful, but independent testing will matter more than launch-day charts.

How to use Qwen3.8-Max with Claude Code

QwenCloud provides an Anthropic-compatible API, allowing Claude Code to use Qwen3.8-Max without replacing the Claude Code interface.

First, install Claude Code:

npm install -g @anthropic-ai/claude-code
Enter fullscreen mode Exit fullscreen mode

Then configure it to use Qwen:

export ANTHROPIC_MODEL="qwen3.8-max"
export ANTHROPIC_SMALL_FAST_MODEL="qwen3.8-max"
export ANTHROPIC_BASE_URL="https://dashscope-intl.aliyuncs.com/apps/anthropic"
export ANTHROPIC_AUTH_TOKEN="YOUR_QWEN_API_KEY"

claude
Enter fullscreen mode Exit fullscreen mode

You will need a QwenCloud API key. The international endpoint may differ depending on your account or deployment region, so check the current QwenCloud documentation before configuring it.

And please do not paste your real API key into a public DEV article. Becoming an involuntary cloud-compute philanthropist is rarely part of the content strategy.

Can you run it locally?

Not casually.

Although only 95 billion parameters are active during each pass, the full model contains 2.4 trillion parameters. Open weights do not magically convert that into something your laptop can run between Chrome tabs.

Once the weights are released, practical deployment will likely require substantial multi-GPU infrastructure, aggressive quantization, or a hosted inference provider.

For most individual developers, QwenCloud will be the realistic way to use the full model.

The verdict

Qwen3.8-Max does not kill Claude.

It does something more consequential: it brings frontier-scale agent capabilities closer to the open-weight ecosystem.

Its strongest argument is not a single benchmark score. It is the combination of long-horizon execution, terminal performance, multimodal feedback, research reproduction, and a public 16-day development trace.

Based on the evidence available today:

  • I would consider Qwen3.8-Max for long-running agents, terminal workflows, research automation, visual tasks, and jobs that benefit from repeated feedback.
  • I would still prefer Claude Fable 5 for the hardest repository-level engineering work, especially when first-pass reliability matters.
  • I would wait for independent evaluations before treating Alibaba's benchmark table as the final verdict.

Qwen3.8-Max is not the model that makes Claude irrelevant.

It is the model that makes the frontier race significantly less comfortable, and that is far more interesting.

Have you tested Qwen3.8-Max in QwenCloud or Claude Code? Share the task, harness, and result in the comments. "It felt smarter" is emotionally valid, but logs are sexier.

Sources

Top comments (0)