DEV Community

Cover image for Qwen 3.6 vs 3.5: Same 37 tok/s on RTX 4070, +43% on Frontend Generation
Ken Imoto
Ken Imoto

Posted on Originally published at zenn.dev

Qwen 3.6 vs 3.5: Same 37 tok/s on RTX 4070, +43% on Frontend Generation

The first number I saw on Qwen3.6-35B-A3B was 12 tok/s.

Same 37 tok/s on both models, +27% Terminal-Bench, +43% frontend generation

I almost hit publish on "Qwen regressed at generation speed" and moved on. The 3.5 baseline on the same RTX 4070 was 34.6 tok/s. A new generation running at a third of the old one would have been a hell of a headline. It was also completely wrong.

The culprit was not the model. Another process on the box was sitting on 9-11 GB of VRAM, so the layers that were supposed to live on the GPU were spilling to system RAM. The tell was that my sanity-check run of Qwen3.5 slowed down too. When two independent models degrade together, the model is not the variable.

I killed the offending process, re-measured, and got numbers that told a completely different story.

Model Generation speed tg128 (tok/s) Runs
Qwen3.6-35B-A3B 38.76 ± 0.82 avg of 3
Qwen3.5-35B-A3B 36.7 ± 1.4 avg of 3 (range 34.9-38.6)

Both models sit inside the ±1.5 tok/s band on the same RTX 4070. On the tokens-per-second axis, "the new generation" is not a story. Same architecture, same activated-parameter count (3B active out of 35B), same MoE routing pattern. The half-speed regression was a measurement bug, and it lived for about half a day before its own inconsistency killed it.

The lesson I keep re-learning: when the number you got is dramatically convenient for your narrative, measure it again before you write anything. The moment I could sell 12 tok/s as a regression, I should have been suspicious. The version of me that ran the second test earned the version of me that got to keep his self-respect.

So where did the generation move to?

If speed did not change, does the 3.5-to-3.6 bump mean anything? It does. The move lives on a different axis.

The official Qwen3.6-35B-A3B model card publishes benchmarks with a very lopsided shape:

Benchmark Qwen3.5 Qwen3.6 Lift
Terminal-Bench 2.0 40.5 51.5 +27%
QwenWebBench (frontend generation) 978 1,397 +43%
SWE-bench Pro 44.6 49.5 +11%
LiveCodeBench v6 74.6 80.4 +8%
SWE-bench Verified 70.0 73.4 +5%
AIME26 91.0 92.7 +2%
GPQA 84.2 86.0 +2%

Terminal operations: +27%. Frontend generation: +43%. Repository-scale coding tasks: +11%. Single-question knowledge probes like AIME and GPQA: +2%.

The pattern is almost too clean. Every benchmark that rewards tool-calling, long-context reasoning, and multi-turn execution moves double-digit percentages. Every benchmark that fits in one problem statement and one answer barely moves at all.

Why did AIME and GPQA plateau at +2%?

Benchmark saturation is the boring answer, and it is probably the right one. AIME and GPQA at the 90-point range are already near the ceiling of what the eval format can measure. A model that gets 91.0 on AIME26 is not being asked to demonstrate reasoning it cannot do. It is being asked whether it can grind out the arithmetic without slipping.

Terminal-Bench 2.0 and QwenWebBench are not saturated. They score long-horizon behavior: does the model recover from a shell error, does it wire the CSS classes to the right components, does it complete the task instead of writing a plan and stopping. These are the axes where a real capability gap still has room to show up.

Which reframes the release. Qwen3.6 is not a faster 3.5. It is the same footprint with more of the model's weight thrown at agent behavior: tool call stability, long context, and thinking control. If your workload is one-shot QA, you will not see the gap. If your workload is "read this repo and land a PR," you will.

The 7-question test where nothing showed up

To sanity-check the model card claims on my own hardware, I ran the same 7-question standard set from Chapter 5 against both models. Both scored 7/7.

Zero delta. If I had squinted, I could have talked myself into "3.6 gives more polished answers," and I actually started drafting exactly that. Then I put the 3.5 answers next to the 3.6 answers, and 3.5 was often the more thorough one (the capital-cities question, the WebRTC explanation). The comparison collapsed.

The reason is the same reason AIME plateaus. My 7-question set is saturated. When both models nail every question, the "generation gap" gets swallowed by run-to-run sampling noise. If you want to see 3.6 win over 3.5, you need problems where both models can still fail: long agent traces, unfamiliar repos, frontend layouts under a spec.

What this changes if you run local Qwen

Two takeaways I would actually act on:

  1. Do not upgrade to 3.6 for the tokens. On the same 12 GB VRAM budget with the same --cpu-moe MoE-offload setup, the tokens-per-second number is unchanged. If your bottleneck was throughput, 3.6 gives you nothing.
  2. Do upgrade to 3.6 if you were about to hand the model a repo. The +43% on frontend generation and +27% on Terminal-Bench are the numbers that matter for coding-agent, IDE-plugin, and CLI-agent workloads. The gap is real, and it is exactly where you want it if you are treating a local 35B as a workhorse instead of a chatbot.

The generation went sideways on speed and forward on autonomy. That is a more interesting release than "3.6 is 5% faster," even if it makes for a worse tweet.

Notes

  • Setup: RTX 4070 (12 GB VRAM), llama.cpp with -ngl 99 --cpu-moe, GGUF quantization from the lmstudio-community mirror. Measurement via llama-bench tg128.
  • Model card benchmarks are quoted directly from the Qwen3.6-35B-A3B Hugging Face card. Released 2026-04-15 under Apache 2.0.
  • The original Japanese chapter this article is adapted from goes deeper on the VRAM debugging story and the standard 7-question quality set (see the canonical link at the top of this article).

Top comments (0)