Up to 95% more output from one GPU on Qwen3.6

One GPU. Six concurrency levels. Our engine beat vLLM by more than 1.5× at every single one.

At the top end, 1.95×.

The setup

• Model: Qwen3.6-35B-A3B
• Hardware: one NVIDIA RTX PRO 6000 Blackwell Server Edition on Google Cloud
• Baseline: vLLM 0.33.0
• Both engines: BF16 weights, 3 MTP draft tokens, 32K context, greedy decoding

We didn't run vLLM on defaults. We tried a range of memory settings and gave it the best one we found: 95% GPU memory utilization. If you're going to beat a baseline, beat a tuned one.

The prompts are a real mix. Chat, long-document QA, math, code, and tool use, pulled from Dolly, OpenAssistant, LongBench, QuALITY, MATH, GSM8K, MBPP, HumanEval and Glaive.

Output throughput, steady traffic

Concurrency	8	16	32	64	128	256
Backboard vs vLLM	1.63×	1.57×	1.62×	1.65×	1.87×	1.95×

Look at the right side of that row. The gap is widest at the highest load.

That's what you want. Your GPUs aren't busy at concurrency 8. They're busy at 128 and 256.

Time to first token

Throughput is half the story. The other half is how long someone waits before the answer starts.

Concurrency	8	16	32	64	128	256
Median TTFT, faster by	1.70×	1.87×	2.03×	3.39×	5.75×	2.09×
p90 TTFT, faster by	1.72×	1.71×	1.70×	3.57×	3.22×	2.65×

More output and a shorter wait. Usually you trade one for the other. Here you don't.

Each request also generates 1.35× to 1.61× faster once it gets going.

Burst traffic

Steady traffic is one thing. Real traffic spikes.

So we dropped every prompt for a given level on the engine at once, three times per level.

• Output throughput: 1.57× to 1.78× vLLM
• Median time to first token: up to 8.56× faster
• Per-user generation speed: 1.28× to 1.59× vLLM

Why Qwen3.6

Mid-size open models are where a lot of real deployments live. They fit on one card. They're good enough for most jobs. And they get run at scale.

That makes them the best place to prove one thing: a better engine turns the GPU you already have into a bigger one.

Read the full report

Everything is in the Qwen3.6 report on our blog.

Running Qwen in production? Let's put your workload on it.