132 decode tokens a second on one RTX 3090 is the number worth reading twice

A poster on the local model forum reports pushing Qwen3.8-27B to 132 decode tokens per second on a single RTX 3090. They give a prefill figure too. I want to stay on decode, because decode is the part you actually sit and watch.

Prefill is the model reading your prompt. Decode is the model writing the answer, one token at a time. Prefill happens in a batch and it feels like a pause. Decode happens in a stream and it feels like typing. When people say a local model is slow, they almost always mean decode is slow.

Why that one number changes the shape of the work

At 132 decode tokens a second, generation outruns reading. You stop watching words crawl across the terminal and start scrolling to keep up. That sounds cosmetic. It isn't.

Below reading speed, you babysit the model. You wait, you skim as it goes, you catch a wrong turn in the third sentence and kill it. The waiting sets your rhythm, and the rhythm is bad. Above reading speed, the answer just arrives. You read it as a finished thing and decide. That's the same interaction you get from a hosted endpoint, and it's the reason hosted endpoints feel usable in a way most local setups don't.

The second thing 132 a second buys you is agent loops. An agent turn isn't a single answer. It's a plan, a tool call, a result, another plan. Every one of those steps pays the decode cost again. A loop that's tolerable at conversational speed becomes unusable when you chain eight steps behind it. Speed compounds down the chain, and so does slowness.

The part I'd want to know before I trusted it

A single number never travels alone, so here's what I'd check.

Batch size matters enormously. A decode rate measured while serving one request is a different claim from the same rate measured across a batch of concurrent requests. Both are real numbers. They describe different machines.

Context length matters too. Decode slows as the key-value cache grows, so a rate measured on a short prompt won't hold up in the middle of a long session. If you care about long-context work, that's the case you want to see.

And quantization is doing work here that the headline number can't show you. A 27B model on a 3090 is a tight fit. Something got squeezed. Knowing what got squeezed tells you whether the speed came free or came out of quality.

None of that makes the result less interesting. It just means the number is a starting point for your own test rather than a spec you can copy.

What I'd take from it

The useful takeaway isn't the exact figure. It's that a 27B model at that decode speed on consumer hardware puts local inference on the right side of a threshold that used to be reserved for hosted APIs.

That threshold isn't a benchmark score. It's the point where you stop thinking about the model's speed at all. Below it, every design decision routes around latency. You batch things. You cache aggressively. You avoid multi-step reasoning because the waiting kills it. Above it, you just build the thing you wanted to build.

If you've got a 3090 sitting in a box and you wrote off models this size, this is worth an afternoon. Run your own prompts, at your own context lengths, with your own concurrency. See if the number holds for your workload.

The claim I'd make is narrow and I'd stand behind it. Somebody demonstrated that the speed you need for real local work is reachable on hardware plenty of people already own. That's a smaller claim than a benchmark table, and it's a more useful one.