If you have ever thought about replacing a per-seat AI coding subscription with a GPU you control, the first thing you want to know is not the benchmark score. It is how many people can use the box before it feels slow.
We measured that for Strata, an open source engine that runs Qwen3.8-Flash-Next (125B parameters, mixture of experts) on consumer cards. The setup was one rented RTX 5090 with 32 GB, the 1-bit Coder build from IST Austria's DASLab (58.4 GB), and a 256K context. The full numbers and the subscription math are in our Strata on an RTX 5090 research post. This piece is about the parts that matter when you are the one setting it up.
The numbers you will plan around
- 1 developer: 144 tokens per second, first token under 2 seconds
- 4 developers at once: 38 each, 125 total
- 8 developers at once: 19 each, 130 total, first token 2 to 8 seconds
Total throughput stays flat as you add people. That is the shape to remember: the card runs out of concurrent seats, not tokens.
Why one user is faster than four
A single client gets speculative decoding: a small helper drafts four tokens, the big model verifies them in one pass, and about three out of four were accepted in our runs. That is a big part of the 144.
Strata serves concurrent users with parallel slots, and the slots do not use speculative decoding. Every token comes out one at a time. So the jump from one user to four costs you more than you would expect from simply sharing the card: per-user speed drops from 144 to 38 and the total also dips.
Practical consequence: if your team rarely generates at the same moment, one card feels great. If you run several agents in parallel all day, plan for the slot numbers, not the solo number.
The 5x slowdown that was not the GPU
Our first rented host gave about 30 tokens per second instead of about 170. The GPU was fine. The container exposed all 256 host threads but limited the CPU through a cgroup quota of around 30 cores. Strata sized its worker pool for every visible thread, the workers spun, the kernel throttled them, and the CPU half of the work (the experts that are not cached on the card) became the bottleneck.
Check before you benchmark anything on a rented container:
cat /sys/fs/cgroup/cpu.max
If it shows a quota, set --pool-workers to a few less than the number of cores it allows. Generation went back to normal on that host.
The second one showed up with eight slots: the default left less than 400 MiB of VRAM free and the driver started paging GPU memory. --vram-reserve-mib 1536 fixed it.
Pointing Claude Code at it
Strata exposes an Anthropic-compatible endpoint, so Claude Code only needs a base URL and a token:
export ANTHROPIC_BASE_URL=http://127.0.0.1:8080
export ANTHROPIC_AUTH_TOKEN=local
On a single-file task (a Snake game with an autopilot mode) it wrote 408 lines in one Write and fixed a timing bug on its own. One quirk: this heavily quantized model sometimes says it finished without calling the Write tool. Naming the Write tool in the prompt solved it for us.
Limits
One card model, one host for the published numbers, one agent task that is a single file rather than a real repository. The quality figure (75.6% on SWE-bench Verified, against 82.8% for the full model) comes from the model card, not from us.
The video of the whole run, the cost per million tokens and the comparison with Claude Code plans are in the full RTX 5090 write-up on inferencekey.com.
Top comments (0)