Engineers at imec, the Belgian nanoelectronics research institute, ran open-weight coding models through the same agent harness as commercial APIs and published the numbers. The finding that matters: the best open model no longer fits on the hardware most organizations would consider buying. Kimi K3, at roughly 1.4 terabytes of weights, needs an eight-GPU B300 node -- and even then serves only 16 concurrent sessions, about eight times slower per task than a hosted frontier coding agent.
Key facts
- K3 required an eight-GPU B300 node because 1.4 TB of weights leaves no room for the key-value cache on a B200 node's 1.5 TB.
- On that node it served 16 concurrent sessions at 122 tokens per second aggregate, with a 38-minute median task time.
- Reported enterprise GPU utilization for internal developer tools runs 15 to 22 percent, and rarely exceeds 25 to 35 percent even when well run.
- Primary source: imec's self-hosting analysis.
The reason to read this rather than another benchmark table is that imec priced the whole stack, not the model. Their summary is unusually direct: "Your own box, priced at the hours it actually spends working, lands in the same ballpark as renting one." And on whether to buy: "Probably not to save money."
The trap is utilization, and it has nothing to do with model quality. Token demand from a developer team is spiky -- near zero overnight, ramping through the morning, dipping at lunch, peaking mid-afternoon. You buy hardware for the peak and pay for it 24 hours a day. imec cites published figures putting average GPU utilization for internal developer inference at 15 to 22 percent, with even a well-run deployment rarely above 25 to 35 percent. A cluster that is idle three quarters of the time costs four times its apparent hourly rate.
What makes this newly urgent is that model sizes have outrun the obvious hardware step. Kimi K3, released by Moonshot in July, is a 2.8-trillion-parameter mixture-of-experts model with a million-token context -- the first open model at that scale. imec's earlier runs used an eight-GPU B200 node, which carries about 1.5 TB of memory in total. K3's weights alone are 1.4 TB, leaving nothing for the key-value cache that every concurrent session needs. They moved to a B300 node at 288 GB per GPU, roughly 2.3 TB, about 20 percent more expensive. The mixture-of-experts design that makes K3 cheap to compute does nothing for the memory bill, because every expert has to be resident even when only 16 of 896 are used per token.
The performance picture is genuinely mixed rather than damning. K3 resolved 86.4 percent of imec's benchmark tasks, 24 percentage points above both GLM-5.2 and Claude Opus 4.8 at 62.5 percent -- with imec flagging honestly that the tasks come from a public suite that may have been in K3's training data. Against that, it served 16 concurrent sessions where GLM-5.2 managed 24, ran about 30 percent lower aggregate throughput, and took a median 38 minutes per task against 26. The smaller model that fits on a single GPU solved roughly a third of 64 tasks where the frontier model solved 40.
The spend distribution imec publishes explains why anyone considers this at all. Their median developer costs about $140 a year in AI API usage -- trivial. The 90th percentile is near $7,300, and the 99th approaches $90,000. The cost problem is not the average; it is a tail of heavy agent users, and it arrives suddenly. imec notes that over 70 percent of recurring revenue at major model providers now comes from coding use cases, which is precisely why the bills are volatile: the more of a workflow you hand to an agent, the more tokens it consumes on your behalf without asking.
Why it matters: "just run it yourself" has been the standard answer to frontier API pricing, and this is the most careful public accounting of what that costs in practice. The answer is that self-hosting buys you sovereignty -- data that cannot leave the building, a stack nobody can rate-limit -- and that you should not expect it to buy you savings. It also marks a threshold: open-weight models at the frontier are no longer things an individual or a small team can run, which quietly narrows what "open" delivers in practice.
The caveats: this is one organization's harness, one benchmark suite with a possible contamination problem, and rental prices that move constantly. Hardware assumptions age in months. But the structural point -- that memory, not compute, sets the floor -- is not going to reverse.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)