A project called Deltafin runs the complete, uncompressed Kimi K3 language model — 2.8 trillion parameters — on a single MacBook Pro, sustaining almost exactly one token per second. It does it by keeping the model's always-active parameters in the laptop's 128 GB of memory and streaming 1.45 TB of expert weights from four external SSDs as each token requires them. Nothing is quantized and nothing is pruned.
Key facts
- The headline number: 1.00 token per second sustained over 512-token responses, on a consumer laptop.
- The hardware: an M5 Max MacBook Pro with 128 GB of RAM and four external SSDs holding 1.45 TB of expert weights.
- Who: Argonaut Labs AI, as a fork of upstream work by gavamedia.
- Primary source: the Deltafin repository.
The reason this is possible at all is mixture-of-experts architecture. A model like Kimi K3 does not use all 2.8 trillion parameters for every token. It is built as a large collection of specialised sub-networks, and a router activates only a small handful for any given piece of text. The total parameter count is enormous; the per-token working set is not.
That gap is the opening. Conventional wisdom says a model must fit in memory, because fetching weights from disk mid-generation is catastrophically slow. Deltafin accepts the catastrophe and engineers around it: a native Rust binary loads the base parameters into RAM once, then fetches expert weights on demand from SSD as the router calls for them. The reported decode speeds are 1.00 tokens per second over 512-token responses and 1.13 over shorter ones, against 0.68 for the upstream implementation on a public benchmark.
The analogy is a chef working from a pantry too large for the kitchen. Most cooking needs the same handful of staples, which stay on the counter. Anything else requires a walk to the storeroom. If the recipe is predictable the walks are rare and the meal arrives; if it is not, you spend the evening in the corridor. Deltafin's engineering is largely about making the walks shorter and better-timed — the README notes that "every 1% improvement is very hard-won."
The honest number, and the developers present it as the finding rather than a footnote, is that one token per second is roughly a word per second. A 500-word answer takes about eight minutes. Time to first token on a 512-token prompt is 6.3 minutes before generation even begins. This is not a usable assistant by any normal standard.
So why does it matter? Because the claim being tested is not "this is practical" but "this is possible, unmodified." The usual route to running a huge model on small hardware is quantization — compressing the weights and accepting some quality loss — or pruning, which removes parts of the model outright. Both change what you are running. The Deltafin developers are pointed about avoiding that: "K3 itself decides every token," and "nothing pruned." What runs on the laptop is the actual model, at full fidelity, just slowly.
That distinction matters for anyone thinking about where frontier capability physically lives. If the largest open models can be executed, however slowly, on hardware a person can buy, then the ceiling on local inference is set by patience and storage rather than by access to a datacentre. Storage is cheap and getting cheaper; patience is a choice. It is the same trajectory that made offloading and streaming weights a mainstream technique rather than a curiosity.
The caveats are real. This is one team's benchmark on one machine, and the numbers are self-reported without independent replication. Four external SSDs plus a high-end 128 GB MacBook is not cheap hardware, even if it is consumer hardware. Sustained multi-hour streaming at these rates puts meaningful wear on consumer SSDs, which the README does not quantify. And the speed makes it a demonstration rather than a tool — nobody is going to do useful work at 6.3 minutes to first token.
But demonstrations of this kind have a habit of aging well. The gap between "technically possible on a laptop" and "annoying but usable on a laptop" has repeatedly turned out to be a couple of years of engineering, and this is the first end of that gap.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)