TL;DR: Moonshot AI released Kimi K3 on July 16, a 2.8 trillion parameter open-weight model with a 1 million token context window. This post explains what those numbers actually mean, goes one level into the architecture (MoE, Kimi Delta Attention), and ends with my take: the era of dirt-cheap Chinese models may be ending, and "open" now means something you can inspect but probably never run.
I have not used K3 myself yet. This is me reading the launch material and independent coverage carefully so you get the picture in ten minutes instead of an afternoon.
What just happened
A Google News notification about "the largest open-source model ever" is what pulled me in. The claim sounded big enough to be worth checking properly.
Moonshot AI, the Beijing startup behind the Kimi models, released Kimi K3. The headline claims from their launch post:
- 2.8 trillion total parameters, which they call the first open model in the 3T class
- A 1 million token context window
- Native multimodality: it processes images and video in the same model, not through a bolted-on vision encoder
- Benchmarks that, by Moonshot's own admission, still trail Claude Fable 5 and GPT 5.6 Sol, but beat everything else they tested, including Claude Opus 4.8 and GLM-5.2
The weights are not out yet. They are promised by July 27. Until then, nobody outside Moonshot can independently verify or run the model. Keep that in mind for everything below: these are launch-day numbers from the vendor, partially confirmed by early independent testing from Artificial Analysis.
The numbers, translated
2.8 trillion parameters does not mean 2.8 trillion at once. K3 is a Mixture of Experts (MoE) model. Think of it as 896 specialist sub-networks ("experts") sitting behind a router. For each token, the router picks 16 of those 896 to actually do the work. So the model has a huge library of knowledge on disk, but only a small slice of it fires per token. That is how you make a 2.8T model affordable to serve at all. This is the same broad trick behind most large open models today; K3 just pushes the sparsity further than usual.
A 1 million token context window means roughly 700,000+ words of input. That is an entire large codebase, or several books, in one prompt. The catch with long context has never been "can you fit it" but "can the model actually use token 900,000 when answering," and that is exactly what K3's attention changes are aimed at (next section).
"Open" here means open weights, later. You will be able to download the model and run it yourself. But at this scale, "yourself" means a serious GPU cluster. Moonshot recommends deploying on nodes with 64 or more accelerators. So it is open in the sense that researchers and companies can inspect, fine-tune, and self-host it. It is not open in the sense that you or I will run it on anything we own.
One level deeper: what is actually new in the architecture
Two named components carry the release, and both are about how information flows through the model.
Kimi Delta Attention (KDA). Standard attention gets expensive as context grows, because every token can look at every other token. KDA is Moonshot's answer for scaling attention efficiently to very long sequences. They also mention it breaks conventional prefix caching, and that they contributed a fix to vLLM, which tells you they expect people to actually serve this thing, not just benchmark it.
Attention Residuals (AttnRes). In a deep network, information from early layers has to survive dozens or hundreds of layers to influence the output. AttnRes, per Moonshot, lets the model selectively retrieve representations from earlier depths instead of accumulating everything uniformly. Intuition: rather than a message being whispered down a long chain of people, later layers can go back and re-read earlier notes directly.
On top of that, a stack of training tricks with names like Quantile Balancing (keeping the 896 experts evenly loaded without a fragile hyperparameter) and quantization-aware training in MXFP4/MXFP8 (so the released weights run efficiently on a wide range of hardware). Moonshot claims all of this together gives roughly 2.5x better scaling efficiency than their previous K2 generation, meaning more capability per unit of compute.
The full technical report is not out yet, so the honest status of every claim above is "vendor-described, plausible, unverified." The July 27 weights release is when the community gets to check.
The parts of the launch post worth raising an eyebrow at
The demos are genuinely impressive on paper: a Triton-like GPU compiler built from scratch, a chip designed in a 48-hour autonomous run, a research pipeline reproducing astrophysics results in two hours. But launch-day case studies are always the best runs, selected by the vendor. The Decoder's coverage notes that independent testing broadly confirmed the benchmark story, but also that K3's hallucination rate went up compared to its predecessor. Moonshot's own limitations section admits the model can be "excessively proactive," making decisions on your behalf when instructions are ambiguous. For an agent meant to run unsupervised for hours, that cuts both ways.
My take: two quiet shifts hiding in this release
First, cheap Chinese models may be over. K3 costs $3 per million input tokens and $15 per million output tokens. That is not DeepSeek-style pricing; that is Western mid-tier pricing. The old story was that Chinese labs compete on being 10x cheaper. K3's story is "we are nearly frontier, pay accordingly." If that holds, the pricing gap that made a lot of hobby projects viable is closing.
Second, "open source AI" is splitting into two different things. There is open-as-in-runnable, models like the smaller Llama and Qwen variants that a student with a decent GPU can actually use. And there is open-as-in-inspectable, 2.8T monsters that are technically downloadable but practically only serve as public research artifacts and enterprise self-hosting options. Both are valuable. But when headlines say "the largest open model ever," it is worth being clear that its openness will be experienced by almost everyone through an API, same as a closed model.
Where does that leave me? Honestly, nowhere near K3. It is far too big to run or experiment with locally, and I do not currently have a project that needs an AI API. Which is kind of the point: a model this "open" changes nothing about my day to day, and I suspect that is true for most individual developers reading this.
What I would watch next
- July 27: do the weights actually land, and what does the technical report reveal about KDA and AttnRes?
- Whether independent benchmarks hold up once anyone can run it.
- DeepSeek's response, which reporting suggests is close.
If you have already tried K3 through the API or Kimi Code, I would genuinely like to hear how it compares to whatever you use now. That is the data point launch posts can't give us.
Sources: Moonshot's K3 launch post, VentureBeat, The Decoder, TechCrunch.
Top comments (0)