On July 16, 2026, Beijing-based Moonshot AI released Kimi K3. It is a 2.8 trillion parameter open-weight model with native vision, a 1 million token context window, and a pricing structure that undercuts most competitors.
It also ranked #3 on the Artificial Analysis leaderboard, behind only Anthropic's Claude Fable 5 and OpenAI's GPT-5.6 Sol (per Wikipedia's coverage).
This is not another "almost competitive" open model. This is a frontier-class system that Moonshot plans to release as open weights on July 27, 2026.
The Numbers
Kimi K3 is a Mixture of Experts (MoE) model. Out of 896 experts, only 16 are activated per token. That sparsity ratio is extreme even by MoE standards. For context, DeepSeek-R1 activates 8 of 256 routed experts (plus 1 shared). Kimi K3 takes sparsity even further with its 16-of-896 configuration.
Key specs:
- Total parameters: 2.8 trillion
- Active parameters: ~16 experts (of 896)
- Context window: 1,000,000 tokens
- Vision: Native multimodal
- Open weights: By July 27, 2026
- API pricing (input, cache hit): $0.30 / million tokens
- API pricing (input, cache miss): $3.00 / million tokens
- API pricing (output): $15.00 / million tokens
Kimi K3's cache hit rate exceeds 90% in coding workloads, meaning most input tokens cost $0.30/MTok.
Moonshot claims approximately 2.5x improvement in scaling efficiency compared to Kimi K2, meaning they get more intelligence per unit of compute.
New Architecture: KDA and Attention Residuals
Kimi K3 introduces two architectural changes worth understanding.
Kimi Delta Attention (KDA) is a more efficient attention mechanism that reduces memory usage and improves generation speed at longer context lengths. It was first introduced in Kimi Linear (October 2025) and scales further here. Moonshot has contributed a KDA-compatible prefix caching implementation to vLLM, the popular inference engine.
Attention Residuals (AttnRes) selectively retrieves representations across model depth rather than accumulating them uniformly. Instead of every layer building on the previous one in a linear chain, AttnRes lets later layers directly access useful intermediate representations from earlier layers.
The model also uses a Stable LatentMoE framework with Quantile Balancing for expert routing. At 896 experts, routing stability becomes a first-order challenge. Quantile Balancing derives expert allocation directly from router-score quantiles, removing heuristic updates and sensitive hyperparameters. This solves the "some experts get overloaded, others sit idle" problem that breaks large MoE models.
What It Can Actually Do
The official blog post includes case studies that go beyond standard benchmarks. Here is what stood out.
Built a GPU Compiler From Scratch
Kimi K3 built MiniTriton, a compact Triton-like GPU compiler with its own tile-level IR layer over MLIR, optimization passes, and a PTX code generation pipeline. Across roofline benchmarks, MiniTriton matches or beats Triton and torch.compile on certain workloads. It even sustains end-to-end nanoGPT training with stable convergence.
A model writing a working compiler for GPUs, from scratch, is not trivial. It requires understanding hardware constraints, memory hierarchies, and code generation pipelines simultaneously.
Designed a Chip for Its Own Architecture
In a single 48-hour autonomous run, Kimi K3 designed a chip to serve a nano model built on its own architecture. Using open-source EDA tools on the Nangate 45nm library, the chip closes timing at 100 MHz, packs 1.46M standard cells, and sustains over 8,700 tokens/s decode throughput in simulation. This is a tiny inference chip for a nano-sized model, not a chip that runs K3 itself, but it demonstrates autonomous hardware design capability.
Reproduced Physics Research in 2 Hours
Kimi K3 reproduced the I-Love-Q universal relations from computational astrophysics. It reviewed 20+ papers, implemented the full numerical pipeline, evaluated 300+ equations of state, identified inconsistencies in published formulas, and produced 3,000+ lines of Python code. A human researcher would typically need 1-2 weeks for this.
Edited Its Own Teaser Video
Kimi K3 edited its own release teaser from 56 source clips. It handled clip selection, motion-matched cuts, frame-accurate beat synchronization, and audio processing. Moonshot estimates this would take an experienced editor 1-2 days.
Benchmark Performance
Moonshot is honest about where K3 stands. Their own blog says:
"While its overall performance still trails the most powerful proprietary models, Claude Fable 5 and GPT 5.6 Sol, Kimi K3 demonstrated frontier-level performance across our evaluation suite, consistently outperforming other tested models."
Key benchmark results (all with reasoning effort set to max):
- DeepSWE (real-world software engineering): 67.3 with mini-SWE-agent harness
- Terminal-Bench 2.1 (terminal-based agentic tasks): Competitive with Claude Fable 5
- BrowseComp (web browsing accuracy): 90.4 with 1M context, no compaction
- Arena.ai Frontend Web Dev (front-end coding): Beat all competitors
- GPU Kernel Optimization: Competitive with Fable 5, beat Opus 4.8, GPT-5.6 Sol, GPT-5.5
Honest Limitations
Moonshot disclosed three limitations, which is more transparency than most labs offer:
Sensitivity to thinking history. K3 was trained with preserved thinking history. If the agent harness fails to pass back all historical thinking content, or if you switch mid-session from another model, generation quality becomes unstable. Stick with verified harnesses like Kimi Code.
Excessive proactiveness. K3 makes decisions on the user's behalf when it encounters ambiguity. If your application needs strict boundaries, Moonshot recommends explicit constraints in the system prompt or AGENTS.md.
Noticeable UX gap. Despite being competitive on benchmarks, Moonshot acknowledges a "noticeable gap in user experience compared with Claude Fable 5 and GPT-5.6 Sol."
What This Means
The open-source AI gap keeps closing. In 2023, open models were months behind proprietary ones. In 2025, the gap was weeks. Some observers argue Kimi K3 brings it to days, though this assessment depends on which benchmarks and use cases you weigh most heavily.
When the weights drop on July 27, anyone with enough hardware (Moonshot recommends 64+ accelerators in a supernode configuration) can run a frontier-class model locally. For everyone else, the API pricing makes it accessible: $0.30/MTok for cached input is aggressive.
The bigger story is what Moonshot proved about scaling. They took MoE sparsity to an extreme (16 of 896 experts), solved the routing stability problem, and demonstrated that architectural efficiency can compensate for raw parameter count. K3's 2.5x scaling efficiency over K2 means the next iteration does not necessarily need 5 trillion parameters to be better. It needs better architecture.
That is good news for everyone who does not have infinite compute budgets.
Sources:
Top comments (0)