Moonshot AI released Kimi K3, a 2.8T-parameter mixture-of-experts model with 1M token context window and $3/M input pricing, claiming autonomous chip design and research capabilities.
Moonshot AI's Kimi K3 packs 2.8 trillion parameters with 896 experts, activating just 16 per token. It offers a 1 million token context window and API pricing of $3 per million input tokens.
Key facts
- 2.8 trillion total parameters, 896 experts, 16 active per token.
- 1 million token context window, natively multimodal.
- API pricing: $3/M input, $15/M output tokens.
- Delta Attention enables 6.3x faster decoding in million-token contexts.
- Autonomously designed a chip with 1.46 million standard cells in 48 hours.
Beijing-based Moonshot AI released Kimi K3, a mixture-of-experts model with 2.8 trillion total parameters and 896 experts, activating only 16 (roughly 1.8% of the expert pool) per token According to @rohanpaul_ai. The model is natively multimodal and supports a 1 million token context window, equivalent to approximately 750,000 words of code or documentation in a single prompt.
Benchmarks and performance
K3's benchmark results place it in the territory of Opus 4.8, GPT 5.6 Sol, and Fable 5, though specific scores were not fully detailed in the source. Its Delta Attention mechanism enables up to 6.3x faster decoding in million-token contexts, addressing a key bottleneck for long-context inference.
Autonomous chip design and self-optimization
K3 autonomously designed, optimized, and verified a working AI chip in a single 48-hour run, specifically to serve a smaller model built on K3's own architecture. The simulated chip reportedly reached 8,700+ tokens/second, contained 1.46 million standard cells, and fit within 4 mm². An early version of K3 handled the majority of the kernel-optimization work used to develop K3 itself. K3 also built a GPU compiler from scratch—creating MiniTriton, optimization passes, PTX generation, and runtime—matching or beating Triton on some workloads and successfully training nanoGPT end to end.
Infrastructure requirements
K3's 2.8T MXFP4 weights require roughly 1.4 TB before quantization metadata and runtime overhead. At an assumed 115 GB usable per GB10 node (NVIDIA's compact Grace Blackwell AI chip), 14–16 nodes are expected to hold the raw weights realistically. Moonshot recommends supernodes containing 64 or more accelerators for production deployment.
Autonomous research capabilities
In a 15-hour autonomous run, K3 redesigned a production-scale training kernel and cut forward-plus-backward time from 283.6 ms to 114.4 ms. For a 42-year semiconductor-industry report, K3 performed 2,800+ web searches/fetches, 1,100+ terminal data pulls, processed 11,000+ pages, and recursively improved the work over 120+ rounds. K3 also reproduced a computational-astrophysics research workflow in roughly two hours, versus an estimated one to two weeks for an experienced researcher, reviewing 20+ papers, evaluating 300+ equations of state, finding inconsistencies in published formulas, and writing 3,000+ lines of Python.
Pricing
K3 is available via API at $3 per million input tokens and $15 per million output tokens.
What to watch
Watch for independent third-party benchmarks on standard NLP and coding tasks (e.g., MMLU, HumanEval, SWE-Bench) to validate K3's claimed performance against Opus 4.8 and GPT 5.6 Sol. Also track enterprise adoption and whether Moonshot publishes a formal technical paper with ablation studies.
Originally published on gentic.news
Top comments (0)