Kimi K3 Shipped 1.56TB of Open Weights. Good Luck.

On July 27, Moonshot AI delivered on its promise: the full Kimi K3 weights landed on HuggingFace. All 2.8 trillion parameters. 96 shards. 1.56 terabytes. The largest open-weight model ever released — 75% bigger than DeepSeek V4-Pro at 1.6 trillion.

📖 Read the full version with charts and embedded sources on ComputeLeap →

Within 30 minutes, the model page was number one on HuggingFace's trending list. Within an hour, r/LocalLLaMA had a 3,000-point thread asking the question everyone was thinking: can I actually run this?

The short answer: almost certainly not. And that fact exposes a tension at the heart of the open-weights movement that nobody wants to talk about.

When we covered the K3 announcement on July 16, the weights were still a promise. Two labs had shipped frontier-class open models in a single day, and prediction markets repriced the Chinese AI race overnight. Now the weights are real, and the reality check begins.

View on Reddit →

The VRAM Math Nobody Wants to Hear

Here is the uncomfortable arithmetic.

Kimi K3 is a Mixture-of-Experts model: 2.8 trillion total parameters, 896 experts, 16 active per token, 104 billion parameters active during inference. That MoE sparsity sounds efficient — only 3.7% of the model fires for each token. But MoE has a dirty secret that the marketing decks never mention: all 896 experts must be loaded into VRAM simultaneously.

You don't get to page in 16 experts at inference time and leave the other 880 on disk. The routing decision happens at compute time. Every expert must be resident in memory, waiting.

Moonshot trained K3 with quantization-aware training (QAT) using MXFP4 weights and MXFP8 activations. This means the model already ships at roughly 4.49 stored bits per parameter. The usual trick of "just quantize it to 4-bit" has already been spent — the weights are already at the practical four-bit floor.

⚠️ The quantization dividend is already spent. Kimi K3 ships natively at MXFP4 (~4.49 bits per parameter) via quantization-aware training. A hypothetical 2-bit quant — with no validated community implementation — would still land past 700GB. There is no "just GGUF it" escape hatch for a 2.8T model.

The minimum VRAM footprint for inference via vLLM is 1,680 GB. Here are the configurations that clear that bar:

Configuration	Aggregate VRAM	Approx. Cost/Hour
8x NVIDIA GB300 (288GB each)	2,304 GB	$59–$142
16x NVIDIA H200 (141GB each)	2,256 GB	$64–$160
16x NVIDIA B200 (180GB each)	2,880 GB	Not widely available
32x H100 80GB	2,560 GB	$160–$320
8x AMD MI355X (288GB each)	2,304 GB	AMD reference only

A 24-hour test run costs $1,419 to $3,840. A month of continuous serving costs $43,158 to $116,800. This is not a hobbyist's model.

View on Hacker News →

The Unsloth Paradox

Unsloth, the community's go-to quantization shop, turned around Kimi K3 GGUFs within hours of release. Their Q4 variant is 1.51TB. Their Q8 lossless is 1.56TB — just 50GB larger than Q4.

Read those numbers again. The gap between 4-bit and 8-bit is 50 gigabytes — essentially a rounding error on a 1.5TB model. This is what quantization-aware training does: it pre-compresses the model to the point where further compression yields almost nothing.

One developer on X captured the absurdity perfectly. Kuber built a Kimi K3 GGUF in 15 minutes for $4.43 of cloud compute. The MXFP4 format makes conversion trivial. But as he noted: "Sadly, it can't be ran yet because the model is too large for any consumer hardware."

View original post on X →

You can build the file. You can hold the file. You cannot run the file.

What Delta Attention Actually Changes

If you can't run K3 locally, why should you care that the weights are open? Because the architecture matters more than the checkpoint.

Kimi Delta Attention (KDA) is the real technical contribution — and it will filter down to models you can run. Here is how it works, and why it matters.

Standard transformer attention computes all key-query pairs across the full context window. For a 1-million-token context, that means the KV cache alone can consume hundreds of gigabytes of VRAM, and decode cost grows linearly with context length. Every new token must attend to every previous token.

KDA replaces this with a fixed-size recurrent state. Instead of a growing KV cache, it maintains a d-by-d matrix that maps key space to value space. The critical insight is the delta rule: when writing a new key-value pair to memory, KDA first queries what the memory currently associates with that key, then writes only the prediction error — the difference between the actual value and what was already stored.

This means writes behave like assignment, not accumulation. The state stays accurate without growing.

But the Kimi team's specific contribution goes further. Previous linear attention work (Gated DeltaNet) used a single scalar forgetting gate — all dimensions decay at the same rate. KDA introduces channel-wise forgetting: a vector-valued gate where each dimension in the feature space has its own independent retention rate. Syntax cues can persist longer. Noisy channels decay fast.

View on Hacker News →

The practical results: up to 6.3x faster decoding in million-token contexts. Up to 75% KV cache reduction. And because the recurrent state is fixed-size, decode cost is constant regardless of context length — the 1,000th token costs the same as the 1,000,000th.

ℹ️ Kimi Delta Attention's constant-cost decode is the architectural primitive that makes million-token inference economically viable. Without it, serving a 1M-context model at API prices of $3/$15 per million tokens would be financially impossible. The architecture subsidizes the price.

Sebastian Raschka, one of the most respected ML educators in the field, broke down the architecture and made a key observation: K3 is essentially a scaled-up production version of the Kimi Linear model from 2025, taken from 48 billion to 2.8 trillion parameters. The new addition is LatentMoE — compressing large linear layers similar to multi-head latent attention — but the architectural DNA traces back to published, reproducible work.

View original post on X →

This is what makes the open weights valuable even if you never download them: every architectural detail is now inspectable. Researchers can trace exactly how KDA scales, how the attention residuals (AttnRes) connect depth information, and how the LatentMoE experts are initialized. The next generation of 7B and 14B models will inherit these techniques.

The "Nobody Can Ban It" Pitch Falls Apart

The strongest argument for open weights has always been sovereignty: if you possess the weights, no API provider can cut you off, no government can embargo your inference pipeline, no terms-of-service update can break your product overnight. Sayash Kapoor framed it precisely: "Our interventions for resilience should be focused on a world where advanced AI is abundant rather than scarce."

But sovereignty requires the ability to run the model, not just store the files. A 1.56TB checkpoint sitting on cold storage is a political statement, not a deployment strategy.

Consider the practical math:

• A startup with a single NVIDIA A100 (80GB) can run the weights it possesses: zero percent of K3
• A well-funded team with 8x H100s can serve K3 — if they are comfortable spending $160+/hour on inference alone
• A hyperscaler can serve K3 at commodity prices — which means the "sovereignty" argument loops back to depending on a cloud provider anyway

The irony is thick. Moonshot released K3 under a custom "Kimi K3 License" that requires model-as-a-service businesses with more than $20 million in revenue to negotiate a separate agreement. The weights are open, but the commercial terms have a gate. Artificial Analysis flagged this as "Commercial Use Restricted" — a meaningful distinction from MIT or Apache 2.0.

View original post on X →

⚠️ Contrarian take: "Open weights" has become a marketing category, not a technical guarantee. If the weights require $60-$160/hour in GPU rental to serve, and the license requires a separate commercial agreement above $20M revenue, the practical difference from a closed API is smaller than the community wants to admit. The real beneficiaries are cloud inference providers and well-funded research labs — not independent developers.

Where the Community Actually Lands

The r/LocalLLaMA thread — 3,090 upvotes, 593 comments — is the most honest assessment of K3's open weights. The top comments are not celebrating sovereignty. They are doing arithmetic.

The consensus: K3 is a research artifact and an infrastructure play, not a local-inference model. The community that built its identity around running models on consumer hardware is confronting a model that requires a small data center.

But the reaction is not purely negative. Several threads highlight the API economics as genuinely disruptive:

• Together AI serves K3 via HuggingFace Inference Providers at $3/M input tokens, $15/M output tokens, 54 TPS
• OpenRouter, Fireworks AI, Modal — all announced K3 endpoints within 48 hours
• With 90% cache hits, blended cost drops to roughly $3.46 per million tokens

For comparison, Claude Fable 5 via Anthropic's API runs $15/$75 per million tokens. GPT-5.6 via OpenAI is $12.50/$50. K3 at $3/$15 is dramatically cheaper — and on the Artificial Analysis Intelligence Index, it scores 57, comparable to Opus 4.8.

The Emotional Download

The most viral reaction to K3's weight release was not technical. It was emotional.

Thomas Unise captured it: "Downloading the kimi k3 weights on huggingface right now. It's crazy to think this is what escaping the permanent underclass looks like. It's even more crazy that Moonshot AI — a Chinese AI lab — is letting me — an American — download frontier intelligence that costs tens of millions of dollars to train."

View original post on X →

The sentiment is real, and it reveals why "open weights" resonates even when self-hosting is impractical. The psychological value of possessing frontier intelligence — of knowing you could, in theory, stand up your own inference pipeline — matters to a community that watched OpenAI pivot from open to closed, watched Google restrict Gemini's weights, and watched Anthropic never release weights at all.

K3's 1.56TB download is, for many people, an insurance policy they may never need to cash.

What This Means for You

If you are building products: Use K3 via API. Together AI at $3/$15 per million tokens is the best value-to-capability ratio in the current market. OpenRouter and Fireworks give you fallback routing. The open weights mean you will never face a sole-provider lockout — if Together goes down, someone else will serve the same model.

If you are doing ML research: Download the weights and study the architecture. Kimi Delta Attention's channel-wise forgetting is the most significant attention innovation since Multi-Head Latent Attention in DeepSeek V2. The technical report is on GitHub. The detailed KDA explainer on Doubleword is excellent.

If you are running local models: K3 is not for you — yet. But watch for KDA to land in smaller models. When someone trains a 14B model with channel-wise delta attention and constant-cost decode, that is your upgrade. The techniques will trickle down. They always do. We have seen this pattern before with models like GLM-5.2 — the architectural innovations arrive at frontier scale first, then compress into something practical.

If you care about the open-weights movement: Celebrate this release, but be honest about what "open" means at 2.8 trillion parameters. Possession is not sovereignty. Sovereignty requires the infrastructure to run what you possess. The real test of open-weights ideology comes not with K3, but with whatever efficient distillation Moonshot or the community produces at 70B or below.

The Bottom Line

Kimi K3's open weights are historically significant: the largest model ever released publicly, with a genuinely novel attention architecture, at a price point that undercuts every closed competitor. Moonshot delivered what they promised, on schedule.

But the gap between "open weights" and "you can run it" has never been wider. At 1.56TB, with the quantization budget already spent, K3 lives in data centers. The community that downloads it is making a statement. The companies that serve it are making money. And the architecture inside it — Kimi Delta Attention, constant-cost decode, channel-wise forgetting — is making the future of efficient inference.

That last part is the story that matters.

Originally published at ComputeLeap