The small number in an MoE announcement is tempting. Kimi K3.1 Pro lists 2.8 trillion total parameters and 104 billion active parameters. It’s easy to read that second figure as the amount of model you need to store. It isn’t. [1]
I worked through the weight arithmetic because that distinction changes a deployment discussion immediately. At an ideal four bits per parameter, the full payload is 1,400 GB. The selected 104B subset works out to 52 GB. Both calculations are correct. They answer different questions. 🧮
Computed payloads from the reported parameter counts. These are arithmetic scenarios, excluding metadata and runtime allocations; no Kimi checkpoint was loaded.
The experts you didn’t select still exist
Mixture of Experts, or MoE, routes a token through a selected subset of expert networks. That reduces the work associated with evaluating every expert for every token. It doesn’t erase the other experts’ learned weights.
Think of a workshop with many specialist benches. Sending today’s part to one bench saves work. It doesn’t make the rest of the workshop disappear from the floor plan. The storage department remains annoyingly literal. 📦
Original meme. Emoji: Twemoji, CC BY 4.0; graphics arranged with original captions.
The K3.1 Pro launch makes this worth understanding. Moonshot says it is post-trained from K3, offers a million-token context window, and is available through its products and API. At launch, the full weights were promised by 4 November. That release milestone matters if you're choosing between API access and self-hosting. [2]
This gives two separate decisions: whether to evaluate the hosted model, and how to assess a future self-hosted checkpoint. A parameter headline alone settles neither.
The calculation, with the units attached
For a hypothetical uniform representation:
raw weight bytes = total parameters × bits per parameter ÷ 8
decimal GB = raw weight bytes ÷ 1,000,000,000
The word raw matters. Quantized formats may store scales, group information and some tensors at higher precision. A four-bit calculation doesn’t establish that a suitable four-bit checkpoint exists, that the runtime supports it, or that it preserves the quality you need.
Here is a complete Python check. Save it as payload.py and run python3 payload.py; it uses the standard library.
from decimal import Decimal
from itertools import product
total = 2_800_000_000_000
active = 104_000_000_000
def gb(parameters, bits):
return Decimal(parameters) * bits / 8 / 10**9
for bits in (16, 8, 4):
print(f"{bits:2d} bit: total={gb(total, bits):.0f} GB, "
f"active subset={gb(active, bits):.0f} GB")
experts = 8
for tokens in (1, 2, 4):
expected = experts * (1 - (1 - 1 / experts)**tokens)
exhaustive = sum(
len(set(route))
for route in product(range(experts), repeat=tokens)
) / experts**tokens
assert abs(expected - exhaustive) < 1e-12
print(f"toy batch={tokens}: {expected:.6f} unique experts")
My output:
16 bit: total=5600 GB, active subset=208 GB
8 bit: total=2800 GB, active subset=104 GB
4 bit: total=1400 GB, active subset=52 GB
toy batch=1: 1.000000 unique experts
toy batch=2: 1.875000 unique experts
toy batch=4: 3.310547 unique experts
✅ The payload arithmetic uses decimal GB throughout. Dividing by 2**30 instead would produce GiB. Switching units halfway through a hardware estimate is an excellent way to manufacture a mysterious memory shortage.
A batch visits more of the workshop
There’s another trap in “active per token”: a batch contains different tokens, which may select different experts.
My small example above assumes eight experts, one selected per token, independent uniform routing. Those are teaching assumptions, not a description of Kimi’s router.
For a particular expert, the probability that one token misses it is 7/8. The probability that all B tokens miss it is (7/8)**B. So its probability of appearing at least once is 1 - (7/8)**B. Adding that probability across the eight experts gives:
expected distinct experts = 8 × (1 - (7/8)**B)
The exhaustive enumeration checks every possible routing assignment for those small batches. A single token selects one expert; four tokens touch an average of about 3.31 distinct experts. The union grows even though each token still selects only one. 💡
Real routers have skew, multiple selections, shared components and scheduling constraints. The example’s purpose is narrower: a per-token active count cannot be substituted for the set of weights a serving system must make available over time.
What I’d ask before estimating a deployment
⚠️ The practical mistake is to enter 104B into a weight-memory calculator and call the result a Kimi deployment estimate.
I’d start with the actual checkpoint inventory and its tensor dtypes. Then I’d check how the chosen runtime places experts, which tensors are replicated, what gets offloaded, and how much temporary and sequence state the workload creates. vLLM’s expert-parallel documentation is useful background: distributing experts across devices and adding redundant experts have explicit memory consequences. That documentation alone doesn’t certify support for this new checkpoint. [3]
Offloading changes where weights live and when they move. It introduces a transfer and scheduling problem; it doesn’t turn total parameters into active parameters. Likewise, a context-window limit describes what a model accepts, not a measured memory allocation for your prompt.
For this article I checked arithmetic and a synthetic router, rather than running K3.1 Pro inference. The useful result is the distinction to carry into a real evaluation: active parameters help explain sparse computation; total weights still belong in the storage plan.
References
- Moonshot’s K3.1 Pro announcement and parameter counts.
- Kimi K3.1 Pro technical launch post.
- vLLM: expert-parallel deployment and memory overhead.
Asset credits: original chart generated from the checked arithmetic in this article. Original meme uses Twemoji graphics under CC BY 4.0, composed with original text; graphics by Twitter and other contributors.
AI disclosure
AI was used to draft the article.
Top comments (0)