On 2026-07-31, DeepSeek quietly shipped V4-Flash-0731 under an MIT license. Instead of reading the model card, VIDRAFT's Darwin model-inspection platform read the actual config.json and every weight shard — 72,317 tensors measured directly — to reconstruct how the model is built and, more interestingly, what it is built for.
The shape, from the weights (not the docs)
| Component | Specification |
|---|---|
| Total parameters | 284B |
| Active per token | ~12.8B (4.5%) |
| Layers | 43, all MoE |
| Context | ~1.05M tokens |
| Experts | 256 routed + 1 shared, 6 selected/token |
| Attention | MLA — 64 heads compressed into a single 512-slot latent KV |
| Routing |
sqrtsoftplus + noaux_tc (score-based, no auxiliary-loss penalty) |
| Residual path | MHC (Hyper-Connections) replacing the plain residual |
| Precision | fp8 with block scaling |
| Decoding | DSpark speculative heads on layers 40-42 |
Two findings worth your attention
1. The first three layers do not route — they use fixed allocation tables
Most MoE models let a router learn which expert handles each token. In V4-Flash, layers 0-2 instead ship with pre-assigned allocation tables. The result is near-perfect load balance: 2,703-3,397 tokens per expert, a ~5% spread. That design choice structurally removes the router collapse failure mode (a few experts hogging traffic while others die) that plagues trained routers early in the stack.
2. How you allocate decides how experts specialize (a 9x gap)
Comparing the table-assigned layers (0-2) against the competitively-routed layers (20-40), expert specialization diverges by ~9x — 0.4% vs 3.4%. In other words, the allocation mechanism itself, not just the data, governs how sharply experts differentiate. Fixed tables keep early layers general; competitive routing lets deeper layers specialize.
What it is actually built for: harnesses, not chat
Put the pieces together — a 1M-token context, sustained long outputs, DSpark speculative decoding, and a sparsity profile tuned for throughput — and the target isn't conversation. It's coding agents: SWE-bench-style automated repair loops that run inside an agent harness, holding a large repository in context and emitting long, structured edits. V4-Flash reads as a model shaped by the harness it will live in.
Why it matters
284B parameters buys frontier-class capability, but only ~12.8B activate per token, so the real compute cost sits in a small class. Combined with fp8 block scaling and MXFP4-friendly layout, that makes a code-oriented, near-home deployment plausible — a different bet from chat-first frontier models.
Methodology note: every number above comes from direct tensor measurement, not vendor documentation. Full teardown (Korean, 72,317-tensor report): https://livegpt.tistory.com/20
Analysis by VIDRAFT — Darwin model-precision inspection.
Top comments (0)