DEV Community

Breach Protocol
Breach Protocol

Posted on • Originally published at groundtruth.day

Kimi K3 topped a fullstack coding board at maximum effort

Moonshot's Kimi K3 took first place on Code Arena's WebDev leaderboard in the July 23 snapshot recorded in its technical report, beating Claude Fable 5 and GPT-5.6 Sol among 99 ranked models. Because the board entrant is the released open-weight model served through Moonshot's API at maximum reasoning effort - not a separate closed tier - this is the first time a downloadable model has led that board. On the live board as of July 28 it has slipped to second, behind a preliminary Claude Opus 5 Max entry.

Key facts

  • K3 ranked first among 99 models on the Code Arena WebDev board in the July 23 snapshot recorded in its technical report.
  • The entrant kimi-k3-max is the open-weight model at its highest reasoning setting - Moonshot's API docs list one model ID.
  • On the live board on July 28 it sits at #2, still above the listed Fable 5 and Sol entries.
  • Weights are public at 1.56 TB on Hugging Face, released by Moonshot AI.

The first thing to get right is what the model actually is, because the most common objection to this result was wrong. When kimi-k3-max appeared on the board, a widespread reading was that Moonshot had entered a secret hosted model with no relation to the weights it published - which would make the win meaningless as an open-weights milestone. Moonshot's own API documentation says otherwise. There is one K3 model ID. "Max" is the highest of three reasoning-effort levels, and it is the default. The board tested the published model, thinking as hard as it is configured to think.

One caveat survives that correction: Code Arena evaluates a hosted endpoint, not a checksum of a downloaded file. Nobody has verified byte-for-byte that Moonshot's serving stack matches the public checkpoint.

The second thing to get right is what the board measures. Code Arena's Fullstack environment is not a conventional software-engineering benchmark. It asks models to build end-to-end web applications - with databases, authentication, third-party APIs, persistent servers, terminal and web-search access, and deployment - and ranks them by anonymous pairwise human preference. Real people compare two outputs and pick one. That measures something genuine and narrow: whether the thing it builds is what a person wanted. It does not measure correctness on a large existing repository, and it is not a general coding crown.

Moonshot is unusually candid about this. Its own report says K3 trails Fable 5 and GPT-5.6 Sol overall. The coding picture is mixed: K3 leads on some program-synthesis and kernel-oriented work, roughly matches Sol on terminal tasks, but trails both on DeepSWE and trails Fable on FrontierSWE. The report also lists comparison caveats most vendors would bury - different agent harnesses, imported baseline numbers, and H20-versus-H100 hardware in some tests.

What makes the model interesting is the architecture, which attacks efficiency along three separate axes rather than simply scaling up.

Across the sequence, K3 alternates three layers of Kimi Delta Attention with one global attention layer. Delta attention is a recurrent scheme - it carries a running summary forward instead of comparing every token to every other token - and it is normally hard to make numerically stable. Moonshot's fix is a decay gate with a floor on how much the running state can fade, which keeps the arithmetic well-behaved enough to run as dense matrix operations on tensor cores instead of a slow sequential path. See linear attention.

Across depth, "Attention Residuals" let a layer selectively reach back and retrieve the output of specific earlier layers, rather than only reading the accumulated residual stream. It is attention applied vertically through the network instead of horizontally across tokens.

Across width, Stable LatentMoE routes each token through a small subset of a very large expert pool - the standard mixture-of-experts trade of enormous total capacity for small per-token compute. The novelty is Quantile Balancing, which corrects traffic imbalance using the distribution of router scores, producing fixed execution shapes and removing a per-layer synchronization with the host CPU that normally throttles these models.

One numerical detail is routinely misreported. The routed expert weights genuinely use four-bit MXFP4, with eight-bit activations. But attention, routers, and shared experts stay at higher precision. K3 is not a four-bit model; it is a model whose bulkiest component is four-bit. See quantization.

The community response was fast and, to its credit, self-correcting. Unsloth published converted builds within hours - a 1.51 TB four-bit file, a 1.56 TB eight-bit file, and separate multimodal projector files that preserve an image-input path. Then people tried to run them. A LocalLLaMA user with two 96 GB RTX 6000 Pro cards, 512 GB of RAM and NVMe RAID reported waiting about half an hour for one short correct answer - roughly a token every four seconds - and later suspected an offload bug. Someone streaming experts from disk on a Mac measured about one token per minute.

That is the honest caveat, and it deserves to sit next to the headline. "Open" here means auditable, inspectable and adaptable - not cheap, not free, and not interactive on hardware you own. Moonshot also warns in its own release notes that K3 can destabilize if a harness fails to preserve its full thinking history, that it can act over-eagerly under ambiguous instructions, and that it still has a noticeable usability gap against Fable and Sol.

The defensible summary: an open-weight model, at its documented maximum reasoning setting, won a dated public preference contest for building web applications against the two leading closed frontier models. That is a real milestone, and it is not the same as leading the field.

See also: Moonshot ships Kimi K3 weights, the hardware floor for Kimi K3 is eight datacenter GPUs, and how AI is benchmarked.


Originally published on Ground Truth, where every claim is checked against the primary source.

Top comments (0)