DEV Community

Cover image for Mozilla Measured the Open-Weight Gap: 5 Points, 8 of 10, 4% of Revenue
Khasky
Khasky

Posted on

Mozilla Measured the Open-Weight Gap: 5 Points, 8 of 10, 4% of Revenue

Four datasets went into Mozilla's State of Open Source AI report. A developer survey fielded with SlashData, OpenRouter's traffic panels, Epoch's Capabilities Index, and METR's time-horizon data. Read separately they support the usual arguments. Read together they describe open-weight models that are close on capability, ahead on routed traffic, and nearly absent from the revenue.

The measurements

The three measurements the report rests on, with the window each one was taken in.

Epoch Capabilities Index, September 1 data
Claude Fable 5, Claude Opus 5 (closed)     162
Kimi K3 (open)                             157
gap                                        5 points
Epoch's average gap since January          8 points (90% CI 7-11)
in calendar time                           about 4 months (Epoch), about 4.4 months (Mozilla, from METR data)
Enter fullscreen mode Exit fullscreen mode
OpenRouter, August 1-31, token volume
open weights in the top 10                 8 of 10
Chinese-built among those 8                7
first open model to lead weekly requests   DeepSeek, from August 3, after Google's 51 weeks at #1
Enter fullscreen mode Exit fullscreen mode
OpenRouter, May-September 2025, model layer
usage      open ~20%    closed ~80%
revenue    open ~4%     closed ~96%
price      closed about 6x per call, at about 90% capability parity
Enter fullscreen mode Exit fullscreen mode

The traffic figures count routed OpenRouter requests only, first-party use inside ChatGPT, Gemini and Doubao sits outside them, and the revenue split is the last one published, from a window the report says usage has since moved past.

Why a 5-point gap and a 4-month gap are the same fact

The 5 points on the September chart run under Epoch's own average of 8 since January, and that average is what Epoch converts into roughly four months of calendar lead.

Mozilla ran the conversion a second way, fitting METR's raw time-horizon data, and got about 4.4 months, with open capability doubling every 3.9 months against 5.5 for closed. The report labels its own fit a simplified reproduction rather than a result. 📐

Release timing keeps the number stable. Every model on the chart shipped between June and August, so the margin is redrawn each cycle instead of compounding across them.

Usage on OpenRouter

DeepSeek's V4 Flash 0731 led the August token table at 45.1 trillion tokens, DeepSeek held three of the top ten places, and only three US entries made the list at all. On requests rather than tokens, August 3 was the first day an open model took the top spot, and DeepSeek covered the distance from third to first in eight weeks.


What the numbers do not cover

Taking the limits before the economics is deliberate, because the revenue figure below is the one most likely to be quoted without them.

Routed traffic is not all traffic. Anything served first-party inside ChatGPT, Gemini or Doubao never appears in OpenRouter's panels, which leaves a large share of real usage outside every figure here.

The revenue split is older still. It covers May to September 2025 and has not been re-measured, and the report states that directly: usage has moved since that window, and the 4% is simply the last figure published.

The measured gap is a floor. Epoch gives two reasons it may be understated: open models optimize on benchmarks more aggressively, and unpublished closed models are absent from the baseline.

The economics

The report's own reading of the revenue table is one clause long: capability is near parity, and price drives the split. Among developers who choose open models, 30% name lower cost as a top reason and 28% name privacy, and the Linux Foundation's estimate of unrealized annual savings from that asymmetry is $24.8B.

Where open stalls

79% of surveyed developers use open models, yet 51% of open deployments reach production against 63% for closed, a gap the survey traces to tooling and trust rather than capability.

firms using open components            89%
developers using open models           79%
open models reaching production        51%
closed models reaching production      63%
Enter fullscreen mode Exit fullscreen mode

One figure underneath it carries the section. Vendor-partnered deployments reach production 67% of the time and internal builds 33%.

Capability without pretraining

DeepSeek's July 31 post-training pass lifted V4-Flash 10 points on the Artificial Analysis index on an unchanged architecture, and the August 13 pass added 8 to V4-Pro, with no new pretraining run behind either.

V4-Flash-0731   Jul 31   AA index ~40 -> ~50   +10   post-training pass, same architecture
V4-Pro-0813     Aug 13   AA index 45 -> 53     +8    post-training pass, two weeks later
Enter fullscreen mode Exit fullscreen mode

The V4-Flash pass moved Terminal-Bench 2.1 from 61.8 to 82.7 and cut eval-suite output tokens from 234M to 206M, so the score did not come from longer answers. Post-training costs a fraction of pretraining, which puts this kind of gain inside reach of anyone with an open base and a budget. 🔁

Any targeted use of another model's outputs would also happen in post-training, so the report treats the cheap-gains question and the provenance question as one.


Follow me for more on AI, LLMs, and Software Development:

@khasky — LinkedIn / Patreon / GitHub / Bluesky / Mastodon

@khaskydev — X / Threads / Instagram / Pinterest / Facebook

Top comments (0)