TL;DR
- On October 19, 2026, BenchLM and Artificial Analysis both put MiMo-V2.6-Pro first among open-weight models (73.7 on BenchLM, Intelligence Index 46.3 on Artificial Analysis).
- That agreement is thinner than it looks. BenchLM tags the #1 row Estimated with a conditional range of 63.92 to 83.37, and the ranges of the top nine rows all overlap each other.
- Below #1 the orderings drift: on the nine models I could match, Kendall's tau is 0.61 (29 concordant pairs, 7 discordant). DeepSeek V4.1 Flash 1017 moves from 6th to 3rd.
- The bigger disagreement is membership. Kimi K3 is missing from BenchLM's open list (weights marked "pending") while its Hugging Face repo is public with 96 safetensors shards. Qwen3.8 Max is #2 on BenchLM's open list but flagged not open-weights on Artificial Analysis.
- Before you cite "best open model", audit the label, the evidence tag, and the checkpoint name. A short script for each is below.
All numbers are from pages and APIs I pulled on October 19, 2026. Leaderboards change daily, so treat this as a dated snapshot.
Which open-weight model is #1 right now?
Searching for "best open source LLM" returns several aggregators that look alike but compute different things. I pulled two of them directly:
- BenchLM publishes a composite called BenchAlign v5.8. Its open-source page ranks 103 open-weight models out of 955 tracked models and 640 tracked benchmarks.
-
Artificial Analysis publishes an Intelligence Index. The page ships its row data as JSON, with fields such as
intelligenceIndex,isOpenWeights,hle,gpqa,scicode,terminalBench21andtau2.
Both lists put MiMo-V2.6-Pro on top. That is the headline most people will quote. Here are the BenchLM top six as published:
| Rank | Model | Score | Evidence tag | Published range |
|---|---|---|---|---|
| 1 | MiMo-V2.6-Pro | 73.7 | Estimated | 63.92 to 83.37 (conditional) |
| 2 | Qwen3.8 Max | 69.7 | Supported | 65.62 to 73.78 (90% interval) |
| 3 | Step 5 Preview | 69.5 | Estimated | 59.80 to 79.24 (conditional) |
| 4 | Ling 3.1 Flash | 68.1 | Estimated | 53.73 to 82.43 (conditional) |
| 5 | GLM-5.3 | 67.9 | Supported | 63.45 to 72.40 (90% interval) |
| 6 | DeepSeek V4.1 Flash 1017 | 65.6 | Supported | 60.65 to 70.62 (90% interval) |
How wide is the uncertainty behind that #1?
The ranges are the part of the page that most readers skip. BenchLM's methodology page defines the two tags in one sentence: "Supported means the row clears the independent-evidence rules. Estimated means it relies on thinner evidence." The #1 row is in the second group.
Three things follow from the published numbers:
- The #1 range is 19.45 points wide. The Supported rows in the same table are 8.2 to 10.0 points wide.
- Every pair overlaps. Among the nine models I matched across both leaderboards (more on that below), all 36 pairs of ranges overlap. Including the tenth BenchLM row, it is 45 of 45.
- BenchLM says so itself. The page states that MiMo-V2.6-Pro "has the higher point estimate" and that "conditional ranges do not establish rank confidence."
So the correct reading is "MiMo-V2.6-Pro has the highest point estimate among open-weight rows, on thin evidence." That is a weaker claim than "MiMo-V2.6-Pro is the best open model", and it is the only one the page supports.
Note also that a conditional range is not a 90% interval. The two kinds of row use different constructions, so comparing the width of an orange bar with a blue bar is informative but not apples to apples.
Do the two leaderboards agree below rank 1?
Partly. I matched BenchLM's top ten open rows against Artificial Analysis rows by model name. Nine matched; "dots3-note Preview" has no row on Artificial Analysis in the data I pulled. Ranking only those nine by each source gives:
| Model | BenchLM rank (of 9) | Artificial Analysis rank (of 9) | AA Intelligence Index |
|---|---|---|---|
| MiMo-V2.6-Pro | 1 | 1 | 46.3 |
| Qwen3.8 Max | 2 | 2 | 45.4 |
| Step 5 Preview | 3 | 5 | 43.7 |
| Ling 3.1 Flash | 4 | 6 | 41.1 |
| GLM-5.3 | 5 | 4 | 44.8 |
| DeepSeek V4.1 Flash 1017 | 6 | 3 | 45.1 |
| MiMo-V2.6-Flash | 7 | 8 | 37.9 |
| DeepSeek V4 Pro 0813 | 8 | 9 | 36.0 |
| Qwen3.8-Flash-Next | 9 | 7 | 39.8 |
Rank correlation is a one-screen computation. This script reproduces the numbers above:
from itertools import combinations
# name: (benchlm_score, benchlm_low, benchlm_high, aa_index)
rows = {
"MiMo-V2.6-Pro": (73.7, 63.92, 83.37, 46.3),
"Qwen3.8 Max": (69.7, 65.62, 73.78, 45.4),
"Step 5 Preview": (69.5, 59.80, 79.24, 43.7),
"Ling 3.1 Flash": (68.1, 53.73, 82.43, 41.1),
"GLM-5.3": (67.9, 63.45, 72.40, 44.8),
"DeepSeek V4.1 Flash 1017": (65.6, 60.65, 70.62, 45.1),
"MiMo-V2.6-Flash": (64.5, 54.82, 74.26, 37.9),
"DeepSeek V4 Pro 0813": (63.2, 48.81, 77.51, 36.0),
"Qwen3.8-Flash-Next": (62.8, 48.48, 77.18, 39.8),
}
def kendall_tau(a, b):
conc = disc = 0
for i, j in combinations(range(len(a)), 2):
s = (a[i] - a[j]) * (b[i] - b[j])
conc += s > 0
disc += s < 0
return (conc - disc) / (conc + disc), conc, disc
def overlapping_pairs(ranges):
n = sum(1 for (l1, h1), (l2, h2) in combinations(ranges, 2) if l1 <= h2 and l2 <= h1)
return n, len(ranges) * (len(ranges) - 1) // 2
bl = [v[0] for v in rows.values()]
aa = [v[3] for v in rows.values()]
tau, conc, disc = kendall_tau(bl, aa)
print(f"Kendall tau={tau:.3f} concordant={conc} discordant={disc}")
print("range overlaps:", overlapping_pairs([(v[1], v[2]) for v in rows.values()]))
Output: Kendall tau=0.611 concordant=29 discordant=7 and range overlaps: (36, 36).
A tau of 0.61 on nine items is "broadly similar", not "the same". The practical consequence is that the middle of the table is not stable. If your shortlist cutoff is "top 5 of these nine", then Ling 3.1 Flash is in on BenchLM and out on Artificial Analysis, while DeepSeek V4.1 Flash 1017 is the reverse.
Artificial Analysis has its own tight cluster. Its open-weight top five are 46.3, 45.1, 44.8, 43.7 and 43.6, so adjacent gaps are 1.2, 0.3, 1.0 and 0.1 index points, and the whole top five spans 2.7 points. The page I pulled shows one number per row, so I cannot tell you whether a 0.1 gap is real. Neither can the table.
Why is Kimi K3 missing from one "open" list and Qwen3.8 Max flagged closed on the other?
This is the finding I would put ahead of the rank noise. "Open-weight" is a label that each aggregator assigns, and the labels disagree.
Kimi K3. On BenchLM it scores 69.92 and ranks #18 of 226 ranked models overall, #14 of 88 on source-verified rows. Its profile says "Weight access pending" and "no verified publication yet for weight access". It is therefore absent from the 103-row open list. By point estimate it would sit at #2 there, just ahead of Qwen3.8 Max at 69.7. Its profile also notes that only 56 of 682 tracked benchmark slots have displayable evidence, so the score itself rests on partial coverage.
On Artificial Analysis the same model, "Kimi K3 (Max)", is flagged isOpenWeights: true with an Intelligence Index of 43.6. And on the Hugging Face Hub, moonshotai/Kimi-K3 is public and not gated, ships 96 safetensors shards, carries a custom license named kimi-k3 (the Hub tag is license:other), and showed 1,012,867 downloads when I queried it.
Qwen3.8 Max. BenchLM lists it as Open Weight at #2. Artificial Analysis lists "Qwen3.8 Max (0902)" with an index of 45.4 and isOpenWeights: false. I could not resolve Qwen/Qwen3.8-Max or Qwen/Qwen3.8-Max-0902 on the Hub API from an anonymous request (HTTP 401, which the Hub also returns for repos that do not exist). The two leaderboards may be naming different snapshots, which is a problem in itself: the same display name without a checkpoint identifier is not a stable key.
MiMo-V2.6-Pro. The Hub has XiaomiMiMo/MiMo-V2.6-Pro-RL (MIT license, public, 132 safetensors files). A repo named exactly MiMo-V2.6-Pro does not resolve anonymously. Whether the benchmarked row is the -RL checkpoint is not stated on either leaderboard page I read.
Here is the check I ran, so you can repeat it on any model you plan to cite:
import json
import urllib.error
import urllib.request
def hub_status(repo: str) -> dict:
url = f"https://huggingface.co/api/models/{repo}"
req = urllib.request.Request(url, headers={"User-Agent": "leaderboard-audit"})
try:
with urllib.request.urlopen(req, timeout=30) as r:
info = json.load(r)
except urllib.error.HTTPError as e:
return {"repo": repo, "resolves": False, "http": e.code}
files = [s["rfilename"] for s in info.get("siblings", [])]
card = info.get("cardData") or {}
return {
"repo": repo,
"resolves": True,
"gated": info.get("gated"),
"license": card.get("license"),
"safetensors_files": sum(f.endswith(".safetensors") for f in files),
"downloads": info.get("downloads"),
}
for repo in ["moonshotai/Kimi-K3", "Qwen/Qwen3.8-Max",
"XiaomiMiMo/MiMo-V2.6-Pro", "XiaomiMiMo/MiMo-V2.6-Pro-RL"]:
print(hub_status(repo))
On October 19 it printed resolves: True, gated: False, license: other, safetensors_files: 96 for Kimi K3, resolves: False, http: 401 for the two names without a public repo, and license: mit, safetensors_files: 132 for the -RL repo.
Are the two leaderboards independent evidence?
Not necessarily. BenchLM's methodology page says BenchAlign "calibrates benchmark protocols, human-preference signals, and external composite indices before combining them." If one of those external composite indices is the other leaderboard, then agreement between the two is partly circular, and a tau of 0.61 would overstate how much independent confirmation you have. The pages I read do not name the external indices, so I cannot tell either way. That is itself worth knowing: an agreement you cannot decompose is weaker evidence than a disagreement you can.
The inputs also differ in kind. Artificial Analysis publishes one index built from a fixed set of evaluations. BenchLM tracks 640 benchmarks, scores a subset of them across six pillars, keeps vendor-run numbers out of the rank, and fills gaps with a learned estimate. Different inputs plus similar outputs is mildly reassuring. Different inputs plus different open-weight labels is a reason to check the label yourself.
A five-step audit before you quote an open-model rank
- Read the evidence tag. "Estimated" or "conditional range" means the position leans on thin evidence. Do not write "best" on top of it.
- Compare ranges, not points. Run the overlap function above on the top rows. If everything overlaps, say "no resolvable difference".
- Resolve the checkpoint. Find the exact Hub repo or API model ID the row refers to. Display names such as "Max" or "Pro" do not identify a checkpoint.
- Verify the open-weight label. Check that weights exist, whether the repo is gated, and what the license actually allows. A custom license is not the same as MIT or Apache-2.0.
- Cross-check one other source. Compute a rank correlation, then look at the individual disagreements, because the interesting rows are the ones that move.
Limitations of this audit
- It is one snapshot, taken October 19, 2026. Both pages update continuously.
- The Artificial Analysis values come from the row data embedded in its leaderboard page, not from a documented API. Parsing that JSON is fragile and I did not see an index version number on the page.
- Name matching between sites was manual. Nine of ten BenchLM rows matched, and the ranks in the table are positions within those nine, not on the full lists.
- Nine items is a small sample for a rank correlation. Treat tau = 0.61 as a rough indicator.
- I did not run any model myself. Every score is as published by the aggregators.
FAQ
What is the best open-weight LLM in October 2026?
It depends on the leaderboard and on how you define open. MiMo-V2.6-Pro has the top point estimate on both BenchLM (73.7) and Artificial Analysis (46.3), but BenchLM tags that row Estimated with a 19.45 point range, so the lead over the next rows is not statistically resolved.
Why does Kimi K3 not appear on BenchLM's open-source list?
BenchLM records its weight access as pending and says it has no verified publication of the weights, even though the Hugging Face repo is public. Artificial Analysis flags the same model as open-weights.
What does "Estimated" mean on a leaderboard row?
On BenchLM it means the row relies on thinner evidence than a "Supported" row, which clears the site's independent-evidence rules. The range shown for Estimated rows is a conditional range, which BenchLM states does not establish rank confidence.
How do I check whether a model is really open-weight?
Resolve its exact repo on the Hugging Face Hub, confirm the weight files exist and the repo is not gated, and read the license text. The hub_status function above automates the first three checks.
Is a rank correlation of 0.61 between two leaderboards good?
It means most pairs are ordered the same way (29 of 36 here) but a meaningful minority flip. It also may be inflated if one leaderboard uses the other as an input.
Further reading
- How to Read a Public Leaderboard Without Fooling Yourself
- When a 0.4-Point Lead Means Nothing: Reading Open-Model Leaderboard Margins Like a Statistician
Top comments (0)