Open models cut the gap to 6 points in 2026 — then lost half their accuracy at the endpoint

Summary. On 30 April 2026, Artificial Analysis measured the best open-weight models at 54 on its Intelligence Index against 60 for GPT-5.5 — a 6-point gap, down from roughly 13 points a year earlier. Adoption followed: Chinese open-weight models took 41% of Hugging Face downloads in spring 2026, and open models handled nearly a third of AI requests on Vercel's gateway in June 2026. Enterprise spend did not follow. Menlo Ventures put open-source at 11% of enterprise LLM usage in its December 2025 survey of 495 US decision-makers, down from 19% a year before. On 4 August 2026 Artificial Analysis published the missing variable: gpt-oss-120b scored 22% on the BFCL-500 tool-calling evaluation on some serverless endpoints against 37% for the reference deployment of the identical weights. The model is not the gap any more. The serving stack is.

That last number is the one to sit with. Two teams can download the same checkpoint, point at two different providers, and get results 15 percentage points apart on tool calling — with no line of application code changed between them.

The benchmark gap really did close

The capability argument for staying closed is weaker than it was, and the numbers are specific.

Artificial Analysis measured Moonshot AI's Kimi K2.6 (Reasoning) and Xiaomi's MiMo V2.5 Pro (Reasoning) tied at 54 on the Intelligence Index on 30 April 2026, with DeepSeek V4 Pro (Reasoning, Max Effort) at 52. The leading proprietary models sat at 60 for OpenAI's GPT-5.5 (xhigh) and 57 for both Google's Gemini 3.1 Pro Preview and Anthropic's Claude Opus 4.7 (Adaptive Reasoning, Max Effort). Twelve months earlier the best open-weight model was DeepSeek V3 0324 at 22, about 13 points behind Claude 3.7 Sonnet (Reasoning) at 35.

Price moved further than capability. Artificial Analysis found 9 of the 13 models on its intelligence-versus-price Pareto frontier were open weights, delivering "comparable intelligence to leading proprietary models at between half to one-sixth of the price." By 31 July 2026, DeepSeek V4 Flash 0731 scored 50, one point behind GPT-5.6 Luna (max) at 51 — and even after OpenAI's 80% price cut on Luna that same day, DeepSeek's cost per task on its first-party API came in roughly 60% lower. A large part of that is DeepSeek's ~98% cache-hit discount against the 90% most of the industry offers. DeepSeek V4 Flash 0731 lists at $0.14/$0.28 per 1M input/output tokens with a 1M-token context window, at 284B total parameters and 13B active.

Model (April–July 2026)	Intelligence Index	Weights	Notable
GPT-5.5 (xhigh), OpenAI	60	Closed	Leading proprietary score, 30 Apr 2026
Gemini 3.1 Pro Preview, Google	57	Closed	Tied with Claude Opus 4.7
Kimi K3 (max), Moonshot AI	57	Open	Open-weights frontier as of 31 Jul 2026
Kimi K2.6 (Reasoning), Moonshot AI	54	Open	1T total / 32B active, 256K context
MiMo V2.5 Pro (Reasoning), Xiaomi	54	Open	1T total / 42B active, 1M context
DeepSeek V4 Pro (Reasoning, Max Effort)	52	Open	1.6T total / 49B active, 1M context
GPT-5.6 Luna (max), OpenAI	51	Closed	80% price cut on 31 Jul 2026
DeepSeek V4 Flash 0731	50	Open (weights pending)	$0.14/$0.28 per 1M tokens

Source: Artificial Analysis Intelligence Index, 30 April 2026 and 31 July 2026.

Where the 6 points still live

An average hides the shape of the remaining gap, and the shape is what decides whether a workload can move.

On Humanity's Last Exam, the three leading open-weight models scored 34–36% against 44% for GPT-5.5 (xhigh) and 45% for Gemini 3.1 Pro Preview. On CritPt, a research-level physics evaluation, they scored 4–12% against 27% for GPT-5.5 (xhigh). On TerminalBench Hard, covering agentic coding and terminal use, they scored 43–46% against 61% for GPT-5.5 (xhigh) and 54% for Gemini 3.1 Pro Preview.

The widest split is hallucination. On the AA-Omniscience index, which combines knowledge with hallucination penalties, DeepSeek V4 Pro (Reasoning, Max Effort) scored -10, MiMo V2.5 Pro +4 and Kimi K2.6 +6, against +20 for GPT-5.5 (xhigh), +26 for Claude Opus 4.7 and +33 for Gemini 3.1 Pro Preview. DeepSeek V4 Flash 0731 improved its Omniscience index to -16 from -23, and Artificial Analysis was explicit that the improvement came entirely from a lower hallucination rate — 84%, down 11 points — with accuracy unchanged at 37%.

That distribution has a direct operational reading. A retrieval-grounded summariser with citations in the loop cares less about unprompted recall than a customer-facing assistant answering from parametric memory does. The former is a reasonable open-weight candidate today. The latter is where the remaining points bite.

The August 2026 finding: same weights, different accuracy

Until 4 August 2026, model leaderboards implicitly assumed a checkpoint behaves the same wherever it runs. The Artificial Analysis Endpoint Accuracy Index tests that assumption directly, and it does not hold.

The method is worth stating because it is what makes the result usable. Artificial Analysis self-hosts the official weights at the lab's recommended precision, following the lab's serving recipe, and publishes the commands. Each serverless endpoint is then scored as a percentage of that reference across three equally weighted areas: tool calling on BFCL-500 (500 questions, 3 repeats), scientific reasoning on HLE-250 (250 questions, 10 repeats) and long-context recall on AA-LCR-25 (25 questions, 10 repeats). An endpoint counts as at reference parity when its result falls inside the 95% confidence interval of the reference. Coverage launched with GLM-5.2, gpt-oss-120b and DeepSeek V4 Pro, with Kimi K3 to follow.

The findings, in Artificial Analysis's own words: "Providers trade off accuracy to optimize for speed and cost. They quantize weights, write custom kernels and tune their inference stacks, and sometimes they simply ship bugs."

Three results matter for anyone picking a provider.

On gpt-oss-120b, tool-call handling separated endpoints: "Providers parse and format tool calls differently, and some endpoints score 22% on BFCL-500 against 37% for the reference." Serving configuration also changed behaviour at identical requested settings, with some endpoints producing far fewer reasoning tokens at the same configured level.

On GLM-5.2, output token limits were the binding constraint: restrictive limits cut responses off before the model finishes reasoning, and "the most restrictive endpoints score half the reference or less on HLE-250."

On DeepSeek V4 Pro, most endpoints were at reference parity, and DeepSeek's own first-party endpoint scored slightly above the reference.

The common signature is token count. Artificial Analysis found that endpoints scoring below the reference generally produce fewer output tokens per task, with the lowest-scoring endpoints on both models producing roughly half the reference's output tokens.

Failure mode at the endpoint	What it looks like in production	Measured effect
Tool-call parsing and formatting	Agent loops stall, retries climb, silent argument drops	gpt-oss-120b: 22% vs 37% reference on BFCL-500
Restrictive output token limits	Answers truncate mid-reasoning on hard tasks	GLM-5.2: most restrictive endpoints at half the reference or less on HLE-250
Reduced reasoning effort at same setting	Same config, fewer reasoning tokens, weaker answers	Lowest-scoring endpoints produce roughly half the reference's output tokens
Reduced context window	Long-document recall degrades without an error	Long-context tasks truncated on gpt-oss-120b endpoints
Quantisation and custom kernels	Quality drift with no version change to point at	Cited by Artificial Analysis as a general provider trade-off

Source: Artificial Analysis Endpoint Accuracy Index, 4 August 2026.

The real cost of an open-weight migration is usually not the model swap. It is discovering, three months in, that the cheap endpoint you benchmarked in a notebook is not the one serving your traffic at peak.

Adoption is high, enterprise spend is not

Two datasets look contradictory until you separate requests from budgets.

On the adoption side, Hugging Face data reported by TechCrunch put Chinese open-weight models at 41% of downloads on the platform in spring 2026, ahead of US models. The top six models on OpenRouter's rankings were all open models from Chinese firms including Tencent, Xiaomi, DeepSeek, MiniMax and Z.ai, with Claude Opus 4.7 in seventh at the time TechCrunch reported it on 14 July 2026. Vercel's data showed open-weight models absorbing much of the volume-heavy infrastructure of AI apps, handling nearly a third of AI requests on the platform in June 2026. Hugging Face itself hosts almost three million public models and one million public datasets, with a new repository created every seven seconds.

On the spend side, Menlo Ventures surveyed 495 US enterprise AI decision-makers between 7 and 25 November 2025 and reported that open-source share of enterprise LLM usage fell "from 19% last year to 11% today," attributing part of it to Llama's stagnation. Chinese open-source models collectively accounted for just 1% of total enterprise LLM API usage, roughly 10% of the enterprise open-source slice. Over the same period Anthropic reached an estimated 40% of enterprise LLM spend, OpenAI 27% and Google 21% — 88% between them, out of $12.5 billion spent on foundation model APIs in 2025.

Both readings are accurate. Open weights win where volume is high and per-call value is low: classification, extraction, embeddings, first-pass drafting, internal tooling. Closed APIs keep the workloads where an error is expensive and the buyer is not the developer. Menlo's own methodology note matters here — its $37 billion figure excludes chips, inference and model serving, so a rupee spent renting GPUs to self-host an open model does not appear in the 11%.

Signal	Figure	Date	Source
Chinese open-weight share of Hugging Face downloads	41%	Spring 2026	Hugging Face via TechCrunch
Open-model share of AI requests on Vercel	Nearly one third	June 2026	Vercel AI Gateway
Open-source share of enterprise LLM usage	11% (from 19%)	Nov 2025 survey	Menlo Ventures
Chinese open-source share of enterprise LLM API usage	1%	Nov 2025 survey	Menlo Ventures
Anthropic / OpenAI / Google share of enterprise LLM spend	40% / 27% / 21%	2025	Menlo Ventures
Foundation model API spend	$12.5 billion	2025	Menlo Ventures

Hugging Face CEO Clem Delangue framed the split this way on TechCrunch's Equity podcast: "Maybe in a few years, the frontier models will be for experimenting and [for] some really high-value tasks, and most of the production workloads will actually be powered either by private models within companies or by open source models." His argument for the shift is ownership rather than benchmarks: "If you're an AI company or a technology company, you don't want to outsource your core capabilities to another company, to a black box API that you don't control, don't have any visibility on, and don't really have any sort of ownership."

A decision procedure that survives the endpoint problem

The endpoint data changes the order of operations. Picking a model first and a provider later is now backwards, because provider choice can cost more accuracy than the model gap it was meant to close.

Step 1: score the workload, not the model

Sort by how expensive an error is. Grounded extraction with a schema and validation tolerates a weaker model. Unsupervised agentic work with tool calls and money at the end does not. The tool-calling spread on gpt-oss-120b endpoints, 22% against 37%, is larger than the 6-point index gap between the open and closed frontiers.

Step 2: benchmark the endpoint you will actually use

Run your own evaluation set against the specific provider and the specific configured reasoning level, not against a leaderboard row. Log output token counts per task alongside accuracy; Artificial Analysis found reduced token production is the leading indicator of a degraded endpoint, and it shows up before quality complaints do.

Step 3: pin the serving contract, not just the model name

Record context window, maximum output tokens, reasoning level, quantisation and tool-call format for every endpoint in use. A provider can change any of these without a version bump on the model.

Step 4: keep a reference deployment

Self-hosting the official weights at the lab's recommended precision for evaluation only — not for production traffic — gives a baseline to detect drift against. This is exactly what Artificial Analysis does, and it is the cheapest way to know whether a regression came from your prompt or from your provider.

Step 5: route, do not migrate

A gateway that can send high-stakes calls to a closed model and volume calls to an open one, with per-route evaluations, captures most of the price advantage without betting the product on one vendor. Our own view of LLM hybrid routing and API spend decisions covers the routing mechanics.

Step 6: recheck quarterly

DeepSeek V4 Flash gained 10 index points between April and July 2026. GPT-5.6 Luna's price fell 80% in a single day on 31 July 2026. Any procurement decision written to survive a year will be wrong within a quarter.

Teams weighing the infrastructure side of this should read the open-weight self-hosting decision for Kimi K3, DeepSeek V4 and GLM-5.2 alongside the running comparison of Gemini 3.5 Pro, GPT-5.6 and Claude Fable 5, and the practical guide to running local LLMs in production with vLLM, Ollama and LM Studio.

India-specific considerations

For Indian teams the arithmetic differs on two axes, and both push toward open weights earlier than a US buyer would move.

Engineering cost is the first. A senior inference engineer in India costs a fraction of the US loaded cost, so the crossover volume at which self-hosting beats an API arrives at lower token throughput. That advantage disappears if the team cannot staff on-call for a GPU fleet, which is the more common failure than the maths being wrong.

Data residency is the second. The Digital Personal Data Protection Act 2023 and its rules push personal data processing toward architectures where the processing location is a controlled variable. An open-weight model served from Indian infrastructure removes a category of question that a cross-border API call keeps raising. That is a design argument, not a compliance guarantee — eCorpIT designs applications aligned with DPDP requirements rather than certifying anyone compliant.

The endpoint finding cuts across both. An Indian buyer choosing a low-cost serverless endpoint to serve an open-weight model is exposed to exactly the accuracy variance Artificial Analysis measured, and the cheapest endpoint is the one most likely to be trading accuracy for cost. Price per million tokens is not the comparison. Price per correctly completed task is, and that requires measuring the endpoint. Teams comparing the economics should also read our analysis of Chinese open models and enterprise AI cost.

What this means for a 2026 model strategy

The strategic question stopped being "open or closed" some time in the first half of 2026. Kimi K3 at 57 sits level with Gemini 3.1 Pro Preview and Claude Opus 4.7 at 57. When the frontier open model matches the frontier closed model on an aggregate index, aggregate indices stop being the deciding input.

Three things decide it now. First, the hallucination and hard-reasoning gap, which is real and concentrated: 4–12% against 27% on CritPt is not a rounding error. Second, endpoint quality, which is measurable for the first time as of 4 August 2026 and varies more than most buyers assumed. Third, ownership, which is a governance decision rather than a benchmark one — Microsoft CEO Satya Nadella argued in July 2026 that "it's imperative that we distribute the learning infrastructure to every firm so that they can control their own learning loop."

The teams that will get this right are the ones treating the inference provider as a component with its own quality bar, versioned and monitored like any other dependency.

FAQ

How large is the gap between open and closed models in 2026?

Artificial Analysis measured a 6-point gap on its Intelligence Index on 30 April 2026, with the best open-weight models at 54 against 60 for GPT-5.5. Twelve months earlier the gap was roughly 13 points. By 31 July 2026 Kimi K3 reached 57, matching Gemini 3.1 Pro Preview and Claude Opus 4.7.

Why does the same open model score differently on different providers?

Providers quantise weights, write custom kernels and tune inference stacks to optimise for speed and cost. Artificial Analysis found some gpt-oss-120b endpoints scored 22% on BFCL-500 tool calling against 37% for its reference deployment of identical weights, and that low-scoring endpoints produce roughly half the reference's output tokens.

If open models are this good, why is enterprise share only 11%?

Menlo Ventures surveyed 495 US enterprise decision-makers in November 2025 and found open-source share fell from 19% to 11%, partly attributing it to Llama's stagnation. Its figure also excludes inference and model serving spend, so GPU rental for self-hosted open models does not appear in that percentage at all.

Which workloads should stay on closed models?

Work where errors are expensive and the model answers from parametric memory rather than retrieved context. The AA-Omniscience gap is wide: DeepSeek V4 Pro scored -10 and Kimi K2.6 +6 against +33 for Gemini 3.1 Pro Preview. Research-level reasoning also lags, at 4-12% versus 27% on CritPt.

What does the Endpoint Accuracy Index actually measure?

It scores each serverless endpoint as a percentage of a self-hosted reference deployment of the official weights, across three equally weighted areas: tool calling on BFCL-500, scientific reasoning on HLE-250 and long-context recall on AA-LCR-25. An endpoint reaches parity when its result falls inside the reference's 95% confidence interval.

Is self-hosting open weights cheaper than a closed API?

It depends on sustained volume, not list price. DeepSeek V4 Flash 0731 costs $0.14/$0.28 per 1M tokens with a 98% cache-hit discount, and its cost per task ran roughly 60% below GPT-5.6 Luna even after an 80% price cut. Engineering and on-call cost decide whether that saving survives contact with production.

How often should a model decision be revisited?

Quarterly at minimum. DeepSeek V4 Flash gained 10 Intelligence Index points between its April and July 2026 releases, and OpenAI cut GPT-5.6 Luna pricing by 80% on 31 July 2026. Procurement written to hold for a year will be stale within one quarter on both capability and price.

What should Indian teams check before moving to open weights?

Two things. Whether the team can staff on-call for a GPU fleet, since lower engineering cost only helps if the capacity exists. And whether the chosen endpoint has been measured rather than assumed, because the cheapest serverless endpoint is also the one most likely to be trading accuracy for cost.

How eCorpIT can help

eCorpIT builds and operates production LLM systems for teams that need the cost profile of open weights without the accuracy variance. Our senior engineering teams set up per-route evaluation harnesses, reference deployments for drift detection, and gateway routing that keeps high-stakes calls on the model that earns them. We are CMMI Level 5 and ISO 27001:2022 certified, and we design applications aligned with DPDP requirements for teams with data residency constraints. To review your current model and endpoint mix, talk to our team.

References

1. Artificial Analysis, Recent open weights model launches, 30 April 2026.
2. Artificial Analysis, Launching the Endpoint Accuracy Index: Same Model, Different Accuracy, 4 August 2026.
3. Artificial Analysis, DeepSeek V4 Flash 0731 scores 50 on the Artificial Analysis Intelligence Index, 31 July 2026.
4. Artificial Analysis, Endpoint Accuracy Index methodology, accessed 6 August 2026.
5. Menlo Ventures, 2025: The State of Generative AI in the Enterprise, 9 December 2025.
6. Rebecca Bellan, The real AI race may no longer be at the frontier, TechCrunch, 14 July 2026.
7. Artificial Analysis, LLM leaderboard, accessed 6 August 2026.
8. OpenRouter, Model rankings, accessed 6 August 2026.
9. Menlo Ventures, 2025 State of Generative AI in the Enterprise (PDF), December 2025.
10. Artificial Analysis, GLM-5.2 providers, accessed 6 August 2026.
11. Artificial Analysis, gpt-oss-120b providers, accessed 6 August 2026.
12. Artificial Analysis, DeepSeek V4 Pro providers, accessed 6 August 2026.

Last updated: 6 August 2026.