Open models now score close to frontier models on benchmarks
An open model is a model whose weights you can download. DeepSeek, Alibaba's Qwen, Moonshot's Kimi, Z.ai's GLM and Google's Gemma publish their weights. OpenAI published gpt-oss in 2025. You can run these models on your own servers or rent them from a hosting company.
For years, the difference between open and closed models was simple: closed frontier models scored higher on benchmarks, and open models were cheaper. But now the benchmark score difference is very small. Artificial Analysis runs every major model through the same tests. When Kimi K3 came out, it measured the difference between the best closed model and the best open model at 4 points on its index. Mozilla's State of Open Source AI report, published in July 2026, puts the difference at about 3%.
Companies now use open models in production. In Mozilla's survey, 79% of developers use open models, and 51% have put them in production. In October 2025, Airbnb's chief executive said the company's customer service agent relies a lot on Qwen: "It's very good. It's also fast and cheap."
Example: an AI assistant for an accounting software company
Say you sell software to small accounting firms and want to add an assistant. It has five jobs:
- Sort incoming email into four groups: questions, bugs, billing, spam.
- Read an uploaded invoice and return the vendor, date, total, and tax.
- Summarize a long email thread.
- Draft a reply. A person reviews it before it goes out.
- Answer customer questions in a chat window, using your help articles.
Jobs 1 and 2 run thousands of times a day. Their answers are short and easy to check. Jobs 3 to 5 need a model that writes well and follows instructions. None of the five needs the most expensive model on the market. The table gives one model for each job, on a closed (frontier) API and on open models.
| Job | Closed model | Open model |
|---|---|---|
| Sort incoming email | Claude Haiku 5.5 or GPT-6 Luna | gpt-oss-120b or GLM 5.3 Flash |
| Read an invoice image | Claude Haiku 5.5 or GPT-6 Luna | GLM 5.3 Flash (reads images) |
| Summarize a thread | Claude Sonnet 5.5 or GPT-6.1 Sol | DeepSeek V4.1 Flash |
| Draft a reply | Claude Sonnet 5.5 or GPT-6.1 Sol | GLM 5.3 |
| Support chat over help articles | Claude Sonnet 5.5 or GPT-6.1 Sol | GLM 5.3 or DeepSeek V4.1 Flash |
Price per million tokens
The table shows list prices per million tokens on 11 October 2026.
| Model | Best for | Input | Output |
|---|---|---|---|
| GPT-6 Astra or Claude Fable 5.1 | Hardest tasks, long agent runs | 10.00 | 50.00 |
| Claude Opus 5.5 | Agent runs, coding | 4.00 | 20.00 |
| GPT-6.1 Sol or Claude Sonnet 5.5 | Everyday work: chat, drafts, summaries | 2.00 | 10.00 |
| GPT-6 Luna or Claude Haiku 5.5 | Simple jobs at high volume | 0.10 | 0.50 |
| Kimi K3 (open) | Hardest tasks, agents, images | 3.00 | 15.00 |
| GLM 5.3 (open) | Everyday work, coding | 1.40 | 4.40 |
| DeepSeek V4.1 Flash (open) | Fast everyday work, images | 0.30 | 1.20 |
| GLM 5.3 Flash (open) | Simple jobs at high volume, images | 0.15 | 0.50 |
| gpt-oss-120b (open) | Simple jobs at high volume | 0.10 | 0.60 |
Sources: OpenAI pricing, Anthropic pricing, Keln model list. Those are standard rates for prompts of normal length (they go higher with longer prompts) on 11 October 2026.
The table has three price levels.
- Cheapest models. Luna, Haiku, gpt-oss-120b and GLM 5.3 Flash cost $0.10 to $0.15 per million input tokens.
- Mid-price models. Most products use this level. GLM 5.3 costs 30% less than Sonnet or Sol on input and 56% less on output.
- Most expensive models. GPT-6 Astra or Claude Fable 5.1 cost 7 times more than GLM 5.3 on input and 11 times more on output.
Let's calculate the price of a specific task
The task: summarize a customer email thread of about 2,000 tokens into a 200-token summary.
| Model | One summary | 10,000 summaries |
|---|---|---|
| GPT-6 Astra or Claude Fable 5.1 | $0.030 | $300 |
| Claude Opus 5.5 | $0.012 | $120 |
| Claude Sonnet 5.5 or GPT-6.1 Sol | $0.006 | $60 |
| GLM 5.3 (open) | $0.0037 | $36.80 |
| DeepSeek V4.1 Flash (open) | $0.0008 | $8.40 |
Each model got the same email thread, word for word, and the same email summary request.
Monthly bill for a support chat
Take job 5, the support chat. It handles 5,000 requests a day. Each request sends about 2,000 tokens: the customer's message, the last few turns and one help article. The model returns about 300 tokens. In a month, that is 300 million input tokens and 45 million output tokens.
| Model | Monthly bill |
|---|---|
| GPT-6 Astra or Claude Fable 5.1 | $5,250 |
| Claude Opus 5.5 | $2,100 |
| GPT-6.1 Sol or Claude Sonnet 5.5 | $1,050 |
| GLM 5.3 (open) | $618 |
| DeepSeek V4.1 Flash (open) | $144 |
At this volume, only for this task, the switch saves a few hundred to a few thousand dollars a month. The savings grow with traffic.
Four ways to start running open models
1. Self-host
You run open models on GPUs you own or rent.
Pros: your data stays on machines you fully control.
Cons: you pay for the GPUs whether you use them or not, and you handle updates and uptime. Worth it at high volume or under strict data rules.
2. Dedicated endpoint
A hosting company runs the model on GPUs reserved for you and gives you a URL. You pay per GPU hour.
Pros: only your traffic runs on your GPUs, so speed is steady, and the host company does the serving work.
Cons: you still pay for idle time, so it needs enough traffic to keep the GPUs busy.
3. Shared provider
A hosting company runs the model for many customers and charges per token. You sign up, get a key, change two lines of code.
Pros: cheapest way to start, no contract, nothing to pay when traffic is quiet.
Cons: you depend on one company, and demand for open models is high, so hosts often go down or slow at peak. Some serve a compressed copy of the model to cope, and do not tell you. You should also read the privacy page before starting: not every provider promises not to store or train on your data.
4. Gateway
One endpoint in front of several shared providers. It picks the provider for each request: the cheapest or fastest.
Pros: it keeps working when one provider goes down or slows, and you can compare lots of models.
Cons: a gateway with 100+ models can't check each one at each provider closely, so quality and reliability can vary.
Can I use open models but at the quality and reliability of frontier labs?
There is a newer kind of company: a reliability layer for open models. Their goal is to serve open models with the reliability of a frontier lab's API.
Compared with a gateway, these offer:
- A short list of the most-used open models, each one consistently checked for quality at every provider.
- One fixed price per model, whichever provider serves the request.
- Mid-stream failover. If a provider fails while a response streams, another provider continues the same response.
- Each request goes to the fastest checked provider, not the cheapest.
- One format for tool calls across all providers.
Keln also has zero data retention at every provider and doesn't bill for a failed request. It accepts both the OpenAI and the Anthropic request format, so you switch by changing two lines. You can also run open models in Claude Desktop (setup guide) and Codex (setup guide) through Keln.
The result is open models at up to an eighth of the frontier price, a few points behind on benchmarks, and reliability built to match OpenAI's or Anthropic's.
How to start using open models in less than 2 minutes
Most open-model providers accept OpenAI's request format. To start using open models you change three settings: the provider's address, the access key and the model name. The example below uses Keln; other providers work the same way with their own address.
- Sign up with a provider and create a key in its dashboard.
- Pick a model for each job. Use the "Best for" column in the pricing table above. Copy the model's exact name from the provider's model list.
-
Swap the provider's address and access key:
client = OpenAI( base_url="https://api.keln.ai/v1", api_key=os.environ["KELN_API_KEY"], ) -
Add a model name:
response = client.chat.completions.create( model="zai-org/glm-5.3", messages=[{"role": "user", "content": "Sort this email: question, bug, billing or spam?"}], ) Run the product's usual requests on the open model and compare the answers with what you get today.
Move a small share of real traffic first, then the rest.
👋 Hey, I'm Kris Holysheva. I'm a co-founder at Keln, a reliability layer for open models, and I write about how teams can cut their AI bill without losing frontier-lab quality and support. Reach out with any questions.
Top comments (0)