A 0.9-billion-parameter model beats a 27-billion one — as long as you hand it a scanned document instead of a conversation. That single fact is why "top 10 local models" lists are useless: they rank by size, and size is not what you pick a model by.
I spent a weekend going through the Ollama library sorted by task rather than by parameter count. Here is what the landscape looks like at the end of September 2026, and — more importantly — the three rules that will still hold when every name below has been replaced.
Rule 1: read the tag, not the name
Sort the Ollama library by newest and the top rows are deepseek-v4.1-flash, glm-5.3, kimi-k3, minimax-m3. All four carry a cloud tag.
That means ollama run ships your prompt to someone else's servers. It is a great way to try a model that would never fit on your machine. It is not local inference, and if you came here for privacy or offline work, those rows get crossed out first.
Rule 2: specialisation beats size on its own turf
The clearest example of the month is glm-ocr: 0.9B parameters, 2.2 GB on disk, and the top spot on OmniDocBench V1.5 at 94.62. Feed it a messy invoice with nested tables, footnotes and formulas and it holds. A general-purpose 27B vision model does not — nobody trained it for document structure.
The right pipeline is boring and effective: glm-ocr turns the document into structured text, then a general model reasons over that text.
Rule 3: the benchmark is not your task
SWE-bench, the number everyone quotes, measures fixing a bug in an unfamiliar repository from an issue description. That is very different from "write me this function" and nothing at all like "explain what this code does". A high SWE-bench score on a model you keep around for autocomplete tells you close to nothing.
Coding
Qwen3.8-27B landed on 14 August: 27.78B dense parameters, Apache 2.0, 17.7 GB in the Ollama build, a 256K context window, vision included. Qwen reports 73.0 on Terminal-Bench 2.1 (up from 63.4), 61.7 on SWE-bench Pro, 90.3 on LiveCodeBench v6 and 84.3 on OSWorld-Verified, up from 63.9. Its Artificial Analysis index went up 14 points over the previous generation on an identical architecture — all of it from post-training.
If 18 GB of weights is out of reach, the choice is between two smaller ones:
| Model | What it is | Speed | Strength |
|---|---|---|---|
qwen3.8:27b |
dense 27B, vision, 256K | ~7 tok/s on a Mac mini M4 | terminal agent, long tasks |
devstral-small-2:24b |
dense 24B from Mistral, vision | 139.2 tok/s | responsiveness, multi-file edits |
qwen3-coder:30b |
MoE, 3B active of 30 | 90.6 tok/s | quality per gigabyte |
Those throughput numbers come from Artificial Analysis on provider hardware — at home they will be lower, the ratio is what matters. One detail most reviews skip: Devstral's context costs roughly twice as much memory, about 5 GB per 32K tokens versus 3 GB for the MoE Qwen. On a machine where RAM is tight, that decides it.
Agents: Meta is back, and that is the surprise of the month
Three weeks ago Meta was the cautionary tale of open weights — the company that invented the strategy, down to under one percent of tokens on OpenRouter. Today muse-glimmer sits in the Ollama library: 30B parameters, Apache 2.0, 18.2 GB, 128K context, and over two hundred thousand pulls in three weeks.
The model card is refreshingly unambitious: tuned for "tool use, long tasks, and failure recovery". Not smartest — the one that finishes. Meta reports 75.5 on MCP Atlas, 51.2 on SWE-Bench Pro and 94.7 on AIME 2026.
That framing is the actual lesson. An agentic task is not one answer, it is a hundred in a row. A model that is right 90% of the time but never notices its own mistake loses to one at 85% that catches and redoes it.
Next to it sits NVIDIA's nemotron-3.5-lightning (11 August): 30B total, 3B active, interleaved Mamba-2 and MoE layers, 51.56 on SWE-bench Verified, 75.44 on GPQA Diamond, 81.94 on MMLU Pro. The headline 1,200 tokens per second is a provider-hardware measurement, not a laptop one — but the architecture is genuinely built to emit tokens fast.
Writing: the honest answer is still "not locally"
On EQ-Bench Creative Writing v3 the top of the board is closed: Claude Opus 5 at 2121 Elo, GPT-5.6 Sol at 1963. The best open-weights score belongs to Kimi K3 at 2071 — a 2.8-trillion-parameter model that needs a multi-node cluster. "Open" in the same sense an operating system's source code is open to someone without a chip fab.
What does work locally is working text: email, summaries, tightening a paragraph, turning a call transcript into structure. The pick there is gemma4, with 25.3M pulls the most downloaded model in the library.
A Russian-language test from May on an RTX 5070 Ti ran Gemma 4, Qwen 3.6 and Qwen3-Coder through twelve practical tasks. Gemma 4 in fast mode scored 12/12, Qwen 3.6 got 9/12. And a finding worth more than any index: turning thinking mode on made instruction-following worse — 11/12 instead of 12 for Gemma, 7 instead of 9 for Qwen.
Medicine: the one area where a specialised model is mandatory
Start with the boundary, not the models. Open medical models ship under the Health AI Developer Foundations terms, and Google says plainly that they are not clinical grade and require task-specific validation. This is a building block for a developer, not a patient-facing chatbot and not a second opinion.
Within that boundary they are genuinely useful. medgemma1.5:4b is 3.3 GB with a 128K context and vision; version 1.5 added whole-slide histopathology, longitudinal imaging, anatomical localisation and document understanding. It scores 89.6% on EHRQA and turns a lab report into structured JSON at 91.0 macro-F1. The text-only medgemma:27b reaches 87.7% on MedQA. For broad health conversations graded by physicians, gpt-oss-120b leads open models on HealthBench at 0.576 — but that is 65 GB of weights.
The realistic use for a private person: put your own records into readable shape, merge several documents into one table, prepare questions for an appointment — all on your own machine, without shipping medical data into someone's cloud.
Retrieval: embeddings matter more than the generator
The most common mistake in a homegrown RAG setup is spending all the memory on a big generator and grabbing whatever embedding model came first. It works the other way round: if retrieval returns the wrong chunks, the model on top will confidently summarise the wrong thing.
| Model | Size | When |
|---|---|---|
qwen3-embedding:0.6b |
0.6 GB | best quality per gigabyte, tunable dimensions |
qwen3-embedding:8b |
4.7 GB | maximum quality, 70.58 MTEB multilingual |
bge-m3 |
1.2 GB | multilingual corpora, hybrid dense+sparse, 8K context |
embeddinggemma:300m |
0.6 GB | high-volume indexing |
nomic-embed-text |
0.3 GB | simplest start, 86M pulls |
The Qwen3-Embedding family lets you set the vector dimension anywhere from 32 to 1024, which is a direct saving on your vector store — you choose the trade-off instead of inheriting it.
For the generator on top, something modest does the job. IBM's granite4.2 (3B/8B/30B, Apache 2.0, 128K) was tuned for exactly this: RAG, tool calls and structured JSON output.
The table
Sizes below are not approximations — I pulled them from the Ollama registry manifests, so this is the sum of layers you will actually download, as of 20 September 2026. In memory it will be more, by the size of your context.
| Task | Model | Command | Size |
|---|---|---|---|
| Terminal agent, long coding tasks | Qwen3.8-27B | ollama pull qwen3.8:27b |
17.7 GB |
| Fast coding assistant | Devstral Small 2 | ollama pull devstral-small-2:24b |
15.2 GB |
| Coding on tight memory | Qwen3-Coder 30B-A3B | ollama pull qwen3-coder:30b |
18.6 GB |
| Tool-using agent | Muse Glimmer (Meta) | ollama pull muse-glimmer:30b |
18.2 GB |
| Many small steps, speed matters | Nemotron 3.5 Lightning | ollama pull nemotron-3.5-lightning:30b |
25.4 GB |
| Writing, editing, email | Gemma 4 | ollama pull gemma4:12b |
7.6 GB |
| Same on weak hardware | Gemma 4 E2B | ollama pull gemma4:e2b |
7.2 GB |
| Medical documents and imaging | MedGemma 1.5 | ollama pull medgemma1.5:4b |
3.3 GB |
| Medical text only | MedGemma 27B | ollama pull medgemma:27b |
17.4 GB |
| Scans, PDFs, tables, formulas | GLM-OCR | ollama pull glm-ocr |
2.2 GB |
| Images on weak hardware | MiniCPM-V 4.6 | ollama pull minicpm-v4.6:1b |
1.6 GB |
| Search over your own documents | Qwen3-Embedding | ollama pull qwen3-embedding:0.6b |
0.6 GB |
| Same, multilingual corpus | BGE-M3 | ollama pull bge-m3 |
1.2 GB |
| Answers over documents, strict JSON | Granite 4.2 | ollama pull granite4.2:8b |
5.3 GB |
What is deliberately missing
Translation. I meant to write that section, but the honest answer is that the best model in the niche is closed. Qwen3.8-LiveTranslate, released 19 September, does simultaneous interpretation at 2.3 seconds of lag across 60 input languages — API only, no weights. Locally there is nothing but general models that translate "fine".
Ornith-1.5. Sizes of 9B, 35B and 397B, 256K context, a self-improvement loop that generates its own training tasks, three hundred thousand pulls in a month. The most interesting row in the library, and exactly why it is not in my list: there are no comparable independent numbers yet.
What I actually run
On a 16 GB MacBook: gemma4:12b as the generalist, plus glm-ocr and qwen3-embedding:0.6b for documents and search over my own archive. Everything heavier is a different machine or a rented endpoint.
If I had to keep exactly one model forever, it would be Gemma 4 at whatever size fits. Not because it wins benchmarks — it does not — but because it does what you asked instead of what it found more interesting. Twenty-five million pulls is a lot of people arriving at the same conclusion independently.
Originally published at klukyanov.ru.
Shorter weekly write-ups (in Russian) — on Telegram.
Top comments (0)