I don't normally do model reviews or comparisons anymore, as I have moved on to much larger technical projects. I was scrolling through Ollama’s model list for a current testing model and was surprised at the lack of the smaller models in their list.
Why has everything moved on to cloud-based models?
So I went on a hunt in Hugging Face to find this model MiniCPM5–2B with very impressive numbers in their benchmarks, and thought, "Why not give it a spin on the new upgrades we have made to Vektors' desk app this week?"
And I have to say I’m reasonably impressed with the results. After fine-tuning our harness, the search functions, and the reranking functions, the results this model gives are excellent for its size.
A 2.5-billion-parameter language model is answering a tool-calling prompt on a machine with Serp search perfectly. Under 3-gigabyte GGUF file on disk, doing something that 6 months ago would have required a model much larger.
Two years ago we assumed tasks like this needed hundreds of billions of parameters just to produce coherent output, let alone correct code. A model small enough to fit in a phone’s storage now beats models twice its size at exactly that. It’s a compression of capability that changes what “small” is allowed to produce.
That model is MiniCPM5–2B, was released by OpenBMB on September 7, 2026.
It completes a pattern I’ve been watching for two years: the point where “small open-source model” stopped meaning “toy” and started meaning “legitimate production choice.” This article uses one model as a case study, but it’s really about what changes when a model this size can do this much.
What the benchmarks actually say
On the Artificial Analysis Intelligence Index, MiniCPM5–2B ranks first among open-source models under 4 billion parameters. Across an internal suite of 34 benchmarks spanning code, math, long-context understanding, tool use, and agentic tasks, it averages 53.9.
That beats not just other 2B-class models but several 4B-class ones too, including Qwen3.5–4B and Nemotron-3-Nano-4B, whose best score in the comparison set tops out around 51.1.
It’s a dense, 42-layer transformer using the standard LlamaForCausalLM structure, not some exotic new block. Attention is grouped-query, 16 query heads to 2 key/value heads, which is what lets it hold a native 128K-token context window without KV cache size becoming absurd on consumer hardware.
It’s released under Apache 2.0, no usage restrictions, no revenue-threshold clauses. Quantized builds run from a 1.04 GB Q2_K file up to a 2.68 GB Q8_0 file, with the commonly recommended Q4_K_M landing around 1.56 GB.
That’s a model you can fit on a mobile phone or smaller VRAM GPU if needed.
Vektor Desk tool call
Intelligence density is the story scale can’t tell
For most of the last five years the dominant AI narrative was scale: bigger models, bigger datasets, bigger clusters, bigger bills. GPT-3 to GPT-4 and beyond is a story about more. There’s a second story running quieter in parallel, and it’s the one that matters if you’re trying to ship a product instead of win a leaderboard: intelligence density, meaning how much capability you can pack per parameter, per gigabyte, per watt.
MiniCPM’s release history makes this concrete. The original MiniCPM-2B, back in 2024, was pitched as competitive with Mistral-7B and reportedly outperformed Llama2–13B, MPT-30B, and even Falcon-40B on several benchmarks despite being a small fraction of their size.
MiniCPM5 continues that same trajectory rather than reinventing it: a 1B checkpoint in May 2026, then this 2B checkpoint in September, described by OpenBMB as the same training recipe scaled up. No new architecture, no new bet. Just more disciplined execution of a formula that treats parameter count as a cost to minimize instead of a number to maximize.
OpenBMB reports training on roughly 550 billion tokens of curated code data (a set they call UltraData-Code) plus around 500,000 agentic and tool-use samples.
They built a reinforcement learning stack tuned for small models specifically, rather than shrinking a frontier-scale RL pipeline down and hoping it still worked. And they made conservative architectural choices, favoring the well-understood Llama backbone over something novel, which matters more than it sounds like it should because it means every serving engine that already supports Llama models supports this one too.
The part I find most telling isn’t the weights releasing themselves. It’s that OpenBMB published the training data, the recipe, and the RL stack alongside the model. It’s an invitation to reproduce and audit the work, which is what makes something feel like infrastructure rather than a product demo.
Production readiness isn’t a benchmark score. It’s a bundle of unglamorous requirements.
Can it run on hardware you already own?
Can you audit what it was trained on?
Can your infrastructure team integrate it without forking a codebase?
Can you fine-tune it without a dedicated research team?
Is the license actually permissive, with no asterisk hiding in section four?
MiniCPM5–2B answers yes to all of those in ways frontier models structurally can’t. Because it uses the standard Llama architecture, vLLM (version 0.21.0 and up) loads it natively with no custom kernels and no model-code fork. A single line, vllm serve openbmb/MiniCPM5-2B --port 8000, gets it running, and at 2.5B parameters it fits comfortably on one GPU with tensor parallelism of 1.
It ships native tool calling using an XML-style format, with a dedicated parser already merged into vLLM under the name minicpm5, so the usual weeks-long slog of getting an agent framework to actually parse a model's tool calls correctly is mostly closed on day one.
For teams doing high-throughput serving, OpenBMB also released MiniCPM5-2B-DSpark, a purpose-built speculative decoding draft model trained specifically to pair with the 2.5B base rather than pointing users at a generic off-the-shelf drafter. That's someone thinking about latency in production, not just accuracy on a leaderboard.
What changes when the model fits on your machine
When a model is small enough to run on-device, whole categories of problems dissolve. Data residency stops being a legal negotiation and becomes a default. Latency stops being a network round trip and becomes a function call. Cost stops being a per-token line item and becomes hardware you already paid for.
There’s a specific kind of risk that comes from building a product on top of an API you don’t control: the quiet awareness that your cost structure and sometimes your product’s core capability are token costs that are uncontrolled by you.
Small open models don’t erase every version of that risk. You can still depend on a hosting provider, or a fine-tune you never validated, or a community fork that goes stale. But they hand you the option of self-sufficiency. Whether or not you take that option, having it changes your negotiating position with every vendor you do choose to rely on.
Even as a fallback in your waterfall of model providers locally, having a workhorse smaller model that performs consistently is reassuring.
None of this means frontier, closed, API-served models are going away, or that they should. There are problems, genuinely open-ended reasoning, tasks needing the broadest possible world knowledge, and work where being wrong costs far more than a few extra cents per token, where the biggest available model is still the right engineering call.
A 2B model isn’t going to replace a frontier model at the edge of what language models can do. But that’s a narrower slice of real production workloads than the industry’s default assumptions suggest.
Structured extraction, tool-calling agents with a bounded action space, code completion inside a known codebase, summarization, classification, on-device assistants: none of that needs the biggest model available. It needs the smallest model that reliably clears the bar, because every parameter above that bar is pure cost. More latency, more GPU spend, more attack surface, more dependency.
Where this leaves us
What OpenBMB shipped is a specific model with a specific benchmark table, and benchmark tables age badly. Some other lab will publish a 2B model in six months that beats this one, the way this one beat its own predecessors.
That's progress in the game of technology, an unstoppable organism.
The gap between “biggest model available” and “good enough model” has been shrinking steadily, and it shrank again this week, in public, with the training recipe attached.
After completing all of our adjustments in the desk code files, the multiple tool calls this model handled correctly on a 3060 12 GB VRAM test machine that is a very average testing bench.
There was no cloud dashboard telling me what it cost in tokens, which is a real stress. There was just a small file doing a job very well on hardware I already owned.
Just a small whine from the GPU coils doing their job…
If you are interested in learning more about Vektor:
Full setup and docs are at vektormemory.com/docs and 1.9.0 info at https://vektormemory.com/docs/changelog
Sources
Hugging Face: huggingface.co/openbmb/MiniCPM5–2B
GitHub: github.com/OpenBMB/MiniCPM
Top comments (0)