Your smallest local model might be your best one - I measured 4 of mine

#python #machinelearning #ai #opensource

I had seven models sitting in Ollama and no idea which one to use for what. So I stopped guessing and measured.

152 generations on a 16 GB laptop. Greedy decoding, deterministic grading - exact number, exact string, JSON field, regex. No LLM judge, so there is no second model's bias to audit.

The result

No model won every category.

task type	deepseek-r1:1.5b (1.1 GB)	llama3.2:3b (2.0 GB)	gemma:2b	codellama 7b
arithmetic (12)	10/12	2/12	2/12	3/12
extraction (9)	4/9	9/9	7/9	8/9
classification (8)	6/8	7/8	7/8	2/8
code output (6)	5/6	3/6	1/6	4/6
median latency	14.6 s	1.6 s	2.7 s	6.3 s

The 1.1 GB reasoning model is five times better at arithmetic than the 2 GB general model - and less than half as good at structured extraction, at nine times the latency. The smallest model I own scored highest overall. The largest came third.

Size did not predict accuracy.

One thing that nearly broke the measurement

Reasoning models emit a <think> block before answering. My first grader read the whole response, so it kept scoring the model's intermediate working instead of its final answer - marking correct answers wrong. Stripping reasoning blocks and anchoring on the final answer changed deepseek's score dramatically.

A second bug: gemma answers "Negative." then explains itself unprompted. I was only checking the last line, so a correct answer scored zero. That would have penalised every chatty small model across the whole benchmark.

Both are now covered by tests. If you build evals, check this before you trust your numbers.

Caveats, up front

3-12 prompts per category, so a single item moves a percentage a lot. Only auto-gradable short-answer tasks, which excludes the open-ended writing bigger models are usually better at. One machine, Q4 quants, backend-reported timings with an uncontrolled OS cache.