An Ollama tag that pulls and runs and has no working code in it. The same model at the same quantisation, elsewhere, is fine.

TL;DR: ollama pull qwen2.5-coder:3b-instruct-q3_K_M succeeds, ollama run streams at normal speed, and nothing anywhere reports a problem. On three trivial HumanEval-style tasks it produced no implementation at all — 0/3. The obvious conclusion is that 3B at q3 is simply too degraded to code. That conclusion is wrong, and one control disproves it: the official Qwen GGUF at the same q3_K_M level, imported with the library model's own template so only the weights differ, scored 3/3 on the same tasks. The failure is in that conversion, not in the quantisation level.

The symptom

Debian 13, Ollama 0.33.3, CPU only. Ask for the simplest function in HumanEval:

Please provide a self-contained Python script that solves the following
problem in a markdown code block:

def sum_to_n(n: int) -> int:
    """ sum_to_n is a function that sums numbers from 1 to n. """

qwen2.5-coder:3b-instruct-q3_K_M answers, deterministically, with 85 characters:

.
00











 to_n: function to_n:

A stray ., a stray 00, two floods of newlines, and to_n: function to_n: —
the identifier from the prompt echoed back. No def. No return. Nothing to run.

It is not a fluke or a sampling artefact. At temperature 0 and seed 42, two
runs produced byte-identical output (sha256 ec98f76dfeeda2a3…).

Everything around it reported success. The pull completed and verified, the
model is 1.6 GB on disk as advertised, load time and token rate look normal, and
the HTTP API returns 200 with a populated response field. The only thing wrong
is the content.

Why "q3 is just too small" is the wrong conclusion

That was the first hypothesis here, and it is the one most people will reach,
because it is usually right. Low-bit quantisation of a small model genuinely
does fall apart — it is a known trade-off, it is discussed everywhere, and a
3B model at 3 bits is exactly where you would expect it.

Two controls, same machine, same prompts, same temperature 0 / seed 42,
generated code extracted and actually executed against assertions rather
than eyeballed:

model                                                    result
qwen2.5-coder:3b-instruct-q3_K_M   (Ollama library)      0/3
qwen2.5-coder:3b-instruct-q4_K_M   (Ollama library)      3/3
Qwen official qwen2.5-coder-3b-instruct-q3_k_m.gguf      3/3

The third row is the one that settles it. Same model, same quantisation
level, a different conversion — and it works. Here is what it returns for the
prompt above:

def sum_to_n(n: int) -> int:
    """ sum_to_n is a function that sums numbers from 1 to n. """
    return n * (n + 1) // 2

So "3B at q3 cannot code" is false on this machine. Something specific to that
artifact is broken.

The control had to be chosen carefully

The obvious control is one of the popular community re-quantisations, and it is
the wrong one. Those repositories ship an .imatrix file — they use
importance-matrix quantisation, which is a different process. If the working
control is an imatrix quant and the broken one is not, then a difference between
them supports two stories at once: "this conversion is broken", and "imatrix
matters enormously at 3B/q3". You cannot tell which you are looking at.

The official Qwen GGUF repository has no .imatrix, which makes it the control
that answers only one question:

curl -s https://huggingface.co/api/models/bartowski/Qwen2.5-Coder-3B-Instruct-GGUF \
  | grep -o 'imatrix'          # present
curl -s https://huggingface.co/api/models/Qwen/Qwen2.5-Coder-3B-Instruct-GGUF \
  | grep -o 'imatrix'          # nothing

One more thing had to be held constant. A raw GGUF imported into Ollama needs a
chat template supplied by hand, and a wrong template produces garbage that
looks exactly like a broken quantisation. So the template was not written from
scratch — it was lifted from the working library model, verbatim:

ollama show --modelfile qwen2.5-coder:3b-instruct-q4_K_M \
  | grep -v '^#' \
  | sed 's|^FROM .*|FROM /path/to/qwen2.5-coder-3b-instruct-q3_k_m.gguf|' \
  > Modelfile
ollama create qwen-official-q3 -f Modelfile

With that, the only difference between the 0/3 model and the 3/3 model is the
weights.

What you can check, and what it costs

The artifact is unchanged since it was first reported. All the relevant tags
still read "1 year ago" on the model's tag page, and the manifest being served
right now hashes to the digest that page shows:

$ curl -s https://registry.ollama.ai/v2/library/qwen2.5-coder/manifests/3b-instruct-q3_K_M \
    | sha256sum | cut -c1-12
65ff2bc170f3

ollama list shows the same 65ff2bc170f3 after pulling, so what is described
here is what you would get today.

Reproduced on Ollama 0.33.3, which is newer than the 0.32.6 in the original
report — upgrading is not the fix.

The fix

There is no flag for this. The artifact is what it is, and the repair belongs
upstream. What you can do on your own machine is stop using that tag:

Move up one level. q4_K_M from the same library works and costs about
300 MB more. If you were on q3 for size reasons, check whether you actually
needed to be.

Or convert from a source you can name. The model author's own GGUF at the
same level works. Import it with the library model's template as above, so you
are not trading one silent failure for another.

And check before you trust it. The whole test is three prompts and running
what comes back:

curl -s http://127.0.0.1:11434/api/generate -d '{
  "model": "your-tag-here",
  "prompt": "Please provide a self-contained Python script that solves the following problem in a markdown code block:\n\ndef sum_to_n(n: int) -> int:\n    \"\"\" sums numbers from 1 to n \"\"\"\n",
  "stream": false,
  "options": {"temperature": 0, "seed": 42}
}' | python3 -c 'import json,sys; print(json.load(sys.stdin)["response"])'

If what comes back has no def and no return in it, you have this. It takes
about ten seconds and it is the only signal you are going to get.

The generalisable habit

The narrow lesson is that a quantised model is a build artifact, not a
property of the model. "Qwen2.5-Coder-3B at q3_K_M" names a recipe; the file you
downloaded is one execution of that recipe by one party, and it can be wrong on
its own without the recipe being wrong. When a model underperforms, the question
"is this quantisation level too low?" and the question "is this file bad?" feel
like the same question and are not. Converting it yourself, or fetching
another party's conversion at the same level, separates them for the price of
one download.

The wider one cost me the most time today, and it is about the order you run
things in.

The first control run reported the working model as 0/3. The harness was
broken: this model emits its language tag on the line after the code fence
rather than on the fence line, so the extractor was handing python to the
interpreter as if it were code. Twenty minutes of a perfectly good model looked
like a second broken one.

I only caught it because the control ran before the suspect. Had I started
with q3_K_M, I would have seen 0/3, matched it against a report that predicted
0/3, and stopped — with a broken measurement and a conclusion that happened to
agree with it. The error would have survived, because the evidence for it looked
exactly like the evidence for the truth.

So: run the case you expect to pass first, and treat its failure as a bug in
your instrument until proven otherwise. A test that has never been seen to
pass has not been shown to work — it has only been shown to produce the answer
you were hoping for. That is the same trap as
a build that confidently reports a commit hash from a repository it has never
heard of:
the thing doing the reporting is not the thing you are trying to measure, and
when they disagree, you will believe the wrong one unless you have arranged in
advance to tell them apart.

Toolkit

This post's fix is available as a tested, ready-to-run script in the toolkit.

See the toolkit →