Ollama went from 0.33 to 0.40 in six weeks, and most of what changed is invisible unless you
read the responses with a magnifier. I did, because I maintain a tool that has to. Here are
four numbers the daemon reports now that it didn't before, what each one actually means, and
in one case why you should not believe it.
Everything below is two curl calls on your own machine. The tool is at the end.
1. A prefill rate the server computes for you — wrongly
Since 0.35.1 the OpenAI-compatible endpoint attaches a timings block to the final usage
chunk. Send one prompt twice, read the second answer:
"usage": {"prompt_tokens": 31, "prompt_tokens_details": {"cached_tokens": 30}, "completion_tokens": 11},
"timings": {"prompt_n": 31, "prompt_ms": 8, "prompt_per_second": 3875,
"predicted_n": 11, "predicted_ms": 83, "predicted_per_second": 132.5}
One token was evaluated, in 8 milliseconds. The block says 3875 tokens per second.
prompt_n is the whole prompt, cached tokens included. prompt_ms is the time spent on the
uncached ones only. Divide the first by the second and you get a rate inflated by exactly the
cache hit ratio: count / (count - cached), so 31x here, and anything up to a few hundred on a
long system prompt that matched. The honest figure sits in the same chunk:
(prompt_tokens - cached_tokens) / prompt_ms = 1 / 8 ms = 125 tok/s
If that looks familiar, it is the formula bug that the native API acquired in 0.33.3, when
prompt_eval_duration started timing only uncached tokens while prompt_eval_count stayed the
total. Now the server emits the result of that division itself, labelled as a rate. The decode
half, predicted_per_second, is fine; generated tokens are never cached.
What LLMxRay does with it: the Protocol Observatory's envelope diff now has three rows for
prefill: the time, the rate as the daemon states it, and the rate over evaluated tokens. Run a
prompt twice and the stated cell turns amber on the second run. That is the whole feature; the
point is to see the two numbers diverge on your own daemon.
2. A probability instead of a "yes"
0.35.1 added decision models: nimble, tev1, clef, clef-flash. They do not write text.
You give /v1/systemone a state and typed questions, and you get back the model's own
distribution over the candidates:
curl localhost:11434/v1/systemone -d '{
"model": "nimble",
"state": "Our checkout has returned 500 errors since 9am. Customers cannot pay.",
"questions": {
"label": {"type": "choice", "instructions": "Which label fits this ticket?",
"criteria": {"billing": "Payments and refunds", "bug": "Software errors", "account": "Login and account access"}},
"urgent": {"type": "noul", "instructions": "Is this ticket urgent?"},
"severity": {"type": "score", "instructions": "How severe is the impact?",
"criteria": ["cosmetic", "minor", "major", "full outage"]}
}}'
"label": {"type": "choice", "choice": "bug", "probabilities": {"billing": 0.038, "bug": 0.907, "account": 0.055}, "confidence": 0.66},
"urgent": {"type": "noul", "noul": 0.9897},
"severity": {"type": "score", "score": 2.37, "probabilities": {"0": 0.005, "1": 0.022, "2": 0.569, "3": 0.403}}
Three things worth knowing before you build on it. score criteria must be an array; a map is
rejected with a clear message. Limits are 2 to 26 candidates and 1 to 64 questions per request.
And a score is the expected position on your scale, so 2.37 on a four-step scale means
"major, leaning to full outage", which is more information than either label alone.
The trap: these models report decision as their only capability, and /api/chat does not
refuse them. Ask nimble "hi" and it streams a reply about the schema it was not given. Any
client that builds its model list from /api/tags without reading capabilities will offer a
decision model for chat.
What LLMxRay does with it: decision models are kept out of the chat and embedding selectors
by capability, and a Decision Lab page sends the request above and draws each answer as
probability bars with the chosen candidate highlighted, plus the token usage. The sample on the
page is the ticket above, asked three ways.
3. Which runtime actually ran your model
0.40 made MLX the default on Apple Silicon for the architectures it supports, which means the
same model tag can now run on two different engines. The daemon says which:
/api/tags -> "details": {"runner": "mlx"} // or "llamacpp", or "ggml" for models pulled before 0.40
/api/ps -> "runner": "mlx"
On the Mac mini I measured with, nomic-embed-text sat on llamacpp and embeddinggemma-2 on
mlx at the same time. A benchmark result that only records the model name is now ambiguous on
that machine.
What LLMxRay does with it: the runner is shown on each model and each running model, and
every new benchmark result is stamped with it, so two results under one name are only compared
when the runner matches. Results recorded before 0.7.0 carry no runner, and the page says so
rather than guessing.
4. An image costs 258 tokens to embed, on one platform
Also in 0.40: /api/embed accepts an image next to the text, on models that report both
embedding and vision:
curl localhost:11434/api/embed -d '{"model": "embeddinggemma-2",
"input": {"text": "a cat on a windowsill", "image": "<base64 png>"}}'
| Input | prompt_eval_count |
Vector |
|---|---|---|
| short sentence | 7 | 768 dims |
| same sentence + a 1x1 PNG | 265 | 768 dims |
A one-pixel image pays a ~258-token block, so the cost is per image, not per pixel. Sending an
image to a text-only embedding model returns HTTP 501, model does not support media, a status worth matching on rather than a generic 400.
embeddings
The catch is in the model, not the API: embeddinggemma-2 is the only multimodal embedding
model in the library, and it requires MLX. On Windows the pull fails with "this model requires
MLX support". So in October 2026, image embeddings in Ollama are an Apple Silicon feature. I
measured the table above by forwarding a Mac mini's daemon over SSH.
What LLMxRay does with it: on the Embeddings page, when the selected model reports embedding
and vision, an image can be attached in the playground and on both sides of the similarity
calculator: photo against sentence, or photo against photo. Every result shows the prompt tokens
it cost. Text-only models show no image controls at all.
One more thing the daemon does quietly
0.40.2 upgrades models in the background the first time you run them and keeps the original
copy on disk, so a downgrade stays safe. The release notes give a jq pipeline to find and
delete the backups by digest. LLMxRay's Models tab does the same selection and offers a remove
button per backup. In fairness: no daemon I have access to has produced a backup yet, 0.40.3
paused the upgrades, so that part is built from Ollama's own recipe and tested against its shape,
not against a live backup. The section stays hidden until one exists.
The tool
LLMxRay is a local observability UI for Ollama:
Apache-2.0, nothing leaves localhost, npx llmxray or docker pull djovaneli/llmxray. Version
0.7.1 is the one with everything above, verified against Ollama 0.40.3. The guides are in
English, French and Chinese.
None of the four numbers needs it. What it adds is the habit of putting the daemon's figure and
the honest figure on the same screen, which is how the first one was caught.
Top comments (1)
Official Platform Update
Security protocols have been updated for all developer accounts.