mtop is a terminal view for local model servers. One window shows the models you have loaded and the VRAM they hold, the GPU, and the requests going through with their tok/s. It works with Ollama, llama.cpp (including router mode and llama-swap), LM Studio, vLLM, Lemonade and SGLang.
Requests while they stream
Until now a request only showed up once it finished. In 1.6 it appears at the top of the requests pane as soon as it starts, with how long it's been going, tokens so far and a live tok/s. At 100 columns you also get time to first token and decode time, which is usually where the answer to "why did that feel slow" is.
For streamed OpenAI-style requests the final tok/s is now decode only. If llama.cpp sends its own predicted_per_second, mtop uses that. Otherwise it counts from the first token, so prompt processing and model loading don't drag the number down.
Naming the reason
The part I wanted most: when a model is slow and mtop can tell why, it says so under the models table.
-
38% of model on CPU: part of the model didn't fit in VRAM and runs from system RAM. -
context 94% fullorprompt cut to fit context: ollama quietly drops the start of a prompt that doesn't fit, which is easy to miss. -
3 requests queued: more requests than the server has slots (llama.cpp, vLLM and SGLang report this; ollama doesn't).
If an NVIDIA card is throttling itself for heat or power, the GPU pane shows that too, but only after it holds for a few polls with the card actually busy, so normal laptop power management doesn't set it off.
Try it without a server
mtop --demo plays a 40-second loop on simulated models, traffic and a GPU. It opens no port and doesn't talk to any server, so it's a quick way to see whether the tool is useful to you.
How the numbers get there
Most local servers don't expose per-request numbers anywhere, so mtop runs a small pass-through proxy on 127.0.0.1:4321 and reads them off the stream. Point your client at it (OLLAMA_HOST=127.0.0.1:4321, or base_url = "http://127.0.0.1:4321/v1") and it gets the same bytes back. The models and GPU panes work without the proxy.
One llama.cpp detail: mtop never polls /slots, because that counts as activity and keeps a server started with --sleep-idle-seconds from sleeping.
Install
brew install eladser/tap/mtop
scoop install mtop
winget install eladser.mtop
Or grab a binary from the releases page. It's a single Go binary, MIT licensed.
- GitHub: https://github.com/eladser/mtop
- Release notes: https://github.com/eladser/mtop/releases/tag/v1.6.0

Top comments (0)