NeuronDump: look inside any local LLM, follow a token, and make it smaller

If you run models locally with Ollama or llama.cpp, you've probably downloaded a few gigabytes of GGUF and treated it as a black box. I wanted to open that box: see the layers, the attention heads, the vocabulary, the chat template, and then ask two simple questions: why did the model pick this token, and which parts does it not actually need?

So I built NeuronDump, a single Rust binary (MIT) with a CLI and a local web UI. It reads GGUF files directly, with nothing hard-coded per model, and everything (including the 3D engine) is bundled, so it works offline.

👉 Repo: https://github.com/aymen94/NeuronDump
👉 Docs & illustrated intro: https://aymen.xyz/NeuronDump/

What it does

• Inspect: architecture, layers (including hybrid SSM/attention models), attention heads, the vocabulary decoded and classified, the chat template as a decision tree, and weight statistics.
• Graph: the model as a 3D (or 2D) dataflow graph you can open from sections down to single attention heads, neuron buckets and MoE experts. WebGPU, with an automatic WebGL 2 fallback.
• Follow a token: the logit lens per layer, attention heatmaps per head, the most active FFN neurons, and a breakdown of how much each layer and head pushed the chosen token against the runner-up.
• Edit live: scale or zero any part, skip a layer, ban or boost tokens, steer, patch activations, change the template, and compare baseline vs edited, with undo / redo and snapshots.
• Cut and see the impact: select any parts and instantly see the size saved, what gets disconnected, and whether llama.cpp still loads the file.
• Sweep and slim: cut each layer (or head) in turn, rank them by damage, and write the least important ones out as a smaller GGUF plus an Ollama Modelfile.

The model graph

neurondump web --open

Here's qwen3.8:27b in the 3D city view. Every part is coloured by whether you can remove it: safe, risky, or breaks the model.

Click any block to see its size, quantization and importance, expand it into its children, or add it to the cut selection. Type a tensor name like blk.3.ffn_down and the graph opens to show it.

Follow one token through the model

This is the part I've been using the most. Pick a prompt and a token, and NeuronDump shows what every layer "thinks" the next token is.

Here's llama3.1:8b on "Peanut butter and". The final answer is " jelly" (0.74). But up to layer 29, the logit lens reads "and" as grammar ("and/or"). " PB" shows up from layer 22, and only the last two layers switch to food.

Below that are the attention heatmaps and the decision breakdown. Every layer mostly attends to <|begin_of_text|> (an attention sink), and in layer 30 a handful of heads decide it: " jelly" beats " banana" by 2.2 logits, with head 24 pushing for it and head 25 pushing against it.

The generation timeline does the same for each generated token, and the uncertain steps stand out by their entropy. On "The Mediterranean Sea is located between", llama3.1:8b hesitates twice (" three" vs " Europe", then " Europe" vs " Africa"), and clicking a later step shows " Africa" winning at 0.99, with the push per layer and the heads behind it.

From the CLI:

neurondump tokens llama3.2:1b "fn main() {" --chat     # tokens, template tokens, nearest tokens
neurondump journey llama3.2:1b --prompt "The name of the star that Earth orbits is"
neurondump generate llama3.2:1b --prompt "The name of the star that Earth orbits is" --steps 8 --explain

Change the model and see what happens

Every change runs in memory, next to the original: you get the text, top-k, KL divergence, perplexity, logit lens and decision breakdown for both. You can scale or zero a layer, block, head or single neuron, skip a layer, edit a token's embedding, ban or boost tokens, add a steering vector, patch activations from another prompt, or swap the system prompt and template.

Edits stack, with undo / redo / restore all, named snapshots, and a portable edits.json:

{
  "schema_version": 1,
  "model": "llama3.2:1b",
  "edits": [
    { "id": 1, "enabled": true, "label": "zero layer 7", "op": "zero", "target": "layer.7" },
    { "id": 2, "enabled": true, "op": "steer", "layer": 6, "a": "Dear Sir or Madam, I am writing to formally request", "b": "hey dude, just wanna ask", "strength": 4 }
  ]
}

neurondump edit llama3.2:1b --apply edits.json --prompt "The name of the star that Earth orbits is"   # baseline vs edited
neurondump edit llama3.2:1b --apply edits.json --bake llama-edited.gguf --no-compare

Bake writes the edits into a new GGUF (plus a Modelfile). The original file is never touched.

Cut, sweep, slim

Slimming is a two-step process: analyze first, then apply a plan you can review.

neurondump analyze llama3.2:1b --out plan.json
neurondump slim llama3.2:1b --plan plan.json --only vocab --out small.gguf

The plan is a plain JSON file, so you can inspect or edit it before anything gets written. --only lets you restrict what gets cut (here, just the vocabulary).

Or let it measure which layers matter:

neurondump ablate llama3.2:1b --cut layer.7 --static                # direct impact on the file
neurondump ablate llama3.2:1b --cut layer.7 --prompt "The name of the star that Earth orbits is"   # simulated impact
neurondump sweep llama3.2:1b --corpus texts.txt --save-plan plan.json --top 2
neurondump slim llama3.2:1b --plan plan.json --out llama-14.gguf

Some things it showed me

• DeepSeek-R1 8B is Qwen3 8B with every weight slightly changed. It has the same neurons in the same places, including one neuron in layer 6 that makes the internal state about 33× larger. Cut that layer and both models produce noise.
• On llama3.2:1b, cutting the last layer made the model more confident on one prompt, even though, apart from the first two layers, no layer hurts perplexity on real text more.
• DeepSeek's chat template emits tool-call markers that aren't in its vocabulary, so each one gets split into 11 ordinary tokens.
• llama3.1:8b knows the answer before it says it. On "The name of the star that Earth orbits is", " Sun" is the top prediction from layer 21 to 26, then the last layers switch to " called", so it writes "…is called the Sun" instead of answering in one word.

Install

Prebuilt Windows and Linux binaries are on the releases page, with everything included (CLI, web UI, and running models). Unzip (or untar) and run neurondump list.

Or build from source (Rust 1.88+):

git clone https://github.com/aymen94/NeuronDump
cd NeuronDump
cargo install --path crates/neurondump --features trace
neurondump list

Leave out --features trace if you only want the static tools (inspect, graph, impact, slim, tokens, bake) and don't need to run models.

Run neurondump with no arguments to choose between the interactive CLI and the web UI.

Limits

It's early, and it has been tested mostly on Windows with Ollama models.

• Inference is CPU-only, with no KV cache, and all weights are loaded into RAM (an 8B Q4_K model needs about 5 GB).
• Running models (journey, timeline, simulation, live edits) covers the llama, qwen2/3, gemma (1–3), phi3 and bert families. MoE and hybrid models (like Qwen 3.5 / 3.8) get static analysis, static impact and bake only for now.
• Run-time-only edits (ban, boost, steering, patching) can't be baked into a GGUF.
• Zeroing heads or neurons keeps the file loadable but doesn't shrink it.

Known gaps are listed in the limitations page and in TEST.md. If you try it on a model I haven't, I'd love to hear what you find: https://github.com/aymen94/NeuronDump/issues