Building a Terminal Based LLM Inference Internals Explorer

Part 1: The Entropy Tracker

Part 1 of a 4-part series on system-level LLM inference internals.

---

What This Series Builds

Most LLM tooling treats inference as a black box. Hosted APIs make this worse; they strip away logits, attention weights, and intermediate activations entirely. What's left is just surface behavior.

This project goes the other direction. Running a 3B model locally on Apple Silicon means getting everything: raw logit distributions at every decode step, full attention weight tensors during prefill, and direct control over the generation loop.

The unifying thesis:

Context quality shapes attention distribution during prefill. Attention distribution shapes generation confidence during decode. Generation confidence determines how efficiently speculative decoding can run. These three things are causally linked — and this series builds the tools to try and prove it.

📊 Diagram:

Part	What We Build
1 — this post	Per-token entropy tracker, visualized in real time
2	Attention sink detector, context health scoring
3	Entropy-guided adaptive speculative decoding
4	Empirical study: correlation plots proving the causal chain

---

Hardware & Stack

• MacBook Pro M1 Pro, 16GB unified memory
• Model: Qwen2.5-3B-Instruct, fp16, MPS backend (~17 tok/s warm)
• Python · PyTorch · HuggingFace Transformers · Rich

This project requires local inference. Hosted APIs (Anthropic, OpenAI) don't expose raw logits or attention weights. To see inside the model, you have to run it yourself.

---

Phase 0: Environment Setup

pyproject.toml

[project]
name = "llm-inference-lab"
version = "0.1.0"
requires-python = ">=3.10,<3.13"
dependencies = [
    "torch>=2.3.0",
    "transformers>=4.42.0",
    "accelerate>=0.31.0",
    "sentencepiece>=0.2.0",
    "protobuf>=4.25.0",
    "rich>=13.7.0",
    "numpy>=1.26.0",
    "matplotlib>=3.8.0",
    "pandas>=2.2.0",
]

MPS sanity check

import torch
print(torch.backends.mps.is_available())   # True on M1/M2/M3
x = torch.rand(1000, 1000, device="mps")
print((x @ x).shape)                       # confirms GPU matmul works

Benchmarks on M1 Pro, 16GB

Run	Tokens	Time	Tok/s
Cold (MPS kernel compilation)	50	5.59s	8.9
Warm	50	2.89s	17.3
Warm, longer	100	5.95s	16.8

The cold-start penalty is a one-time cost per process. All subsequent calls run at ~17 tok/s. All params confirmed on mps:0 in fp16 — no silent CPU fallback.

---

Phase 1: The Entropy Tracker

The Math

At each decode step, the model produces a logit vector over ~32,000 vocabulary tokens. After softmax this becomes a probability distribution. Shannon entropy measures how uncertain the model is:

H(t) = -Σ p(x) · log2(p(x))    over all vocab tokens x

Low H  → peaked distribution → confident
High H → flat distribution   → uncertain

The Hook: LogitsProcessor

from transformers import LogitsProcessor
import torch.nn.functional as F

class EntropyCapture(LogitsProcessor):
    def __init__(self):
        self.entropies = []
        self.top_tokens = []

    def __call__(self, input_ids, scores):
        probs = F.softmax(scores, dim=-1)
        log_probs = torch.clamp(probs.log2(), min=-1e9)
        H = -(probs * log_probs).sum(dim=-1)
        self.entropies.append(H.item())

        top = torch.topk(probs, k=5, dim=-1)
        self.top_tokens.append({
            "values": top.values[0].tolist(),
            "indices": top.indices[0].tolist(),
        })

        return scores  # unchanged -- observing, not modifying

Wiring it in:

processor = EntropyCapture()

outputs = model.generate(
    **inputs,
    max_new_tokens=200,
    do_sample=True,
    temperature=0.7,
    logits_processor=[processor],
)
# processor.entropies now has one float per generated token

Rich Terminal Renderer

from rich.text import Text
from rich.console import Console

def entropy_color(H: float) -> str:
    if H < 1.0: return "green"
    if H < 3.0: return "yellow"
    return "red"

console = Console()
text = Text()
for token_str, H in zip(generated_token_strings, entropies):
    text.append(token_str, style=entropy_color(H))
console.print(text)

Key Finding: Subword Commitment Points

Qwen2.5 uses BPE tokenization — words split into subword units. "civilization" might tokenize as civil + ization. What happens at the entropy level?

• civil → high entropy (the model commits to this word here)
• ization → low entropy (the continuation is already determined)

📊 Diagram:

Entropy spikes mark commitment points — where the model decides among multiple valid continuations. Once the first subword of a new word is chosen, the rest is nearly deterministic. The real decision happens at the leading edge.

Entropy Profiles by Prompt Type

Prompt Type	Mean Entropy	Notes
Factual ("capital of France")	~0.8	Mostly confident, few spikes
Creative writing("name 10 new colors")	~2.6	Frequent uncertainty
Code generation	~1.1	Surprisingly confident — syntax constrains
Math reasoning	~1.4	Spikes at numeric choices
Ambiguous questions("what makes most sense?")	~3.1	Sustained high entropy

Code being "greener" than prose is counterintuitive but makes sense: the model has strong priors about what syntactically valid Python looks like. In prose, almost any word could plausibly follow.

---

How This Connects to Parts 2 and 3

To attention sinks (Part 2): If attention during prefill pools into irrelevant sink tokens, the model enters decode with a degraded state — observable as higher mean generation entropy on poisoned contexts. Part 2 measures both sides and plots the correlation.

To speculative decoding (Part 3): The draft model acceptance criterion is min(1, p_verifier / p_draft). The draft gets accepted most when it's confident — low entropy, peaked distribution. High draft entropy signals a likely upcoming rejection. So instead of always drafting a fixed k tokens, we stop early when entropy spikes.

📊 Diagram:

— decision flow: low draft entropy → keep drafting (likely accepted); high draft entropy → call verifier (rejection incoming).

This is entropy-guided adaptive speculative decoding — the thread connecting all three phases.

---

Links

Resource	Link
GitHub repo	https://github.com/cyprus09/llm-inference-lab
StreamingLLM	https://arxiv.org/abs/2309.17453
Speculative Decoding	https://arxiv.org/abs/2211.17192
Lost in the Middle	https://arxiv.org/abs/2307.03172
The Illustrated Transformer	https://jalammar.github.io/illustrated-transformer/
Making Deep Learning Go Brrrr	https://horace.io/brrr_intro.html
nanoGPT	https://github.com/karpathy/nanoGPT

---

Series:

• Part 1 — Entropy Tracker (you are here)
• Part 2 — Attention Sink Detector (coming)
• Part 3 — Speculative Decoding with Entropy-Guided Draft Length (coming)
• Part 4 — The Empirical Study (coming)