TokenPrint: An Open-Source 3D Visual Debugger for LLMs and Transformers

TokenPrint: An Open-Source 3D Visual Debugger for LLMs and Transformers

Large language models are usually presented as a stack of equations, architecture diagrams, and source code.

That explains what the model is.

But it doesn't always explain what is actually happening to a token as it moves through the model.

So I built TokenPrint.

Try it: https://tokenprint.in/
GitHub: https://github.com/Sudharsanselvaraj/Token-Print

What is TokenPrint?

TokenPrint is an open-source interactive visualization and debugging environment for transformer and LLM inference.

The goal is simple:

Make the internal computation of a transformer something you can actually see and inspect.

Instead of looking at:

x → attention → MLP → x

you can explore the individual operations that produce that result.

You can move through the model layer by layer and inspect things such as:

• token embeddings
• positional information
• RMSNorm
• Q, K and V projections
• Grouped-Query Attention
• RoPE
• attention scores
• softmax
• weighted value aggregation
• output projection
• SwiGLU
• MLP projections
• residual streams
• logits and predictions

Follow a token through the model

One of the ideas behind TokenPrint is to treat inference as a journey.

A token starts as a discrete token ID.

It becomes an embedding.

That representation enters the first transformer layer.

Then it passes through normalization, attention, projections, nonlinear transformations and residual connections.

The representation continues evolving through the remaining layers until the model produces its final logits.

TokenPrint makes that process explorable in 3D.

You can select operations, move between layers, follow the computational path, and inspect the associated tensors and metadata.

See the actual transformer computation

For example, an attention block can be understood as a sequence of transformations:

Q projection

Q = XW_Q

K projection

K = XW_K

V projection

V = XW_V

Then positional information is applied to the relevant representations before attention scores are calculated.

The attention mechanism can then be expressed as:

A = softmax(QKᵀ / √d_k + M)

and the context representation becomes:

C = AV

Finally, the concatenated head outputs are projected back into the model's hidden dimension.

For a model using Grouped-Query Attention, TokenPrint can also expose the relationship between query heads and shared key/value groups instead of treating the attention block as a single opaque operation.

Why 3D?

A transformer is already a computational graph.

The problem is that a traditional diagram becomes difficult to read once you start showing many layers, branches and intermediate tensors.

TokenPrint uses 3D space to make those relationships spatial.

You can move around the computation, focus on a layer, inspect an operation, and follow the data path.

The intention isn't to make the model look futuristic.

The intention is to make the structure easier to understand.

Inspect individual components

Selecting a component opens an inspection view containing information such as:

What it does

A plain-language explanation of the operation.

Equation

The mathematical operation represented by the component.

Dimensions

Input and output dimensions.

Parameters

The number of learned parameters involved.

Model data

The actual tensor path associated with the component.

Provenance

Whether the displayed information is real model data, derived information, conceptual structure, or simulation.

For example, selecting a projection can show a tensor such as:

model.layers.11.self_attn.k_proj.weight

along with its shape, dtype, parameter count, layer and runtime information.

Real model data

A major design goal of TokenPrint is to distinguish between what the model actually provides and what is only being illustrated.

The visualizer uses explicit provenance categories:

REAL — directly obtained from the model/runtime.

DERIVED — calculated from real model information.

CONCEPTUAL — an educational representation of a model concept.

SIMULATION — intentionally simulated behavior.

This distinction matters because an attractive visualization is not useful if it quietly invents model internals.

Explore tensors

TokenPrint also provides a tensor inspection workflow.

Instead of treating tensors as invisible implementation details, you can inspect their metadata directly.

For example:

model.layers.3.self_attn.v_proj.weight

can expose:

Shape: 128 × 896

Dtype: float32

Parameters: approximately 114.7K

Layer: 3

Runtime: hf_local

That creates a connection between the visualization and the actual model implementation.

Learn while exploring

TokenPrint is also intended to be an educational tool.

A component should answer more than:

"What is this box?"

It should explain:

What does this operation compute?

Why does the transformer need it?

What is its input?

What is its output?

How does it connect to the next operation?

Which parameters and tensors are involved?

This makes it possible to move from a high-level explanation to the mathematical and implementation-level details without leaving the visualization.

Built for experimentation

The long-term goal is not only visualization.

TokenPrint is being designed as an environment for inspecting and experimenting with transformer computation.

That includes areas such as:

• attention inspection
• activation analysis
• residual stream analysis
• tensor inspection
• layer and head experiments
• ablations
• activation patching
• trace replay
• model comparisons

The idea is to make the internal behavior of an LLM inspectable rather than treating inference as a black box.

Open source

TokenPrint is open source and built for people who want to learn, experiment, research, and contribute.

There are opportunities across:

• frontend and 3D visualization
• transformer architecture support
• inference backends
• tensor instrumentation
• debugging tools
• visualization UX
• educational content
• research experiments

GitHub:

https://github.com/Sudharsanselvaraj/Token-Print

What's next?

I'm working toward making TokenPrint much more than a visualization of one local model.

Some of the directions include broader Hugging Face model support, better model capability detection, richer execution traces, remote inference, experiment workflows, and shareable transformer traces that can be explored without reproducing the original runtime.

The larger idea is:

What if opening an LLM did not just show you the model's architecture, but let you actually watch computation happen inside it?

That's what I'm trying to build with TokenPrint.

Try TokenPrint → https://tokenprint.in/

Star the project or contribute → https://github.com/Sudharsanselvaraj/Token-Print

AI #MachineLearning #LLM #OpenSource

Explore what happens inside a transformer: follow tokens through embeddings, attention, Q/K/V projections, RoPE, MLPs, residual streams, and logits with an interactive 3D visual debugger.