I traced a single prompt through an LLM engine to see how it actually works (Visual Breakdown)

We all use the API. We send a JSON payload to /v1/chat/completions, wait a few hundred milliseconds, and get a magical response back.

But as an engineer, the "Black Box" nature of AI bothered me. I wanted to understand the actual pipeline—not just the high-level theory, but the mechanical journey of the data.

So, I visualized the life of a single prompt: "Write a poem about a robot."

I traced it through Tokenization, Embeddings, Attention, and the KV Cache to understand how a matrix of numbers becomes a creative output.

Here is the full visual breakdown :

🛠 T*he Architecture Explained (Summary)*
If you can't watch the video right now, here are the core mental models I used to make sense of the math.

1. Tokenization: The "Lego Brick" Phase The engine doesn't read English; it reads integers. Before anything happens, the tokenizer smashes our prompt into chunks.

Input: "Write a poem..."

Output: [1203, 45, 9001, ...]

Think of these as Lego bricks. A simple word is one brick; a complex word might be three.

1. Embeddings: The "Hyper-Grocery Store" This was the biggest "Aha!" moment for me. How does the model know that "King" is related to "Queen"?

It's not a dictionary; it's a Grocery Store. In a grocery store, items aren't sorted alphabetically (Apples aren't next to Antifreeze). They are sorted by concept.

Apples are near Bananas (Fruit aisle).

Shampoo is near Soap (Hygiene aisle).

The model converts our tokens into coordinates in a massive, multi-dimensional space. "Robot" isn't just a word; it's a vector located near "Metal," "Future," and "Technology."

1. The Attention Mechanism: The "Cocktail Party" This is the heavy lifting. Once the tokens are in the store, how do they relate to each other?

I visualized this as a Cocktail Party. Imagine you are at a loud party. You ignore 99% of the noise, but if someone shouts your name or a topic you love, you snap to attention.

The model does exactly this. When processing the word "Bank," it looks back at the entire context window.

If it sees the token "River," it pays attention to the "Nature" meaning of Bank.

If it sees "Money," it pays attention to the "Finance" meaning.

1. The Context Window: The "Carpenter's Workbench"
   We often hear about context limits (8k, 32k, 128k). Think of the context window not as a brain, but as a physical workbench. You can only fit so many tools (tokens) on the bench at once. If you add too many, the oldest ones fall off the edge. This is why the model "hallucinates" or forgets things from the start of a long conversation—they literally fell off the table.

2. RLHF: The Wolf vs. The Dog
   Finally, I dug into why the model is polite. A raw base model (like GPT-3 before training) is a Wild Wolf. It just wants to hunt patterns. If you ask it a question, it might just ask you another question back, because that's what the training data looks like.

RLHF (Reinforcement Learning from Human Feedback) is the process of domesticating that wolf into a helpful Labradoodle. We don't make the wolf smarter; we just train it to behave in a way that humans find useful.

Final Thoughts
Tracing the data path removed a lot of the "magic" for me, but it made me appreciate the engineering even more. It’s not a mind; it’s a probabilistic engine that is terrifyingly good at predicting the next Lego brick.

If you want to see the animations for the KV Cache and Temperature, check out the video above!

Let me know if these analogies click for you, or if you have a better way to visualize the Attention Mechanism! 👇