About this series
Agentic AI from First Principles is a hands-on series where we build an AI agent from scratch, without relying on frameworks. Instead of starting with abstractions, every abstraction appears only after we've hit the problem it solves.
If you're joining from the beginning, welcome. If not, you can find the previous chapters in the series here.
You ask ChatGPT a question.
A few seconds pass.
And suddenly, a well-structured answer appears on your screen.
It feels almost magical.
The model seems to understand your question, reason about it, and respond intelligently.
But what actually happens inside?
Before we start building AI agents, we need to understand the engine that powers them: the Large Language Model (LLM).
Instead of diving into equations and research papers, let's follow a single prompt as it travels through the model.
Our prompt:
"Explain how a large language model actually works."
Let's see where it goes.
1. The Gate: Tokenization
Your prompt arrives at the entrance of the model.
But there's a problem.
LLMs don't understand text.
At least, not directly.
They only understand tokens.
A tokenizer sits at the gate and breaks your sentence into smaller pieces from the model's vocabulary.
For example:
Explain how a large language model actually works.
becomes:
["Explain", " how", " a", " large", " language",
" model", " actually", " works", "."]
Each token is then converted into a numeric ID.
| Token | Ex | plain | how | a | large | language | model | actually | works | . |
|---|---|---|---|---|---|---|---|---|---|---|
| Token ID | 849 | 21435 | 1268 | 264 | 3544 | 4221 | 1646 | 3604 | 4375 | 13 |
The model never sees your original sentence again.
From this point onward, it only sees numbers.
Why tokenization exists
- Gives the model a fixed vocabulary
- Handles rare words efficiently
- Converts messy human language into predictable units
At this stage, the model doesn't understand meaning.
It simply has a sequence of token IDs.
The journey continues.
2. The Embedding Hall
The token IDs now enter a massive chamber called the embedding layer.
Here, every token is transformed into a vector.
Something like:
"model"
↓
[0.18, -0.42, 0.07, ...]
Think of embeddings as coordinates in a giant semantic map.
Words with similar meanings tend to end up near each other.
But there's still a limitation.
The embedding for "model" doesn't yet know whether we're talking about:
- a fashion model
- a machine learning model
- a mathematical model
- a 3D model
The token has some notion of meaning.
But it doesn't yet know the meaning in this sentence.
For that, we need context.
And context lives inside the Transformer.
3. The Transformer Core
This is where the real magic happens.
Imagine a giant room where every token can look at every other token.
The token "model" starts asking questions:
Which other tokens matter to me?
It notices:
- "large"
- "language"
- "actually"
- "works"
Those surrounding words provide clues.
Suddenly, the meaning becomes clear.
We're talking about a large language model, not a fashion model.
This process is powered by something called self-attention.
You don't need to understand the math yet.
The intuition is enough:
Every token continuously gathers information from other relevant tokens.
As information flows between them, the sentence becomes clearer and clearer.
What the Transformer accomplishes
- Resolves ambiguity
- Understands relationships
- Captures context
- Builds a representation of meaning
After several transformer layers, the model has a rich understanding of your prompt.
Now it's ready to respond.
4. The Response Factory
This is the part that surprises most people.
The model does not generate an entire paragraph at once.
It generates one token at a time.
That's it.
The process looks like this:
- Look at the prompt
- Predict the most likely next token
- Add it to the conversation
- Repeat
For example:
"A"
becomes
"A large"
then
"A large language"
then
"A large language model"
and so on.
Every newly generated token is fed back into the model.
The model repeatedly asks:
Given everything I've seen so far, what token should come next?
Thousands of times per response.
This simple loop is responsible for everything:
- explanations
- stories
- code
- essays
- conversations
Everything emerges from repeated next-token prediction.
5. The Exit Gate
Eventually the model decides it has finished.
Now we have a sequence of generated tokens:
["A", " large", " language", " model", " is", ...]
These tokens pass through the detokenizer.
The pieces are stitched back together into readable text.
The result becomes:
"A large language model is..."
And that's the answer you see on your screen.
6. The Entire Journey
Let's compress everything into a single flow:
Step 1: Tokenization
Text → Tokens
Step 2: Embeddings
Tokens → Vectors
Step 3: Transformer
Vectors → Contextual Understanding
Step 4: Generation
Predict the next token repeatedly
Step 5: Detokenization
Tokens → Human-readable text
Step 6: Final Response
Answer appears on your screen
What Most People Miss
At this point, something important should stand out.
Throughout this entire journey, we never gave the model:
- memory
- tools
- a browser
- a calculator
- access to files
- the ability to take actions
All it did was:
- Process tokens
- Understand context
- Predict the next token
That's it.
An LLM is incredibly powerful.
But by itself, it's still just a next-token prediction engine.
This realization becomes extremely important once we start building AI agents.
Because many of the capabilities we associate with modern AI systems don't come from the LLM alone.
They come from the systems built around it.
Final Recap
If you remember only one sentence from this article, make it this one:
An LLM converts text into tokens, uses attention to understand context, and generates a response one token at a time.
Or even shorter:
Text → Tokens → Context → Prediction → Text
That's the core idea.
And now that we understand what an LLM actually does, we can start exploring what it doesn't do.
In the next article, we'll build a chatbot from scratch and discover a surprising limitation that every LLM has.
One that ultimately leads us toward agents.
Top comments (0)