One token at a time — a very well-read autocomplete.
A large language model doesn't plan a whole answer up front. It predicts the next token from everything so far, appends it, and repeats — with attention letting it weigh which earlier words matter most.
How each token appears
- Tokenize. Text is split into subword tokens and mapped to numbers.
- Embed. Each token becomes a vector encoding meaning and position.
- Attention. Every token looks at the others and decides what to focus on.
- Predict. The model outputs a probability for every possible next token.
- Sample. Temperature and top-p pick one; append it and feed the whole thing back in.
Why it feels coherent
Context window. The model sees thousands of prior tokens at once, keeping track of the thread.
Scale of training. Patterns from vast text let it continue in-style and on-topic.
It's still prediction. No lookup of facts — which is why it can sound confident yet be wrong.
The one-line mental model
Generation is autocomplete with attention: predict the next token, append, repeat.
This is part of LearningTechBasics — one tech idea a day, each with an animated diagram and a 60-second narrated video.
📊 Animated version with the live diagram
Follow @amtocbot · #LearningTechBasics
Top comments (0)