Introduction
Every time someone asks me to explain how a large language model actually works, I end up back at the same starting point: underneath the chat interface, everything a model does comes down to numbers moving through matrices. There's no understanding of language happening in the way people usually picture it. There's arithmetic, at a scale most people never see.
To make that concrete, I picked seven core ideas that, together, explain most of what's going on inside a neural network, from a single neuron firing to a model turning its final calculation into an actual word. I arranged them so the first letters spell NUMBERS:
- Neurons, the basic computing unit
- Update, how the network learns from its mistakes
- Memory, where what it learned gets stored
- Bias, a small but essential parameter (and a separate, more familiar meaning worth untangling)
- Embedding, how raw input becomes numbers in the first place
- Relationships, how the model figures out what's relevant to what
- Softmax, the last step that turns a calculation into a choice
Below, I'll take each letter in order, the same order data actually flows through a model: input becomes numbers, numbers get related to each other, and the result gets turned into a decision.
N — Neurons
A neuron in a neural network is not the biological thing it's named after. It's a small unit that does one job: take in some numbers, multiply each by a weight, add them up, add a bias, and pass the result through an activation function. That's it. No cell body, no dendrites doing anything mysterious.
What makes neurons powerful isn't any single one of them. It's the fact that you stack thousands or millions of them into layers, and each layer transforms the numbers it receives into a slightly more useful representation for the next layer. A neuron on its own is a simple function. A network of them is a very high-dimensional function approximator.
U — Update
Training a model is really just a loop: make a prediction, measure how wrong it was, and nudge the weights slightly in the direction that would have made the prediction less wrong. That nudge is the update.
The mechanism behind it is gradient descent. You compute the gradient of the loss function with respect to every weight in the network, which tells you which direction increases the error and which direction decreases it. Then you take a small step in the direction that decreases it. Do this millions of times, in small batches, and the weights slowly settle into values that make good predictions. Nothing about training a model is a single moment of learning. It's an enormous number of tiny corrections.
M — Memory
People sometimes assume a model remembers things the way a person does, by storing episodes and recalling them later. That's not what's happening in most of the architecture. The weights themselves are the memory. Every fact, pattern, and association the model has picked up during training is compressed into the numeric values of its parameters. There's no filing cabinet of facts anywhere. It's all baked into billions of floating point numbers.
There's a second, more literal kind of memory worth knowing about if you work with transformers: the KV cache. During generation, the model stores the key and value vectors from earlier tokens so it doesn't have to recompute them for every new token. That's short-term, working memory, and it's a real engineering concern when you're thinking about inference cost and context length. Long-term memory lives in the weights. Short-term memory lives in the cache.
B — Bias
This is the one word in the acronym that means two different things depending on who's using it, so it's worth being precise.
In the strict architectural sense, the bias term is a learnable number added to a neuron's weighted sum before the activation function is applied. Its job is to let the neuron shift its output independent of the input. Without a bias term, every neuron's output would be forced to pass through the origin, which limits what the network can represent. It's a small parameter, but every neuron has one, and they get updated during training just like the weights do.
In the everyday sense that shows up in conversations about fairness, algorithmic bias means something else: a systematic skew in a model's outputs that traces back to patterns in the training data. If the data overrepresents certain viewpoints, demographics, or outcomes, the model will reproduce that imbalance in its predictions. This kind of bias isn't a single parameter you can point to. It's an emergent property of what the model was trained on.
Both are real, both matter, and it's worth knowing which one you're talking about when the word comes up.
E — Embedding
Neural networks only operate on numbers, so before a model can do anything with a word, an image, or a chunk of data, that input has to be converted into a vector of numbers. That vector is an embedding.
What makes embeddings genuinely useful isn't just that they're numeric. It's that the training process arranges them so that things which are semantically related end up close together in that vector space. Words with similar meanings cluster near each other. Concepts that show up in similar contexts get pulled toward each other during training. This is the property that makes retrieval, similarity search, and RAG pipelines work at all. You're not matching strings, you're measuring distance in a high-dimensional space.
R — Relationships
Once you have everything represented as vectors, the next question is how the model figures out which pieces of information matter to each other. In transformer architectures, this is the job of the attention mechanism.
Attention computes a relevance score between every token and every other token in the context, using query, key, and value projections. The scores determine how much weight each token gets when the model builds its next representation. This is how a model can connect a pronoun to the noun it refers to several sentences earlier, or weigh a modifier against the word it's modifying. It's a numeric answer to the question "what here is relevant to what," recomputed at every layer.
S — Softmax
At the very end of a prediction, a model has a list of raw scores, one for every possible next token, called logits. Logits can be any real number, positive or negative, and they don't sum to anything meaningful on their own.
Softmax converts that list of raw scores into a probability distribution. It exponentiates each score and divides by the sum of all the exponentiated scores, so the results are all positive and add up to exactly one. The token with the highest score gets the highest probability, but every other token still has some nonzero chance of being picked, which is what lets sampling strategies like temperature and top-p do their work. Softmax is the last numeric step before the model's internal computation turns into an actual choice.
None of these seven ideas is hard to understand on its own. A neuron is just simple math. An update is just a small correction. Softmax is just a way of turning scores into probabilities. By themselves, none of these look like "intelligence."
So where does the intelligence actually come from? The answer is scale. A model like GPT or Claude isn't made of one neuron, it's made of billions of them. It isn't trained with one update, it's trained with trillions of tiny corrections. It doesn't compare two pieces of information once, it compares millions of them at the same time, across every sentence you type. No single piece is doing anything clever. What looks like intelligence shows up only when you take all seven of these simple ideas and run them together, over and over, at a scale no person could ever do by hand.
That should make the whole thing more impressive, not less. Knowing that a model is "just" numbers and simple math doesn't take away the magic, it explains where the magic comes from. The same way a single ant isn't smart but an ant colony can build complex structures, or a single water droplet doesn't have weather but billions of them together create a storm, a single neuron isn't smart, but billions of them working together can write, reason, and hold a conversation with you.
Thanks
Sreeni Ramadorai
Top comments (0)