Let me start with a deceptively simple question:
Why can AI answer our questions at all? How does it "know" anything?
There are papers and textbooks that answer this in exhaustive technical detail — and most of them are genuinely hard to get through. But for the vast majority of developers and curious non-specialists, you only need to understand three mechanisms. Once these click, a lot of AI behavior that seems magical starts to make sense.
Mechanism 1: Parameters (Compressed Knowledge)
At the macro level, every large language model is essentially a mathematical model of human knowledge.
Here's how it works: the model takes all the text it's trained on and breaks it down into small units called tokens (roughly: words or word fragments). It then calculates the mathematical relationships between every pair of tokens. This process is called training.
Those relationships are encoded as parameters — the famous billions of numbers you see cited in model announcements. For example, GLM 5.3 has 744 billion parameters. Each one is a tiny weight that captures some aspect of how tokens relate to one another.
Training, in essence, is the process of finding those 744 billion weights — the ones that best represent the structure of human knowledge as seen in the training data.
Once training is done, when you ask the model a question, it uses those weights to find the most probable sequence of tokens to generate as a response.
The mental model to hold here: LLMs are a compression-and-generation engine. Human knowledge gets compressed into parameters during training, then decompressed into language when you query the model.
Mechanism 2: Reasoning (Derived Knowledge)
More parameters generally means better performance — more parameters mean finer-grained representations of more knowledge, which translates to more accurate outputs.
But there are two hard limits on just scaling parameters forever:
Cost. Training and inference costs scale fast. So does latency.
Redundancy. Not all knowledge needs to be memorized. A lot of it can be derived.
Consider a simple example: if a model knows a city's birth rate and death rate, it doesn't need to have the net population growth rate stored as a separate fact — it can calculate it on the fly.
This is the reasoning mechanism. Given the knowledge encoded in its parameters, the model applies logical rules to infer things it was never explicitly trained on. Modern "reasoning models" (o1, DeepSeek-R1, etc.) are largely an amplification of this mechanism — they're designed to spend more compute at inference time working through chains of logic before producing an answer.
Mechanism 3: Web Access (Retrieved Knowledge)
No matter how many parameters a model has, and no matter how strong its reasoning, there will always be questions it can't answer.
The classic example: "What was the closing price of the S&P 500 today?"
That fact wasn't in the training data. It can't be derived through logic. The model simply doesn't know — and it shouldn't pretend to.
This is where the third mechanism kicks in. Via agents or application frameworks, the model can reach out to the internet, query an API, or search a database to fetch what it doesn't know internally.
Tools like web search, code execution, and external API calls are all expressions of this mechanism. Rather than the model "knowing" everything, it knows how to find what it doesn't know.
Putting It Together
These three mechanisms form a layered system:
| Mechanism | What it provides | When it's used |
|---|---|---|
| Parameters | Knowledge baked in during training | The baseline — always active |
| Reasoning | Inferences from existing knowledge | When the answer can be derived |
| Web Access | Real-time or external knowledge | When parameters + reasoning fall short |
Most AI answers you get are a blend of all three. The model pulls from its parameters, reasons across them, and if equipped with tools, fetches what's missing.
Understanding this stack doesn't just satisfy curiosity — it changes how you prompt, how you architect AI features, and how you debug why a model gets something wrong. A hallucination is often a parameter problem (the training data was wrong or sparse). A stale answer is a web-access problem. A logical error is a reasoning problem. The diagnosis changes based on which mechanism failed.
Top comments (0)