What an LLM Actually Is (and Isn't)

An LLM doesn't "know" that Paris is the capital of France the way a database row knows it. It doesn't understand your question, and it isn't reasoning before it answers.

Here's the one sentence that explains almost everything "weird" about how these models behave:

A large language model is a function that takes in some text and outputs a probability distribution over what token is most likely to come next. That's it.

It has seen so much text where "The capital of France is" is followed by "Paris" that "Paris" gets an overwhelmingly high probability in that spot. Ask it something its training data genuinely didn't cover well, and it still has to output something with high probability — which is the actual mechanism behind hallucination.

Next-token prediction, made concrete

Feed the model a sequence of tokens (words and word-pieces). The model outputs a probability for every token in its vocabulary being the next one. A sampling step picks one, appends it, and the whole thing runs again to predict the token after that — one token at a time, until it hits a stop token or a length limit.

Here's a wildly simplified stand-in for the mechanism (real models score tens of thousands of candidate tokens with a neural network; this hardcodes a tiny probability table just to make the shape of it visible):

const nextTokenProbabilities = {
  "The capital of France is": { Paris: 0.91, Lyon: 0.03, France: 0.02, "?": 0.01 },
  "2 + 2 =": { "4": 0.97, "5": 0.01, "22": 0.01 },
};

function predictNext(promptSoFar) {
  const distribution = nextTokenProbabilities[promptSoFar] ?? { "...": 1.0 };
  const [bestToken] = Object.entries(distribution).sort((a, b) => b[1] - a[1])[0];
  return bestToken;
}

console.log(predictNext("The capital of France is")); // "Paris" — highest probability, not "known fact"
console.log(predictNext("2 + 2 ="));                   // "4" — same mechanism, correct answer

Notice: nothing in this function checks an answer against reality. Both correct outputs came from the identical process — "what token has the highest probability given everything before it" — as any hallucinated output would. There's no separate "is this true" step. That's not a bug you patch; it's the architecture.

What this rules out

No understanding, no reasoning "before" answering, in the way those words imply for a person. The model isn't silently forming an opinion and then describing it — the text is the computation. It's also why chain-of-thought prompting works at all: forcing the model to generate intermediate reasoning tokens genuinely changes what gets predicted next, because each new token becomes part of what the next prediction conditions on.

No live knowledge of anything after its training cutoff, and no access to your data, by default. Ask a base model what today's date is, and it can't know. Closing that gap is the entire premise of retrieval (RAG) and tool calling.

No persistent memory between separate API calls. Every request is stateless — "remembering" a conversation is really "re-sending the whole conversation so far," which is exactly why context windows become a hard, practical constraint the moment a conversation runs long.

Keep this mental model — text in, next-token probabilities out, sampled and repeated — in mind for everything else: tokenization, context windows, hallucination, prompting techniques, and even why agents get stuck in loops all trace back to this one mechanism.

---

This is the opening topic of the AI/LLM Engineering pillar on discoveringCode — a free, ad-free notebook that goes beginner-to-expert on frontend, backend, system design, infra, and AI/LLM engineering, with live interactive code playgrounds. If this was useful, the next topic on tokenization builds directly on it.