Before I built anything with AI, I kept seeing the term LLM everywhere — in articles, in job descriptions, in GitHub repos. I nodded along like I understood it. I didn't. Not really.
I knew it stood for Large Language Model. I knew ChatGPT was one. But when someone asked me "how does it actually work" — I couldn't explain it. I had the label, not the understanding.
This article is the explanation I wish I had before I started building.
What does "Large Language Model" actually mean
Let's break the name down.
Language — it works with language. Text in, text out. It reads your words and writes words back.
Model — it is a mathematical model. Millions (actually billions) of numbers, carefully calculated during training, that together capture patterns in language.
Large — the model is enormous. GPT-4 has an estimated 1.8 trillion parameters. Gemini, Claude, Llama — all in the hundreds of billions. These numbers are what make them capable. The "large" isn't marketing — it's what separates these models from earlier, simpler ones.
So: a Large Language Model is a very large mathematical model trained to understand and generate language.
You've already used one. When you typed something into ChatGPT, you used GPT-4o — an LLM made by OpenAI. When you used Gemini on Google, that's an LLM. Claude (made by Anthropic), Llama (made by Meta, open source), Mistral — all LLMs. Different companies, different training data, different sizes — but the same core idea.
That's the definition. But definitions don't give you intuition. Let's go deeper.
What is the model actually doing
Here is the single most important thing to understand about how an LLM works:
It is predicting the next word.
That's it. That's the core task. You give it text, it predicts what word (technically token, but we'll get to that in the next article) comes next. Then it predicts the next one. Then the next. It keeps going until it decides to stop.
Let me show you what I mean. Say you type:
"The capital of France is"
The model looks at those five words and predicts what comes next. "Paris" is overwhelmingly the most likely next word based on everything it learned during training. So it outputs "Paris."
Now it has: "The capital of France is Paris"
It predicts what comes next again. Maybe a period. Maybe "and" followed by something else. It continues predicting, one token at a time, until the response is complete.
This is why LLMs can write code, answer questions, summarise documents, translate languages — all with the same underlying mechanism. It is always just: given this text, what comes next?
How does it "know" things
The model didn't come with knowledge pre-loaded. It learned by reading.
During training, the model was fed an enormous amount of text — web pages, books, code repositories, research papers, articles. Trillions of words. For each piece of text, it practiced the same task: predict the next word. When it got it wrong, the training process adjusted its internal numbers slightly to do better next time. This happened billions of times.
After training, those billions of numbers encode patterns from all that text. When you ask it "what is the capital of France" — it doesn't look it up in a database. It pattern-matches against what it saw during training and produces the most likely answer.
This is why it feels like the model "knows" things. It has absorbed patterns from an enormous amount of human writing. But it's not retrieving facts — it's generating text that fits the pattern.
Why it gets things wrong — hallucination
Here's the uncomfortable part.
Because the model is predicting text rather than retrieving facts, it can generate text that sounds completely confident and is completely wrong.
Ask it about a paper that doesn't exist — it might describe one in detail, with a made-up author and made-up conclusions. Ask it a question whose answer wasn't well-represented in its training data — it'll give you something that sounds right but isn't.
This is called hallucination. The model isn't lying. It's doing exactly what it was trained to do — predict plausible text. But "plausible" and "true" are not the same thing.
For a backend engineer, this is critical to understand because it shapes every architectural decision you make:
- You can't trust the model to know your company's internal data — it was never in the training set
- You can't trust it for real-time information — training has a cutoff date
- You can't trust it for precise facts without verification
This is exactly why RAG (Retrieval Augmented Generation) exists — you give the model the facts it needs in the prompt, so it's not guessing from training memory. But that's a later article. For now, just internalise this: the model generates, it doesn't retrieve.
What the model is not
These comparisons help me think clearly when designing systems:
Not a database. You can't query it for a specific record. It doesn't store facts — it stores patterns. You can't ask it "what did user 123 order last week" and expect an answer unless you tell it.
Not a search engine. A search engine finds documents that match your query. An LLM generates new text. Completely different mechanisms.
Not a person. It has no opinions, no emotions, no beliefs. When it says "I think" or "I feel" — that's pattern matching on how humans write, not an actual inner state. It produces text that sounds like a person because it learned from text written by people.
Not always right. Even when it sounds certain. Especially when it sounds certain.
Why this matters for what we're building
Everything we build in this series — chat apps, RAG pipelines, AI agents — sits on top of this one thing: a model that predicts text, one token at a time, based on everything you give it in the prompt.
Understanding this changes how you design:
- You know you need to give it facts explicitly, not assume it knows them
- You know the quality of your output depends heavily on the quality of your input
- You know it has no state, no memory, no awareness beyond what you send in a single call
The model is a very powerful, very fast text predictor. Your job as a backend engineer is to engineer what it predicts — by carefully constructing what you give it.
That's the whole game.
What's next
Now that you know what an LLM is and how it works, the next question is: how does it measure and process the text you give it? That's where tokens and context windows come in — and that's exactly what the next article covers.
What surprised you most about how LLMs actually work? Drop it in the comments.
Sham Prakash K — Backend Engineer, 4+ years in Java, Spring Boot, and distributed systems. Building AI backend infrastructure. Writing about what I actually learned, mistakes included.
Top comments (0)