Large Language Models (LLMs) like GPT are incredibly powerful, but they work in ways that are often counterintuitive. Here I wanna break down 3 core concepts that explain what's really happening under the hood.
tl;dr
- LLMs are stateless.
- Some models are specifically trained to produce a "thinking trace" before giving a final answer. These are called reasoning models.
- They generate internal reasoning tokens first.
- Then they generate the final output.
- This often leads to better performance on complex tasks.
Stateless LLM
When you send a prompt to an LLM, it doesn't "remember" anything from previous interactions. Each call is completely independent. The model takes your input, predicts the most likely text to follow based on its training data, and returns a response. That's it.
So why does chat.deepseek.com seem to remember what you said five minutes ago?
The trick is simple: the entire conversation history is sent back to the model with every new request.
Simply put, model is predicting what comes next based on the full conversation!
The Power of Reasoning (Chain-of-Thought)
In 2022, researchers discovered something interesting: if you ask an LLM to "think step by step," it produces better results.
This technique is called Chain-of-Thought (CoT) prompting and you can learn more about it here too.
Here's why it works: LLMs generate text one token at a time, and each token is influenced by everything that came before it. When you prompt the model to describe its reasoning process, it generates tokens that explain the problem. Then, when it generates the final answer, that answer is consistent with the reasoning it just laid out.
A Concrete Example
Question: "Toss two coins. One of them is heads. What's the chance the other one is tails?"
If you ask a standard (non-reasoning) model this question, it might incorrectly answer 50%.
But if you ask an LLM whom does reasoning by default, you will get something like this:
To answer this, we have to be very careful about what "one of them is heads" means. There are two common interpretations:
Interpretation 1: "At least one of the two coins is heads"
(This is the standard probability puzzle.)The possible outcomes for two coins are:
- HH
- HT
- TH
- TT
If we know at least one is heads, we eliminate TT. That leaves three equally likely outcomes:
- HH
- HT
- TH
In two of these three outcomes (HT and TH), the other coin is tails.
Probability = 2/3
Interpretation 2: "I randomly pick one specific coin (e.g., the first one), and it is heads"
If you specifically check one coin (say, coin #1) and see it's heads, then the other coin (coin #2) is independent. It has a 50% chance of being tails.
Probability = 1/2
So the correct answer depends on the exact wording:
- If you mean "at least one is heads" → 2/3
- If you mean "a particular coin is heads" → 1/2
Most people interpret the question as the first case, so the classic answer is 2/3.
However, reasoning models aren't always better. For simple tasks or when you're building agentic systems that already guide the model step-by-step, a standard chat model might actually perform just as well or better.
The "Wait" Hack (Thinking Budget)
Here's a hacky technique used to make models think longer and more deeply.
The model generates tokens one at a time. During the reasoning phase, it produces a series of tokens that describe its thought process. But you don't have to just feed it its own generated tokens back 😉. You can inject extra tokens 😁.
Top comments (0)