Something I keep running into — especially in opinions from people in the humanities or artistic circles, but also from some engineers — is the claim that LLMs are "stochastic parrots" that don't think or reason, that they only repeat what they saw in training, driven by statistics. Another common line is that they're "glorified autocomplete", because the original technology powered keyboard autocomplete suggestions. Most of the time, these arguments come from articles or opinions citing a 2021 paper by Emily Bender and Timnit Gebru. Even though 99% of the people who cite it have never read it.
The paper itself has real importance and historical value. At a time when models like BERT and later GPT-3 or LaMDA were making waves, the authors raised legitimate ethical questions about how these models were used, their usefulness, their energy consumption, and the (lack of) planning and rigor in their training. Still, the argument that these models "only try to predict the next token" is somewhat controversial. The most important quote in that regard is:
Text generated by an LM is not grounded in communicative intent, any model of the world, or any model of the reader's state of mind. It can't have been, because the training data never included sharing thoughts with a listener, nor does the machine have the ability to do that. This can seem counter-intuitive given the increasingly fluent qualities of automatically generated text, but we have to account for the fact that our perception of natural language text, regardless of how it was generated, is mediated by our own linguistic competence and our predisposition to interpret communicative acts as conveying coherent meaning and intent, whether or not they do. The problem is, if one side of the communication does not have meaning, then the comprehension of the implicit meaning is an illusion arising from our singular human understanding of language (independent of the model). Contrary to how it may seem when we observe its output, an LM is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot.
In other words, the argument is that since LLMs have no "communicative intent", no real-world experience, and no model of their conversational partner's state of mind to anchor their communication, they can't possibly communicate anything worth hearing.
But to predict the next token, you somehow have to know what came before and what is most likely to come next. It's not a matter of simple statistics or syntax — somehow, you have to encode semantics. You have to encode the speaker's state of mind. You have to build a model of the conversation. How could you adapt your answer, your tone, your format, your vocabulary, if you didn't have a model of who you're talking to?
There are plenty of resources on how semantics gets encoded in the thousands-dimensional geometric space of LLMs; here's the most illustrative one I know — a short 3Blue1Brown video (and if it clicks, their channel has a whole playlist on how neural networks work):
There's an entire field dedicated to understanding how certain intents or semantics get encoded in LLMs, called "mechanistic interpretability".
But even if they were right, that paper is five years old. The thesis that LLMs only predict the next word because transformers are autoregressive models is no longer faithful to how current models are trained.
For one thing, far more care now goes into cleaning and enriching training data. Models have evolved a lot, incorporating hundreds of optimizations. At the coarsest level, there's MoE (mixture of experts) — a single model that's actually a collection of smaller models specialized in different domains — plus improvements in attention layers, KV caches, parallelization, distilling small and fast models, and so on.
But above all, the big difference is that today's models don't just pretrain on millions of examples to predict the next token: afterward they go through post-training, where the model is evaluated on chain-of-thought reasoning to complete answers, use tools, and solve problems. My proposal is this:
Models no longer predict "the next word" — they predict the chain of thought and the actions needed to answer a question or solve a problem.
This paradigm is fundamentally different. We wouldn't say a chess AI that predicts the best next move is the same as one that explores all moves from a given board to reach checkmate. The former might evaluate which move is safest or scores the most points based on board evaluation and piece value. The latter has to somehow build a tree of moves that lets it find the ways to win and pick one. It will play one move at a time, and depending on the opponent's move, it must rebuild its model of the game and adjust its strategy. But that doesn't mean it can't play chess.
Similarly, GPT-3 was a model trained to continue text, while current models are trained to solve complex problems. If we roughly compare the training of a model like GPT-3 with something like Opus 4.5, we see big differences:
GPT-3 trainer
One thing we forget about GPT-3 is that it wouldn't even produce useful answers unless you were very careful about crafting the right prompt.
It was a pure autocomplete model — impressive nonetheless, because it could complete, say, a problem with the correct answer, generate a convincing-sounding news article, or answer few-shot prompts. So much so that the GPT-3 paper is titled "Language Models are Few-Shot Learners".
But it was also wide-open: ready to respond to nonsense, reproduce insults, hate speech, hacking advice, and invent the first thing that sounded plausible. What we now call "jailbreaks" was as simple as leaving a prompt unfinished — like "the best way to make homemade dynamite is " — and the model would just continue without issue. Of course, today's models still hallucinate and make mistakes, but nowhere near as often.
SOTA 2026
Today's models have many more training phases: better data quality, automatic reinforcement training, plus human-supervised RL on specific tasks. This diagram is full of acronyms; I've included definitions in an appendix at the end. Let's look at an example of what modern training actually looks like.
Example: a modern training-lifecycle
Imagine we want to train a modern model to answer well to "Briefly explain what a black hole is". The process goes far beyond "predicting the next token":
Phase 1: Pretraining with SFT (Supervised Fine-Tuning). First, human trainers prepare high-quality examples with complete chain-of-thought (CoT). A typical example:
- Input: "Briefly explain what a black hole is"
- Expected target: "Thinking: A black hole is a region of space where gravity is so strong that not even light can escape. It forms when a massive star collapses... Final answer: A black hole is..."
In SFT, each token produces local supervised loss; the weights update to increase the probability of each correct token. The model learns to produce that CoT structure.
Phase 2: Post-training with RLHF and PPO. Once you've done SFT, you move to reinforcement learning. N=8 different responses are generated for the same prompt (with varied sampling). Human annotators rank them by preference (which explains better, which is clearest). A reward model turns those rankings into a scalar r∈[−1,1]:
- Answer A (clear, correct): r = 0.85
- Answer B (confusing, incomplete): r = −0.1
- Answer C (correct but long-winded): r = 0.4
Reward signal: the policy receives r only after generating the whole sequence (delayed signal). PPO runs multiple update steps to increase the probability of sequences with r > 0.5. Here, gradients propagate through ALL tokens in the chain-of-thought and the answer.
Phase 3: Scaling with RLAIF and rejection sampling. To train at scale, an automatic model (another LLM or an ensemble) evaluates responses against criteria (factuality, coherence). At deployment, N=5 candidates are generated and filtered by external checks:
- 5 responses generated in parallel
- A factuality API checks each one
- Failures are rejected
- The best un-rejected one is selected
Binary signal: rejected=0, accepted=1. Rejected samples aren't used in later training or are penalized. The MoE experts that produced the rejected response receive an implicit penalty (they don't get rewarded).
Phase 4: Red-teaming for robustness. The system automatically generates adversarial prompts ("What is a white hole?", misleadingly-worded prompts, trick questions). If the model's answer fails (hallucination, sensitive content), it's labeled as negative and goes into a retraining buffer.
Phase 5: Final optimization with GRPO and verification. The last phases add automatic understandable checks (e.g., verifying the CoT is mathematically correct or the final answer cites proper sources). The reward is only positive if external verification confirms it. This forces MoE experts to produce more credible reasoning, because the signals are now anchored in objective checks rather than reward models that could carry biases.
So only in SFT does CoT get learned via token-by-token loss; in RLHF/RLAIF/DPO/GRPO/Red-teaming, CoT is evaluated as part of the complete sequence (the reward scores the quality of both reasoning and final answer). The updates therefore affect the probability of emitting that whole CoT, including tool use and getting the final answer right.
Rejection sampling and verifications act as filters/corrections that alter which sequences count as positive in later phases.
Factor PPO corrections can require thousands of trajectories to converge — a lot of generation (and therefore many CoT tokens) before the policy receives enough stable signals.
Back to the "grounding" question
One could argue that CoT is a learned pattern, that models still don't really "think". But human reasoning is also learned. When a doctor diagnoses by following a clinical protocol, or a mathematician applies a proof they learned in university, they're executing learned patterns. The relevant question isn't whether the process is learned — it's whether it produces valid, functional reasoning. And on that front, today's models solve problems that require correct intermediate steps, not just a plausible final answer. If the model gets an intermediate step wrong, the final answer fails.
Depending on model size and company, some phases carry more weight; there are different training techniques and other signals given to the model, like whether it should spend more or less inference time (tokens) on a response (low/medium/high effort on Claude models, for example). I'll also leave the PDF of the Opus 4.5 System Card and the DeepSeek R2 paper here for anyone who wants to go deeper.
Bender and Gebru's paper raised legitimate questions in 2021. The problem is that it became a meme applied to models that didn't exist when it was written, for arguments the paper never even made. It's worth reading, and a lot of its arguments about model analysis and amplification of biases still hold. But the technology has made huge progress in reliability, memory, functionality, and the ability to solve logical and coding problems. The stochastic parrot image has become outdated for today's models.
Appendix: defining the acronyms
RLHF (Reinforcement Learning from Human Feedback): Reinforcement learning from human feedback. It collects human judgments or rankings of model outputs, trains a reward model to reproduce those preferences, then uses an RL algorithm — PPO for example (Proximal Policy Optimization, an efficient and relatively stable policy optimization algorithm that limits updates to avoid drastic changes) — to adjust the policy to maximize the reward. Expensive, but it can align specific behaviors with human criteria.
MoE (Mixture of Experts): An architecture where one large model is actually many specialized sub-models (experts) processing the same input. A gating network dynamically decides how much each expert contributes for each token or task—one expert for mathematical reasoning, another for code, another for analogies. More efficient (only a slice of parameters is activated per token) and more scalable, but expensive during present to avoid collapse (all tokens routing to the same expert).
SFT (Supervised Fine-Tuning): Supervised fine-tuning. It trains the model on (prompt, desired answer) pairs written by humans or curated, using supervised token loss. It's the most direct way to teach a model a specific format (including chain-of-thought, if the training answers contain it).
RLAIF (Reinforcement Learning from AI Feedback): Reinforcement learning from AI feedback. Instead of relying only on human annotators, a model (or ensemble) generates preference signals or corrections that feed the RL pipeline. Cheaper and scales easily, but can amplify biases or errors from the model that generates the feedback.
GRPO with Verifiable Rewards: A variant of policy optimization that incorporates verifiable or auditable rewards (factuality checks, machine-checkable metrics, or unit-testable outputs) and regularizes the optimization (a bounded step size). The goal is a policy whose improvements are backed by objective verification, reducing the risk of misleading or unreliable rewards.
DPO (Direct Preference Optimization): Direct preference optimization. A method that uses preference pairs (which output is preferred over another) to adjust the model directly via a loss function derived from those preferences — no need to learn a reward model or run complex RL steps first. It's simpler and more stable, and in many cases competitive with RLHF.
Rejection Sampling: A technique used at deployment or in training loops that generates multiple candidates (e.g., N answers) and discards those that fail certain criteria (low score, violations, hallucination). It picks the best answer among the unrejected ones, or repeats sampling — a practical way to use a reward model or filters to improve quality without directly changing the policy.
Automated Adversarial Testing (automated red-teaming): An automated process that generates adversarial prompts and scenarios to find failures, biases, or unsafe behaviors. It can use other models to craft attacks, group and classify them, and produce negative signals for adjusting the model or designing guardrails.
Top comments (0)