Almost every LLM you've used — GPT, Claude, Llama — writes text the way a person types: predict the next token, append it, predict the next one from everything so far. One token per forward pass, strictly left-to-right. It's so standard that most people assume it's just what generating text means. It's a design choice, not a law. A diffusion language model (LLaDA, Mercury) makes a completely different bet: it starts from a whole sentence of [MASK] and denoises it, predicting every blank at once, keeping only the most-confident ones, and repeating. I built a demo with a real confidence schedule computed live in the browser to show what that buys you — and what it costs.
The catch with left-to-right: you can't unsee a bad token
Because it's strictly left-to-right, an autoregressive model must commit token t using only tokens 1…t-1 — it has no access to the future. If a later word would have implied a better choice earlier, too bad: the earlier token is already written and frozen. Beam search only shuffles which greedy path you keep; it doesn't let a committed token change once the sentence moves on. That one-way, no-eraser property is the specific limitation diffusion sets out to remove.
Start from all-[MASK] and clear the fog
Diffusion flips the setup. Instead of building the sentence up from nothing, you begin with the whole sequence already there — but every position is noise, a special [MASK] token. Generation is no longer "append the next word"; it's "take this fully-corrupted sequence and clean it up," exactly like image diffusion starting from static. Each denoise step looks at the entire current sequence — masks and revealed words together — and predicts a token with a confidence for every masked position at once. Crucially it's bidirectional: a blank in the middle already sees hints from words on both sides.
Keep only the confident ones, and let context resolve the rest
If you accepted every guess at once you'd get garbage — the hard slots are still unsure. So the schedule commits only the most-confident K predictions each step and re-masks the rest:
function denoiseStep(seq, K){
const masked = [];
for (let i=0;i<N;i++) if (!seq[i].revealed) masked.push(i);
masked.sort((a,b) => confidence(seq,b) - confidence(seq,a)); // most-confident first
for (const i of masked.slice(0, K)) reveal(seq, i); // K at once, any order
}
The easy anchors — function words, obvious continuations — reveal first. And every token you reveal becomes context for the ones still masked. In the demo, a slot's confidence is its own predictability plus a bonus for each already-revealed neighbour:
function confidence(seq, i){
return TOKENS[i].prior + 0.26 * revealedNeighbours(seq, i) + noise[i];
}
So an isolated content word is a coin-flip, but once the words around it land its confidence jumps and it gets revealed next. The sentence resolves outward from its certain points, a few tokens per step, instead of guessing hard words left-to-right before the right-hand context exists.
The superpower: revise an early guess
Because tokens aren't frozen on contact, a diffusion model can re-mask and rewrite a slot it now believes is wrong. A word revealed early with little context is only a tentative guess; each step, before unmasking, re-check every tentative slot — now that neighbours have arrived, its confidence has climbed past the threshold, so overwrite the guess with the correct word. In the demo that's an amber tentative token flipping green once its context lands. It's the correction an autoregressive model structurally cannot make.
Fewer steps, and where the bet stands
Add it up. An N-token sentence costs an autoregressive model N forward passes, always. A diffusion model commits several tokens per step, so it needs only about ⌈N/K⌉ — a big-K schedule drafts a whole sentence in a handful of passes. That's the headline pitch behind fast text-diffusion like Mercury. The honest catch: each pass predicts all positions (more compute per step), and because positions aren't generated left-to-right it can't reuse a KV-cache the way a decoder does — so "fewer steps" means lower latency, not automatically fewer FLOPs. It's worth lining up against Mamba, the other big challenger to the transformer default: Mamba keeps the left-to-right order but makes it linear-time; diffusion keeps attention but throws the order out entirely, buying parallelism and revisability. Two different bets against "autoregressive transformer is the only way," and no single winner yet — which is exactly why text-diffusion is the challenger to watch.
Run the denoiser and race it against the left-to-right writer:
https://dev48v.infy.uk/ai/days/day45-diffusion-language-models.html
Top comments (0)