Imagine two writers tackling the same sentence.
The first writes one word at a time, checking each new word against everything already written. The second sketches several blanks, fills in what it can, then revises the sentence until it makes sense.
The steady storyteller: autoregressive models
Most familiar chatbots generate text autoregressively. They predict the next token, add it to the text, and predict the next one. Each choice depends on the growing conversation.
This step-by-step approach works well when order matters: answering a question, writing a long explanation, or completing code from a cursor. It also makes streaming feel natural the model can start showing you its answer while it continues writing.
The trade-off? Those generation steps happen in sequence. The model has to produce one token before it can move on to the next.
The collaborative editor: diffusion models
A diffusion language model starts with a partly blank or noisy sequence. It predicts several missing tokens, then refines the result over multiple passes. Think of it as filling in a draft with context from both sides.
That makes diffusion appealing for tasks such as filling a gap in a document or editing several known regions. It can also predict multiple positions at once, creating the possibility of faster generation.
But parallel guesses can clash. If a model fills several parts of a list simultaneously, each item might look reasonable on its own while the overall list breaks the requested pattern.
| Autoregressive | Diffusion | |
|---|---|---|
| How it writes | One token after another | Fills and revises multiple positions |
| Good fit | Chat, streaming, code continuation | Infilling, editing, structured drafts |
| Watch out for | Sequential generation can take time | Parallel guesses may need extra refinement |
What researchers are learning
Recent papers show that diffusion’s potential speed advantage depends on the task. ParallelBench, an ICLR 2026 study, found that parallel decoding can lose quality when tokens depend strongly on one another. A useful reminder for developers: test on the patterns your users actually care about, not just on average benchmark scores. ParallelBench, ICLR 2026
Researchers are also trying to make diffusion models smarter about when to generate in parallel. Learn2PD, another ICLR 2026 paper, uses a lightweight filter to decide which token predictions are ready to accept. Its reported speedups are promising, but they come from specific experiments your results may differ. Learn2PD, ICLR 2026
So, which one should you choose?
For general chat, streaming, and tasks where every next word depends on the last, autoregressive models are a strong place to start. For filling gaps, revising multiple regions, or generating outputs with room for parallel work, diffusion models are worth testing.
The interesting question isn’t simply which model is faster. It’s whether your task gives a model enough room to work on several parts at once without losing the connections that make the answer correct.
Papers and further reading
- Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution (SEDD), ICML 2024.
- Shansan Gong et al. Scaling Diffusion Language Models via Adaptation from Autoregressive Models, ICLR 2025.
- Daniel Israel, Guy Van den Broeck, and Aditya Grover. Accelerating Diffusion LLMs via Adaptive Parallel Decoding, 2025.
- Wonjun Kang et al. ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMs, ICLR 2026.
- Scaling Behavior of Discrete Diffusion Language Models, ICLR 2026.
Top comments (0)