Mercury 3: How Inception Labs Scaled Diffusion LLMs to 1,000 Tokens Per Second

When most people think of large language models, they picture the same basic loop: generate one token, append it to the context, generate the next. Autoregressive decoding has been the default for years, and for good reason — it works. But it also has a hard ceiling on throughput, because each token depends on every token before it. Inception Labs has been quietly building a different kind of model, and with Mercury 3, released on October 19, 2026, they've pushed diffusion-based language models to a scale where that ceiling starts to matter.

What Makes a Diffusion LLM Different

Standard autoregressive models generate text left-to-right, one token at a time. Each step is conditioned on the full sequence so far, which means you can't parallelize across output positions — you have to wait for position n before you can compute position n+1. This sequential dependency is the fundamental bottleneck.

Diffusion language models flip the process. Instead of building a sequence incrementally, they start with a fully masked or noisy draft of the entire response and iteratively refine all positions in parallel. The model runs multiple denoising passes, each time improving the quality of the whole output simultaneously. Think of it as the difference between writing a document word-by-word versus sketching a rough outline and progressively sharpening every section at once.

Mercury 3 is built on a Transformer backbone — the same architectural foundation as GPT-style models — but replaces the autoregressive training objective with a denoising diffusion loss. This means the model learns to predict clean tokens from corrupted ones, rather than predicting the next token from a clean prefix. The Transformer handles the heavy lifting of attention and representation; the diffusion process defines how generation actually works.

The Throughput Advantage

The practical payoff is speed. Mercury 3 achieves approximately 1,068 tokens per second on NVIDIA GPUs — a figure Inception claims is 4 to 8 times faster than speed-optimized autoregressive models like GPT-6 Luna and Claude Haiku 5.5. The reason is arithmetic intensity: because the model refines many positions simultaneously, it can saturate GPU compute more efficiently than a sequential decode loop that spends most of its time waiting on memory bandwidth for a single token at a time.

This isn't just a benchmark number. For applications that need to generate large volumes of text quickly — automated evaluation pipelines, document drafting, high-throughput API services — the difference between 150 tokens/sec and 1,000 tokens/sec changes what's economically feasible. Inception prices Mercury 3 at $0.10 per million input tokens and $0.35 per million output tokens (a 50% launch discount from list price), positioning it well below the cost of frontier reasoning models. Full pricing details are on the Inception pricing page.

Scale and Context

Mercury 3 is described as more than 40 times larger than the original Mercury 1 model. Inception has not disclosed an exact parameter count — the company notes that raw parameter comparisons between diffusion models and autoregressive models can be misleading, since the two architectures use compute differently during inference. What they have confirmed is a 1-million-token context window, up from 260K in Mercury 2.5.

The scaling story matters because it addresses a longstanding criticism of diffusion LLMs: that they work well at small scale but don't improve predictably as you add parameters. Inception's position, backed by their Mercury technical report, is that the same scaling laws that govern autoregressive models apply to diffusion models — you just need to run the experiment at sufficient scale to see it. Mercury 3 is their evidence.

Where Diffusion Helps (and Where It Doesn't)

The parallel refinement process gives diffusion models structural advantages in specific task types.

Infilling and editing are natural fits. Because every position in the output is revisable at each denoising step, the model can natively fill in the middle of a document or revise a specific section without the "fill-in-the-middle" workarounds that autoregressive models require. Constrained generation — producing output that must conform to a JSON schema or a specific format — also benefits, since late-stage constraints can propagate back to earlier positions during refinement.

Agentic coding tasks showed strong results in early testing. In one documented case, Mercury 3 diagnosed and fixed a non-obvious logic bug in a donor-registry application without explicit error messages, demonstrating grounded reasoning rather than pattern-matching on surface cues. Inception also reports that running "swarms" of Mercury 3 agents for recursive self-improvement increased effective throughput by an additional 16%.

The limitations are real, though. Long-horizon, multi-step tasks — building an interactive graphic from scratch, complex open-ended generation — exposed weaknesses. The model slowed down, experienced looping issues, and struggled with tool calling as task complexity increased. This is consistent with what you'd expect: diffusion models excel when the output space is well-defined enough that parallel refinement converges quickly. Open-ended generation with many interdependent decisions is harder to parallelize effectively.

Availability and Integration

Mercury 3 is available through the Inception API, OpenRouter, Baseten, and Vercel. It exposes an OpenAI-compatible API, which means existing code that calls GPT-style endpoints can switch to Mercury 3 with minimal changes. The model identifier is mercury-3 on the Inception platform.

Inception has sunset Mercury 2 and 2.5 for new users, with existing access ending November 15, 2026. The company has indicated interest in making the launch pricing permanent, though no formal commitment has been made.

What This Means for the Field

Mercury 3 doesn't replace frontier reasoning models — it's not trying to. What it demonstrates is that diffusion-based language models can scale to competitive quality levels while maintaining a throughput advantage that autoregressive architectures can't match by design. For practitioners building high-volume pipelines, automated evaluation systems, or latency-sensitive applications, that's a meaningful option that didn't exist at this quality level a year ago.

The deeper question is whether diffusion scaling continues to hold as models get larger. If Inception's claim about scaling laws is correct, the throughput gap between diffusion and autoregressive models should widen as both approaches scale — because the sequential bottleneck in autoregressive decoding doesn't get better with more parameters, but the parallelism in diffusion decoding does. That's a hypothesis worth watching as the next generation of models arrives.