Today Inception Labs dropped Mercury 3: a diffusion-based LLM with a 1M-token context window doing 1,068 tokens/sec on NVIDIA GPUs, claiming to beat GPT-6 Luna and Claude Haiku 5.5 on benchmarks — at $0.20/$0.70 per million tokens, half price during launch. It's live on OpenRouter, Baseten, Vercel, and behind an OpenAI-compatible API, so you can try it by changing a base URL.

If you're a builder, here's the part that matters: this is not "a faster autoregressive model." It's a different generation paradigm, and it changes what your inference code can assume.

The 30-second architecture primer

Autoregressive decoding: for i in range(n): token[i] = model(tokens[:i]). Serial. Unfixable — once token 5 is emitted, it's baked in.

Diffusion decoding: start with a fully masked/noisy sequence of length n, then iteratively denoise all positions in parallel over K refinement steps. Position 5 can be revised in step 12 based on what position 500 looks like. The model gets K parallel passes over the whole sequence instead of n serial commits.

That parallelism is where 1,068 tok/s comes from. On a GPU, parallel refinement across the sequence exploits the hardware's actual strength (wide vector math) instead of its weakness (serial dependency chains).

Why this should change your evals

Three workload classes where diffusion's architecture is structurally advantaged:

  1. Infilling and editing. Code completion mid-file, document revision, structured-output repair. Autoregressive models fake this with fill-in-the-middle hacks; diffusion models do it natively because every position is refinable.
  2. Constrained generation. JSON schemas, formats with global consistency requirements, anything where a late constraint should rewrite an early choice. Serial models can't go back; diffusion can.
  3. Self-correction loops. Exposure bias — the "one wrong token poisons everything downstream" problem — is an autoregressive artifact. Parallel refinement doesn't have it, at least not in the same form.

The flip side, from the research literature (Diffusion-LM, SEDD, LLaDA): open-ended long-form coherence and precise instruction-following have historically lagged autoregressive models. Mercury 3 claims to close the gap. Claims. Test it yourself.

Trying it: the 5-minute eval 

from openai import OpenAI client = OpenAI( base_url="https://api.inception.ai/v1", # OpenAI-compatible endpoint  api_key="YOUR_INCEPTION_KEY", ) resp = client.chat.completions.create( model="mercury-3", messages=[{"role": "user", "content": "Refactor this function..."}], ) print(resp.choices[0].message.content)

What to actually measure — don't just vibe-check:

  • Time to first usable token vs. full completion time. Diffusion models refine the whole sequence; "first token" semantics differ. Measure what your UX actually needs.
  • Throughput at YOUR context length, not the headline 1,068 tok/s (measured at favorable conditions). Plot tok/s against 4K / 32K / 128K / 1M context. The 1M-context + max-speed pairing is almost certainly not simultaneous.
  • Edit/infill accuracy on your real tasks: give it broken JSON, half-written functions, documents with a paragraph to rewrite. This is where diffusion should win — verify it does.
  • Instruction-following precision on multi-constraint prompts. This is where diffusion historically loses — verify it doesn't.

The honest caveats (day-one edition)

  • Vendor benchmarks, vendor hardware, vendor conditions. Independent evals will land in days; don't migrate production on launch-day numbers.
  • The entire inference ecosystem (KV-cache tricks, speculative decoding, every serving framework's optimizations) was built for autoregressive decoding. Diffusion uses different kernel patterns — expect immature tooling and surprising failure modes.
  • Prompt engineering behaves differently. Techniques tuned for left-to-right generation (chain-of-thought scaffolding, "let's think step by step") may not transfer. Budget time to re-learn what works.

Why I'm paying attention anyway

October has been the month the LLM monoculture cracked: open weights counter-attacking (816B-param models last week), inference prices collapsing (DeepSeek's $0.50 agent yesterday), and now a non-autoregressive architecture shipping at frontier-adjacent quality. Even if Mercury 3's numbers turn out optimistic, the direction is set — and the builders who understand diffusion decoding now will have a head start when (not if) every lab ships one.

Try it this week. Report back what breaks. That's how new architectures earn trust.

Source: Inception announcement via PANews, Oct 20, 2026. Benchmark figures are vendor-reported. Written by Shivam Kumar (VisionQuantech).