A 2B Drafter Made a 27B Model 3.4x Faster. Same Exact Output.

TL;DR

DFlash 2 landed on August 18 from Inco AI: a rebuilt block-diffusion drafter for speculative decoding, plus two Apache 2.0 drafter checkpoints on Hugging Face, a 2B for Qwen3.8-27B and a 3B for Meta's Muse Glimmer 30B. On a single H200 the Qwen pairing runs 2.67x to 3.43x faster than plain autoregressive decoding at batch size 1, and the output is provably unchanged. That is the headline number. The more interesting one is what happens at 32 concurrent streams, where the incumbent methods stop being faster at all.

What actually shipped

Two drafters, incoai/Qwen3.8-27B-DFlash2 (2B parameters) and incoai/Muse-Glimmer-30B-DFlash2 (3B), both under Apache 2.0. The implementation lives at z-lab/dflash under MIT, currently around 5.8k stars.

Runtime support is the unusually complete part. SGLang, vLLM, Transformers and text-generation-inference on day one, with quantized builds for llama.cpp, Ollama, LM Studio and Jan, plus an MLX path for Apple Silicon. A speculative-decoding method that only works in one server is a paper. This one is a dependency.

Lineage matters here. DFlash 1 shipped in June as a Z Lab, Modal and SGLang collaboration. Version 2 is the same authors, continued at Inco AI. If you have run a Muse Glimmer local build recently, you have probably already used version 1 without noticing.

What a block-diffusion drafter is, and why it kept breaking

Ordinary speculative decoding pairs a big target model with a small autoregressive draft model. The drafter guesses a run of tokens one at a time, the target verifies the whole run in a single forward pass, and you keep the matching prefix. The catch is that the drafter is itself serial, so you are paying serial latency to save serial latency.

Block diffusion drafting removes that: predict the entire block of draft tokens in one parallel pass. Excellent in theory. In practice each position was predicted more or less independently of its neighbours, so you got seven individually plausible tokens that did not read as a sequence, and the target rejected most of them.

Inco measured the failure instead of hand-waving at it, and the two numbers that come out are the whole release:

• Selection headroom. The correct token sits somewhere in the drafter's top 16 candidates 99.5% of the time. The top-1 pick is right only 85.4% of the time. The information was already there. The drafter just was not reading it.
• Suffix decay. Recall@16 falls from 99.5% at the first position to 87.8% by position 6. The block starts confident and trails off toward the end, the way a sentence does when its author loses the thread partway through.

The two fixes

The path selector keeps the top 16 candidates at every position and scores adjacent pairs with a low-rank bilinear attention, then traces one coherent route through that lattice. Everything still runs in parallel; the selector is choosing a path through a grid, not generating tokens one by one.

Picture your phone's predictive text, but with sixteen suggestions above every future word rather than three. Accepting the top suggestion at each slot independently gives you seven words that are each defensible and together mean nothing. The path selector is the step that reads down the grid and picks the chain that actually forms a sentence.

The two-tap dynamic convolution attacks the decay. It mixes each position's representation with its immediate predecessor's, giving the tail of the block the short-range local context it was starved of.

The costs are the reason this is interesting rather than merely clever. The selector adds 2.0M parameters and 0.6% of draft-verify cycle latency. The convolution adds 16.5M parameters, roughly 3% of the drafter, and 0.7% of cycle latency. Together: about 1.3% more latency per cycle, for over 20% more accepted output per verification pass.

How many tokens survive

Acceptance length is the metric that matters: the mean number of draft tokens the target actually keeps per verification pass. Higher means fewer expensive target forward passes per unit of output.

On Qwen3.8-27B, DFlash 2 averages 4.80 against 4.28 for MTP and 3.62 for DSpark. On Muse Glimmer it averages 5.70 against 4.44 for the original DFlash and 4.48 for DSpark. Per task, the Qwen drafter ranges from 4.10 on MT-Bench to 5.46 on GSM8K; Glimmer ranges from 4.42 to 6.57. Reasoning and math predict well. Open-ended chat does not, which is exactly what you would expect and rarely what vendors publish.

Throughput at batch size 1

Across the five tasks Inco reports at batch size 1, DFlash 2 lands between 2.67x (MT-Bench) and 3.43x (GSM8K). At concurrency 8 it holds 2.27x to 2.85x. The Muse Glimmer drafter is faster still, up to 4.62x at concurrency 1 and up to 3.99x at concurrency 8.

The table nobody usually publishes

Speculative decoding is a trade: you burn extra compute to save memory bandwidth. At batch size 1 a GPU is bandwidth-bound and its compute is effectively free, so the trade is a gift. At high concurrency the GPU is already saturated with real work, and every rejected draft token is compute you stole from a paying request. This is where speculative decoding usually quietly stops helping, and where vendor blog posts usually quietly stop having tables.

On MT-Bench at 32 concurrent requests, MTP scores 0.77x and DSpark 0.74x, which is a polite way of saying they made the server slower. DFlash 2 scores 1.01x. On GSM8K at the same concurrency it is 1.45x against 1.04x for MTP and 1.13x for DSpark.

Read that honestly: 1.01x is break-even, not a win. But "does not actively harm your throughput under load" is a materially different operational story from "turn it off above batch 8," and it is the difference between a laptop trick and something you leave enabled in production.

Running it

The drafter is not a standalone model. It attaches to a specific target. In SGLang:

python -m sglang.launch_server --model-path Qwen/Qwen3.8-27B --speculative-algorithm DFLASH --speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 --speculative-num-draft-tokens 8

Draft-token counts differ by runtime: 8 in SGLang, 7 in vLLM, a maximum of 7 in llama.cpp, and a block size of 5 in the MLX path. The published H200 benchmarks used 7 draft tokens per verification step with FlashAttention 3 on both target and draft attention.

Caveats before you rebuild your serving stack

• All numbers are Inco's own. One H200, FlashAttention 3, five benchmarks. No third-party reproduction existed at publish time.
• Two drafters, two targets. Drafters are trained per target model. If you serve something other than Qwen3.8-27B or Muse Glimmer 30B, you are waiting or training your own.
• Licenses differ. The weights are Apache 2.0, the implementation is MIT. Check which one applies to what you are shipping.
• Losslessness is the real claim. Greedy output matches the target exactly and sampling preserves its distribution. Verify that on your own eval before you treat it as free, because it is the entire reason to prefer this over a quantized model.

Key Takeaways

• DFlash 2 shipped August 18 from Inco AI with two Apache 2.0 drafters on Hugging Face: 2B for Qwen3.8-27B, 3B for Muse Glimmer 30B. Code is MIT at z-lab/dflash.
• Two additions do the work: a 2.0M-parameter path selector that traces a coherent route through the top-16 candidates at each position, and a 16.5M-parameter two-tap convolution that stops the block decaying at the tail.
• Combined overhead is about 1.3% of cycle latency for over 20% more accepted output per verification pass.
• Mean acceptance length hits 4.80 on Qwen3.8-27B (vs 4.28 MTP, 3.62 DSpark) and 5.70 on Muse Glimmer (vs 4.44 DFlash, 4.48 DSpark).
• Single-stream throughput on one H200 runs 2.67x to 3.43x; at 32 concurrent streams DFlash 2 stays at or above break-even while MTP and DSpark fall to 0.77x and 0.74x on MT-Bench.
• Day-one support across SGLang, vLLM, llama.cpp, Ollama, LM Studio and MLX, with output provably identical to the target model.

Sources: Inco AI, "DFlash 2: Keep Drafting Parallel", Qwen3.8-27B-DFlash2 model card, Muse-Glimmer-30B-DFlash2 model card, z-lab/dflash on GitHub, LMSYS, "The Next Generation of Speculative Decoding: DFlash and Spec V2"