Qwen3.8-27B Is 2.3× Faster With One Flag — Do You Need DFlash2?

A 27-billion-parameter model on one consumer graphics card, running at less than half the speed it was capable of, for weeks — because of a single sentence nobody checked. That sentence was ours, and we published it.

Qwen3.8-27B is a good local model. On an RTX 5090 it reads documents, writes code, and answers at about 66 tokens per second, which is roughly what a 17 GB quantised checkpoint should manage against the memory bandwidth of that card. That number sat unquestioned in our notes for weeks, next to a comment explaining why it could not be improved.

It could be improved. It is now between 147 and 159 tokens per second, and a five-image vision suite that took 92 seconds finishes in 42. Nothing was downloaded to achieve that. No second model sits in VRAM, no weights were re-quantised, and the fix is two command-line flags.

The speedup was inside the model file the whole time.

What follows is what those flags do, what they cost — and they do cost something, in a way that matters if you extract documents — and why the more famous alternative, DFlash2, is the wrong answer for this particular model despite being measurably faster.

Test rig. Every figure here comes from one RTX 5090 with 32,607 MiB of memory, running llama.cpp 153d324bc with Qwen3.8-27B in unsloth's UD-Q4_K_XL quantisation (17.9 GB) at a context of 65,536 tokens with vision loaded, over 21–22 August 2026. A handful of numbers come from a second, locally built llama.cpp; those are labelled, and the two are never compared against each other.

The flag that was already there

Modern large language models increasingly ship with a small extra head trained to guess the next few tokens ahead of the main network. The main model then verifies those guesses in a single batched pass, keeping the ones it would have produced anyway. When the guesses are good, you get several tokens for the price of one forward pass. When they are bad, you have lost almost nothing. (The mechanism, and why it wins at all, is laid out in our Muse Glimmer write-up.)

Qwen3.8-27B carries such a head. It is declared in the config as mtp_num_hidden_layers: 1, and it survives quantisation into the GGUF as fifteen tensors at block 64 — nextn.eh_proj, nextn.enorm and the rest. It is not an optional download. If you have the model, you have the speculator.

Turning it on takes this:

llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf -ngl 99 -c 65536 --jinja \
             --spec-type draft-mtp -fa on \
             --mmproj mmproj-F16.gguf --image-min-tokens 1024

--spec-type draft-mtp points llama.cpp at that head. There is no -md, no draft file, no second set of weights. -fa on enables flash attention, and it is in that command line for a reason worth being explicit about: every measurement below was taken with it, and MTP has never been benchmarked without it. Dropping it does not adjust these numbers, it invalidates them.

The last flag is unrelated to speed but matters for any comparison involving images. Qwen-VL models need a minimum of about 1,024 image tokens to behave properly on grounding tasks, and llama.cpp warns about it on load; without --image-min-tokens 1024 an OCR comparison measures the engine's image tokenisation rather than the model.

Three speculators, and where they live

It is worth being clear about the names, because they describe two genuinely different approaches and the difference is the whole argument of this article.

DFlash1 is the classic shape: a separate, small draft model that you download alongside the real one. Its trick is block diffusion — rather than proposing one token at a time, it emits a whole block per forward pass, with the main model's hidden states injected into its attention so its guesses stay anchored to what the big model is actually thinking.

DFlash2 is that same draft model with two components added — a path selector, which scores adjacent candidates pairwise, and a local convolution that models dependencies within a proposed block. Its authors report roughly 21 percent longer accepted runs as a result. It is still a separate file to fetch, and it still has to be version-matched to the model it drafts for. When llama.cpp says it expects 81 tensors and found 58, those 23 missing tensors are the selector and the convolution.

MTP — multi-token prediction — inverts the arrangement. Instead of a companion model, the drafter is a small head trained into the main checkpoint itself and shipped with it. There is nothing to download, nothing to version-match, and nothing extra to keep in sync when the model is updated. Qwen3.8-27B has one, whether or not you use it.

That last idea is not a Qwen peculiarity, and it is spreading. On the pair of DGX Spark (GB10) boxes we run DeepSeek-V4-Flash on, the model speculates using DSpark — again a draft module living inside the checkpoint, five tokens per block across three draft layers — and measures 77.2 percent acceptance, about 4.86 accepted tokens for every six proposed. Same principle as MTP, different vendor, and the same operational payoff: no second model to source, load, or version-match.

The rest of this article is, in effect, a contest between the two arrangements on one particular model.

How a prediction became a fact

The reason this took weeks to find is more interesting than the flag.

Our own notes on this model contained a confident claim that llama.cpp "ignores the MTP head — it is unimplemented", and the server launch script carried a header predicting decode would "sit at the bandwidth floor, around 69 tok/s". Both statements were wrong. Neither had ever been tested.

Worse, one of them shipped. Our comparison Qwen3.8-27B vs Muse Glimmer 30B told readers that llama.cpp "discards it at load" and framed the whole result around what would change "if llama.cpp ever implements the MTP head." It already had. That article's measurements are unaffected — they were taken without speculation, and they are what a non-speculating Qwen3.8 does — but its explanation of why Qwen3.8 was slower is now wrong, and the ~3× gap it reports closes to roughly parity once this flag is on.

The claim came from reading a loader message, drawing a reasonable inference, writing it down, and then quoting it forward for weeks as though it were a measurement. The flag existed in the binary the entire time. The head was in the file. They find each other with no help at all.

The cost of that unchecked inference was the whole gap between 66 and 150 tokens per second, on hardware that was sitting right there. A prediction that never gets checked does not stay a prediction. It becomes a fact by attrition.

What it buys, on text

Three prompts — code, free prose, and a word problem — at temperature 0:

Individual requests measured after a reboot the following day landed at 162 and 165 tokens per second, at or slightly above the top of that band.

Notice the spread. The same configuration produced 146.80 on one day and 158.69 on another, an eight percent difference, and the temptation is to blame something — a background process, thermal throttling, a flag that quietly differed. The last column rules that out. Those two runs returned the same three answers, byte for byte: identical SHA-256 prefixes on all three. Same binary, same flags, the same tokens emitted in the same order, eight percent apart in wall clock. Whatever moves it lives somewhere below the model, and it has not been identified.

Which is why the honest headline is a range rather than a number. 2.21× to 2.39×, of which 2.3× is the midpoint — not a reading. Quoting the top of that band because you happened to catch a fast session is how a benchmark stops being one.

The gain also depends heavily on what you ask for:

That ordering is exactly what the mechanism predicts. Predictable syntax is easy to guess ahead; free-form prose is not. A benchmark built on any single one of these prompts would have reported somewhere between 1.9× and 2.5× and been perfectly truthful each time.

What it buys, on vision

Qwen3.8-27B reads images, and this is where the wall-clock difference is most visible. Five images — a photograph, a scanned invoice, two dense screenshots and a text-heavy report — each processed twice per configuration:

The second pass ran 93.1 s against 38.5 s. Across both, the suite lands at 41 to 45 percent of its baseline wall clock — comfortably better than halved.

The number in that table you should not trust

The invoice's 3.29× is the most quotable figure in this article and the least honest one, and we include it having briefly quoted it ourselves.

The baseline emitted 1,868 tokens describing that invoice. MTP emitted 1,071. Wall clock fell by 3.29×, but the decode rate only rose by 1.89×. The remainder is not speed. It is MTP simply writing less.

The same effect runs through the whole suite, more mildly: 6,443 tokens against 5,935, so a 2.21× wall-clock figure sits on top of a 2.04× rate figure. Both are real. They answer different questions. Reporting only the first sells output length as though it were throughput.

Shorter is not automatically worse here, and checking what went missing is illuminating. The baseline emits a Branch field that both MTP runs omit, and MTP reads S/L No where the baseline reads S/ No. Which is correct is a judgement about pixels that no amount of JSON can settle — an earlier spot-check called both for MTP — but the shorter read is at least not shorter through dropping content. What can be said without qualification is that "3.3× faster at OCR" would be a claim about verbosity wearing a claim about speed.

The case that breaks other servers

There is a known failure mode in llama.cpp where speculative decoding and image input destroy each other. Issue #27408 reports that combining a draft model with an mmproj vision projector makes every image request stall and return HTTP 500: image chunks advance the main model's context without being mirrored into the draft's cache, and the draft then rejects the positions as non-consecutive.

A five-step sequence alternating images and text is the shape that triggers it:

Two runs, zero draft-cache errors in either log. MTP is not affected. Which brings us to the alternative.

So do we need DFlash2?

DFlash2 is the one people have actually heard of, and its authors publish claims of 2.7× to 3.4× on this exact model. It is the obvious thing to reach for, and we reached for it first.

For Qwen3.8-27B, the answer is no. That is worth stating plainly, because DFlash2 is genuinely the faster speculator and it loses anyway.

Measured on a single locally built llama.cpp — PR #27342, compiled for sm_120, because DFlash2 does not load on the release build at all:

DFlash2 is 10.6 percent faster than MTP on text. It also requires a separate 2.06 GB download and a locally compiled llama.cpp, because on the release build the draft refuses to load:

error loading model: done_getting_tensors: wrong number of tensors; expected 81, got 58

The twenty-three tensors it cannot place are the path selector and the local convolution — precisely the two components that make DFlash2 what it is.

Set the download and the custom build aside, though. Two measured problems settle this without them.

It needs 2.6 GiB more memory, on a card that is already nearly full

All three configurations at a 65,536-token context, on the same 32,607 MiB card:

DFlash2 costs 2,635 MiB more than MTP — the draft weights, plus the draft's own context — and leaves 1.1 GiB of headroom on a card that is also driving a desktop. That desktop's own memory use has been measured swinging by around 600 MiB when a video player opens. There is not much room left for a bad afternoon.

The comparison is, if anything, unfair in DFlash2's favour. MTP's 28,805 MiB is a steady-state figure, taken after an image had gone through and the roughly 2.2 GiB of vision buffers had actually been allocated. DFlash2's 31,440 MiB cannot be a steady-state figure, because DFlash2 never successfully processes an image — those buffers are never allocated at all. The true like-for-like gap is wider, by an amount that configuration cannot be made to reveal.

The ceiling also caps the draft block at 4. Requesting 8 dies on a 535 MiB allocation at context sizes of 65,536, 32,768 and 16,384, with and without the vision projector. Block 8 becomes reachable only by dropping the context to 8,192 — where it turns out to be no faster anyway, 2.43× either way with identical output. The cap costs nothing in speed. It is just a reading on how little room is left.

It breaks vision completely

Not degrades. Not intermittently. Every image request fails:

E process: llama_decode(ctx_dft) failed rc=-1 (n_tokens=512, offset=512)
E srv    send_error: failed to process mtmd chunk        →  HTTP 500

This is #27408 reproducing on this hardware, though it fails in about a second here rather than the roughly 500-second stall the issue describes. There is nothing to tune. It survives removing --image-min-tokens 1024, so that flag is not the cause.

The shape of the failure is the worst possible one. Text continues to serve at 2.43× while every image returns a 500, so the server looks perfectly healthy until someone sends a picture. MTP has no such problem: the same invoice reads correctly in 8.0 seconds, and the interleaved sequence passes twice.

Ten percent of text throughput is not worth a blind model, and this is a vision model. 2.20× that can see beats 2.43× that cannot. DFlash2 would only become the right answer for a text-only deployment of Qwen3.8 with nothing else on the card — and even there it is stuck at block 4.

A coincidence that will mislead you

MTP's fastest recorded session reached 158.69 tokens per second. DFlash2 measured 158.92. Those are 0.14 percent apart, and it means nothing whatsoever: they come from different builds of llama.cpp. Comparing them would be comparing two versions of an inference engine and calling the difference a drafting result.

On the one build where both were measured together, DFlash2 leads 158.92 to 143.66. Every configuration has to be re-measured on whichever binary you are reporting, every time.

None of this is a verdict on DFlash2 in general, either. On Muse Glimmer 30B the same draft generation speculates and reads images simultaneously, on both builds, with much larger gains than anything here. The breakage is specific to Qwen3.8, not a property of DFlash2 and not of speculative decoding.

For this model on this card, though, the decision is settled: the free speculator that can see beats the paid speculator that cannot.

Does quality survive?

Every graded suite already recorded for this model, re-run with MTP on:

Twenty-four graded items, no regressions.

What cannot be claimed is bit-identity. Speculative decoding is often described as provably lossless — the main model verifies every guess, so the output should be exactly what it would have produced alone. That is not what happened. MTP's text differs from non-speculative text on all three prompts.

The oddity is that both configurations are individually deterministic. The baseline reproduces byte-for-byte across a full server restart; MTP reproduces byte-for-byte across a full server restart. They simply disagree with each other. The likeliest explanation is that verifying several tokens in one batched pass uses different floating-point reduction orders than decoding one at a time, flipping an argmax at a near-tie, after which the two continuations diverge for good. That is a hypothesis. Confirming it needs logit-level comparison that has not been done.

Different text, same quality is supported by the evidence. Same text is not.

The cost nobody advertises: vision stops being reproducible

On text, MTP is deterministic. On images, it is not — and this, rather than memory, is the finding that should decide whether you turn it on.

Same server, same five images, two runs each:

The baseline is byte-identical on all five images across a full restart. MTP differs from itself on four of the five, and the magnitude is strikingly bimodal: the report moves by two characters, while three images move by twelve to sixteen percent.

Two things make that less alarming than sixteen percent sounds. The divergence is table structure rather than different readings — the same figures grouped into cells differently. And where MTP differs from the baseline on the invoice it is not obviously the worse read: it omits a field the baseline invents, and declines to guess a name off an illegible stamp.

One thing makes it worse, and it is decisive. OCR that changes between runs is not a reading; it is a sample. A pipeline that extracts the same invoice twice and gets two different table shapes cannot be diffed, cached, or audited — regardless of which shape is better.

There is one clean exception in that table, and it is the least comfortable result of the lot. The benchmark table is byte-identical under MTP across both runs and identical to the baseline. Determinism is not uniformly lost here; it is lost depending on the content. That is harder to plan around than losing it outright, because nothing about the input tells you which case you are in.

The practical rule that falls out: run without speculation when an extraction has to be reproducible, and with it when a human reads the output once.

What memory it actually costs

Both configurations, same card, same session, measured after an image had been processed so the vision buffers genuinely existed:

The draft context costs 2.4 GiB. An earlier version of our own memory budget put that at 3.7 GiB, by comparing a whole-card total against a per-process figure — over-stating the cost by 1.3 GB, and nearly refusing to start the server on a slightly busy desktop for no reason at all.

Two sizing rules that this model punishes you for ignoring. First, the vision buffers do not exist when the server loads. They appear when the first image arrives, and they cost about 2.2 GiB. Size from the startup footprint and 128K context looks like it fits; size from the steady state and 65,536 is the ceiling on a 32 GB card. Second, the KV cache runs at 65.0 KiB per token — far less than the layer count suggests, because only 16 of the 64 layers use full attention and the other 48 are Gated DeltaNet holding a fixed state of around 160 MB. (Why that per-token figure is the number that decides your context limit is the subject of our KV cache explainer.)

With vision and MTP at a 64K context, the card sits at roughly 27.1 of its 32.6 GB once the desktop is counted. It fits. It is not roomy.

How to tell whether it is actually on

Four checks, cheapest first, because a speculator that silently fails to engage looks exactly like one that was never configured.

The server log is the fastest test. llama.cpp prints a draft-acceptance line for every request, but only when a drafter is wired up:

eval time = 4240.25 ms / 699 tokens ( 6.07 ms per token, 164.85 tokens per second)
draft acceptance = 0.70536 ( 474 accepted / 672 generated), mean len = 3.12

A log from a non-speculative server contains no such line at all, so its mere presence is the wiring test. The mean len figure is the mechanism made visible: about three tokens per verification step instead of one. A line reading mean len = 1.0 would mean the drafter is running and earning nothing.

The rate alone is diagnostic. Decode here is bimodal with no overlap — 66 to 75 tokens per second means no drafter, 147 to 165 means MTP. Compare like with like, though: the baseline itself moves with content, and the non-speculative invoice ran at 71, not 66.

Cumulative counters are available at the /metrics endpoint, if the server was started with --metrics:

llamacpp:spec_decode_num_draft_tokens_total      1428
llamacpp:spec_decode_num_accepted_tokens_total    958
llamacpp:spec_decode_num_drafts_total             476

The real test is an A/B at temperature 0 with the outputs hashed, which measures the speedup and checks for losslessness in one pass.

The measurement lied for weeks, in two ways at once

That third check deserves a warning, because it was quietly broken here and the way it broke is instructive.

/metrics is disabled by default; without the flag it returns 501 This server does not support metrics endpoint. Separately, our benchmark harness was scraping counter names that do not exist in any build of llama.cpp — llamacpp:n_drafted, where the engine actually exports spec_decode_num_draft_tokens_total. Those names had been guessed and never checked against a live response.

Either fault alone produces the identical result: a null acceptance figure. And the scraper reports a missing counter as null by design — deliberately, so that a zero can never be misread as "nothing was accepted" when the truth is "nothing was drafted". That careful decision is exactly what hid the bug. A wrong counter name is indistinguishable from a server that is not speculating. Every result file written before the fix carries a null acceptance, including runs that were demonstrably drafting at between 0.57 and 0.88 according to the server's own log.

With both faults fixed, the two independent sources agree exactly on a single request:

log       draft acceptance = 0.67087 (958 accepted / 1428 generated), mean len = 3.01
/metrics  958/1428 = 0.67087             958/476 + 1 = 3.013

The +1 is llama.cpp's convention rather than padding: each verification step also emits one token from the main model that was never drafted and cannot be rejected.

What this does not tell you

Five things this measurement cannot support, stated plainly because the gaps matter as much as the results.

Acceptance was never recorded during the headline runs. The 2.21× to 2.39× figures are derived from wall clock; the acceptance rates quoted here come from later individual requests. The counters only began working after the fix above, so a fresh pass would be the first to carry both in one file.

Flash attention is unbenchmarked in isolation. Every number assumes -fa on.

The eight percent session-to-session spread is unexplained. Identical output, identical flags, eight percent apart.

Nothing here predicts behaviour under load. Every measurement is single-stream, one request at a time. Speculative decoding typically loses its advantage under concurrency, because drafting competes with real work for the same compute.

The vision divergence has a characterisation but no root cause. Why the benchmark table stays perfectly deterministic while the cityscape does not is unknown.

Reproducing it

Two servers, identical but for one flag:

# baseline — no speculation
llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf -ngl 99 -c 65536 --jinja --metrics \
             --mmproj mmproj-F16.gguf --image-min-tokens 1024

# the checkpoint's own MTP head
llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf -ngl 99 -c 65536 --jinja --metrics \
             --spec-type draft-mtp -fa on \
             --mmproj mmproj-F16.gguf --image-min-tokens 1024

Send the same prompts to each at temperature 0. Three things are worth capturing per request, and the third is the one usually skipped:

• Rate, as usage.completion_tokens divided by wall clock, with "stream": false. Streaming rates measure your client as much as the server.
• A hash of the answer text, which is what turns a speed test into a losslessness test. At temperature 0 the two configurations should agree, and here they do not.
• Draft acceptance, from the server log or the /metrics counters — the number that explains why a speedup is or is not there, rather than merely that it is.

Use prompts that span content types. Acceptance is strongly content-driven, and a single-prompt benchmark on this model can honestly report anything between 1.9× and 2.5× depending on which prompt you picked.

Run each configuration twice. On this model a single run cannot distinguish "the two configurations differ" from "this configuration differs from itself" — and on vision, that distinction turned out to be the entire story.

Related reading

• Qwen3.8-27B vs Muse Glimmer 30B: Same 10/10, One Is 10× Faster — the comparison this flag rewrites, and the article that carried the wrong claim
• Muse Glimmer 30B: 4× Faster Than a 304B Model, and 22% Slower — By Default — the same drafter family, on a model where it does work with vision
• DeepSeek-V4-Flash on Two GB10s: 304B Params, 1M Context, 83 Watts — DSpark, the in-checkpoint drafter that works the same way
• The KV Cache: the trick that makes LLMs fast — and slow — where the 65.0 KiB per token goes
• Prompt Processing vs Token Generation: the Two Speeds of an LLM — why decode is the half a drafter can help