DeepSeek V4.1 Flash on Two DGX Sparks: 2.9-Bit, 1.9× Faster Than V4 Flash, and What the Bits Cost

In September we sized DeepSeek-V4.1-Flash against our two GB10 boxes and said no: 475 GiB of weights against 242 GB of memory, and the only way in, a re-quantisation to two or three bits, would "give back concurrency and the KV pool, move us to a forked vLLM, and decode no faster than today, with no published quality table". Nine days ago we repeated the verdict for the 1017 retrain: two Sparks, V4-Flash only.

We were wrong on two of those four counts, and this article is the measurement that shows it. On 25 October we ran V4.1-Flash-1017 at 2.9 bits per weight on the same two DGX Sparks, tensor parallel over both RoCE rails, with a native serving engine instead of vLLM. It decodes 1.5 to 1.9× faster per token than the V4-Flash that has served us since August. It ties V4-Flash on every quality harness we own, beats it on hard code, and finds a needle at 976,000 tokens. The two counts we had right are the costs: it gives back most of the memory headroom and the concurrency, and it draws about 1.6× DS4's typical busy power under sustained load. It also reads documents, which DS4 cannot.

Test rig. Two GB10 workstations, one 200 Gb cable, both RoCE rails. "DS4" is DeepSeek-V4-Flash-0731 in NVIDIA's NVFP4 DSpark build on vLLM, tensor parallel 2, the pair's engine since August; its quality rows are the August and September runs. "The lane" is DeepSeek-V4.1-Flash-1017 in LibertAI's EXL3 pack at 2.9 bits per weight, served by TensorFold tensor parallel 2 following tonyd2wild's two-Spark recipe of 18 October, with the 196B Engram lookup tables read from each node's NVMe; measured 25 October with the same harnesses, prompts and haystacks. "The API" is DeepSeek's own deepseek-flash endpoint, which serves the same 1017 weights unquantised: the control that separates "the newer model" from "the 2.9-bit quantisation". Every number is ours.

What runs on each box

The pack was not made for this engine; LibertAI built it for a vLLM-based two-Spark kit, and its card reports perplexity 1.8% worse than the unquantised model on code and 0.4% on prose. The fit arithmetic from September still holds, which is why the experts had to go to 3 bits: at their native 4 bits they alone exceed the pair.

Speed: the bigger model is the faster one

One stream, nothing else running, through the same chat front end both engines serve:

With four prose streams in flight the lane aggregates 106 tok/s at 29 per stream; the recipe's own four-stream figure on a mixed set with code in it is 136. DS4 has served six streams at about 148 aggregate on structured prompts, so at full concurrency the two are closer than the single-stream rows suggest; per user, the lane is faster at every count it allows.

The same V4.1 under vLLM on Sparks, in tonyd2wild's measurements, decodes at 27 prose and 40 code. The lane doubles that on the same boxes. None of this is the model being smaller, because it is not. It is the serving path.

Why

Decode on a GB10 is bound by memory bandwidth and by how many scheduler steps a second the engine completes; the tokens per step come from the DSpark drafter, which both models have. The chat's own statistics give both numbers for each run:

Acceptance is the same or a little lower. The whole gain is in the step, which the native engine finishes in 31 to 43 ms where vLLM takes about 70. Four reasons, in order of weight:

1. CUDA graphs. On GB10, vLLM must run eager, because the NVFP4 mixture-of-experts kernels crash on this chip with graphs enabled, a fix we recorded in August. Every DS4 step is launched kernel by kernel from Python. The lane replays one captured graph per step.
2. A native engine. Zig and CUDA, no Python in the decode loop, the all-reduce over RDMA on both rails, fused kernels compiled ahead of time for exactly these bit widths.
3. Fewer bytes per step at 2.9 bits, despite more active parameters: about 6 GB against 7, so the bandwidth-bound part is a little shorter, not longer.
4. A quarter of the KV per token, 890 bytes against 3,514, so attention reads less per step, and V4.1's sparse indexer reads only selected blocks at long context. That is also why prefill at 538K is 1.6× faster and why the 976K prompt still runs at 1,620 tok/s.

This is the correction to our September sentence. We assumed a re-quantised V4.1 would inherit vLLM's step time. The step time was vLLM's, not the model's.

Quality: three columns, with the unquantised model as the control

Three readings. The newer model's gain over DS4 is real in one place, code: unquantised, V4.1 solves 23 more LiveCodeBench problems, 17 of them hard. Everywhere else the two models tie, and V4.1 writes shorter prose than asked, at the API as much as on the lane, so that is the model and not the bits. And the quantisation shows in exactly one row, hard code: the lane equals the API on deep knowledge, prose length, GSM8K, recall, traps and every needle, and is 10 hard problems behind it with thinking off.

Reading past the counts: the lane's deep-knowledge misses are the same facts the unquantised model misses, Milanković's insolation cycles, Enigma's daily keys, H.265's tiles, not new errors. The agent loop is tighter by one turn per repair and a third fewer tokens, which is where DeepSeek says the 1017 retrain's gains are, and that arrives on the pair intact. And the invoice row is new for the pair: DS4 cannot read an image at all, so until now a document went to the API or to a Qwen3-VL 32B on the desktop, which scored 88.4% on the same fixture in 92 s. The lane reads it in 6.8 s with every critical field exact, and nothing leaves the network.

What the 2.9 bits cost, taken apart

On the 80 hard problems: DS4 20, the lane 27, the API 37. The newer model is worth 17 hard problems over DS4 at full precision; the lane delivers 7 of them. Where did the other 10 go?

• Noise. At temperature 0.2 the lane and the API disagreed on 34 problems. Rerun on those 34, the API went from 24 to 21 and flipped on 17 of them; the lane went from 10 to 14 and flipped on 22. Averaging both runs gives lane 106 against API 116.5 on the full set, a six-point gap with a standard error near 3.6 per run. One run on 80 hard problems carries a standard error of about four, so the lane's +7 over DS4 is at the edge of what a single run proves; the two-run estimate on the whole set points the same way.
• Budget, not loops. The lane hit the 6,000-token cap on 38 problems, the API on 27. Those answers are not degenerate; they are the model working the problem out in prose before writing code, which V4.1 does with thinking off and DS4 does not, median 1,200 output tokens against 356. Rerun at a 16,000-token budget, the lane solved 6 of its 38; the same 38 on the API at 6,000 tokens gave 7. These are problems both endpoints mostly fail, and given the room, the lane reaches the API on them. That is worth five to six problems of the gap.
• One missing kernel. One problem fails in 0.13 s on every no-thinking run with a kit error: the ahead-of-time kernel set has no recorded variant for that prompt shape, and the engine errors instead of falling back. It is counted as a fail in every lane number above. With thinking on, a different prompt shape, the same problem passed, which confirms the fault is the kernel table and not the model.
• The rest, seven problems or so, is spread over the other 137 problems, inside one run's noise. That is the most the bits themselves can be shown to cost here.

With thinking on the picture is the same but closer: lane 119, API 127, a gap of 8 instead of 14. Per problem the two agree on 161 of 175; the API alone passes 11, the lane alone 3. Against DS4, the lane with thinking on solves 24 more problems, 13 of them hard, on the same two boxes.

What thinking costs on the pair. The decode rate does not change; the answer gets longer, and the time follows the tokens exactly: median output per problem 1,223 tokens against 3,943, wall for 175 problems on four streams 2,623 s against 8,415 s, both 3.2×. A coding question that answered in a quarter of a minute takes a minute, and scores better. On short Serbian questions the reasoning ran to a few hundred tokens and the cost is small.

The needle grid, to a million

Same filler as the pair's August grid and the API's September one, byte for byte: one needle at three depths, then two needles 90% apart whose sum is the answer, then one needle at a million.

Twelve of twelve single needles to 807K, the needle at a million found, and the two-needle sums 4/4 to 807K where the pair had been run to 540K. Recall precision is unchanged by the 2.9-bit backbone and by the sparse indexer at every length tested, up to the full window. DS4 ran the 538K haystack in 460 s and the 269K in 179 s; the lane did them in 289 and 130.

What it costs

These are the whole case against, and the tables above are the case for.

1. Memory headroom of 6 to 8 GiB per node. The model takes 105 of the 121 GiB. Nothing else fits beside it, and the page cache for the Engram rows is small, so those reads go to NVMe every step, 2 to 3 ms each.
2. Fewer people in the window. Four streams against six, and a pool of 1,048,576 positions shared by all of them against 2,873,231. Every request on either engine is capped at the model's 1M; the pool is the total in flight. DS4 holds two full-window requests at once with room to spare; the lane holds one, or four averaging 260K. A fifth request waits. For a household chat that is no constraint; several concurrent agents each resending 140K would feel it as queueing.
3. No scheduler priority. vLLM takes a per-request priority and the chat used it to put interactive turns ahead of long document reads. TensorFold serves in arrival order.
4. No engine metrics. No Prometheus endpoint, so the decode, KV and acceptance charts stay dark while it runs.
5. Shorter prose than asked, a property of V4.1 itself: the business letter came in at 47% of the asked length, the argument essay at 50%. A length that matters needs saying twice.
6. More power under sustained load. 165 W for the pair at four busy decoding streams, 82 W per node at 95% GPU and 84 to 85 °C. DS4 over two months of snapshots draws about 100 W whenever anything is running, with peaks of 165 to 170 W during long prefills. So the lane runs at DS4's peak continuously under full load, about 1.6× DS4's typical busy draw, because the native engine keeps both GPUs busy where vLLM's eager loop left them waiting. Per token of decode it is less efficient, 0.64 tokens per joule on prose against DS4's 1.08 to 1.67; per answer it is faster. Idle is the same, 22.4 W against 23.1.
7. A younger stack. A community engine at a pinned commit with three patches, measured in hours of uptime against DS4's seventy-one days. Rollback is DS4's own start script, two minutes.

The lever on the first two, if it ever matters, is bits: the recipe's newer version serves a 2.5 bpw pack with 2,097,152 positions and eight streams on the same pair, because smaller weights give the pool its room back. Whether 2.5 bits still passes the harnesses is a test like this one, not a given.

The verdict

• V4.1-Flash fits two DGX Sparks, at 2.9 bits per weight with a native engine, and it is the faster model on them: 1.6 to 1.9× per token, 1.4 to 1.6× on prefill. We said otherwise in September and October; the fit was right, the speed was not.
• The bits cost hard code and nothing else we can measure. The lane keeps 7 of the 17 hard problems V4.1 gains over DS4 with thinking off, 13 of 20 with thinking on, and ties the unquantised model on every other row, the invoice included.
• The price is headroom: four streams and one million positions in flight instead of six and 2.87 million, and about 1.6× the power when busy. One user with a million-token window loses nothing; a busy multi-user box loses a third of its seats.
• If you run one or two people on a pair of Sparks, this is now the engine to run. If you run six, DS4 on vLLM still holds more of them, and the day vLLM gets CUDA graphs on GB10 and a sub-3-bit expert format, that arithmetic changes again.

Related reading

• Should You Upgrade to DeepSeek V4.1 Flash? We Tested It on Two DGX Sparks — the September verdict this article corrects
• DeepSeek V4.1 Flash 1017 vs V4 Flash vs Muse Spark Open: Which Fits Two or Four DGX Sparks? — the fit ladder, now with a two-Spark V4.1 row
• DeepSeek-V4-Flash on Two GB10s: 304B Params, 1M Context, 83 Watts — how DS4 runs
• Best Model for a Dual DGX Spark: DeepSeek-V4-Flash vs Qwen3.8-Flash-Next vs GLM-5.3-Flash — the earlier LiveCodeBench rows
• 4-Bit Quantization Decoded: INT4 QAT, MXFP4, and NVFP4 — what 2.9 bits per weight means
• Prompt Processing vs Token Generation: the Two Speeds of an LLM — why steps per second decide decode