Should You Upgrade to DeepSeek V4.1 Flash? We Tested It on Two DGX Sparks

Three weeks ago we wrote that running DeepSeek-V4.1-Flash on the 128 GB GB10 class was the next measurement worth taking. We have now taken it, from both sides: what it would cost to put V4.1 on our two DGX Sparks, and what V4.1 actually does better than the V4-Flash they run today, measured on our own content with the API on the same afternoon. The answer to the title is no. The reason is not the one we expected, and the way we found it is the most useful thing in this article.

Test rig. The "pair" is DeepSeek-V4-Flash-0731 with its DSpark drafter on two GB10 workstations, tensor parallel over one 200 Gb cable, as described in our first article on it. V4.1-Flash ran on DeepSeek's own API, key opened on 25 September 2026, effort high unless stated. Every number below is ours unless it sits in the section marked as other people's.

Eight tests, and a tie

We wrote eight harnesses, most of them in Serbian because that is what our readers write in and nobody else tests it. Each was built to find a difference. Here is what they found.

The coding test saturated: it separates a broken model from a working one and nothing finer. The repair test did not reproduce the twenty-point DeepSWE gap DeepSeek reports, because those tasks finish in five turns and a model that holds a plan for fifty has nowhere to show it. The recall test never reached the long tail, because a question setter has to be certain of the answer, and certain answers are the well-known ones. The false-premise test is the reassuring one: handed a novel nobody wrote, an EU membership Serbia does not have and an element that does not exist, neither model invented anything, in plain Serbian, five times out of five.

Two differences did show. With thinking on, at twelve times the cost, V4.1 writes tighter Serbian prose and names things: on a lease-versus-leasing question it cited the actual statutes, where the pair offered a symbolic one-euro buyout that belongs to equipment leasing. On the three explanations, the checklist gave the pair the edge and reading the answers reversed it: V4.1 gave Milanković's birthplace, his Vienna doctorate and both key works under their original German titles, and named Merge mode, AMVP and the interpolation filter taps for H.265, where the pair explained the mechanisms better and named none of them. Named specifics rather than better argument is exactly what a 196B lookup table added to a model should look like, and it was the only trace of Engram anywhere in the day.

Eight harnesses, within a point or two on almost everything. We were ready to write "modest step up, mostly on knowledge, not worth the hardware".

Then we used what they built

The harness we use most is not a harness. It is Crush, the coding agent this project runs on, given a real task with no single right answer. So we gave both models four: a Flappy Bird, a bouncing-ball animation, a to-do list with persistence, and a unit converter. Each ran in a fresh sandbox. Each artefact was checked mechanically, syntax under node, a feature checklist, a "finish" checklist for the things a careful implementation does and a rushed one skips. Then a person opened all eight.

The game. Both files parse, both have every mechanic, both scored 10/10 on the feature list. The pair's version advances the physics once per drawn frame, so its speed is whatever the display's refresh rate happens to be; it is silently tuned for 60 Hz. The desk it was opened on runs a 240 Hz monitor, and there the bird fell four times too fast and the game was over in a second. V4.1's version advances the physics by elapsed time, which nobody asked for, and plays the same on any screen.

You can try both, unmodified: the pair's game and V4.1's game. What you see depends on your monitor. On a 60 Hz screen, or in a browser that caps animation frames, the pair's game plays normally, which is exactly the point: the same file is fine on one desk and unusable on the next, and no check that does not involve a person and a fast display would ever tell you.

The converter. This one scored 6/6 on features and 4/4 on the finish list that caught the game, and it is broken on any screen. One value is handed between two functions in the wrong form, so temperature converts and length and weight throw an error on every keystroke. Every ingredient the prompt asked for is in the file; the wiring between two of them is wrong, and no parser or checklist can see an error that only fires when the thing is used.

The one the pair won. On the animation, built to see whether the refresh-rate bug would recur, the pair used the more rigorous timing technique of the two. The pair can do this. On the game it did not.

That is the finding, and it is bigger than which model won. Both of the pair's failures are the same species: not missing knowledge, not a wrong formula, but integration. It knows the physics and the conversion factors and does not consistently wire them up. And six harnesses of careful, saturating, tie-producing measurement could not see either failure. One afternoon of opening files could. Seven times in this exercise a number disagreed with an artefact, five times because a scorer was too strict, twice because every scorer was too generous, and seven times the number was wrong.

Four artefacts is four. Two of four is not a failure rate; it is two failures, and two genuine ties. But it is enough to say that on this sample the cheaper model needed checking and the paid one did not, and that "scores the same" deserves less trust than something you have used.

The efficient model needs three times the memory

Before the quality question there was a fit question, and it has a counter-intuitive answer that the spec sheets bury.

V4.1 is the more efficient model per token of context. It is also three times bigger. Those are two budgets moved in opposite directions, and on a GB10 they land differently. The pair has about 242 GB usable across both boxes. V4-Flash takes 155 GiB and leaves 85 GB for context, which is why its KV pool holds 2.87 million tokens. V4.1's routed experts alone come to about 300 GiB at their native 4-bit precision, more than the whole pair before a single byte of cache.

The active-parameter figure is what misleads. Eight to sixteen billion active against thirteen sounds smaller. It describes compute, not memory: in a mixture of experts any expert may be needed for any token, so all 552B must be loaded. Fewer active parameters buys speed, never a smaller footprint. And on unified memory "just offload the cold experts" has nowhere to offload to that is not the same pool the KV cache lives in.

So the two-box options are all re-quantisations to 2 or 3 bits, which give back concurrency and the KV pool, move us to a forked vLLM, and decode no faster than today, with no published quality table. We declined that for V4 on Blackwell and the reasoning has not changed. The one native-precision run on a single GB10 streams experts from NVMe at 2.68 tokens a second. Four boxes hold the model natively, and that was the upgrade we expected to recommend.

Why four Nvidia GB10s do not help either

The most decision-relevant number of the day has nothing to do with quality.

A 538K-token question takes seven and a half minutes on the pair and eleven seconds on DeepSeek's API. Both answered every needle. That is not a model difference; it is two GB10s against a datacentre. It is also the difference between a long-document workflow that is interactive and one that is a batch job, and it is natural to read it as "buy more hardware".

The four-box recipe's own measured numbers say otherwise. The best published run of native V4.1 on four DGX Sparks prefills at 1,194 tokens a second on a 93K prompt. The pair prefills V4-Flash at 1,798 on 90K today. Four boxes would prefill V4.1 no faster than two boxes prefill V4 now, and slower at short lengths. Decode is the same order too: 74 tokens a second on code for the four-box V4.1 against the pair's 88 on structured output. The thing worth buying at length is not for sale in this form factor.

Then there is the price. The break-even case for the first pair was made against Claude Opus at $25 per million output tokens. Two more boxes, priced against DeepSeek's own V4.1 API at $1.20 per million output, with the pair's marginal cost near four cents, break even after about 6.9 billion generated tokens. At the four-box aggregate of 132 tokens a second that is 600 days of nonstop generation, and GB10 boxes are not getting cheaper. What four boxes would genuinely buy is vision that never leaves the LAN, the effort dial as a local control, and finish on artefacts nobody checks. That is a real list. It is not an $8,000 list, and the first two are available other ways.

Other people's numbers, for the record

Two independent measurements exist, both on DeepSeek's or a lab's serving, none on a GB10. Artificial Analysis scores V4.1-Flash 39 on its Intelligence Index against 34 for V4-Flash, rank 7 against 10 of 115, with both spending about 250 million output tokens to run it, so V4.1 is not cheaper in tokens. Arena's WebDev leaderboard puts V4.1 at 1620 against 1582. DeepSeek's own table shows the gap concentrated in agentic work: DeepSWE 54.4 to 74.2, Terminal-Bench 2.1 82.7 to 90.6, while GPQA moves one point, 89.9 to 90.9. A model that added a giant knowledge store and moved a graduate-reasoning benchmark by one point is either not helped by it or being measured by the wrong test. Our three explanation questions suggest the latter.

The verdict

Keep the pair. It is paid for, draws about 83 W, meters nothing, keeps every document on the LAN, and matched a 748B model on recall, honesty, retrieval at 807K tokens, invoice reading through a stand-in, and repair tasks.

Use the API for the two things it wins. Finish on generated artefacts, at a third of a cent a file, or cheaper still by opening the file or having a second model review it, which is what the failures above actually needed. And latency at length: under a dollar for a five-million-token needle sweep that would have been an hour on the pair.

Do not buy two more boxes for V4.1. The decision that would change that is not a better checkpoint. It is a serving stack that prefills the pair an order of magnitude faster, or a four-node recipe that reports tens of thousands of tokens a second rather than 1,194.

And treat a tie with suspicion. Eight harnesses said these two models are the same. The output said they are not.

Related reading

• DeepSeek-V4-Flash on Two GB10s: 304B Params, 1M Context, 83 Watts — the pair, and the measurements this article extends
• DeepSeek V4.1 Flash on Nvidia 5090? — the other end of the hardware question, and the memory arithmetic in full
• GB10, DGX Spark, HP ZGX Nano, ASUS Ascent GX10: Eight Names for One 128 GB Box — what a third and fourth box would be
• Best Model for a Dual DGX Spark — where V4-Flash sits against the other two-box options
• Mixture of Experts (MoE), Explained — why active parameters describe compute, not memory