The question in the title has a short answer and a long one. The short answer is yes: DeepSeek-V4.1-Flash — 552 billion parameters in the backbone, another 196 billion in a lookup memory, a 502 GB file on disk — generates text on a single RTX 5090 with 32 GB of VRAM. It does so at 5.1 tokens per second on content it has not seen before, and 21 tokens per second on content it has. Nothing about that first number is going to replace an API subscription.
The long answer is the interesting one, and it is not ours. The port was built and measured by JigSawPT and published on 11 September 2026 as a technical report with a DOI, an MIT-licensed llama.cpp fork, a set of measurement tools, and the raw results. We have read all of it, and it is the most carefully measured "will it fit" report we have seen for a model this size on a consumer card. Its value is not the headline. Its value is a list of things that do not help — more VRAM, more RAM, a faster SSD, a smarter cache, the model's own speculative decoder — each with a number attached, plus one genuine bug that produced fluent text for weeks and was caught by a single suspicious constant.
Whose numbers these are. Every figure in this article comes from JigSawPT's report and its llama.cpp discussion thread, measured on one RTX 5090 (31.8 GiB of VRAM) with 125.7 GiB of system RAM and a PCIe 5 NVMe drive, over 10–11 September 2026. We have not reproduced them on our own hardware. Where we compare against numbers we did measure — DeepSeek-V4-Flash on two GB10s — we say so.
What V4.1 Flash is, in bytes
DeepSeek-V4.1-Flash is the successor to the V4-Flash model we ran on two GB10 workstations in August. On paper the two are close relatives — a sparse mixture of experts with a million-token context and a built-in speculative decoder. In bytes they are not close at all.
| V4-Flash-0731 | V4.1-Flash | |
|---|---|---|
| Backbone parameters | 304B | 552B |
| Conditional memory ("Engram") | — | 196B, read by lookup |
| Active per token | ~13B | 8B prefill / 16B decode |
| Layers | 43 | 40 (20 encoder + 20 decoder) |
| Routed experts per layer | 256, 6 active | 384, 6 active, plus 1 shared |
| KV cache per token | ~3.5 KB | 890 bytes (FP4) |
| Context | 1M | 1M |
| On disk | 155 GiB (vendor FP4/FP8) | 502 GB GGUF, 11 shards |
| Licence | MIT | MIT |
The vendor's benchmark claims — GPQA Diamond 90.9%, Terminal-Bench 2.1 at 90.6%, DeepSWE 74.2% — are DeepSeek's own and we have not verified them; they are the reason anyone wants to run this model at home, not the subject of this article.
The line that matters for the hardware question is the second one. Engram is new in V4.1: two hash tables of 384 million rows each, indexed by n-grams of the token ids rather than by anything the network computes. Every token reads 48 rows. It is memory addressed by text, and it does not compress — in the GGUF it is 189 GiB of raw fp8 bytes, two shards of 98 GB apiece. A 299 GB variant without the tables exists, and the report is blunt about it: its outputs do not match the reference. The tables are not optional.
So the accounting for one RTX 5090 is: 269 GiB of routed experts, 189 GiB of engram tables, a few GiB of everything else, against 31.8 GiB of VRAM and 125.7 GiB of RAM. The model is roughly four times the size of the machine.
How 502 GB runs on a 32 GB card
Three tiers, and one property of the model that makes tiers work at all.
The property is sparsity. Each of the 40 layers picks 6 of its 384 experts per token, so the bytes a token actually touches are a small slice of the 269 GiB. The fork's --moe-stream path keeps a cache of expert weights in VRAM (18 GiB in the recommended configuration), a second, larger tier in pinned system RAM (72 GiB), and reads everything else from the NVMe as the router asks for it. Once per layer per token, a remap call pulls that layer's six chosen experts into the VRAM cache. Where those six come from — card, RAM, or disk — decides how long the token takes.
The engram tables are never loaded. They are memory-mapped from disk and read by the CPU, 48 rows per token; because the row addresses depend only on token ids, they are known the instant a token exists, and the port prefetches them through the operating system. That one change took the engram cost from 10.4 ms per token to 0.9 ms, an 11.4× improvement that is also the only place in the whole report where look-ahead pays without a predictor. We will come back to why.
The numbers, and the trap in the second row
Measured on a resident llama-server with four prompts of mixed content — a short question, a code task, Portuguese prose, a reasoning puzzle — three rounds each, decode rate taken between the first and last streamed token:
| decode | time to first token | |
|---|---|---|
| cold — content seen for the first time | 5.12 tok/s | 8.56 s |
| resident — the same prompt again | 21.27 tok/s | 0.27 s |
| DeepSeek's reference implementation, same card, with a cache written for it | 4.34–4.67 tok/s | 29 s |
The second row is real and it is a trap. With greedy decoding, repeating a prompt regenerates the same tokens, which route to the same experts, which are still in the cache from last time. Nothing was fetched, so the card ran at its compute ceiling: 21.27 tok/s is what this model does on a 5090 when every byte it needs is already there. It is also what a conventional benchmark would have reported. The usual practice — discard the first request as warm-up, keep the rest — was written for engines that start cold and stay warm. This engine does not warm up; the answer does. Publish the resident number and you have quoted four times what any conversation will see.
We made the mirror-image mistake on Qwen3.8-27B — a number carried forward for weeks without being checked — so this is not a criticism of anyone. It is a warning that on models whose residency depends on content, "warmed engine" and "seen this prompt before" are the same thing.
Where the token goes
The report's central measurement is a decomposition of a remap call at the 72 GiB host tier: 20% compute, 26% PCIe transfer, 54% NVMe. Half of every token is spent waiting for the disk, and a quarter waiting for the bus.
From that decomposition, four ceilings:
| if you could… | tok/s |
|---|---|
| do nothing (today, 72 GiB host tier) | 4.3–5.1 |
| know the next 5 tokens' routing in advance (prefetch oracle, measured) | 5.6 |
| eliminate disk misses entirely | 6.2 |
| hold the whole working set in VRAM | 21.3 — unreachable, 105 GiB against 31.8 |
The gap between 6.2 and 21.3 is PCIe, and PCIe is only removed by already having the expert on the card. The gap between 5.1 and 6.2 is the disk, and the disk turns out to be the easier of the two to misread.
The author's own earlier estimate was "the disk is saturated; the way forward is a faster NVMe." Then the disk was measured with the real access pattern: it delivers 10.04 GB/s at the queue depth decode produces, and the port draws 4.33 — 43% of what is available. The shortfall is not bandwidth. It is that only 3.78 disk requests exist in flight per layer, because a layer's requests cannot leave until that layer's router has run, and the queue fills and drains 40 times per token. No thread pool can invent requests the graph has not issued yet.
What does not help
This is the part of the report we would hand to anyone about to spend money on the problem. Every row is a measurement.
| lever | result | why |
|---|---|---|
| host tier above 72 GiB | worse — 88 GiB gives 3.9–4.1 tok/s | it steals page cache from the engram tables |
| VRAM cache 13 → 22 GiB | flat at 4.3 tok/s | hit rate rises 59 → 66%, but more VRAM removes PCIe misses, not disk misses |
| pinning the hottest experts in VRAM | null, three separate measurements | at 72 GiB everything hot already fits |
| a smarter eviction policy | 0% margin | LRU, LFU, random and the optimal Belady policy give identical misses |
| more RAM | does not help | the working set grows with the content: 26 new experts per token in the tail, 105 GiB by 128 tokens |
| 2-bit experts, skipping layers | unnecessary | bytes are not the bottleneck; declining them costs no speed |
| a faster NVMe | not expected to help | 43% of the current drive is used |
| a layer-level routing predictor | negative (−7 to −19%) | the speculative read is still in flight when the real demand arrives, and is paid twice |
The last row is the one that would have cost the most engineering time. A predictor that guesses the next layer's experts from the current hidden state has decent recall — 70.9% at top-6 — but the oracle sweep shows that the depth such a predictor can reach loses, and the gain only appears one full token or more ahead. Knowing the next token's routing means knowing the next token, which is speculative decoding by another name. That closes the loop on the engram prefetch above: it is the one lookup whose address is known before the network runs, so it is the one lookup that can be fetched early for free.
The defect that never raised an error
The correctness section is where this report earns its DOI.
The fork is built on deepseek4.cpp, the existing V4 implementation, and V4.1 differs from V4 in nine places that all failed silently: the fp8 scale block is 32×32 rather than 128×128, the compression ratios are per-layer rather than compile-time constants, four layers produce compressed caches that the others share, the indexer projects keys directly and skips the Hadamard rotation, and so on. Shapes matched, the model loaded, the text was fluent. All nine were found by comparing per-layer dumps against DeepSeek's Python reference, none by a crash.
Then, after everything ran, layer 0 was still receiving an exact input and returning a divergent output. Bisecting inside the layer:
| sub-step | correlation vs reference | norm, port | norm, reference |
|---|---|---|---|
qr_norm |
0.999912 | 40.015 | 40.034 |
q |
0.949370 | 181.019 | 375.363 |
181.019 is √(64 × 512) to the last digit. There is exactly one way a 64-head query vector of dimension 512 gets that norm: every head has been normalised to RMS 1. V4 normalises each attention head of q after the wq_b projection; V4.1 does not. The line came with the inheritance and nothing ever objected to it — the model still produced sensible text, because a per-head RMS norm on q changes attention sharpness, not attention direction. It just makes it a slightly different model.
| first-step logits | before | after |
|---|---|---|
| correlation vs reference | 0.761 | 0.927 |
| argmax | equal | equal |
Note that last row. The most likely token was the same before and after the fix. A test that checks "does it produce the expected next token" passes both. A test that checks "is the text fluent" passes both. What caught it was a norm that happened to be a recognisable number — the report's own lesson 37: a probe that produces an identifiable constant finds the defect by itself. The fix is gated on the architecture rather than removed, because V4 still runs on the same fork.
We would add one thing. Anyone porting a "point-one" release onto its predecessor's code should assume this class of bug exists until per-layer dumps say otherwise. Fluent output is not evidence of correctness in a 552B model; a 552B model is fluent through a remarkable amount of damage.
What remains is the reference rounding itself
After the fix, the port and the reference still disagree at the fourth decimal, and here the report does something most ports skip: it measures how much disagreement DeepSeek's own arithmetic produces. The reference runs its linear layers in fp8 with block-wise scales and rounds its activations; the port runs attention in Q8_0 and experts in MXFP4 blocks that are bit-for-bit the released weights (480 of 480 sampled blocks identical after a round trip). So the port is, if anything, the more precise of the two.
| what the reference rounds and the port does not | round-trip correlation | relative error |
|---|---|---|
| one fp8 linear layer | 0.999896 | 1.45% |
| windowed KV, fp8 block 32 | 0.999613 | 2.80% |
| compressed latent, fp4 block 16 | 0.994630 | 10.39% |
| indexer q and k, fp4 block 32 | 0.987746 | 15.64% |
The port's 0.999912 at the point where a single fp8 layer's floor is 0.999896 means it is at the floor. The remaining gap is DeepSeek's quantisation, not JigSawPT's implementation — and the report is careful to say this does not make the port better: the model was trained with those roundings, and a more precise port is a different model. Only a quality benchmark could settle that, and none has been run.
Two consequences follow, and both matter if you plan to A/B anything on this model.
The MoE gate is a switch, not an adder. At layer 2, the shared expert and the routed experts see the same input and run the same arithmetic. The shared expert loses 0.0022 of correlation to the rounding; the routed path loses 0.0304 — fourteen times more, because one of them makes a discrete choice. Over all 40 layers, 208 of 240 top-6 expert selections agree with the reference (86.7%), and the gap between the 6th and 7th routing score is about 1%, the size of an fp8 error. Token-for-token agreement over a long generation is therefore unattainable by architecture. At 1,401 tokens the port correlates 0.9967 with the reference and 0.9959 with a second run of itself: the distance to DeepSeek is the same as the distance to yesterday.
Above about 1,024 tokens, the port is not reproducible by default. The cause is pinned: expert-cache slot assignment depends on I/O timing, which changes the accumulation order inside the MoE matmul, and the gate amplifies it. --moe-stream-io-threads 1 makes two runs bit-identical, at a cost of 4.3 → 3.6 tok/s. If you are comparing outputs — ours, DeepSeek's, anyone's — that flag is not optional. We learned the same lesson the hard way on OCR with speculative decoding: a configuration that differs from itself cannot be diffed against anything.
The draft head that works and does not pay
V4.1 ships its own speculative decoder, DSpark: three full V4.1 blocks with 128 routed experts, a bigram head, a confidence head, trained to propose blocks of five tokens. JigSawPT exported it as an 8 GB GGUF and measured it against the same server with the draft off.
| draft length | content | acceptance | cold | resident |
|---|---|---|---|---|
| 2 | verbatim repetition | 97% | +12% | +15% |
| 2 | code with explanation | 82% | +1% | +12% |
| 2 | Portuguese prose | 51% | −14% | −13% |
| 5 (the trained block) | code with explanation | 49–70% | 0% | −8% |
| benchmark median, draft of 2 | mixed | −4% | 0% |
The head is fine. Acceptance tracks content exactly as it should — half on prose, nearly all on repetition — and greedy speculation reproduces the non-speculative text token for token on the verbatim case. It simply does not translate into speed on this machine, and the reason is the same disk.
A verification step processes K+1 tokens at once and must fetch the union of the experts those tokens route to. The report measured that union directly by feeding a known continuation in blocks — the best possible draft, 100% acceptance, no draft model at all — and got +17–20% at K=4 and a plateau after. At the trained block of five, a step that accepts 5.67 tokens costs about 5.3 single-token steps. At a draft of two, the accepted length barely covers the cost. Each prompt lands within ±14% of the control according to its own acceptance, and the median is a wash.
Compare the same mechanism on a machine where the working set is resident. On our two GB10s, V4-Flash's DSpark accepted 4.86 of 6 tokens per step and produced a 2.7× spread between structured output and prose — because there, verification is nearly free and every accepted token is pure gain. Same idea, opposite result, and the report's lesson 49 puts it well: the draft length is a property of the machine, not the model. The block of five is the optimum on 8×H800. On a disk-bound 5090 the optimum is two, and even two is neutral. Sweep it before you publish it, and publish the sweep.
An ordinary n-gram draft, for the record, is null in the best case and −33% on realistic content: it proposes a lot when it is wrong, and every wasted verification pays the expert union of the block. Measuring it did expose a real bug in the fork — rollback of the recurrent state was only enabled for model-based drafts — which is now fixed for every draft type.
So, is it usable?
At 5 tok/s with an 8.6-second first token: for interactive chat, no. For a coding agent, which reads far more than it writes, the prefill side is not reported and we would not guess. For a batch job that can run overnight on a machine you already own, with a 552B model whose outputs correlate 0.9967 with the vendor's, on a card that draws a fraction of a rack — that is a different question, and the report answers the part of it that can be answered on one card.
What it really is, in our view, is a map. The single most useful number in it is 105 GiB: the expert working set after 128 tokens of a realistic prompt. The port's own ceilings say that the whole distance from 5 to 21 tok/s is residency — not compute, not bytes, not the SSD. On paper that points at the 128 GB unified-memory class of machine, the GB10 boxes we have been measuring all summer, where the experts could sit in the same memory the GPU reads. We want to be precise about what we are and are not claiming: nobody has run V4.1 Flash on a GB10, the engram tables would still have to live on NVMe, and 105 GiB of experts plus everything else in 128 GB is tight enough that "on paper" is doing real work in that sentence. It is the next measurement we would take, not a result.
Against that, the 5090 has a ceiling of 21.3 tok/s at full residency, which is a compute number a GB10 would not match. The right hardware for this model at home is not obvious, and that is exactly why a report that measures the ceilings instead of the throughput is worth more than one that reports a single tok/s.
Reproducing it
The 502 GB GGUF — eleven shards, MXFP4 experts, engram tables included — requires the dsv41-porte branch of JigSawPT's llama.cpp fork — 31 commits on top of b10269. Upstream llama.cpp has a conversion PR for V4.1 but no runtime yet, and it stores the engram tables differently, so the files are not interchangeable. Ollama and LM Studio cannot load it.
llama-server -m DeepSeek-V4.1-Flash-MXFP4-engram-00001-of-00011.gguf \
-ngl 99 -c 8192 \
--moe-stream --moe-stream-cache 18 --moe-stream-l2 72 \
--reasoning off \
--host 127.0.0.1 --port 8080
# bit-identical runs for A/B tests, at 3.6 instead of 4.3 tok/s:
# --moe-stream-io-threads 1
# the model's own draft head, neutral on mixed content:
# -md DeepSeek-V4.1-Flash-DSpark.gguf --spec-type draft-dspark \
# --spec-draft-n-max 2 -ngld 99 --spec-draft-n-cpu-moe 3--reasoning off is there because thinking mode at temperature 0 loops on vague requests. The 502 GB download deduplicates well on the Hub — the author notes the lossless repack moved about 2 GB of genuinely new data — but it is still 502 GB on your drive, plus 8 GB for the draft head. The measurement tools (bench_server.py, stall_decomposition.py, cache_oracle.py and the rest) are in the report repository under MIT, and every number above names the tool that produced it.
Bottom line
- It runs. 552B backbone + 196B engram, 502 GB on disk, on one RTX 5090 with 128 GB of RAM: 5.1 tok/s cold, 21.3 resident, 0.9967 logit correlation with DeepSeek's reference at 1,401 tokens.
- The second number is a trap. Repeating a prompt warms the answer, not the engine. Discard-the-first-request benchmarking would report 4× the truth.
- More hardware of the kind you can buy does not help. VRAM cache size, RAM above 72 GiB, SSD speed, eviction policy and expert pinning are each measured at zero or negative. The bottleneck is residency of a 105 GiB working set.
- One real bug, caught by a constant. A per-head query norm inherited from V4 produced fluent text with the same argmax and 0.76 logit correlation; 181.019 = √(64×512) gave it away. Everything left after the fix is DeepSeek's own fp8/fp4 rounding.
- The built-in draft head is neutral here and excellent on resident machines. Draft length is a property of the machine.
- It needs a fork. Upstream llama.cpp cannot run it yet; the engram layouts have to be reconciled first.
Related reading
- DeepSeek-V4-Flash on Two GB10s: 304B Params, 1M Context, 83 Watts — the predecessor, on a machine where the working set is resident
- AI Workstation Comparison: RTX 5090 vs GB10 (HP ZGX) — bandwidth versus capacity, which is the whole question here
- Mixture of Experts (MoE), Explained — why 6 of 384 experts per token makes streaming possible at all
- 4-Bit Quantization Decoded: INT4 QAT, MXFP4, and NVFP4 — the MXFP4 format the experts ship in
- Qwen3.8-27B Is 2.3× Faster With One Flag — Do You Need DFlash2? — in-checkpoint drafters, and what happens to reproducibility
- Best Model for a Dual DGX Spark — the 128 GB class this report points at