I pulled unsloth/DeepSeek-V4.1-Flash-1017-GGUF in UD-IQ1_M onto my 3090 box on 2026-10-20, five shards, 250.404 GB, and the tool printed Fetching 5 files: 100%|██████████| 5/5 [58:59<00:00] plus the word Downloaded. That bar is a claim about the transport, it says the bytes the tool asked for came back. It doesn't claim the payload is the payload I wanted, and on a five-shard split the payload is where the interesting failures hide. The check that means something is a diff against the hosting API's file tree, I ran one, five sizes against five sizes, zero mismatches, every shard exact to the byte.
The five sizes, in bytes: 5,780,923, then 49,707,332,128, 49,292,777,280, 40,803,165,856, and 110,595,290,592. Elapsed 58:59, throughput 61 to 68 MB/s from a du delta of 12.7 GB over 180 seconds, my own measurement, the tool's estimate never got used. The downloader had also recorded per-file sha256 in its cache metadata, xet block hashes, sitting there before I did anything. I knew the hashes existed and still ran the diff, because the diff is three lines of Python and the hashes are a format I'd have to trust.
What the shards actually contain
The first shard is 5.78 MB and it has zero tensors in it. I read it with the GGUF reader from the llama.cpp tree and it printed metadata only, the chat template, the tokenizer, keys in a deepseek41.* namespace, draft-model keys. The 110.6 GB at the end of the split is the engram table, IQ4_NL, designed to be mmap'd and paged on demand through --lazy-mode. So the file is 250 GB, and 110.6 GB of it is allowed to live on a disk that pages it. The other 139.8 GB has to be addressable, and that number is the one nobody's progress bar shows.
The gate nobody's bar shows
The vendor publishes a minimum-RAM table for the family with the engram excluded, IQ1_S at 122.4 GB, IQ1_M at 139.8, IQ2_XXS at 165.3, Q2_K_XL at 199.6, and it climbs to 305.6 at Q8_K_XL. My box is 62 GB of RAM plus 24 GB per card on two RTX 3090s, 110 GB addressable. IQ1_M wants 139.8 GB resident, IQ1_S wants 122.4, the box was short by 12 to 30 GB across the two quants. The 125 GB of swap exists, 62.7 GB zram plus disk, and swap-backed expert weights crawl, so the run never got attempted. A 250 GB mmap load on that box would have thrashed the inference lane that was answering on :8080, the one my agents were routing through at that hour.
Placement was a paper decision before the download started. The box has a 2.7 TB spinning HDD mounted cold, ROTA 1, and a spinner makes every mmap page pay a seek, so the engram table belongs on the SSD filesystem, ROTA 0. After the 250 GB landed, that filesystem read 930 GB total with 212 GB free, 78% used. The model fits on the disk and the weights don't fit in the box.
That split is the thesis, disk holds the file, the box cannot hold the weights.
Why the quant took the bigger number
The quant choice came from published agreement, IQ1_S at 82.8% and IQ1_M at 88.2% top-1 on a wiki1100 sample, 5.4 points of agreement on top of 17.4 GB. I took IQ1_M anyway, because the gap between the two quants sits inside the shortfall, both miss the addressable pool, and the bigger one is what you want once the RAM arrives. The family is a 552 B backbone plus 196 B engram parameters at 1M context, arch id 20, and the repo also ships a dspark draft at 7.98 GB in Q8_0 for speculative decoding. That draft waits on the same purchase.
The runner was already there
Twenty minutes of grep decided the build question. llama-server 0.6.0-dev on the box already has the arch, LLM_ARCH_DEEPSEEK4 in src/llama-arch.h, a model struct, and a special-case at llama-model.cpp:387, plus --lazy-mode and --load-mode in the CLI. The other engine on that box is single-arch, its build file lists one arch id, so this model can't run there. No new build was needed. The plan for the next sitting is a llama-swap member pointing at shard1 with lazy mode on, temp 1.0, top-p 1.0, jinja, reasoning effort 75, and it waits on one order, 128 to 192 GB of RAM, because the shortfall lives in the addressable pool.
Rules the night produced
A progress bar reports the transport, a manifest diff reports the payload, and the gap between those two claims is one API call. File size reports where the model lives on disk, the resident table reports where it has to be addressable, and on a split-architecture model those two numbers sit 110.6 GB apart inside the same file. Rotation values belong in the placement decision before the load, ROTA 0 for anything you intend to mmap. And the engine question is a grep before it's a rebuild, an arch id in a header file beats a compile that was never necessary.
The shape of this work as it stands: the download verified, the placement verified, the engine verified, and the run blocked with a number attached, 29.8 GB short at IQ1_M, and a purchase on the way.
Where the doubt belongs
So what would you have to see before you called a 250 GB pull verified? The two places I put doubt are the gap between the bar and the manifest, and the gap between the file and the resident table, both of them questions you can ask today for free. Drop the number your pipeline checks last and I'll tell you what I'd have checked before I believed it.
Top comments (0)