DeepSeek shipped the official V4 Flash build today (July 31), and it is the rare frontier release that real hardware can hold: 284B parameters as a Mixture of Experts, but only 13B active per token, MIT licensed, with GGUF quantizations on Hugging Face within hours. Here is the practical picture for anyone who wants it running locally this weekend.
The shape of the model decides everything
MoE models split the hardware question in two. Total parameters (284B) set your memory bill: every expert has to live somewhere. Active parameters (13B) set your speed: each token only touches a small slice, so the compute load resembles a 13B model. That is why V4 Flash runs interactively on machines that would crawl with a dense model of the same size.
The numbers
The unsloth GGUFs of the 0731 build come in two sizes: UD-Q4_K_XL at 155 GB and UD-Q8_K_XL at 162 GB. The 8 bit file is only 7 GB bigger because expert weights dominate and quantize well, so if you clear the bar for Q4 you should check whether Q8 fits too.
Realistic machine classes:
- A Mac Studio with 192 GB unified memory runs Q4 comfortably.
- A workstation with 192 to 256 GB DDR5 plus a 24 GB GPU works well: attention and shared layers on the GPU, experts in RAM. Expect roughly 5 to 15 tokens per second depending on memory bandwidth.
- Anything smaller: take the Qwen 3.5 9B distill of V4 Flash, a 5.3 GB file that carries much of the reasoning style and runs on a 6 GB GPU or plain CPU.
One line serves the full model with a current llama.cpp:
llama serve -hf unsloth/DeepSeek-V4-Flash-0731-GGUF:UD-Q4_K_XL
The no terminal route
If you would rather click than type, the free and open source local AI studio Locally Uncensored has the V4 Flash family in its one click model catalog (the full build, the popular abliterated variant, and the 9B distill), checks your memory before you download 150 GB, and speaks to 12 local backends including llama.cpp.
The full walkthrough with all download links lives here: How to Run DeepSeek V4 Flash Locally, and the complete memory math per machine class is in Can You Run DeepSeek V4 Flash Locally?
Why bother running it at home
Three reasons. Privacy: your prompts never leave the machine. Cost: the weights are MIT, so a one time hardware investment replaces a metered bill. And control: the model you benchmark today is the model you run next year, no silent upstream swaps. With agent scores like 82.7 on Terminal Bench 2.1, this is the first time the model everyone is hyping on release day is also one you can own.
Top comments (0)