How much VRAM do open LLMs really need? Real GGUF sizes for 35 models (October 2026)

"Parameters × bits ÷ 8" is the rule everyone quotes for how much GPU memory a model needs. It gets you in the right neighbourhood and then lets you down exactly where it matters: on the card that is 2 GB too small, or on the 32k-token chat that suddenly no longer fits.

So I stopped estimating and pulled the numbers from the source for 35 current open models:

• Weights: the real size of the published file — GGUF builds by unsloth, ggml-org, bartowski and the model authors (Q4_K_M unless the model ships quantised natively, like GPT-OSS in MXFP4), read from the Hugging Face API.
• Context (KV) cache: computed from each model's own config.json — number of layers, KV heads and head size — including the architectures where the naive formula is wrong (more on that below).
• Runtime: CUDA context and buffers, about 0.75 GiB plus the logits buffer of a 512-token batch (it adds up for 256k-token vocabularies).

All figures are GiB (1024³ bytes), the unit GPU memory is sold in, so they compare directly with "24 GB". Hugging Face lists decimal gigabytes, which is why its file sizes look ~7% larger.

The table

One GPU, one request, FP16 cache. "Smallest single card" is the smallest common card size where the total fits in 97.5% of the memory (the driver and CUDA take the rest).

Model	Params	Weights (format, GiB)	VRAM @ 8k ctx	VRAM @ 32k ctx	Smallest single card (8k)
Qwen3.8 27B	27.78B	Q4_K_M 15.3	17.2	18.7	24 GB
Qwen3.5 9B	9.65B	Q4_K_M 5.3	6.8	7.6	8 GB
Qwen3.5 4B	4.66B	Q4_K_M 2.6	4.1	4.8	8 GB
Qwen3 8B	8.19B	Q4_K_M 4.7	6.8	10.2	8 GB
Qwen3 14B	14.77B	Q4_K_M 8.4	10.7	14.4	12 GB
Qwen3 32B	32.76B	Q4_K_M 18.4	21.4	27.4	24 GB
Qwen3 30B-A3B	30.53B (3.3B active)	Q4_K_M 17.3	19.1	21.3	24 GB
Qwen3 Coder 30B-A3B	30.53B (3.3B active)	Q4_K_M 17.3	19.1	21.3	24 GB
Qwen3 Next 80B-A3B	81.32B (3B active)	Q4_K_M 45.2	46.5	47.0	48 GB
Qwen3 235B-A22B	235.09B (22B active)	Q4_K_M 132.4	134.9	139.3	141 GB
Qwen3 Coder 480B-A35B	480.15B (35B active)	Q4_K_M 270.1	273.1	278.9	288 GB
Gemma 4 12B	11.96B	Q4_K_M 6.6	8.3	8.7	12 GB
Gemma 4 26B-A4B	25.81B (4B active)	Q4_K_M 15.8	17.4	17.9	24 GB
Gemma 4 31B	31.27B	Q4_K_M 17.1	19.7	21.6	24 GB
Gemma 3 27B	27.43B	Q4_K_M 15.4	17.7	19.6	24 GB
GPT-OSS 20B	20.91B (3.6B active)	MXFP4 11.3	12.6	13.2	16 GB
GPT-OSS 120B	116.83B (5.1B active)	MXFP4 59.0	60.5	61.3	80 GB
DeepSeek R1 671B	684.53B (37B active)	Q4_K_M 377.1	378.7	380.3	multi-GPU
DeepSeek V3.2	685.36B (37B active)	Q4_K_M 377.6	379.1	380.7	multi-GPU
DeepSeek V4 Flash	290.94B	UD-Q4_K_XL 144.4	145.5	145.8	180 GB
DeepSeek R1 Distill Qwen 32B	32.76B	Q4_K_M 18.5	21.5	27.5	24 GB
DeepSeek R1 0528 Qwen3 8B	8.19B	Q4_K_M 4.7	6.8	10.2	8 GB
Llama 3.1 8B	8.03B	Q4_K_M 4.6	6.6	9.6	8 GB
Llama 3.3 70B	70.55B	Q4_K_M 39.6	43.1	50.6	48 GB
Mistral Small 3.2 24B	24.01B	Q4_K_M 13.3	15.6	19.3	16 GB
Devstral Small 2 24B	24.01B	Q4_K_M 13.3	15.6	19.3	16 GB
Ministral 3 14B	13.95B	Q4_K_M 7.7	9.9	13.7	12 GB
Mistral Nemo 12B	12.25B	Q4_K_M 7.0	9.2	13.0	12 GB
GLM-4.7 Flash	31.22B (3B active)	Q4_K_M 17.1	18.5	19.8	24 GB
GLM-4.5 Air	110.47B (12B active)	Q4_K_M 68.0	70.4	74.8	80 GB
GLM-5.3 Flash	321.32B	UD-Q4_K_XL 186.0	187.3	187.6	288 GB
Kimi K2 1T	1026.47B (32B active)	Q4_K_M 578.6	580.2	581.8	multi-GPU
Hermes 4.3 36B	36.15B	Q4_K_M 20.3	23.3	29.3	24 GB
Phi-4 14B	14.66B	Q4_K_M 8.3	10.8	12.3	12 GB
Hunyuan Hy3	298.79B	FP8 279.3	282.8	290.3	multi-GPU

What surprised me

Hybrid and sliding-window attention change the picture. Qwen3 32B and Qwen3.8 27B are about the same size on disk, but going from an 8k to a 32k context costs Qwen3 32B +6 GiB and Qwen3.8 27B +1.5 GiB. Qwen3.8 keeps a KV cache on only 16 of its 64 layers; the other 48 are linear-attention layers with a fixed state per sequence. Gemma 4 does something similar with a 1024-token sliding window on 5 of every 6 layers. If you plan for long contexts, the architecture matters as much as the parameter count.

MoE models need all of their weights in memory. Qwen3 30B-A3B uses 3.3B parameters per token, but all 30.5B have to be loaded: it needs the same 24 GB card as a dense 27B model. It just runs much faster on it. (llama.cpp can keep experts in system RAM with --n-cpu-moe, at a speed cost.)

GPT-OSS is the best fit for its size class. GPT-OSS 20B ships in MXFP4 and fits a 16 GB card with room for context; GPT-OSS 120B fits a single 80 GB H100.

70B still means 48 GB. Llama 3.3 70B in Q4_K_M is 43 GiB at 8k context: a 48 GB card, or two 24 GB cards with tensor parallelism. At 32k it no longer fits in 48 GB without a quantised KV cache.

DeepSeek's MLA cache is tiny. DeepSeek V3.2 and R1 cache one 576-wide latent per token and layer instead of full keys and values: about 70 KB per token for the whole model. The weights are the problem there, not the context.

The formula, if you want to check a model yourself

weights   = file size of the quantisation you run (or params × bits ÷ 8)
kv_cache  = tokens × layers_with_full_attention × 2 × kv_heads × head_dim × 2 bytes
          + min(tokens, window) × sliding_layers × 2 × kv_heads × head_dim × 2 bytes
runtime   ≈ 0.75 GiB + 512 × vocab × 4 bytes
total     = weights + kv_cache × parallel_requests + runtime

A q8_0 KV cache (-ctk q8_0 -ctv q8_0 in llama.cpp, --kv-cache-dtype fp8 in vLLM) roughly halves the second line. vLLM preallocates most of the card for its cache by default (--gpu-memory-utilization), so nvidia-smi will always look full there; the numbers above are what the model needs, not what vLLM grabs.

Calculator

I turned this into a free calculator: pick a model (35 LLMs plus FLUX, Qwen-Image, Wan 2.2, HunyuanVideo and other image and video models), a quantisation, the context length and the number of parallel requests, and it shows the memory breakdown and the GPUs that fit: gpu.nz/en/tools/vram. Every model also has a page with the full quantisation table: gpu.nz/en/models.

Disclosure: I run gpu.nz, a per-minute GPU rental, so the calculator also lists hourly prices of the cards that fit. The numbers themselves don't depend on that — if you spot a model where they look wrong, tell me and I'll check it against the files.