DEV Community: Gosha

Which Qwen model to run: versions, sizes and VRAM for 11 models

Gosha — Wed, 21 Oct 2026 03:26:07 +0000

The Qwen model you can run depends on your GPU memory. At Q4_K_M with an 8K context, Qwen3.5 9B needs 6.8 GB, Qwen3.8 27B 17.2 GB, Qwen3 Next 80B-A3B 46.5 GB and Qwen3 Coder 480B-A35B 273.1 GB. These are single-request figures for the 11 family models. Below: what separates the generations, which model fits which card and how to run it.

6.8 GB — Qwen3.5 9B, Q4_K_M, 8K context
17.2 GB — Qwen3.8 27B, Q4_K_M, 8K context
46.5 GB — Qwen3 Next 80B-A3B, Q4_K_M, 8K context
256K — context of Qwen3.8, Qwen3.5, Next and Coder, tokens; Qwen3 has 40K

Computed by the VRAM calculator, data as of 2026-10-19.

Which Qwen models are there?

Qwen is Alibaba's family of open models. The names are easy to mix up: versions, sizes and suffixes all run together. Five lines matter for a GPU in our data.

Qwen3 — dense models at 8B, 14B and 32B plus two Mixture-of-Experts (MoE) models, 30B-A3B and 235B-A22B. They work with text and think before answering by default.
Qwen3 Coder — programming models, 30B-A3B and 480B-A35B. The authors built them for agentic coding, and they have no thinking mode.
Qwen3 Next 80B-A3B — a MoE with 81.3B parameters, 3B of them working on each token. Its attention is hybrid, more on that below.
Qwen3.5 — the 4B and 9B versions are in the table. They take images along with text.
Qwen3.8 27B — the newest in the table. Its authors call it the next generation after Qwen3.5 and Qwen3.6, with gains in code, professional work and agents. It takes images and video.

“A3B” and “A22B” mean 3B and 22B active parameters. In a dense model every parameter works on every token.

Part of the family is not in the calculator yet: Qwen3.5 has 27b, 35b and 122b sizes in the Ollama library, Qwen3.8 has Flash-Next and 2.4T-A95B on Hugging Face, and Qwen3.6 is not in the data. We do not count these.

How much VRAM does each Qwen model need?

At Q4_K_M and an 8K context, from 4.1 GB (Qwen3.5 4B) to 273.1 GB (Qwen3 Coder 480B-A35B). Quantisation compresses the weights: the Q4_K_M file of Qwen3.8 27B is 3.3 times smaller than the original BF16, with little loss in answer quality. The total covers the weights, the context cache (memory for the text the model has already read) and the CUDA runtime.

Qwen up to 32B fits a 24 GB card at Q4_K_M, Next 80B needs 48 GB

	GB
Qwen3.5 4B	4.1
Qwen3.5 9B	6.8
Qwen3 8B	6.8
Qwen3 14B	10.7
Qwen3.8 27B	17.2
Qwen3 30B-A3B (MoE)	19.1
Qwen3 Coder 30B-A3B (MoE)	19.1
Qwen3 32B	21.4
Qwen3 Next 80B-A3B (MoE)	46.5

8K context, one request, FP16 cache. In bold: models with hybrid attention. Weights are the GGUF files on Hugging Face; data as of 2026-10-19.

The whole family in one table: parameters, file size, memory at two context lengths, the smallest card and the context limit set in each model's config.

Model	Parameters	Q4_K_M weights	Memory at 8K / 32K	Smallest card	Context limit
Qwen3.5 4B	4.7B	2.6 GB	4.1 / 4.8 GB	8 GB	256K
Qwen3.5 9B	9.7B	5.3 GB	6.8 / 7.6 GB	8 GB	256K
Qwen3 8B	8.2B	4.7 GB	6.8 / 10.2 GB	8 GB	40K
Qwen3 14B	14.8B	8.4 GB	10.7 / 14.4 GB	12 GB	40K
Qwen3.8 27B	27.8B	15.3 GB	17.2 / 18.7 GB	24 GB	256K
Qwen3 30B-A3B	30.5B (3.3B active)	17.3 GB	19.1 / 21.3 GB	24 GB	40K
Qwen3 Coder 30B-A3B	30.5B (3.3B active)	17.3 GB	19.1 / 21.3 GB	24 GB	256K
Qwen3 32B	32.8B	18.4 GB	21.4 / 27.4 GB	24 GB	40K
Qwen3 Next 80B-A3B	81.3B (3B active)	45.2 GB	46.5 / 47.0 GB	48 GB, tight	256K
Qwen3 235B-A22B	235B (22B active)	132.4 GB	134.9 / 139.3 GB	141 GB, or 2 × 80 GB, tight	40K
Qwen3 Coder 480B-A35B	480B (35B active)	270.1 GB	273.1 / 278.9 GB	288 GB, or 4 × 80 GB, tight	256K

A3B does not mean “3 GB”. Qwen3 30B-A3B uses 3.3B parameters per token, but all 30.5B load into memory: 17.3 GB for the weights alone. MoE buys answer speed, not memory. The model will not fit a card smaller than 24 GB, and renting a weak card just burns minutes.

How do Qwen3.5 and Qwen3.8 differ from Qwen3?

The biggest difference shows in memory: the new models have a much smaller context cache. In Qwen3 every layer keeps the whole conversation. In Qwen3.8 27B and Qwen3.5 9B (according to the model cards) three layers in four are linear attention, Gated DeltaNet, with a tiny state. Only every fourth layer keeps a cache. That is 64 KiB per token against 256 KiB for Qwen3 32B, 4 times less.

From 8K to 32K context Qwen3 32B takes 6.0 GB more, Qwen3.8 27B only 1.5 GB

	GB
Qwen3 32B	6
Qwen3 14B	3.8
Qwen3 8B	3.4
Qwen3 30B-A3B	2.3
Qwen3 Coder 30B-A3B	2.3
Qwen3.8 27B	1.5
Qwen3.5 4B	0.8
Qwen3.5 9B	0.8
Qwen3 Next 80B-A3B	0.6

Extra memory at 32K tokens over 8K, Q4_K_M, one request. In bold: hybrid attention.

Qwen3 Next is built like Qwen3.8, and Qwen3 Coder like plain Qwen3. The gap widens with long contexts. Qwen3.8 27B at 128K needs 24.7 GB and stays on a 32 GB card; at the full 256K it takes 32.7 GB and a 48 GB card. Qwen3 Coder 30B-A3B at 128K needs 30.3 GB for 17.3 GB of weights: its cache is the ordinary kind.

The new models take 256K tokens of context, about 400 pages of text. Qwen3 is limited to 40K in our data. The Qwen3 cards say YaRN stretches it to 128K, with a warning that quality on short texts can drop. The calculator does not model that mode.

We have not measured answer quality in Russian or any other language. The authors claim support for more than 100 languages in Qwen3 and 201 in Qwen3.5. Try the model on your own texts: an hour on a rented card is enough.

Which Qwen model fits your GPU?

The table shows which models each next memory step adds at 8K, 32K and 128K context (Q4_K_M, one request). A dash means nothing new.

Card memory	New at 8K context	New at 32K	New at 128K
8 GB	Qwen3.5 4B, Qwen3.5 9B, Qwen3 8B	Qwen3.5 4B, Qwen3.5 9B	—
12 GB	Qwen3 14B	Qwen3 8B	Qwen3.5 4B, Qwen3.5 9B
16 GB	—	Qwen3 14B	—
24 GB	Qwen3.8 27B, Qwen3 30B-A3B, Qwen3 Coder 30B-A3B, Qwen3 32B	Qwen3.8 27B, Qwen3 30B-A3B, Qwen3 Coder 30B-A3B	—
32 GB	—	Qwen3 32B	Qwen3.8 27B, Qwen3 Coder 30B-A3B
48 GB	Qwen3 Next 80B-A3B	—	—
80 GB	—	Qwen3 Next 80B-A3B	Qwen3 Next 80B-A3B

8 GB — Qwen3.5 9B: 6.8 GB at 8K and 7.6 GB at 32K. Qwen3 8B fits at 8K too, but for 32K it needs a 12 GB card.
12 GB and 16 GB — Qwen3 14B: 10.7 GB at 8K. A 32K context (14.4 GB) needs a 16 GB card.
24 GB — the widest step: Qwen3.8 27B (17.2 GB), Qwen3 30B-A3B and Coder 30B-A3B (19.1 GB), Qwen3 32B (21.4 GB). All but Qwen3 32B hold a 32K context.
32 GB — Qwen3 32B with a 32K context (27.4 GB) and Qwen3.8 27B with 128K (24.7 GB).
48 GB — Qwen3 Next 80B-A3B: 46.5 GB, almost the whole card. A 32K context no longer fits and needs an 80 GB card.

Within one line, more parameters usually mean better answers. We have no quality comparison across lines: take two or three models that fit and run them on your own tasks.

Which Qwen for code, chat and images?

Code in an editor, agents — Qwen3 Coder. The 30B-A3B needs 19.1 GB and runs on a 24 GB card; the 480B-A35B needs several cards.
Chat, writing, analysis — Qwen3.8 27B (24 GB), or Qwen3.5 9B on a smaller card. Both think by default.
Images as input — Qwen3.5 and Qwen3.8: attach a photo or a screenshot. Qwen3 and Coder take text only.
An API for several people — MoE models such as 30B-A3B and Next: fewer parameters work on each token than in a dense model of the same size. vLLM suits such an API.
Image generation — a different family, Qwen-Image. Setup and use are in Qwen-Image-Edit in ComfyUI.

How many GPUs do Qwen3 235B and Coder 480B need?

The big ones do not fit an ordinary card. Qwen3 235B-A22B at Q4_K_M needs 134.9 GB: almost the whole 141 GB card. Qwen3 Coder 480B-A35B needs 273.1 GB: one 288 GB card or several smaller ones.

Model	Q4_K_M weights	Memory at 8K	One card	80 GB cards	141 GB cards
Qwen3 Next 80B-A3B	45.2 GB	46.5 GB	48 GB	80 GB	141 GB
Qwen3 235B-A22B	132.4 GB	134.9 GB	141 GB	2 × 80 GB	141 GB, tight
Qwen3 Coder 480B-A35B	270.1 GB	273.1 GB	288 GB	4 × 80 GB	2 × 141 GB, tight

The model is split across cards: the calculation halves the weights and the cache, and every card keeps its own runtime memory.

How to run it

With a card that fits, Ollama is the shortest path; the tags match the models above:

ollama run qwen3.5:9b      # 8 GB card
ollama run qwen3:14b       # 12-16 GB
ollama run qwen3.8:27b     # 24 GB
ollama run qwen3-coder:30b # 24 GB

Ollama tags are separate builds, so their sizes differ a little from the Q4_K_M GGUF figures here; if the fit is tight, take the next card size.

For the big models and for an API with several users, use vLLM. This is how the model card runs Qwen3 Next 80B-A3B: four cards, a 256K context, vLLM 0.10.2 or newer.

vllm serve Qwen/Qwen3-Next-80B-A3B-Instruct --port 8000 --tensor-parallel-size 4 --max-model-len 262144

--tensor-parallel-size sets the number of cards and --max-model-len the context length.

No card that big? You can rent one by the minute on gpu.nz (my service): Qwen3.8 27B has a ready template with a chat and an API on a 24 GB card, and the rest run through the Open WebUI + Ollama template.

Check your own setup

Pick the model, quantisation and context length in the free VRAM calculator: it shows the memory breakdown and which GPU sizes hold it. Every model above also has a page with the full quantisation table.

What can go wrong

Thinking uses up context. Qwen3, Qwen3.5 and Qwen3.8 think first by default, and long reasoning grows the cache. In Qwen3 and Qwen3.5 you can switch it off with enable_thinking=False. Coder has no thinking.
Parallel requests multiply the cache. The figures above are for one request. In vLLM every concurrent request keeps its own cache, so an API for several people needs headroom.
A tight fit. Above 92% of the card's memory the model starts, but a long conversation will not fit. The table marks Qwen3 Next 80B-A3B, Qwen3 235B-A22B and Qwen3 Coder 480B-A35B this way.
Licences differ. The cards of Qwen3.8 27B, Qwen3.5 9B, Qwen3 32B and 235B, Next and Coder say Apache-2.0, while Qwen3.8 Flash-Next says qwen-community-1.0. Open the card of your model before a commercial launch.

How we counted

Weights are the real file sizes: GGUF from Hugging Face, BF16 from Qwen's original repositories. The context cache is computed from each model's config.json, counting the linear layers of the hybrid models. The runtime is about 0.75 GB plus the logits buffer. A card counts as enough when the model takes at most 97.5% of its memory. 1 GB here is 1024³ bytes, as on GPUs. Data as of 2026-10-19; the method is in the calculator, and one table for 35 models is in How much VRAM do open LLMs really need?. Ollama tags by memory step are in the Ollama model roundup, and a neighbouring read is Gemma 4: which version fits.

Disclosure: I run gpu.nz, a per-minute GPU rental; the memory figures come from the model files and configs and do not depend on that. The original, with a live calculator, is at gpu.nz/en/guides/which-qwen-model.

Ollama vs LM Studio vs llama.cpp vs vLLM: which to pick, and how much VRAM each needs

Gosha — Wed, 21 Oct 2026 03:25:39 +0000

At home, for yourself, pick LM Studio or Ollama; for an API on a server, pick Ollama for one user and vLLM for many. The choice changes the memory too: Qwen3.8 27B takes 17.2 GB as GGUF Q4_K_M and 52.8 GB as the original BF16 weights that vLLM runs. Below: a feature table checked against the docs on 2026-10-20, the memory for each format and what goes wrong.

17.2 GB — Qwen3.8 27B, GGUF Q4_K_M, 8K context: Ollama, LM Studio, llama.cpp
30.6 GB — the same model at FP8, 8K context: vLLM
52.8 GB — original BF16 weights, 8K context: vLLM
40.2 GB — FP8 with 16 requests at once, 8K each

Computed by the VRAM calculator, data as of 2026-10-19.

How do Ollama, LM Studio, llama.cpp and vLLM differ?

All four run open models, but they split the work differently: LM Studio and Ollama are ready-made programs, llama.cpp is an engine with many flags, and vLLM is a server for an API under load.

Ollama is a program that downloads a model and runs it with one command, ollama run. It has a REST API and an OpenAI-compatible /v1 address.
LM Studio is a desktop app: a chat, a model catalog and a local-server switch. For servers there is a windowless mode (llmster) and the lms command-line tool. It runs models on the llama.cpp engine, and on Macs with Apple Silicon on MLX as well.
llama.cpp is an open C/C++ engine under the MIT licence. It also provides llama-server, with an OpenAI-compatible API and a web interface.
vLLM is a library and server that serves models through an API. It was built for simultaneous requests and has no web chat.

What they can do in practice is in the table. Its rows come from the projects' READMEs and documentation, checked on 2026-10-20.

Feature	Ollama	LM Studio	llama.cpp	vLLM
Interface	Terminal and API, an app on Windows and macOS; Open WebUI adds a web chat	A window with a chat, a model catalog and a server switch	Terminal and API; llama-server has a web interface	API only, no ready web chat
API	Its own REST API and /v1 in the OpenAI format (a subset), port 11434	OpenAI-compatible /v1 and an Anthropic-compatible one; port 1234 in the examples	OpenAI-compatible chat completions, responses and embeddings, plus Anthropic Messages; --api-key	OpenAI-compatible (chat, completions, responses, embeddings) and Anthropic Messages; started with vllm serve
Several users	By default 1 request per model, the rest queue; set with OLLAMA_NUM_PARALLEL	Continuous batching in the llama.cpp engine, 4 requests at once by default	Parallel slots (-np) and continuous batching, on by default	Continuous batching and PagedAttention: built for throughput
Weight formats	GGUF, and safetensors on import; the Ollama library and hf.co/…	GGUF; MLX on Macs with Apple Silicon	GGUF, 1.5- to 8-bit quantisation	Hugging Face weights: FP8, INT8, INT4, GPTQ, AWQ. GGUF is “highly experimental”
Systems	Windows 10 22H2+, macOS, Linux, Docker; NVIDIA, AMD, Apple Metal, Vulkan	Windows (x64, ARM), Linux (x64, ARM64), macOS on Apple Silicon only	Windows, macOS, Linux, Docker; CUDA, HIP, Metal, Vulkan, SYCL, CPU	Linux; Windows only through WSL; NVIDIA from compute capability 7.5, AMD, Intel
Licence	MIT	Free at home and at work (since July 2025)	MIT	Apache 2.0

Sources: docs.ollama.com and the Ollama README, lmstudio.ai/docs, the ggml-org/llama.cpp repository on GitHub, docs.vllm.ai. We did not measure generation speed and do not compare it: it depends on the card, the model and the quantisation.

Where to download Ollama, LM Studio, llama.cpp and vLLM

Ollama, llama.cpp and vLLM install with a terminal command; LM Studio is an ordinary app from lmstudio.ai. The commands come from the projects' documentation.

# Ollama: Linux and macOS
curl -fsSL https://ollama.com/install.sh | sh

# Ollama: Windows, PowerShell
irm https://ollama.com/install.ps1 | iex

# llama.cpp: Linux and macOS
curl -LsSf https://llama.app/install.sh | sh

# vLLM: Linux with an NVIDIA card
uv pip install vllm --torch-backend=auto
vllm download-kernels

LM Studio runs on Windows, Linux and Macs with an Apple Silicon chip; it does not run on Intel Macs. llama.cpp has ready builds on the Releases page of the ggml-org/llama.cpp repository, and a Docker image. vLLM needs Linux and, for NVIDIA cards, compute capability 7.5 or higher: an RTX 20-series card or newer.

How much memory does one model take in each format?

The same Qwen3.8 27B takes from 17.2 GB to 52.8 GB, and the difference is in the weight format, not the program. Ollama, LM Studio and llama.cpp work with quantised GGUF files: Q4_K_M stores about 4.8 bits per parameter instead of 16. vLLM takes Hugging Face weights: the original BF16, or compressed FP8, AWQ or GPTQ.

Qwen3.8 27B: 17.2 GB at Q4_K_M and 52.8 GB at BF16 — a 24 GB card and an 80 GB one

	GB
Q4_K_M (GGUF: Ollama, LM Studio, llama.cpp)	17.2
Q8_0 (GGUF)	28.9
FP8 (vLLM)	30.6
BF16 (original weights, vLLM)	52.8

8K context, one request, FP16 cache. Weights are file sizes; data as of 2026-10-19.

The Q4_K_M weights are 3.3 times lighter than BF16 (15.3 GB against 50.9 GB). Q4_K_M fits a 24 GB card. Q8_0 and FP8 need a 32 GB card, and FP8 almost fills it. BF16 needs an 80 GB card.

Qwen3.8 27B at BF16 does not fit a 48 GB card. The BF16 weights are 50.9 GB, and 52.8 GB with the cache and buffers. They will not start on a 48 GB card: you need an 80 GB card or the FP8 build (30.6 GB). The first vLLM start takes 10–30 minutes or more, and the card is billed the whole time. Work out the memory before you rent.

GPT-OSS 20B looks different: it was released in MXFP4 from the start, and its weights are 11.3 GB. With an 8K context it needs 12.6 GB, a 16 GB card. Expanding the weights to BF16 gives about 40.3 GB (estimated from the parameter count) and a 48 GB card. Before you pick a program, check the format the model was released in: the memory depends on it.

What changes when many requests arrive?

All four can serve several requests at once, but every request has its own context cache, and it takes GPU memory. The Ollama documentation says so directly: memory grows as OLLAMA_NUM_PARALLEL × OLLAMA_CONTEXT_LENGTH.

Qwen3.8 27B at FP8: one request 30.6 GB, 16 at once at 8K 40.2 GB, at 32K 64.2 GB

	GB
1 request (8K)	30.6
4 requests (8K)	32.5
16 requests (8K)	40.2
16 requests, 32K (32K window each)	64.2

FP8 weights, FP16 cache. Every request fills its window, so this is an upper estimate. Data as of 2026-10-19.

For one request with an 8K window, the Qwen3.8 27B cache takes 0.64 GB: the model has a hybrid architecture, so it is modest. For 16 requests that is 10.3 GB, and at a 32K window 34.3 GB. One request fits a 32 GB card, four need 48 GB, and sixteen at 32K need 80 GB.

The number of simultaneous requests is set differently in each tool. In Ollama it is the OLLAMA_NUM_PARALLEL variable, in llama.cpp the -np flag (the number of slots), in LM Studio the Max Concurrent Predictions setting. In vLLM it is limited by the memory left after the weights.

What to pick: a PC, a server or maximum control

One person, a home computer. LM Studio if you want a window with a chat and a model catalog. Ollama if you prefer the command line and need an API for your own programs. On Windows and Mac these two are the easiest.
No GPU of your own, or the model does not fit. Rent a server with Ollama: the template gives a browser chat and an API. It is enough for you and a small team.
An API for many users or a service. vLLM: continuous batching, Hugging Face weights, FP8 and AWQ. It needs Linux; on Windows it runs only through WSL.
You need to squeeze a model into limited memory. llama.cpp: fine control over the cache, layers and experts, described below.

llama.cpp helps when a model barely fits. It can store the K and V cache compressed: the flags -ctk q8_0 -ctv q8_0 make it about half the size. For Qwen3.8 27B at Q4_K_M with a 128K context that is 20.9 GB instead of 24.7 GB, so a 24 GB card is enough instead of 32 GB. Ollama has the same trick (OLLAMA_KV_CACHE_TYPE), but with one cache type for all models. Its documentation calls the precision loss at q8_0 very small.

With MoE models, llama.cpp can keep the experts in system RAM (the --cpu-moe and --n-cpu-moe flags), and a model that does not fit in GPU memory runs in a hybrid CPU and GPU mode. In that mode answers are slower than with the whole model in GPU memory.

Check your own model

Pick a model, a format and a context in the free VRAM calculator: it shows the memory breakdown and which GPU sizes hold it. For vLLM look at the FP8 and BF16 options, for Ollama, LM Studio and llama.cpp at Q4_K_M and Q8_0. The rule of thumb is in the VRAM guide for LLMs.

What can go wrong

Parallel requests multiply the cache. The first tables are computed for one request. For an API with several people, add memory for the cache of every simultaneous request, especially with a long context.
GGUF in vLLM is not the main path. The vLLM documentation calls GGUF support highly experimental and requires a separate plugin. For vLLM take FP8, AWQ or the original weights.
vLLM takes almost the whole card. The --gpu-memory-utilization parameter sets the share, so other jobs on the same card usually cannot run.
The vLLM key does not guard every address. By the documentation, --api-key is checked only on the /v1, /v2 and /inference paths. If you install vLLM yourself, put the port behind a proxy.
The model did not fit in GPU memory. Ollama and llama.cpp put some layers in system RAM and still start, but answers become noticeably slower. In Ollama the share shows in the Processor column of ollama ps.

How we counted

Weights are real file sizes: GGUF from Hugging Face repositories, FP8 and BF16 from the models' original repositories. The BF16 size of GPT-OSS 20B is computed from its parameter count. The context cache is computed from the model's config.json; for several requests it is multiplied by their number, each request filling its whole window. The runtime is about 0.75 GB plus the logits buffer. 1 GB here is 1024³ bytes, as on GPUs. Calculator data as of 2026-10-19; the method is in the calculator, and one table for 35 models is in How much VRAM do open LLMs really need?.

The tools' features were checked against their official documentation on 2026-10-20: the README and docs.ollama.com, lmstudio.ai/docs, the llama.cpp README and server documentation, and docs.vllm.ai. Releases are frequent, so open the project page before you install.

No GPU that big?

If the model you want does not fit your own card, the quickest test is a rented GPU for an hour. On gpu.nz (my service, billed by the minute) Ollama and vLLM have ready templates: Open WebUI + Ollama from an 8 GB card and vLLM from 24 GB, both with an OpenAI-compatible /v1 API. The walkthrough with request examples and common mistakes is in Ollama and vLLM on a rented GPU. LM Studio and llama.cpp you install yourself over SSH.

Disclosure: I run gpu.nz, a per-minute GPU rental; the feature table and the memory figures do not depend on that. The original, with a live calculator, is at gpu.nz/en/guides/ollama-lm-studio-llama-cpp-vllm.

How much VRAM do open LLMs really need? Real GGUF sizes for 35 models (October 2026)

Gosha — Tue, 20 Oct 2026 15:55:43 +0000

"Parameters × bits ÷ 8" is the rule everyone quotes for how much GPU memory a model needs. It gets you in the right neighbourhood and then lets you down exactly where it matters: on the card that is 2 GB too small, or on the 32k-token chat that suddenly no longer fits.

So I stopped estimating and pulled the numbers from the source for 35 current open models:

Weights: the real size of the published file — GGUF builds by unsloth, ggml-org, bartowski and the model authors (Q4_K_M unless the model ships quantised natively, like GPT-OSS in MXFP4), read from the Hugging Face API.
Context (KV) cache: computed from each model's own config.json — number of layers, KV heads and head size — including the architectures where the naive formula is wrong (more on that below).
Runtime: CUDA context and buffers, about 0.75 GiB plus the logits buffer of a 512-token batch (it adds up for 256k-token vocabularies).

All figures are GiB (1024³ bytes), the unit GPU memory is sold in, so they compare directly with "24 GB". Hugging Face lists decimal gigabytes, which is why its file sizes look ~7% larger.

The table

One GPU, one request, FP16 cache. "Smallest single card" is the smallest common card size where the total fits in 97.5% of the memory (the driver and CUDA take the rest).

Model	Params	Weights (format, GiB)	VRAM @ 8k ctx	VRAM @ 32k ctx	Smallest single card (8k)
Qwen3.8 27B	27.78B	Q4_K_M 15.3	17.2	18.7	24 GB
Qwen3.5 9B	9.65B	Q4_K_M 5.3	6.8	7.6	8 GB
Qwen3.5 4B	4.66B	Q4_K_M 2.6	4.1	4.8	8 GB
Qwen3 8B	8.19B	Q4_K_M 4.7	6.8	10.2	8 GB
Qwen3 14B	14.77B	Q4_K_M 8.4	10.7	14.4	12 GB
Qwen3 32B	32.76B	Q4_K_M 18.4	21.4	27.4	24 GB
Qwen3 30B-A3B	30.53B (3.3B active)	Q4_K_M 17.3	19.1	21.3	24 GB
Qwen3 Coder 30B-A3B	30.53B (3.3B active)	Q4_K_M 17.3	19.1	21.3	24 GB
Qwen3 Next 80B-A3B	81.32B (3B active)	Q4_K_M 45.2	46.5	47.0	48 GB
Qwen3 235B-A22B	235.09B (22B active)	Q4_K_M 132.4	134.9	139.3	141 GB
Qwen3 Coder 480B-A35B	480.15B (35B active)	Q4_K_M 270.1	273.1	278.9	288 GB
Gemma 4 12B	11.96B	Q4_K_M 6.6	8.3	8.7	12 GB
Gemma 4 26B-A4B	25.81B (4B active)	Q4_K_M 15.8	17.4	17.9	24 GB
Gemma 4 31B	31.27B	Q4_K_M 17.1	19.7	21.6	24 GB
Gemma 3 27B	27.43B	Q4_K_M 15.4	17.7	19.6	24 GB
GPT-OSS 20B	20.91B (3.6B active)	MXFP4 11.3	12.6	13.2	16 GB
GPT-OSS 120B	116.83B (5.1B active)	MXFP4 59.0	60.5	61.3	80 GB
DeepSeek R1 671B	684.53B (37B active)	Q4_K_M 377.1	378.7	380.3	multi-GPU
DeepSeek V3.2	685.36B (37B active)	Q4_K_M 377.6	379.1	380.7	multi-GPU
DeepSeek V4 Flash	290.94B	UD-Q4_K_XL 144.4	145.5	145.8	180 GB
DeepSeek R1 Distill Qwen 32B	32.76B	Q4_K_M 18.5	21.5	27.5	24 GB
DeepSeek R1 0528 Qwen3 8B	8.19B	Q4_K_M 4.7	6.8	10.2	8 GB
Llama 3.1 8B	8.03B	Q4_K_M 4.6	6.6	9.6	8 GB
Llama 3.3 70B	70.55B	Q4_K_M 39.6	43.1	50.6	48 GB
Mistral Small 3.2 24B	24.01B	Q4_K_M 13.3	15.6	19.3	16 GB
Devstral Small 2 24B	24.01B	Q4_K_M 13.3	15.6	19.3	16 GB
Ministral 3 14B	13.95B	Q4_K_M 7.7	9.9	13.7	12 GB
Mistral Nemo 12B	12.25B	Q4_K_M 7.0	9.2	13.0	12 GB
GLM-4.7 Flash	31.22B (3B active)	Q4_K_M 17.1	18.5	19.8	24 GB
GLM-4.5 Air	110.47B (12B active)	Q4_K_M 68.0	70.4	74.8	80 GB
GLM-5.3 Flash	321.32B	UD-Q4_K_XL 186.0	187.3	187.6	288 GB
Kimi K2 1T	1026.47B (32B active)	Q4_K_M 578.6	580.2	581.8	multi-GPU
Hermes 4.3 36B	36.15B	Q4_K_M 20.3	23.3	29.3	24 GB
Phi-4 14B	14.66B	Q4_K_M 8.3	10.8	12.3	12 GB
Hunyuan Hy3	298.79B	FP8 279.3	282.8	290.3	multi-GPU

What surprised me

Hybrid and sliding-window attention change the picture. Qwen3 32B and Qwen3.8 27B are about the same size on disk, but going from an 8k to a 32k context costs Qwen3 32B +6 GiB and Qwen3.8 27B +1.5 GiB. Qwen3.8 keeps a KV cache on only 16 of its 64 layers; the other 48 are linear-attention layers with a fixed state per sequence. Gemma 4 does something similar with a 1024-token sliding window on 5 of every 6 layers. If you plan for long contexts, the architecture matters as much as the parameter count.

MoE models need all of their weights in memory. Qwen3 30B-A3B uses 3.3B parameters per token, but all 30.5B have to be loaded: it needs the same 24 GB card as a dense 27B model. It just runs much faster on it. (llama.cpp can keep experts in system RAM with --n-cpu-moe, at a speed cost.)

GPT-OSS is the best fit for its size class. GPT-OSS 20B ships in MXFP4 and fits a 16 GB card with room for context; GPT-OSS 120B fits a single 80 GB H100.

70B still means 48 GB. Llama 3.3 70B in Q4_K_M is 43 GiB at 8k context: a 48 GB card, or two 24 GB cards with tensor parallelism. At 32k it no longer fits in 48 GB without a quantised KV cache.

DeepSeek's MLA cache is tiny. DeepSeek V3.2 and R1 cache one 576-wide latent per token and layer instead of full keys and values: about 70 KB per token for the whole model. The weights are the problem there, not the context.

The formula, if you want to check a model yourself

weights   = file size of the quantisation you run (or params × bits ÷ 8)
kv_cache  = tokens × layers_with_full_attention × 2 × kv_heads × head_dim × 2 bytes
          + min(tokens, window) × sliding_layers × 2 × kv_heads × head_dim × 2 bytes
runtime   ≈ 0.75 GiB + 512 × vocab × 4 bytes
total     = weights + kv_cache × parallel_requests + runtime

A q8_0 KV cache (-ctk q8_0 -ctv q8_0 in llama.cpp, --kv-cache-dtype fp8 in vLLM) roughly halves the second line. vLLM preallocates most of the card for its cache by default (--gpu-memory-utilization), so nvidia-smi will always look full there; the numbers above are what the model needs, not what vLLM grabs.

Calculator

I turned this into a free calculator: pick a model (35 LLMs plus FLUX, Qwen-Image, Wan 2.2, HunyuanVideo and other image and video models), a quantisation, the context length and the number of parallel requests, and it shows the memory breakdown and the GPUs that fit: gpu.nz/en/tools/vram. Every model also has a page with the full quantisation table: gpu.nz/en/models.

Disclosure: I run gpu.nz, a per-minute GPU rental, so the calculator also lists hourly prices of the cards that fit. The numbers themselves don't depend on that — if you spot a model where they look wrong, tell me and I'll check it against the files.