Will it fit on my GPU? I built a calculator that finally answers it

StudioTV — Sat, 10 Oct 2026 14:42:27 +0000

Every time a new open model drops, the same question floods Reddit and Discord: will it run on my card? On 24 GB? On two of them? In 4-bit? At what context length?

The usual answer is a back-of-the-envelope formula: parameters times bytes per weight, plus a KV cache computed as if every layer looked at the whole context. For a classic Llama-style model, that works. For most models released in the last year, it is wrong, sometimes by an order of magnitude.

So I built the StudioTV LLM VRAM Calculator. It is free, needs no account, and works with any model on Hugging Face. Here is what it does, and why its numbers may differ from what you are used to.

The KV cache is where naive estimates break

Model weights are the easy part. The hard part is the KV cache, the memory that grows with every token of context, and modern architectures cache very differently:

Sliding-window attention. Gemma 4 31B keeps a 1,024-token window on 50 of its 60 layers. Only 10 layers remember the whole context. At 128K tokens, the real cache is 10.8 GiB. Treat every layer as global and you get 120 GiB, eleven times too much.
Linear attention. Qwen3.6 35B-A3B caches keys and values on only 10 of its 40 layers; the other 30 keep a small fixed-size state. At 128K: 2.5 GiB, not 10.
Latent attention (MLA). DeepSeek R1 stores a compact 576-wide latent per layer instead of full keys and values for its 128 heads. A multi-head formula predicts about 610 GiB at 128K; the real cache is 8.6 GiB.
Compressed attention. DeepSeek V4 Flash compresses the sequence and keeps a 128-token window: about seven times less than the naive estimate.

The calculator reads each model's configuration layer by layer and computes the cache the way vLLM and llama.cpp actually allocate it, including FP8 cache formats and the extra window llama.cpp keeps on sliding-window layers.

One question, answered in four steps

1. Your setup. Pick one of 53 curated models or paste any Hugging Face link. Choose the workload (inference, full fine-tuning, LoRA or QLoRA), the context length, how many requests you serve at once, your GPU and your engine. There are 60 GPUs, from data-center cards to laptop GPUs, Macs and other unified-memory machines, and four engines: vLLM, SGLang, TensorRT-LLM, and llama.cpp / Ollama / LM Studio.

2. Your answer. How many GPUs you need, whether it fits, and what fills each card.

What fills the card, plus a rough speed and cost. Here an API would be cheaper, and the calculator says so.

In this example, Gemma 4 31B in Q4_K_M takes 17.1 GiB of weights, 3.7 GiB of KV cache for one 32K-token request and about 3 GiB of runtime overhead: 23.7 GiB out of the 28.8 GiB usable on the RTX 5090. The calculator also estimates the speed (about 54 tokens per second here) and your cost per million tokens next to API prices. When an API is cheaper than renting the hardware, it tells you.

3. Better quality or a cheaper GPU? The “Does it fit?” grid crosses every weight format with every context length on your hardware. Click a cell to apply it.

Every weight format against every context length on one RTX 5090.

For Gemma 4 31B on one RTX 5090: Q4_K_M and Q5_K_M up to 32K tokens, Q3_K_M up to 128K. GGUF sizes follow llama.cpp's real quantization rules. Checked against the GGUF files actually published on Hugging Face, the median error is under 1% from Q8_0 down to Q3_K_M. Next to the grid, the calculator lists the cheapest GPUs that fit your setup and the other models that fit your card.

4. Rent and deploy. On-demand prices from RunPod, Vast.ai, Verda and Azure, refreshed every hour, with a price history for each GPU, and a ready-to-paste command for vLLM, SGLang, llama.cpp or TensorRT-LLM.

Hourly rental prices with their history, and the command to launch the model.

At the time of writing, the RTX 5090 started at $0.46 an hour on Vast.ai, and the calculator pointed out an even cheaper option: two RTX 3090s for $0.29 an hour in total. The command for this setup:

llama-server -hf unsloth/gemma-4-31B-it-GGUF:Q4_K_M \
  -c 32768 --parallel 1 -ngl 99 -fa on

When it does not fit

Ask for more than your card can hold and you get an offload plan instead of a dead end: how many MoE experts (--n-cpu-moe) or layers (-ngl) to keep in system RAM, and the speed to expect from your RAM, from dual-channel DDR4 to quad-channel DDR5.

A page for every model, updated every hour

Each model has its own page, here Gemma 4 31B.

Each model has its own page: VRAM at every precision and context length, how many GPUs of each type it takes, the cheapest way to run it today, its KV cache curve and the real sizes of the published GGUF files.

New models show up on their own. A watcher checks Hugging Face every hour, reads each new model's config.json and safetensors headers without downloading the weights, and publishes a page as soon as it can model the architecture. The “New & trending” section follows Hugging Face trends, Ollama's most pulled models and the latest releases.

Ask your AI assistant

The calculator is also an MCP server and a JSON API.

The calculator is also an MCP server. Add it to Claude, ChatGPT or any MCP client, then ask “does Qwen3.6 35B fit on my RTX 4090?”: the assistant calls the calculator and answers with the same numbers. Three tools, no API key: estimate_vram, models_that_fit and gpu_prices. In Claude Code:

claude mcp add --transport http studiotv-vram https://studiotvai.com/api/mcp

There is also a static JSON API and a VRAM badge you can paste into a model card.

On your phone too

On a phone, the answer stays pinned at the bottom of the screen while you change the settings.

How it's built

Front end: React 19, TypeScript and Vite. Every page (the calculator, one page per model, one per GPU) is prerendered to static HTML, so it loads fast and search engines see the numbers.
Model watcher: every hour, a script looks for new models on Hugging Face. config.json gives the architecture, and the header of each safetensors file, read with an HTTP Range request (a few hundred KiB instead of gigabytes), gives the exact size of every tensor. That is how vision towers or extra prediction layers that engines skip get counted right.
Serving: the MCP server is a small Node service, and the JSON API is plain static files. Everything runs in Docker on a single VPS.
A layout check in the build: the build opens every changed page in headless Chrome at 320, 390, 768 and 1280 px wide, scrolls it top to bottom, and fails if anything overflows the screen or throws a JavaScript error. It only rechecks pages whose code or structure changed, so it stays quick. I added it after a CSS change silently cut off the right side of the page on iPhones.

Limits, and how you can help

These are estimates. Engine versions and settings change real usage, so keep some headroom and confirm with nvidia-smi. Speed figures are rough (±30 to 50%) and do not model speculative decoding. Some “Rent” links are affiliate links; this is disclosed on the site, and providers are ranked by price only.

If a number does not match what you see, paste your vLLM or llama.cpp startup log into the “Compare with your real usage” box: the calculator lines up the real allocation with its estimate. That feedback is exactly what makes it better.

Try it: studiotvai.com/llm-vram-calculator

DEV Community: StudioTV