DEV Community

Cover image for Which open models actually fit a 12 GB card? We measured instead of guessing
AI Compute Radar
AI Compute Radar

Posted on Originally published at aicomputeradar.dev AI-assisted

Which open models actually fit a 12 GB card? We measured instead of guessing

Numbers below are as published on AI Compute Radar on 20 November 2026, for the 49 open-weights models we track. Everything is recomputable from public files; nothing is a benchmark.

The most common question under any model release is "will this run on my card?", and the most common answers are folklore: "about four gigabytes per billion parameters", "add twenty percent for context", "a 12 GB card runs 13B". Those rules were roughly right in 2023. For the models shipping now they can be off by a factor of four, and for one family by forty.

We run a small site that answers the question with arithmetic from published facts, and this post shows the arithmetic, because the interesting part is where the old rules break.

Three numbers decide it

Whether a model fits a card comes down to three quantities and one margin.

  1. Weights. Not estimated from the parameter count: measured from the byte size of a real, published GGUF file. We quote Q4_K_M where it exists, because that is the size most local guides assume, and we sum the shards of split archives (a three-part file read as one shard understates the model threefold, which is exactly the difference between "fits" and "does not").
  2. Context cache. The memory the conversation itself takes, computed from the model's own config.json: how many layers cache the context, how many key-value heads they have, how wide each head is, how many tokens you allow. This is the number the folklore gets wrong.
  3. Runtime overhead. A declared allowance for the compute buffers the runtime allocates, about a gigabyte on a small model and a bit more on large ones.

The sum has to fit into the usable memory of the card: we keep 12 % of a graphics card as a safety margin, and we plan Apple Silicon at 70 % of unified memory, because macOS caps how much of the pool the GPU may wire down (the kernel default is two thirds of RAM up to 32 GB, three quarters above).

That is the whole engine. Weights are a fact you can check before downloading. The cache is arithmetic. The margin is a stated assumption you can disagree with. Since October every verdict says which is which: each part of the estimate is marked measured, computed or assumed, on the page and in the API.

Where the old rules break

The classic context-cache formula is simple:

bytes = 2 × layers × kv_heads × head_dim × tokens × bytes_per_value
Enter fullscreen mode Exit fullscreen mode

Two for keys and values, times every layer, times the shape of one head, times the tokens, times two bytes for f16. Three things about current models make that formula misleading if you apply it naively.

Not every layer caches the whole context any more. Hybrid attention has become the norm. A config.json now often carries a layer_types array, and the layers behave differently:

Model Layers Layers that cache the full context The rest
Gemma 3 27B 62 10 52 sliding-window layers capped at 1,024 tokens
Gemma 4 12B 48 8 40 sliding-window layers, 1,024 tokens
Qwen3.8 27B 64 16 48 linear-attention layers with a constant state
gpt-oss 20B 24 12 12 sliding-window layers, 128 tokens

Gemma 3 27B at 32K tokens needs about 2.9 GB of context cache, not the 15 GB the every-layer formula gives. Qwen3.8 27B needs 2.0 GB where the formula says 8. If you assume every layer caches everything, you will tell people a card is too small when it is not.

MLA models store one latent, not keys and values. DeepSeek-V4 and the GLM-5 family use multi-head latent attention: each layer keeps a single compressed vector per token (576 dimensions in every MLA model we have measured so far) instead of separate K and V tensors. There is no factor of two, and the head shape does not enter at all. Applied naively, the classic formula overstates GLM-5.2's cache at 128K context by roughly forty times, hundreds of gigabytes instead of about eleven. That single mistake turns a model that fits a large unified-memory machine into one that "needs a data centre".

The head dimension has to be read, not derived. The common shortcut hidden_size / num_attention_heads is wrong for five of the first eight models we curated; they publish head_dim explicitly and it is 128, not what the division gives. One model, GLM-4.7-Flash, returns 102.4 from the division, which is not a dimension any tensor has. When a config does not let us derive the shape, we say unknown rather than round.

A 12 GB card, honestly

Take the RTX 3060 12 GB, still the most common "budget local AI" card. With the 12 % margin it has 10.6 GB usable. Here is what the arithmetic says for a few of the models people actually ask about, at 8K and at 32K tokens of context, f16 cache, Q4_K_M weights (read from the API on 20 November 2026):

Model (Q4_K_M) Weights Cache at 8K Total at 8K Verdict Cache at 32K Total at 32K Verdict
Phi-4 Mini 2.3 GB 1.0 GB 4.2 GB runs easily 4.0 GB 8.0 GB runs
Qwen3 8B 4.7 GB 1.1 GB 6.8 GB runs easily 4.5 GB 10.9 GB spills into RAM
Gemma 4 12B 6.6 GB 0.4 GB 8.1 GB runs 0.8 GB 9.2 GB runs
Qwen3 14B 8.4 GB 1.3 GB 10.8 GB spills into RAM 5.0 GB 15.3 GB spills into RAM
gpt-oss 20B 10.8 GB 0.2 GB 12.2 GB spills into RAM 0.8 GB 13.5 GB spills into RAM
Gemma 3 27B 15.4 GB 1.0 GB 17.7 GB spills into RAM 2.9 GB 20.4 GB spills into RAM

Three things in that table are worth reading twice.

  • Context is the hidden variable, in both directions. Qwen3 8B goes from "runs easily" to "spills into RAM" between 8K and 32K, by 300 megabytes: its cache grows from 1.1 to 4.5 GB, almost as much as its weights. Phi-4 Mini's cache at 32K is 1.7 times its weights. Gemma 4 12B, a bigger file, stays a comfortable fit at 32K because only 8 of its 48 layers cache the full context. Most "it runs on my 3060" claims are true at 4K and false at 32K, and the person making the claim rarely says which, or which model's attention they are talking about.
  • The 14B does not fit, and it is close enough to fool you. Qwen3 14B at 8K wants 10.8 GB against 10.6 usable. It will load, and then the first long conversation spills into system RAM. The old "12 GB runs 13B" rule assumes a smaller quantization and no context at all.
  • gpt-oss 20B is a mixture-of-experts with a tiny cache. Its 0.2 GB at 8K is real: only twelve of its layers cache the full context and its heads are narrow. The weights are what stops it, not the conversation.

Across all 49 tracked models, the 12 GB card runs 15 comfortably at 8K and 12 at 32K. An RTX 4090 with 21.1 GB usable runs 24 at 8K and 19 at 32K. A Mac mini M4 Pro with 64 GB of unified memory, planned at 44.8 GB, runs 31 at 32K. A DGX Spark with 128 GB, planned at 112.6 GB, runs 37, and at that size the context length no longer changes the count. Two models are "unknown" on every machine, because their configs are gated and we do not guess.

What we refuse to publish

  • No tokens per second. Speed depends on the runtime, the quantization, the driver and the day. A single figure would be wrong for most readers within weeks. We publish memory, which does not move.
  • No quality scores. We measure adoption (downloads, trending, API prices) and fit, not intelligence.
  • No street prices for used cards, and no "recommended" sticker that a vendor paid for. Launch prices appear with their source.
  • No rounding up. A verdict of "unknown" is a verdict.

Every number on the site carries either a collection timestamp, a source link, or a stated assumption you can change. If a config file says something different from what we computed, we would rather be corrected than believed.

Try it on your own machine

GET https://aicomputeradar.dev/api/v1/fit?hardware=rtx-3060-12gb&context=32768
Enter fullscreen mode Exit fullscreen mode

There is an MCP server too, so an assistant can ask the same question: npx -y ai-compute-radar for stdio, or the hosted endpoint at https://aicomputeradar.dev/api/mcp. The daily GPU rental prices behind the site's price board are a separate story; they are in the same API, day by day, up to 90 days back.

If your card is not in the list, or a model's shape looks wrong to you, say so. The catalog is curated by hand and every correction that survives checking changes the data.

Top comments (0)