Almost every local LLM buying guide sorts hardware by how many billions of parameters it can "handle." That framing falls apart as soon as you try to act on it. A 27B label tells you nothing about how many gigabytes actually land in memory when the model loads, and the gap between those two numbers is where people waste money.
So I checked. Pulling current download sizes from Ollama's library on September 14, 2026 and applying one stated headroom rule produces a blunt result: an 8 GB card fits none of the models worth running, not even the smallest one. 16 GB fits exactly two. The threshold that actually changes what you can do is 24 GB, and once you pass 32 GB the question stops being which GPU and starts being GPU or Mac.
I wrote the longer version of this, with the full per-model breakdown, over on DevToolLab. This is the condensed argument.
Parameter Counts Are Not a Memory Spec
The number in a model's name is marketing shorthand. What lands in VRAM is a quantized file, and the mapping between the two shifts with quantization level, dense versus mixture-of-experts layout, and vocabulary size.
One page of Gemma 4 tags settles it. The 12b build weighs 7.6 GB while 26b weighs 19 GB, so roughly twice the parameters costs about two and a half times the bytes. Then e2b shows up at 7.2 GB with far fewer parameters, because it is built differently. None of that is predictable from the label. Read the file size.
A Script That Answers the Question
Here is the check, in Node 18 or newer with no dependencies. It reads live sizes from Ollama and reports which machines can hold each model with enough left over to run it.
// fit-matrix.mjs - Node 18+, no dependencies. Run: node fit-matrix.mjs
const MODELS = [
["gemma4", "12b"], ["phi4", "14b"], ["mistral-small3.2", "24b"],
["qwen3.8", "27b"], ["gemma4", "26b"], ["gpt-oss", "20b"], ["gpt-oss", "120b"],
]
// Apple figures are unified memory from apple.com; NVIDIA figures are card VRAM.
const MACHINES = [
["laptop iGPU / RTX 3050", 8], ["RTX 4060 Ti 16GB", 16], ["RTX 4090", 24],
["RTX 5090", 32], ["Mac Studio M5 Max (base)", 36], ["Mac Studio M5 Max (max)", 128],
["Mac Studio M5 Ultra (base)", 96], ["Mac Studio M5 Ultra (max)", 512],
]
// Headroom rule, stated so you can disagree with it:
// - the OS and desktop need room, so reserve 2 GB (Macs share memory with the display)
// - KV cache and activations at a moderate context add roughly 20% on top of weights
const OS_RESERVE_GB = 2
const RUNTIME_OVERHEAD = 0.20
async function sizeOf(model, tag) {
// Sizes live on the model's /tags page, next to each tag link.
const html = await fetch(`https://ollama.com/library/${model}/tags`, {
headers: { "user-agent": "Mozilla/5.0" },
signal: AbortSignal.timeout(25000),
}).then((r) => (r.ok ? r.text() : ""))
for (const block of html.split(`href="/library/${model}:`).slice(1)) {
const name = block.match(/^([a-z0-9._-]+)/)
const size = block.match(/([0-9.]+)GB/)
if (name && size && name[1] === tag) return Number(size[1])
}
return null
}
const rows = []
for (const [model, tag] of MODELS) {
const gb = await sizeOf(model, tag)
if (gb) rows.push({ name: `${model}:${tag}`, gb, need: gb * (1 + RUNTIME_OVERHEAD) })
}
rows.sort((a, b) => a.gb - b.gb)
for (const [name, mem] of MACHINES) {
const usable = mem - OS_RESERVE_GB
const cells = rows.map((r) => (r.need <= usable ? " yes" : " no").padStart(8)).join("")
console.log(`${name.padEnd(28)}${String(mem).padStart(5)}${cells}`)
}
The two constants are the whole opinion here, and they are deliberately visible so you can argue with them. Two gigabytes goes to the operating system, which matters more on a Mac because the display shares that pool. The extra 20 percent covers KV cache and activations at a moderate context length. Push your context window higher and that number climbs with it, which is covered in more depth in the full article.
What it printed on September 14, 2026:
model sizes read from ollama.com on 2026-09-14
fit rule: weights + 20% runtime overhead + 2 GB for the OS
machine GB gemma4 phi4 gpt-oss mistral qwen3.8 gemma4 gpt-oss
7.6GB 9.1GB 14GB 15GB 18GB 19GB 65GB
-----------------------------------------------------------------------------------------
laptop iGPU / RTX 3050 8 no no no no no no no
RTX 4060 Ti 16GB 16 yes yes no no no no no
RTX 4090 24 yes yes yes yes yes no no
RTX 5090 32 yes yes yes yes yes yes no
Mac Studio M5 Max (base) 36 yes yes yes yes yes yes no
Mac Studio M5 Max (max) 128 yes yes yes yes yes yes yes
Mac Studio M5 Ultra (base) 96 yes yes yes yes yes yes yes
Mac Studio M5 Ultra (max) 512 yes yes yes yes yes yes yes
Reading the Grid
8 GB is a wall, not a starting line. The smallest model on the list needs 9.1 GB usable, and an 8 GB card has about 6 GB once the OS takes its share. It cannot hold Gemma 4 12B. Budget guides recommend 8 GB cards for this work constantly, which pushes buyers toward models so small that a hosted API would have served them better.
16 GB unlocks two models. Gemma 4 12B and Phi-4, both solid. For code completion or classification work, stopping here is defensible.
24 GB is where it gets interesting. It picks up gpt-oss:20b, Mistral Small 3.2 and Qwen3.8 27B, which is the first configuration that handles general assistant work alongside coding.
Watch the gap between 18 and 19 GB. Gemma 4 26B sits one gigabyte above Qwen3.8 27B, and that single gigabyte is the difference between a 24 GB card and a 32 GB card. When margins are that thin, parameter counts are actively misleading.
Where Unified Memory Wins
Past 32 GB the price per gigabyte on discrete cards gets ugly, and Apple's shared memory pool becomes the practical answer. Apple's published Mac Studio specs list the M5 Max at 36 GB of unified memory running at 460 GB/s, configurable up to 128 GB at 614 GB/s. The M5 Ultra starts at 96 GB at 1.2 TB/s and goes to 256 or 512 GB at the same bandwidth.
Nothing in consumer GPU territory answers 512 GB. A 65 GB model such as gpt-oss:120b just loads on a Mac Studio. Matching that with cards means several of them, a board that can host them, and a power supply sized accordingly.
Bandwidth is the honest counterargument. GDDR7 on an RTX 5090 clears 614 GB/s comfortably, and once a model fits, generation speed tracks memory bandwidth. That makes the tradeoff easy to state: a GPU gives you more tokens per second on anything that fits in VRAM, and a Mac gives you a wider set of models that fit at all. Under 32 GB, take the card. Above it, the Mac runs the thing and the card does not.
Electricity Is Part of the Price
Inference load on a workstation GPU is a sustained draw rather than an occasional spike, and an agent running all day turns the box into an appliance that never idles. A few hundred watts around the clock at typical US residential rates becomes a monthly bill worth putting next to the API invoice you were trying to cancel.
That is really the break-even calculation. Running locally swaps per-token pricing for capital plus power. Price your current hosted spend first with the LLM Token Cost Calculator, then run the card's sustained wattage through the Electricity Cost Calculator. If your API bill sits under a couple hundred dollars a month, the hardware may never pay for itself.
The Short Version of How to Decide
- Choose the model first, look up its actual download size, add 20 percent plus 2 GB, and treat that as your floor.
- Skip 8 GB entirely for this work. It clears nothing current, so spend the money on an API instead.
- Default to 24 GB. It covers the useful middle of the 2026 open-weight range with context to spare.
- Only cross into 32 GB for a named model. In this list Gemma 4 26B is the one that forces it.
- Go Mac when capacity beats throughput. Above roughly 64 GB of weights, unified memory is the only sane single-box option.
- Add the power bill before you commit, because a 24/7 inference box changes the payback period.
Conclusion
The entire decision reduces to one comparison: the file size of the model you intend to run against the usable memory of the machine you are considering, plus about 20 percent for runtime. On that basis 8 GB fits nothing, 16 GB fits two models, 24 GB covers most of what is worth running, and beyond 32 GB you are trading GPU bandwidth against Mac capacity. Run the script against your own model list before spending anything. The answer usually costs less than the spec-sheet version of the argument.
References
- Local LLM Hardware: GPU vs Mac in 2026 - the original article on DevToolLab, with the full fit matrix and per-model minimums
- Local LLM VRAM Requirements - the memory math behind the 20 percent headroom rule, including how KV cache grows with context
- Best Offline LLM Models and Tools 2026 - which models earn the download once you know your ceiling
- Best GPU Cloud Providers for AI Workloads - the rent-rather-than-buy path and when it wins
- Ollama model library - the source of every file size quoted here
Top comments (0)