I tried to run a 27B parameter model on a 24GB MacBook.
It kernel-panicked my Mac. Three times. Hard reboot, lost work, the whole thing. And the fix every forum gives you made it worse.
Here is the number nobody prints, and how to find yours.
Click to watch: 12 minutes, the three panics, picking the quant, and the model driving a TV over MCP.
Your Mac has 24GB. Your GPU gets 17.76 GiB.
That is about 74% of what is printed on the box. Your weights, your KV cache and your compute buffers all share it.
Every guide comparing a GGUF's file size against your total RAM is doing arithmetic against a number the GPU never sees.
Find yours right now:
ioreg -l | grep recommendedMaxWorkingSetSize
Divide by 2^30.
| Your Mac | Expected GPU working set |
|---|---|
| 16 GB | ~11.8 GiB |
| 24 GB | 17.76 GiB (measured) |
| 36 GB | ~26.6 GiB |
| 48 GB | ~35.5 GiB |
| 64 GB | ~47.4 GiB |
Only the 24GB row is measured on my machine. Run the command and trust your own number.
Worth being precise about what this is, because the popular version is wrong. Apple does not call it a cap. Their own definition is:
An approximation of how much memory, in bytes, this GPU device can allocate without affecting its runtime performance.
It is a threshold Apple recommends you stay under, not a wall that stops you. Nothing refuses your allocation at 17.76 GiB. You are free to walk straight past it, into territory Apple told you to avoid, which on this machine contains a driver bug.
Why it panics instead of erroring
Q4_K_M, the quant everyone recommends, is 15.3GB of weights and peaks at 19.6GB once the projector and runtime buffers load. That is 110% of the budget.
panic(cpu 9): "pending memory object unexpectedly found in non pending hash"
IOGPUGroupMemory.cpp:528
process: WindowServer
The important line is elsewhere in the log:
Compressor Info: 4% of compressed pages limit (OK)
Four percent. The machine did not run out of memory. It hit a bug in Apple's GPU driver.
Not just me, and not just llama.cpp. The same failure is filed repeatedly against MLX: a kernel panic on an M4 Max during a large prefill, and unbounded KV cache growth taking down a Mac Studio. That thread names the mechanism: the memory is wired, so macOS's out-of-memory killer cannot reclaim it and never even sees pressure.
A normal process that asks for too much gets killed. A GPU allocation that asks for too much takes the kernel with it.
Why raising the wired limit backfires
Every forum says sudo sysctl iogpu.wired_limit_mb=20480. It is backwards.
Normal memory is a car in a car park. If the system needs the space, it tows the car.
Wired memory is bolted to the ground. The kernel is not allowed to move it, compress it, or swap it out. That is what the word means.
Raising the limit does not make the car park bigger. It bolts down more cars. You get less room to shuffle, and you remove macOS's ability to dig itself out.
I raised it. It panicked again.
"But I have 64GB and it is still slow"
This is the most common reply I get, so it gets its own answer.
RAM decides what fits. Bandwidth decides how fast. Separate limits, constantly conflated.
Your GPU re-reads the whole active weight set for every token it writes, so speed is roughly bandwidth divided by model size. Your RAM figure tells you nothing about your bandwidth:
| Chip | Bandwidth |
|---|---|
| M4 Pro | 273 GB/s |
| M2 / M3 Max | 400 GB/s |
| Ultra | 800 GB/s |
Three times the spread at the same 64GB. On the Pro tier you are capped and no setting fixes it. You are likely comparing yourself to people with double your bandwidth.
What actually works
llama-server -m Qwen3.8-27B-UD-Q3_K_XL.gguf \
--mmproj mmproj-F16.gguf \
-ngl 99 -c 16384 -fa on -ctk q4_0 -ctv q4_0 --jinja
-ctk q4_0 -ctv q4_0 quantises the KV cache itself to 4 bits, roughly 4x smaller. On this machine it is the difference between 16k context fitting and not. Quantise your cache before you drop a tier on your weights. You lose far less.
| Decode, clean machine | 12.60 tok/s |
| Same config, 4.15GB of swap present | 11.26 tok/s |
| Prefix cache | 14.2x (47.9s to 3.4s on 4,826 tokens) |
| Objective tasks | 21/21 |
| Tool calling | 6/6 |
| Wired memory | 16.37 GiB, 92% of budget |
That second row is the point. Same everything, 11% slower purely because the machine was carrying swap from a long session. Eleven percent of your throughput can be sitting in a browser tab you forgot about.
And one thing that nearly caught me out: my fit tool estimated 90% of budget. Measured, it wires 92%, hitting 94% the moment you show it an image. Treat any fit estimate as a floor, not a ceiling.
Will it fit on your machine
pip install localfit
localfit qwen3.8-27b --gpu-gb 12
--gpu-gb scores every quant against your budget instead of mine. Feed it your own ioreg number.
AnassKartit
/
localfit
Will it fit? GPU toolkit for AI models — MLX + GGUF + Cloud in one menu.
localfit
██╗ ██████╗ ██████╗ █████╗ ██╗ ███████╗██╗████████╗
██║ ██╔═══██╗██╔════╝██╔══██╗██║ ██╔════╝██║╚══██╔══╝
██║ ██║ ██║██║ ███████║██║ █████╗ ██║ ██║
██║ ██║ ██║██║ ██╔══██║██║ ██╔══╝ ██║ ██║
███████╗╚██████╔╝╚██████╗██║ ██║███████╗██║ ██║ ██║
╚══════╝ ╚═════╝ ╚═════╝╚═╝ ╚═╝╚══════╝╚═╝ ╚═╝ ╚═╝
Will it fit? Say what model you want — localfit figures out the rest.
Text and image generation. Fits locally? Run it. Doesn't fit? Free Kaggle GPU. Still too big? RunPod cloud. One command does everything: downloads models, starts servers, configures tools, launches your UI.
pip install localfit
One Command — Everything Works
# Chat + image gen in Open WebUI (gemma4 LLM + Flux Klein 4B images)
localfit launch openwebui --model gemma4:e4b --img klein-4b
# Code with image gen in localcoder
localfit launch localcoder --model gemma4:e4b --img klein-4b
# Claude Code with image MCP tools
localfit launch claude --model gemma4:e4b --img klein-4b
# Model doesn't fit locally? Run on free Kaggle GPU
localfit launch openwebui…The full version
The complete write-up has the agent tool-calling results, the MCP setup where I handed the model control of a television, and the trick that gets llama.cpp decoding video when the server insists it cannot.
If you have hit this, I want your ioreg number and your chip. Drop them in the comments.
Top comments (0)