Run Local LLMs on a Mac in 2026: Which Chip Runs Which Model, and Why Bandwidth Beats Cores

Erik Bagdaer — Thu, 10 Sep 2026 14:03:09 +0000

Originally published on the Macyou blog. Disclosure up front: I run Macyou - we rent dedicated Apple Silicon Macs for AI. This post is about the hardware math, which is the same whether the Mac is on your desk or in a rack.

The short answer: any Apple Silicon Mac with 16 GB of unified memory runs 7B-14B models well, a 64 GB M4 Pro runs 70B-class models at usable speeds, and a 256 GB Mac Studio M3 Ultra runs 200B-class models that no single consumer GPU can hold. Generation speed is set almost entirely by memory bandwidth, so the chip tier matters more than the year.

Which Mac runs which model

The rule that decides everything: a model has to fit in unified memory at the quantization you choose, with a few gigabytes left for the context window and the OS. Once it fits, tokens per second scale with memory bandwidth. The M4 row below is measured (methodology and raw JSON, CC BY 4.0); the other rows are bandwidth-scaled estimates until measured columns land.

Mac	Bandwidth	Comfortable models (Q4)	What to expect
M4 Mac mini, 16 GB	120 GB/s	3B-14B	Measured: Llama 3.2 3B 46.7 tok/s, Llama 3.1 8B 21.2, Qwen 2.5 14B 11.7
M4 Mac mini, 24-32 GB	120 GB/s	14B comfortably, 32B at the edge	Same speeds as 16 GB - extra memory buys model size, not tok/s
M4 Pro Mac mini, 48-64 GB	273 GB/s	32B comfortably, 70B Q4 at 64 GB	~2x the base M4 at equal model size; 70B at ~5-6 tok/s
M4 Max Mac Studio, 128 GB	546 GB/s	70B Q8, 123B Q4	~4x the base M4; 70B Q4 in the low teens tok/s
M3 Ultra Mac Studio, 256 GB	819 GB/s	200B-class Q4, 70B FP16	Largest single-box option; 405B still needs clustering

The RAM math in one line

Weights in GB = parameters (billions) x bits per weight / 8, plus about 15% runtime overhead, plus context. At Q4_K_M (~4.85 bits) an 8B model is ~4.9 GB of weights and ~6 GB in memory; a 32B model is ~20 GB; a 70B model is ~42 GB; a 123B model is ~75 GB. Q8_0 roughly doubles those numbers, FP16 roughly quadruples them. Add 1-4 GB for a 32K context, more for 128K.

That is why 16 GB tops out at 14B, 64 GB is the 70B threshold, and 128 GB is where 100B+ dense models become practical.

Why speed is about bandwidth, not cores

Generating one token means reading every active weight once. A 4.9 GB model on a 120 GB/s bus can therefore never exceed ~24 tok/s - and we measured 21.2, about 87% of the theoretical ceiling. The same arithmetic predicts the M4 Pro (273 GB/s) at ~2x and the M4 Max (546 GB/s) at ~4x for the same model, which matches what the community consistently reports.

Two consequences:

More GPU cores on the same bandwidth barely help generation.
Mixture-of-experts models only read their active experts per token, so they run far faster than their parameter count suggests - a 35B-A3B MoE can out-run a dense 8B.

Prompt processing is the exception: it is compute-bound, and it varies about 2x between model families at equal size. We measured Qwen 2.5 7B at 1,130 prompt tok/s against Llama 3.1 8B at 587. If your workload is long-prompt/short-answer (RAG, classification), that gap matters more than generation speed.

Ollama vs LM Studio vs MLX vs llama.cpp

Ollama - the default for anything headless or scripted: one command to pull a model, a REST API on port 11434, and an OpenAI-compatible endpoint. Runs llama.cpp underneath.
LM Studio - the best GUI: model browser, chat window, and a local server that speaks the OpenAI API. Same engine class as Ollama, so same speeds.
MLX - Apple's own array framework. Fastest on some models and the natural choice for fine-tuning on a Mac; it's a Python library, not an app.
llama.cpp - the raw engine when you want every flag, the newest quant formats, or a C/C++ embed.

Five-minute setup with Ollama

brew install ollama
ollama serve &
ollama pull llama3.1:8b
ollama run llama3.1:8b "Explain unified memory in two sentences."

# OpenAI-compatible endpoint for your existing code:
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"llama3.1:8b","messages":[{"role":"user","content":"Hi"}]}'

Point any OpenAI SDK at http://localhost:11434/v1 with a dummy API key and it works. Before pulling a bigger model, do the RAM math - a 70B build on a 16 GB machine downloads 40 GB and then fails to load.

Common mistakes

Buying GPU cores instead of memory. A 24 GB Mac with more GPU cores runs the same 8B model no faster than a 16 GB one on the same chip; the next tier of bandwidth is what changes speed.
Ignoring the context window. A model that "fits" with 1 GB to spare will swap and crawl the moment you paste a long document. Leave 2-4 GB free, more for RAG.
Running production on a laptop. Thermal throttling, sleep, and a residential uplink turn a 21 tok/s machine into an unreliable one. Anything that needs to be up 24/7 belongs on a desktop-class Mac with a real network connection - yours or rented.

When a cloud Mac makes more sense than buying

If you need a local-class model available around the clock, reachable by your team or your app, the honest comparison is not laptop vs cloud but buying a Mac mini vs renting one. Owning wins on raw hardware cost over three years; renting wins the moment you count hosting, a static IP, power, remote-access setup, and being your own on-call. That's the niche we built Macyou for - a dedicated M4 Mac mini from $79/mo with Ollama pre-installed and the OpenAI endpoint already exposed, up to a 256 GB M3 Ultra.

Per-model requirements, quant tables, and the cheapest chip that fits each model are in our model guides. Questions about a specific model or Mac - ask in the comments, I'll answer with numbers where we have them.

DEV Community: Erik Bagdaer