DEV Community

Chennarao Vemula
Chennarao Vemula

Posted on

Local LLM on a 16GB Mac Mini: Replacing GitHub Copilot with Ollama + Qwen

#ai #opensource #macos #productivity

I kept paying a monthly subscription for a cloud coding assistant while a 16GB M4 Mac mini sat on my desk idling most of the day. So I ran the obvious experiment: can a 16GB Mac mini run a coding assistant entirely offline — no code leaving the machine, no subscription — and is it actually usable for real work?

Short answer: yes, with one hard constraint (RAM) and one soft one (context length). This article is the written version of the video above, with every command, config file, and benchmark number so you can reproduce it.

Table of contents

Why bother running locally

Three reasons, in the order that actually mattered to me:

  1. Privacy. Client code, internal repos, anything under NDA — none of it leaves the machine. This is the one thing a hosted assistant cannot offer you at any price tier.
  2. Cost. A coding assistant subscription is roughly $100–240/yr depending on tier. The Mac mini was already bought.
  3. Offline. Flights, bad hotel wifi, coffee shop dead zones. The assistant just works.

The reason not to: raw capability. The frontier hosted models are better at large multi-file reasoning, and it isn't close. More on that below.

The hardware constraint nobody mentions

On Apple Silicon, the GPU and CPU share one pool of unified memory. A model has to fit in that pool alongside macOS, your browser, VS Code, and whatever containers you're running. On a 16GB machine, macOS + a normal dev environment eats 6–8GB before you've loaded anything.

That leaves you roughly 7–9GB of realistic headroom for the model. This single number determines everything else, and it's why "just run the 30B model" advice from people on 64GB machines doesn't transfer.

By default macOS allows the GPU to use about 75% of total RAM as VRAM. You can check what you're actually working with:

# total RAM in bytes
sysctl hw.memsize

# current memory pressure — the number that actually matters
memory_pressure | tail -5

# what's using it
top -o MEM -n 10 -l 1 | head -20
Enter fullscreen mode Exit fullscreen mode

Step 1: Install Ollama

Two options. Homebrew is easier to script and update:

brew install --cask ollama
Enter fullscreen mode Exit fullscreen mode

Or download the app directly from ollama.com. Either way, verify the daemon is up:

ollama --version
curl -s http://localhost:11434/api/tags | head
Enter fullscreen mode Exit fullscreen mode

If that curl returns JSON, the local API server is live on port 11434. That endpoint is what VS Code will talk to — it is OpenAI-API-compatible enough for most tooling.

If it isn't running:

# start the server in the foreground to see logs
ollama serve
Enter fullscreen mode Exit fullscreen mode

LM Studio alternative: if you'd rather have a GUI with a model browser and a built-in chat window, LM Studio does the same job and also exposes an OpenAI-compatible server (default port 1234). Everything below works with either — swap the apiBase port.

Step 2: Pick a model that fits in 16GB

This is where most local-LLM writeups go wrong. Here's the actual size on disk (and roughly in memory) for the Qwen coder family:

Model Download size Fits in 16GB? Use it for
qwen2.5-coder:1.5b 986 MB ✅ Trivially Autocomplete only
qwen2.5-coder:3b 1.9 GB ✅ Easily Autocomplete, light chat
qwen2.5-coder:7b 4.7 GB ✅ Sweet spot Chat + edit + autocomplete
qwen2.5-coder:14b 9.0 GB ⚠️ Tight — close other apps Best quality you can get
qwen2.5-coder:32b 20 GB ❌ No —
qwen3-coder:30b (a3b) 19 GB ❌ No Needs 32GB+

The 16GB recommendation: qwen2.5-coder:7b for chat and edits, qwen2.5-coder:1.5b for inline autocomplete. Running a small dedicated autocomplete model alongside the bigger chat model is the trick that makes the whole thing feel responsive — autocomplete needs to answer in milliseconds, and a 7B can't.

Pull them:

ollama pull qwen2.5-coder:7b
ollama pull qwen2.5-coder:1.5b

# optional: embeddings for codebase indexing
ollama pull nomic-embed-text

ollama list
Enter fullscreen mode Exit fullscreen mode

If you have the RAM headroom and want to try 14B, pull an explicit quantization rather than the default — q4_K_M is the best quality-per-gigabyte tradeoff:

ollama pull qwen2.5-coder:14b-instruct-q4_K_M
Enter fullscreen mode Exit fullscreen mode

Step 3: Run and verify 

ollama run qwen2.5-coder:7b
Enter fullscreen mode Exit fullscreen mode

Then, to see actual timings instead of vibes, use verbose mode:

ollama run --verbose qwen2.5-coder:7b "Write a Python function that parses an ISO 8601 duration string into seconds. Include edge cases."
Enter fullscreen mode Exit fullscreen mode

--verbose prints total duration, prompt eval rate, and eval rate (tokens/sec) after every response. That's your benchmark instrument — no extra tooling needed.

While it's generating, watch memory in another terminal:

ollama ps        # shows loaded models, size, and CPU/GPU split
Enter fullscreen mode Exit fullscreen mode

The PROCESSOR column in ollama ps should say 100% GPU. If it says anything with CPU, the model spilled out of unified memory and your tokens/sec just fell off a cliff — drop to a smaller model or quantization.

Step 4: Wire it into VS Code

Install the Continue extension, then edit ~/.continue/config.yaml:

name: Local Mac Mini Config
version: 0.0.1
schema: v1

models:
  - name: Qwen2.5 Coder 7B
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit
      - apply
    defaultCompletionOptions:
      contextLength: 8192
      maxTokens: 2048

  - name: Qwen2.5 Coder 1.5B (autocomplete)
    provider: ollama
    model: qwen2.5-coder:1.5b
    apiBase: http://localhost:11434
    roles:
      - autocomplete
    defaultCompletionOptions:
      contextLength: 2048
      maxTokens: 256

  - name: Nomic Embed
    provider: ollama
    model: nomic-embed-text
    roles:
      - embed

context:
  - provider: code
  - provider: diff
  - provider: terminal
  - provider: currentFile
Enter fullscreen mode Exit fullscreen mode

Two things worth calling out:

  • model: must match ollama list exactly. A tag mismatch fails silently with an empty response, which is a miserable 20 minutes of debugging.
  • contextLength: 8192 is deliberate. Qwen2.5-Coder supports 32K, but on 16GB the KV cache for a 32K context costs you more memory than the model weights saved you. 8K covers a file and its imports, which is what a local assistant is realistically good at anyway.

Restart VS Code, open the Continue panel, and confirm the model dropdown shows your local models. Inline autocomplete should start firing as you type.

Step 5: Tune Ollama for a 16GB box

Three environment variables do most of the work. Set them where Ollama can see them — if you run the app, use launchctl; if you run ollama serve yourself, put them in your shell profile.

# keep the model resident so you don't pay reload cost on every request
launchctl setenv OLLAMA_KEEP_ALIVE "30m"

# only one model in memory at a time — critical on 16GB
launchctl setenv OLLAMA_MAX_LOADED_MODELS "1"

# don't let concurrent requests multiply your memory footprint
launchctl setenv OLLAMA_NUM_PARALLEL "1"
Enter fullscreen mode Exit fullscreen mode

Then restart Ollama for them to take effect.

The counterintuitive one is OLLAMA_MAX_LOADED_MODELS=1. It seems to fight the two-model setup from Step 4 — and it does mean a swap when you jump between chat and autocomplete. But on 16GB, having both a 7B and a 1.5B resident plus a browser open is what pushes you into swap, and swap on a local LLM is catastrophic, not slow. If you have the headroom (nothing else open), set it to 2 and enjoy the snappier switching.

If you want to reclaim memory immediately:

ollama stop qwen2.5-coder:7b
Enter fullscreen mode Exit fullscreen mode

Measuring it on your own machine

I'm deliberately not handing you a table of my numbers. Throughput on Apple Silicon swings with macOS version, thermal state, and whatever else is resident in unified memory — a benchmark from someone else's Mac mini tells you almost nothing about yours. Here's the two-minute version that gives you real figures:

for m in qwen2.5-coder:1.5b qwen2.5-coder:3b qwen2.5-coder:7b; do
  echo "=== $m ==="
  ollama run --verbose "$m" \\
    "Write a Python function that retries an HTTP request with exponential backoff. Include type hints and docstring." \\
    2>&1 | tail -8
done
Enter fullscreen mode Exit fullscreen mode

The number to read is eval rate (tokens/sec). prompt eval rate matters less for interactive coding — it's how fast it ingests your file, and it's rarely the bottleneck at 8K context.

Run it a few times and take the median; the first run of any model includes load time and will look worse than reality.

Rules of thumb that held up across my runs:

  • Above ~15 tok/s — feels conversational. You read the output as it streams and stay in flow.
  • 8–15 tok/s — usable, but you'll notice the wait on longer generations.
  • Below ~8 tok/s — you start context-switching to another window while it thinks, which defeats the entire point of having it inline.

And keep an eye on ollama ps while it runs — if PROCESSOR shows any CPU percentage, the model spilled out of unified memory and the numbers you're reading are meaningless. On 16GB that's the single most common reason people conclude "local models are too slow."

What it does well, what it doesn't

Genuinely good at:

  • Single-function generation and refactors
  • Boilerplate — tests, type hints, docstrings, config scaffolding
  • "Explain this regex / this stack trace / this git diff"
  • Renaming and mechanical edits across a file
  • Anything you'd rather not paste into a hosted service

Falls down on:

  • Multi-file reasoning. It doesn't hold your architecture in its head.
  • Long context. You're on 8K by choice; hosted tools give you hundreds of thousands of tokens.
  • Very recent library APIs — the training cutoff bites, and there's no web access.
  • Agentic multi-step work. The 7B loses the thread.

The honest framing: a local 7B is roughly a competent junior who has read all the docs, works instantly, never leaks your code, and cannot see past the current file.

Should you cancel Copilot?

Depends entirely on your work:

  • Mostly function-level work on code you'd rather keep private → local wins on cost and comfort. Cancel.
  • Greenfield scaffolding across dozens of files, heavy agentic workflows → you'll hit the ceiling in a week. Keep the subscription.
  • Most people → run both. Local for the 80% of routine edits, hosted for the hard 20%. The subscription math still works out if you drop a tier.

I've been running this setup as my default and reaching for the cloud only when the local model visibly struggles. That split has held up.

The full video walkthrough — install, model runs, VS Code hookup, and the live coding tests — is at the top of this post, or here: Goodbye GitHub Copilot? Building a Local AI Lab on a 16GB Mac Mini.

Which model should I benchmark next on 16GB? Drop it in the comments — I'll run it.

Top comments (0)