DEV Community: Bruno Mello

Running a Fully-Local AI Agent on a Mac Studio, OpenClaw + Ollama + MLX

Bruno Mello — Fri, 22 May 2026 06:18:31 +0000

A real-world, copy-paste guide to running a personal WhatsApp AI agent entirely on-device on Apple Silicon, with zero per-token API billing. Two agents from one config (a full-access private assistant and a sandboxed public one), swappable local LLM backends (Ollama and MLX), local voice (TTS + STT), and LaunchAgents so everything survives reboots.

Tested on a Mac Studio M3 Ultra (96 GB unified memory), OpenClaw 2026.7.1-2, Ollama 0.24.0, mlx-lm 0.31.3, mlx 0.32.2, macOS 27.0 beta.

Updated August 2026. This agent has now been running continuously for three months, and the things that broke were not the things I expected. New material is marked [2026]. If you only read one new section, make it 6b: the kernel panic, because it will reboot your Mac and it is not your fault.

TL;DR findings

You don't need the cloud. A 26B-class model (Gemma 4 26B-A4B, a 4B-active MoE) is plenty for a chatty personal agent and runs comfortably in well under half of 96 GB.
Ollama and MLX can coexist as two OpenClaw providers; flip the agent's primary model with a one-line config change.
Benchmark one model at a time. Two large models resident at once throttle each other on memory bandwidth, which nearly halved throughput and produced a totally wrong "Ollama is faster" conclusion until I unloaded the idle one. In isolation the MLX build hit ~73 tok/s vs a contended ~35. [2026] I made this exact mistake again 15 months later. See §7.
[2026] mlx_lm.server can kernel panic your whole Mac. An open Apple GPU driver bug, triggered by the set_wired_limit call mlx-lm makes at startup. One line of Python works around it. §6b.
[2026] For a chat agent, tok/s is the wrong metric. Turning reasoning on raised throughput and made replies 8.8x slower to arrive. §7b.
MLX has no separate "warm-up" problem. mlx_lm.server loads the model at process start and holds it; the LaunchAgent keeps it alive. Ollama lazily unloads, so it needs OLLAMA_KEEP_ALIVE + a tiny warm-up ping.

0. Prerequisites

# OpenClaw (the agent gateway)
npm install -g openclaw@latest

# Ollama (llama.cpp backend)
brew install ollama

# MLX (Apple-silicon-native inference) in an isolated venv
python3 -m venv ~/mlx-env
~/mlx-env/bin/pip install -U mlx-lm

# ffmpeg (audio transcode for STT)
brew install ffmpeg

Throughout, replace these placeholders with your own values:

Placeholder	Meaning
`<YOUR_NUMBER_E164>`	your WhatsApp number, e.g. `+15551234567`
`<YOUR_GATEWAY_TOKEN>`	a random secret (`openssl rand -hex 24`)
`<YOUR_GROUP_ID>@g.us`	a WhatsApp group id (optional)
`you@example.com`	your provider OAuth email (optional cloud fallback)
`/Users/you`	your home directory

1. Architecture

                 ┌───────────────────────────────────────────┐
   WhatsApp ───► │  OpenClaw gateway (loopback :18789)       │
                 │                                           │
                 │  agent "private"  ── full tools           │
                 │  agent "public"   ── sandboxed (no bash)  │
                 └───────┬─────────────────────┬─────────────┘
                         │ model providers       │
            ┌────────────▼─────────┐   ┌─────────▼──────────────┐
            │ ollama  :11434       │   │ mlx  :8080             │
            │ (llama.cpp / Metal)  │   │ mlx_lm.server (OpenAI- │
            │                      │   │  compatible endpoint)  │
            └──────────────────────┘   └────────────────────────┘
            voice:  OmniVoice TTS :17494   ·   mlx-whisper STT :17495

One config defines two agents: a full-access private agent bound to your DM, and a locked-down public agent for everyone else.

2. The two local LLM providers

Both providers live under models.providers in ~/.openclaw/openclaw.json. Ollama speaks its native API; MLX is exposed as an OpenAI-compatible server, so OpenClaw talks to it with api: openai-completions.

{
  "models": {
    "providers": {
      "ollama": {
        "api": "ollama",
        "apiKey": "ollama-local",
        "baseUrl": "http://127.0.0.1:11434",
        "models": [
          { "id": "gemma4:26b-a4b-it-q8_0", "name": "Gemma 4 26B (Q8_0)",
            "contextWindow": 131072, "input": ["text","image"], "reasoning": true,
            "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 } }
        ]
      },
      "mlx": {
        "api": "openai-completions",
        "apiKey": "mlx",
        "baseUrl": "http://127.0.0.1:8080/v1",
        "models": [
          { "id": "mlx-community/gemma-4-26B-A4B-it-qat-4bit",
            "api": "openai-completions", "name": "Gemma 4 26B-A4B QAT-4bit (MLX)",
            "contextWindow": 131072, "input": ["text"], "reasoning": true, "maxTokens": 4096,
            "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 } }
        ]
      }
    }
  }
}

cost: 0 everywhere, these are local, free. It also keeps OpenClaw's usage accounting honest.

3. Wiring models into the agent + aliases

Under agents.defaults, register the models (with short aliases for quick swaps) and pick the primary:

{
  "agents": {
    "defaults": {
      "workspace": "/Users/you/.openclaw/workspace",
      "model": {
        "primary": "mlx/mlx-community/gemma-4-26B-A4B-it-qat-4bit"
      },
      "models": {
        "ollama/gemma4:26b-a4b-it-q8_0":          { "alias": "gemma4-26b-q8",   "params": { "think": true } },
        "mlx/mlx-community/gemma-4-26B-A4B-it-qat-4bit": { "alias": "gemma4-26b-qat", "params": { "think": true } }
      }
    }
  }
}

Swap Mika's brain with a one-liner (then restart the gateway):

openclaw config set agents.defaults.model.primary 'ollama/gemma4:26b-a4b-it-q8_0'
openclaw gateway restart
openclaw config get agents.defaults.model.primary   # verify

4. Two agents from one config (private vs public)

This is the underrated trick: define a locked-down public persona alongside the full-access private one. The public agent denies the dangerous tools.

{
  "agents": {
    "list": [
      {
        "id": "private",
        "name": "Mika",
        "workspace": "/Users/you/.openclaw/workspace"
      },
      {
        "id": "public",
        "name": "Mika (Public)",
        "workspace": "/Users/you/.openclaw/workspace/public",
        "tools": { "deny": ["bash", "process", "web_search"], "exec": {} }
      }
    ]
  },
  "bindings": [
    { "agentId": "private", "match": { "channel": "whatsapp", "peer": { "id": "<YOUR_NUMBER_E164>", "kind": "dm" } } },
    { "agentId": "public",  "match": { "channel": "whatsapp" } }
  ]
}

Your DM hits private (can run shell, browse, etc.); everyone else hits public (chat + safe tools only).

5. Ollama: tuning + keeping the model warm

[2026] Ollama's job in this stack changed. It no longer serves a chat model here: MLX won that role outright (§7), and I retired the duplicate 33 GB Q8_0 build along with the warm-up LaunchAgent below. Ollama now runs for exactly one thing, embeddings for memory search (§8c), which needs no warm-up because a 274 MB embedder loads on demand in well under a second. The section below still stands if you want a chat model on Ollama, or as a second opinion when a quant looks suspicious. Just do not keep both resident, for the reason in §7.

Ollama lazily unloads models after OLLAMA_KEEP_ALIVE (default 5 min), so the next request pays a cold-start. Two fixes: tune the service env, and pre-warm on boot.

5a. Service env (LaunchAgent: `~/Library/LaunchAgents/homebrew.mxcl.ollama.plist`)

<key>EnvironmentVariables</key>
<dict>
    <key>OLLAMA_FLASH_ATTENTION</key>  <string>1</string>
    <key>OLLAMA_KEEP_ALIVE</key>       <string>24h</string>
    <key>OLLAMA_KV_CACHE_TYPE</key>    <string>q8_0</string>
</dict>

launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.ollama.plist
launchctl load   ~/Library/LaunchAgents/homebrew.mxcl.ollama.plist

5b. Warm-up script: `~/ollama-warmup.sh`

#!/bin/bash
# Pre-warm an Ollama model into GPU after service start.
MODEL="${1:-gemma4:26b-a4b-it-q8_0}"
MAX_RETRIES=30; RETRY_INTERVAL=2
echo "[warmup] waiting for Ollama..."
for i in $(seq 1 $MAX_RETRIES); do
  if curl -s http://localhost:11434/api/tags >/dev/null 2>&1; then
    echo "[warmup] loading $MODEL..."
    curl -s http://localhost:11434/api/generate \
      -d "{\"model\":\"$MODEL\",\"prompt\":\"hi\",\"stream\":false,\"keep_alive\":\"24h\"}" >/dev/null 2>&1
    echo "[warmup] $MODEL warm."; exit 0
  fi
  sleep $RETRY_INTERVAL
done
echo "[warmup] Ollama did not start in time"; exit 1

5c. Warm-up LaunchAgent: `~/Library/LaunchAgents/com.ollama.warmup.plist`

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
  <key>Label</key><string>com.ollama.warmup</string>
  <key>ProgramArguments</key>
  <array>
    <string>/Users/you/ollama-warmup.sh</string>
    <string>gemma4:26b-a4b-it-q8_0</string>
  </array>
  <key>RunAtLoad</key><true/>
  <key>StandardOutPath</key><string>/tmp/ollama-warmup.log</string>
  <key>StandardErrorPath</key><string>/tmp/ollama-warmup.log</string>
</dict></plist>

chmod +x ~/ollama-warmup.sh
launchctl load ~/Library/LaunchAgents/com.ollama.warmup.plist
# disable later (keeps the file):  launchctl unload -w ~/Library/LaunchAgents/com.ollama.warmup.plist

6. MLX: a persistent OpenAI-compatible server

mlx_lm.server loads the model at startup and holds it for the life of the process, so the LaunchAgent is the warm-up. Pull a model once (it caches under ~/.cache/huggingface):

~/mlx-env/bin/hf download mlx-community/gemma-4-26B-A4B-it-qat-4bit

LaunchAgent: `~/Library/LaunchAgents/com.mlx-lm.server.plist`

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
  <key>Label</key><string>com.mlx-lm.server</string>
  <key>ProgramArguments</key>
  <array>
    <string>/Users/you/mlx-env/bin/python3</string>
    <string>/Users/you/mlx-env/mlx_server_shim.py</string>
    <string>--model</string><string>mlx-community/gemma-4-26B-A4B-it-qat-4bit</string>
    <string>--port</string><string>8080</string>
    <string>--prompt-cache-size</string><string>8</string>
    <string>--prompt-cache-bytes</string><string>12000000000</string>
  </array>
  <key>EnvironmentVariables</key>
  <dict>
    <key>PATH</key><string>/Users/you/mlx-env/bin:/opt/homebrew/bin:/usr/bin:/bin</string>
    <key>MLX_LM_DISABLE_WIRED_LIMIT</key><string>1</string>
  </dict>
  <key>RunAtLoad</key><true/>
  <key>KeepAlive</key><true/>
  <key>StandardOutPath</key><string>/tmp/mlx-lm-server.log</string>
  <key>StandardErrorPath</key><string>/tmp/mlx-lm-server.log</string>
</dict></plist>

launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.mlx-lm.server.plist
curl -s http://127.0.0.1:8080/v1/models | python3 -m json.tool   # confirm it's serving

[2026] Two changes in this plist. It no longer launches mlx_lm.server directly, it goes through a small shim that disables set_wired_limit (see §6b, not optional for a 24/7 agent), and it bounds the prompt cache. Also note bootstrap rather than the older launchctl load.

[2026] The launchd trap that cost me an hour. launchctl kickstart -k restarts the process but reuses launchd's cached job definition, so an edited plist is silently ignored. The service comes back, the port answers, the log looks healthy, and your change did nothing. After any plist edit, use launchctl bootout gui/$(id -u)/<label> then bootstrap, and verify against the running process rather than the file with ps -o args= -p $(pgrep -f mlx_server_shim.py). Corollary: because KeepAlive respawns a dead server transparently, health-check by comparing the PID, not by asking whether the port answers. A crash-and-respawn looks identical to healthy uptime from outside.

[2026] Why QAT, not OptiQ? The original version of this post used OptiQ-4bit, a mixed-precision community quant that keeps the MoE router at 8-bit and the experts at 4-bit. I have since switched to Google's official quantization-aware-trained qat-4bit, for two reasons. It is faster (~83 tok/s vs ~73, because it is uniformly 4-bit rather than keeping the router at 8-bit), and it removes a single-maintainer dependency in favour of a standard reproducible build. Same ~15-17 GB resident.

6b. [2026] The kernel panic that will reboot your Mac

This is the single most important thing I learned in three months of running this agent 24/7, and it is barely documented anywhere.

If you run mlx_lm.server as a long-lived service, your Mac can kernel panic and reboot. Not crash the Python process: reboot the machine, losing whatever else you had open.

panic(cpu 8 caller 0x...): "completeMemory() prepare count underflow" @IOGPUMemory.cpp:550
Kernel Extensions in backtrace:
   com.apple.iokit.IOGPUFamily(...)

Tracked as ml-explore/mlx#3186, open since March 2026 and still unfixed as of late August, with no response from Apple or the MLX maintainers anywhere in the thread. Reproduced on M1 Ultra, M3 Ultra, M4, M4 Pro, M4 Max and M5, from 32 GB to 256 GB. Not a "wrong hardware" problem, and not an out-of-memory problem.

What actually triggers it

A commenter did the work everyone else skipped: a controlled single-variable isolation, ten fresh-server soak runs per arm (full report). The result overturns the obvious assumption:

Not prefill size. A replay of a crashing session with prefills up to 23k tokens ran 111 minutes clean.
Concurrency plus prompt-cache eviction churn. Two concurrent streams, plus unique prompts overflowing --prompt-cache-bytes, panics in 102 to 108 seconds, 3 of 3 cold boots.
Not OOM. Real exhaustion gives a clean userspace Metal error. The panic hits with headroom to spare.

For a WhatsApp agent this matters more than it looks. Overlapping messages plus scheduled cron jobs is concurrency, and a long conversation churning the prompt cache is the eviction pattern. This exact setup sits squarely inside the trigger profile.

The one mitigation that works

mlx_lm.server calls mx.set_wired_limit(...) at startup, claiming roughly 75% of RAM as wired. Wired pages cannot be reclaimed by macOS. That single call is the discriminating factor:

Change (one variable per arm)	Result
none (stock)	panic x3 @ 102 to 108s
`set_wired_limit` never called	no panic, 10/10 runs, 4.2h
+ also no periodic `clear_cache`	no panic, 10/10 runs, 5.0h
only skip periodic `clear_cache`	panic @ 99.9s
`sysctl iogpu.disable_wired_collector=1` + metal4 async mapping off	panic @ 109.6s
RotatingKVCache / lower wired cap / `MLX_MAX_OPS_PER_BUFFER`	die earlier as clean userspace OOM

Note what fails: the sysctl workarounds you will find on forums, and bounding the KV cache. Also worth knowing, --max-kv-size is silently ignored for architectures that define make_cache.

The cost of the fix is about 3% sequential throughput, because weights become pageable.

The shim

There is no upstream opt-out. As of mlx-lm 0.31.3, mx.set_wired_limit is still called unconditionally in seven places, including server.py. So wrap the server in a launcher, ~/mlx-env/mlx_server_shim.py:

#!/usr/bin/env python3
# Launcher that no-ops set_wired_limit before mlx_lm.server starts.
# Mitigation for ml-explore/mlx#3186.
import os, sys
import mlx.core as mx

def main():
    if os.environ.get("MLX_LM_DISABLE_WIRED_LIMIT") == "1":
        mx.set_wired_limit = lambda *a, **k: 0
        print("[shim] wired limit disabled", file=sys.stderr)
    from mlx_lm.server import main as server_main
    sys.argv = ["mlx_lm.server"] + sys.argv[1:]
    server_main()

if __name__ == "__main__":
    main()

Patch at the mlx.core module level: mlx_lm shares that module object, so every call site is covered, including the one inside server.py that a narrower patch would miss. Confirm the [shim] line appears in your log. Do not assume it applied.

The symptom you are probably already living with

Long before it panics, this bug has a milder form that I tolerated for months without diagnosing: the further a session gets through a large context window, the more the whole Mac drags. Not just the model, everything.

Same root cause. The KV cache grows, wired allocations grow with it, and macOS cannot reclaim any of it, so the entire system starves. With the shim in place that memory is pageable instead: during a heavy session I watched free RAM dip to 44% and then recover to 86% once the task finished. That reclaim is exactly what wiring prevents.

If your Mac gets sluggish at high context, you are not "running a big model", you are on the road to a reboot.

7. Benchmarking: and the gotcha that almost fooled me

Same model family, same prompt, 200-token generation, steady-state, wall-clock:

Backend / build	Decode speed	Resident
MLX `gemma-4-26B-A4B-it-OptiQ-4bit` (isolated)	~73 tok/s	~17 GB
Ollama `gemma4:26b-a4b-it-q8_0`	~60 tok/s	~33 GB
MLX while a second model is also resident	~30-36 tok/s	(contended)

# Ollama: exact decode rate from the API (excludes prompt + load)
curl -s http://localhost:11434/api/generate -d '{
  "model":"gemma4:26b-a4b-it-q8_0","prompt":"Count from 1 to 100 slowly.",
  "stream":false,"think":false,"options":{"num_predict":200,"temperature":0}}' \
| python3 -c 'import json,sys;d=json.load(sys.stdin);print(round(d["eval_count"]/(d["eval_duration"]/1e9),1),"tok/s")'

# MLX: wall-clock over completion_tokens (short prompt ⇒ prompt time negligible)
time curl -s http://127.0.0.1:8080/v1/completions -d '{
  "model":"mlx-community/gemma-4-26B-A4B-it-qat-4bit",
  "prompt":"Count from 1 to 100 slowly.","max_tokens":200,"temperature":0}' >/dev/null

⚠️ The lesson: the first time I ran this, MLX clocked ~35 tok/s and I "concluded" Ollama was 1.7× faster. Wrong. The 33 GB Ollama model was still resident and the two were fighting over memory bandwidth. Unload everything but the model under test (ollama stop <model>), then measure. In its real deployed condition (only the MLX model resident) it runs faster and lighter.
[2026] I did it again, 15 months later. Upgrading MLX, I measured a 2.7x speedup: 30 tok/s before, 84 after. Wonderful, and entirely fake. A second model was resident during the "before" run and not the "after" run. With the GPU actually quiet, both versions do ~83 tok/s and the upgrade changed nothing. Knowing the trap is not the same as remembering to check for it, so build the check into the harness rather than into your intentions.

The tell is variance. A contaminated sweep gave 20.5, 36.6 and 72.4 tok/s for the same config. Wild run-to-run spread means contention, not noise. Clean runs are boring and tight, and if your numbers are not boring, stop and find out what else is resident.

7b. [2026] For a chat agent, tok/s is the wrong metric

I turned reasoning on for the agent because the benchmark said it was faster. It was, and the agent got dramatically worse to use.

With think: true and enable_thinking in the chat template, decode throughput went up: the model produces long reasoning traces at full speed, and a longer generation amortises fixed per-request overhead, so tok/s looks great. Measured on a real inbound WhatsApp message:

Reasoning	Time to reply	Tokens generated
off	0.73s	32
on	6.42s	529

Same question, same quality of answer, 8.8x longer to arrive. The model spent 497 extra tokens thinking about a message that needed one sentence.

Throughput measures how fast tokens come out. What a conversational agent actually lives or dies by is time to first useful reply, and reasoning trades the second for the first. For an agentic coding tool that ratio is often worth it. For someone texting you on WhatsApp, it is not.

If you want a middle ground, OpenClaw's thinkingDefault accepts off, minimal, low, medium, high, xhigh, adaptive, max. adaptive is the interesting one: reason only when the message looks like it needs it.

A related trap from the original §9: with reasoning on and a small max_tokens, the model can spend the entire budget thinking and return empty content. If you enable reasoning, raise the ceiling at the same time.

8. Bonus: fully-local voice (TTS + STT)

OpenClaw treats both as OpenAI-compatible/CLI endpoints, so no cloud and no keys.

TTS: OmniVoice on :17494 (`messages.tts`)

{
  "messages": {
    "tts": {
      "provider": "openai",
      "auto": "always",
      "providers": {
        "openai": {
          "enabled": true,
          "baseUrl": "http://127.0.0.1:17494/v1",
          "apiKey": "omnivoice-local",
          "model": "omnivoice",
          "voice": "female-young-pt"
        }
      }
    }
  }
}

(OmniVoice runs behind a small Python wrapper that exposes /v1/audio/speech, kept alive by its own LaunchAgent, same pattern as the STT server below.)

STT: mlx-whisper on :17495

~/whisper-server.py (FastAPI wrapper around mlx_whisper, loads whisper-large-v3-turbo once):

#!/usr/bin/env python3
"""HTTP wrapper for mlx-whisper STT. Loads model once, serves many requests."""
import os, tempfile, time
from contextlib import asynccontextmanager
import mlx_whisper
from fastapi import FastAPI, File, Form, UploadFile
from fastapi.responses import JSONResponse

PORT = int(os.environ.get("WHISPER_PORT", "17495"))
MODEL_REPO = os.environ.get("WHISPER_MODEL", "mlx-community/whisper-large-v3-turbo")

@asynccontextmanager
async def lifespan(_app: FastAPI):
    # warm the model on a moment of silence so the first real request is fast
    silent = os.path.join(tempfile.gettempdir(), "whisper-warmup.wav")
    if not os.path.exists(silent):
        import wave
        with wave.open(silent, "wb") as w:
            w.setnchannels(1); w.setsampwidth(2); w.setframerate(16000)
            w.writeframes(b"\x00\x00" * 16000)
    mlx_whisper.transcribe(silent, path_or_hf_repo=MODEL_REPO)
    yield

app = FastAPI(lifespan=lifespan)

@app.get("/health")
async def health(): return {"status": "ok", "model": MODEL_REPO, "port": PORT}

@app.post("/transcribe")
async def transcribe(file: UploadFile = File(...), language: str | None = Form(None), model: str | None = Form(None)):
    started = time.time()
    suffix = os.path.splitext(file.filename or "")[1] or ".wav"
    with tempfile.NamedTemporaryFile(suffix=suffix, delete=False) as tmp:
        tmp.write(await file.read()); path = tmp.name
    try:
        kw = {"path_or_hf_repo": model or MODEL_REPO}
        if language and language.lower() not in ("auto", ""): kw["language"] = language
        r = mlx_whisper.transcribe(path, **kw)
        return JSONResponse({"text": (r.get("text") or "").strip(),
                             "language": r.get("language"),
                             "duration": round(time.time() - started, 3)})
    finally:
        try: os.unlink(path)
        except OSError: pass

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="127.0.0.1", port=PORT, log_level="info")

Install + LaunchAgent (~/Library/LaunchAgents/com.mlx-whisper.server.plist):

~/mlx-env/bin/pip install mlx-whisper fastapi 'uvicorn[standard]' python-multipart

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
  <key>Label</key><string>com.mlx-whisper.server</string>
  <key>ProgramArguments</key>
  <array><string>/Users/you/mlx-env/bin/python3</string><string>/Users/you/whisper-server.py</string></array>
  <key>EnvironmentVariables</key>
  <dict>
    <key>PATH</key><string>/opt/homebrew/bin:/usr/bin:/bin</string>   <!-- mlx-whisper shells out to ffmpeg -->
    <key>WHISPER_PORT</key><string>17495</string>
    <key>WHISPER_MODEL</key><string>mlx-community/whisper-large-v3-turbo</string>
  </dict>
  <key>RunAtLoad</key><true/><key>KeepAlive</key><true/>
  <key>StandardOutPath</key><string>/tmp/mlx-whisper-server.log</string>
  <key>StandardErrorPath</key><string>/tmp/mlx-whisper-server.log</string>
</dict></plist>

Hook it into OpenClaw as the audio model (tools.media), transcode to WAV, POST, return text:

{
  "tools": {
    "media": {
      "audio": { "enabled": true },
      "models": [{
        "type": "cli", "command": "bash", "provider": "mlx-whisper", "model": "whisper-large-v3-turbo",
        "capabilities": ["audio"],
        "args": ["-c",
          "TMP=$(mktemp -t stt) && /opt/homebrew/bin/ffmpeg -y -i \"$1\" -f wav \"$TMP\" 2>/dev/null && curl -s -X POST http://localhost:17495/transcribe -F \"file=@$TMP;filename=stt.wav;type=audio/wav\" -F 'language=pt' | python3 -c \"import json,sys;d=json.load(sys.stdin);print(d.get('text',''))\" ; rm -f \"$TMP\"",
          "--"]
      }]
    }
  }
}

Tip: whisper-base hallucinates a trailing phantom phrase on short clips. whisper-large-v3-turbo (≈809M, ~1.5 GB) fixes it at ~0.3 s/clip on an M3 Ultra.

8b. [2026] Voice delivery: the duplicate-message trap

Adding TTS created a bug that took several rounds to pin down. Asking the agent for a voice note produced the audio and a text message repeating the same thing. Once, four identical audios in three seconds.

The structural cause is worth knowing before you build on this: OpenClaw implements "voice message replaces text" only for Feishu. There is a shouldSuppressFeishuTextForVoiceMedia() helper, referenced solely from the Feishu path. WhatsApp has no equivalent. If a turn produces both text and media, WhatsApp delivers both. There is no config switch for this, so it can only be fixed at the instruction level.

Three separate paths produced the same symptom, and all three had to be closed:

The final turn text. The agent sent the audio, then ended the turn with a transcription. The rule has to be absolute: after sending media, end the turn with empty text. My first attempt allowed "one short, different sentence" and it used that latitude to send the transcription anyway. Leave zero escape hatches.
The progress update. With path 1 closed, the order flipped: text first, audio six seconds later. It had switched to sending "one moment, recording that now", which its own instructions permitted. Progress updates need scoping to genuinely long work (multi-step research, browser automation), never to generating a file.
Session history. The 4x audio loop was not fixed by instructions at all. The session store contained the looping pattern and the model was imitating its own recent behaviour. Clearing it fixed it. Note there is no openclaw sessions delete command: stop the gateway first (it rewrites stores from memory on shutdown), then write {} to ~/.openclaw/agents/<id>/sessions/sessions.json.

The general lesson: when an agent misbehaves repeatedly, check whether it is copying itself from history before you keep rewriting the prompt.

8c. [2026] Memory search needs an embedding model, and it fails hard

If you enable memorySearch, OpenClaw embeds the query before searching, so it needs a live embedding endpoint:

{ "agents": { "defaults": { "memorySearch": {
  "provider": "ollama", "model": "nomic-embed-text" } } } }

I had this pointed at a GUI app's bundled embedder, and it was silently broken for weeks. The failure mode is what makes this worth writing down: I assumed it would degrade to keyword search, because the index is hybrid and carries both FTS and vector paths. It does not. The query path embeds first, so a dead embedder kills the entire search:

Memory search failed: fetch failed | connect ECONNREFUSED 127.0.0.1:1234

Ollama is the right home for this. It is already in this stack, it autostarts as a LaunchAgent, and nomic-embed-text is only 274 MB:

ollama pull nomic-embed-text
openclaw memory status --deep                       # "Embeddings: ready"
openclaw memory status --index --agent private      # reindex after changing model
openclaw memory status --index --agent public

Two things to watch. Reindex after any embedder change, since the stored index records which model built it and vector search pauses until rebuilt. And check dimensions match before switching: both nomic v1.5 builds are 768-dim, so my existing vectors stayed compatible.

Verify semantically, not with a keyword you know is present. Query something whose words do not appear in the target text and confirm you still get the right chunk back.

9. Gotchas worth knowing

Co-residency throttles throughput (see §7). Keep one big model resident; ollama stop <model> frees it instantly.
Slower model ⇒ blown cron timeouts. A research-heavy scheduled job that finished on a fast quant can time out on a slower one. Bump the job's timeoutSeconds, or give the cron its own faster model.
Stale "typing…" indicator. If WhatsApp's connection drops mid-turn, the "composing" presence may never get its "stop." It's cosmetic; an openclaw gateway restart clears it.
Reasoning models eat your token budget. With think: true and a tiny max_tokens, the model can spend the whole budget "thinking" and return empty content. Give it headroom.
[2026] Your Mac reboots under load. Not your fault, not your RAM. See §6b, and disable set_wired_limit.
[2026] Rapid gateway restarts revoke your WhatsApp session. I restarted the gateway roughly seven times in an hour while iterating on instructions. WhatsApp treated it as abuse and invalidated the linked device: [whatsapp] auto-restart skipped, terminal disconnect. The agent went fully offline until I re-scanned a QR. Batch your config edits and restart once. Worse, openclaw channels status still reports linked, because the local credentials survive and only the server side revoked them. Check running, connected and health: instead.
[2026] Scheduled jobs confabulate. A cron that asks for facts ("summarise today's crypto news") will happily invent them if the prompt does not name the tool it must use and demand citations. Verify against the gateway log rather than trusting the message you receive. Anything a cron reports that you cannot trace to a tool call did not happen.
[2026] An agent that "tested" something probably did not. Mine confidently reported benchmarks it structurally could not run. Check claimed actions against the tools the agent actually has.
A cloud fallback can stay configured (e.g. an OpenAI/Anthropic OAuth profile) without ever spending API credits, just don't make it primary. OAuth access tokens expire and re-auth is interactive, so don't rely on it for unattended jobs.

10. Service map (what autostarts)

LaunchAgent	Purpose	Port
`ai.openclaw.gateway`	the agent gateway	18789 (loopback)
`homebrew.mxcl.ollama`	Ollama backend	11434
`com.ollama.warmup`	pre-warm Ollama model (optional; [2026]* disabled here, see §5)*	-
`com.mlx-lm.server`	MLX LLM (OpenAI-compatible)	8080
`com.mlx-whisper.server`	STT	17495
`com.omnivoice.server`	TTS	17494
[2026] `com.fm-vision.server`	local vision shim (image input)	17496

launchctl list | grep -E 'openclaw|ollama|mlx|omnivoice|fm-vision'

[2026] Verify reboot-resilience properly. launchctl list shows what is loaded now. What survives a reboot is RunAtLoad plus not being disabled, and those are different things. Check both:
plutil -extract RunAtLoad raw ~/Library/LaunchAgents/<label>.plist
launchctl print-disabled gui/$(id -u) | grep <label>
A service can be running happily today and still be => disabled, in which case it will not come back after a reboot.

Built and debugged interactively with Claude Code. Numbers are from a single M3 Ultra (96 GB); your mileage will vary with chip, RAM, and quant. Updated August 2026 after three months of continuous 24/7 operation, which is where most of the [2026] material came from: none of it showed up in the first week.

Running Local LLMs as Your AI Coding Assistant on Apple Silicon

Bruno Mello — Sat, 07 Feb 2026 04:25:48 +0000

Updated August 2026. The original version of this post ran Qwen3-Coder-30B on mlx_lm.server under mlx-lm 0.30.6. Eighteen months of daily use later, the architecture still holds, but almost everything I learned since then is about the things that only show up after you run this setup continuously: a still-unfixed Apple GPU driver bug, why your Mac gets slower as context fills, why benchmarks lie to you, and why the "easy" GUI option is not actually faster. Original content is preserved below; new material is marked [2026].

Tags: #llm #ai #programming #apple

What We're Building

The objective is to operate "OpenCode", an open-source agentic coding CLI, powered by a large language model running entirely on your Mac, without cloud dependencies, API costs, or external data transmission.

The Architecture (Three Layers)

Three interconnected components work together:

OpenCode (The App) - User interface that sends messages and executes tools like file reading and code editing
mlx_lm.server (The Hub) - Middleware translating between OpenCode and the model via HTTP on localhost:8080
The LLM (The Brain) - Neural network processing numerical representations of text

Key Concepts

Tokens are numerical representations of text. Models process these numbers rather than raw text, with tokenizers handling conversion between text and token sequences.

Tool Calls represent structured requests where the model cannot directly access files but can request actions. The model outputs a tool call, the application executes it, and results feed back to the model for continued reasoning.

Tool Parsers interpret model-specific output formats. Different architectures use different structures (XML tags, JSON, special tokens), requiring appropriate parsers for proper function.

What Is MLX?

MLX is Apple's machine learning framework optimized for Apple Silicon. The key advantage: unified memory architecture allows the CPU and GPU to share RAM, enabling efficient model loading without copying overhead.

mlx-lm provides model loading, inference, and an OpenAI-compatible HTTP server through mlx_lm.server.

[2026] The Bug Nobody Warns You About: Kernel Panics from `set_wired_limit`

This is the most important thing in this post. If you take one thing away, take this.

If you run mlx_lm.server as a long-lived service, your Mac can kernel panic and reboot. Not crash the Python process: reboot the entire machine. The signature is:

panic(cpu 8 caller 0x...): "completeMemory() prepare count underflow" @IOGPUMemory.cpp:550
Kernel Extensions in backtrace:
   com.apple.iokit.IOGPUFamily(...)

This is tracked as ml-explore/mlx#3186, open since March 2026. As of late August 2026 it is still unfixed, with no response from Apple or the MLX maintainers across the entire thread. It has been reproduced on M1 Ultra, M3 Ultra, M4 base, M4 Pro, M4 Max and M5, from 32GB to 256GB, so it is not a "you bought the wrong Mac" problem.

What actually triggers it

The valuable work in that thread is a controlled single-variable isolation by a commenter running ten fresh-server soak runs per arm (full report and shim). The findings overturned the obvious assumption:

It is not prefill size. A faithful replay of a crashing session, sequential requests with prefills up to 23k tokens, ran 111 minutes clean.
It is concurrency plus prompt-cache eviction churn. Add two concurrent request streams and unique prompts overflowing --prompt-cache-bytes, and it panics in 102 to 108 seconds, 3 out of 3 cold boots.
It is not OOM. Genuine memory exhaustion produces a clean userspace Metal error. The panic happens with headroom to spare.

The one mitigation that works

mlx_lm.server calls mx.set_wired_limit(...) at startup, raising the wired memory ceiling to roughly 75% of RAM. Wired pages cannot be reclaimed by the OS. That single call is the discriminating factor:

Change (one variable per arm)	Result
none (stock)	panic x3 @ 102 to 108s
`set_wired_limit` never called	no panic, 10/10 runs, 4.2h
+ also no periodic `clear_cache`	no panic, 10/10 runs, 5.0h
only skip periodic `clear_cache`	panic @ 99.9s
`sysctl iogpu.disable_wired_collector=1` + metal4 async mapping off	panic @ 109.6s
RotatingKVCache / lower wired cap / `MLX_MAX_OPS_PER_BUFFER`	die earlier as clean userspace OOM

Note what fails: the sysctl workarounds you will find on forums, and bounding the KV cache. Also worth knowing, --max-kv-size is silently ignored for architectures that define make_cache (the Qwen3.5 hybrids, for example).

The cost of the fix is about 3% sequential throughput, because weights become pageable.

Implementing it

There is no upstream opt-out. As of mlx-lm 0.31.3, mx.set_wired_limit is still called unconditionally in seven places, including mlx_lm/server.py. So you wrap the server in a launcher that no-ops it before importing:

#!/usr/bin/env python3
"""Launcher that disables set_wired_limit before mlx_lm.server starts.
Mitigation for ml-explore/mlx#3186."""
import os, sys
import mlx.core as mx

def main():
    if os.environ.get("MLX_LM_DISABLE_WIRED_LIMIT") == "1":
        mx.set_wired_limit = lambda *a, **k: 0
        print("[shim] wired limit disabled", file=sys.stderr)
    from mlx_lm.server import main as server_main
    sys.argv = ["mlx_lm.server"] + sys.argv[1:]
    server_main()

if __name__ == "__main__":
    main()

Patching at the mlx.core module level matters: mlx_lm shares that module object, so every call site is covered, including the one inside server.py that a narrower patch would miss.

Run it with MLX_LM_DISABLE_WIRED_LIMIT=1 and confirm the [shim] line appears in your logs. Do not assume it applied.

[2026] Why Your Mac Gets Laggy as Context Fills

I lived with this for months without diagnosing it: the further a coding session got through a large context window, the more the whole machine dragged. Not just the model, everything.

It is the same wired memory, viewed from a different angle. As the KV cache grows, wired allocations grow with it, and wired pages cannot be reclaimed by macOS. So instead of the OS paging something out, your entire system starves.

With the shim in place, the same memory is pageable. Under a heavy multi-compaction session I watched free RAM dip to 44% and then recover to 86% once the task finished. That reclaim is exactly what wiring prevents. Same workload, no lag.

This is worth internalizing: that lag is not "the model is big", it is the early stage of the same unreclaimable-memory pathology that ends in the kernel panic above. If your Mac gets sluggish at high context, you are on the road to a reboot.

Setting Up

Prerequisites

Apple Silicon Mac (M1 or newer)
Minimum 32GB RAM (64GB+ recommended)
macOS with Homebrew

Installation Steps

Step 1: Install Python 3.12

brew install python@3.12

Step 2: Create Virtual Environment

/opt/homebrew/bin/python3.12 -m venv ~/mlx-env
source ~/mlx-env/bin/activate
pip install --upgrade pip
pip install mlx-lm

Step 3: Install OpenCode

npm install -g opencode-ai
# or
brew install anomalyco/tap/opencode

Step 4: Start Model Server

source ~/mlx-env/bin/activate
mlx_lm.server --model mlx-community/Qwen3-Coder-30B-A3B-Instruct-8bit --port 8080

Step 5: Configure OpenCode

Create ~/.config/opencode/opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "mlx": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "MLX (local)",
      "options": {
        "baseURL": "http://localhost:8080/v1"
      },
      "models": {
        "mlx-community/Qwen3-Coder-30B-A3B-Instruct-8bit": {
          "name": "Qwen3-Coder 30B 8bit",
          "tools": { "task": true }
        }
      }
    }
  }
}

[2026] Running It as a Real Service

Step 4 above is fine for trying things out, but a terminal-bound server dies with your terminal. Here is the LaunchAgent I actually use, at ~/Library/LaunchAgents/com.mlx-lm.coder.plist, wired through the shim:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.mlx-lm.coder</string>
    <key>ProgramArguments</key>
    <array>
        <string>/Users/you/mlx-env/bin/python3</string>
        <string>/Users/you/mlx-env/mlx_server_shim.py</string>
        <string>--model</string>
        <string>/Users/you/mlx-models/Qwen3.8-27B-MLX-4bit</string>
        <string>--port</string>
        <string>8081</string>
        <string>--chat-template-args</string>
        <string>{"enable_thinking": true}</string>
        <string>--prompt-cache-size</string>
        <string>4</string>
        <string>--prompt-cache-bytes</string>
        <string>8000000000</string>
    </array>
    <key>EnvironmentVariables</key>
    <dict>
        <key>MLX_LM_DISABLE_WIRED_LIMIT</key>
        <string>1</string>
    </dict>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>StandardOutPath</key>
    <string>/tmp/mlx-lm-coder.log</string>
    <key>StandardErrorPath</key>
    <string>/tmp/mlx-lm-coder.log</string>
</dict>
</plist>

[2026] The launchd Trap That Cost Me an Hour

launchctl kickstart -k restarts the process but reuses launchd's cached job definition. Your edited plist is silently ignored.

The service comes back, the port answers, the logs look healthy, and your change did nothing. I edited that plist three times (added a token cap, changed concurrency, repointed --model to a new path) and none of it applied. I only noticed when the server failed to load after I moved the weights and the log showed it still opening the old path.

After any plist edit:

launchctl bootout gui/$(id -u)/com.mlx-lm.coder
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.mlx-lm.coder.plist

Then verify against the running process, never the file:

ps -o args= -p $(pgrep -f mlx_server_shim.py) | tr ' ' '\n'

A corollary, since KeepAlive respawns a crashed server transparently: when you health-check a long-running service, compare the PID, not whether the port responds. A crash and respawn looks identical to healthy uptime from the outside.

[2026] Do Not Trust Your Own Benchmarks

I nearly published a completely fabricated result, and the mistake is easy to make.

I upgraded MLX, benchmarked, and measured a 2.7x speedup: 30 tok/s before, 84 tok/s after. Wonderful. Except a second model was resident on the GPU during the "before" run and not during the "after" run. With the GPU actually quiet, both versions do about 83 tok/s. The upgrade changed nothing.

On unified memory, a co-resident model does not just compete for RAM, it competes for memory bandwidth, which is the binding constraint for decode. That is enough to halve your numbers.

The tell is variance. A contaminated sweep produced 20.5, 36.6 and 72.4 tok/s for the same config. Wild run-to-run spread means contention, not noise. Clean runs are boring and tight.

So: before trusting any local LLM timing, confirm nothing else holds a model.

pgrep -fl mlx_server_shim.py   # or your equivalent

And build the check into the harness so it aborts rather than producing a plausible wrong number.

[2026] Reading MLX Releases: Does This Commit Help You?

MLX ships fast, and release notes look exciting. Most of it will not touch your workload. A concrete example from 0.32.2:

"Read each K/V byte once in gqa-8 decode attention" looked like a direct win for my chat model. It does nothing for it. That path targets a GQA ratio of 8, and Gemma 4 26B-A4B is 16 attention heads over 8 KV heads, a ratio of 2. It never enters the optimized path.
"Fused full-attention path for head_dim 256" was even more tempting, since that model does have head_dim: 256. But it is gated to NAX devices, and an M3 Ultra reports applegpu_g15d.

Measured before and after on a quiet GPU: identical within noise.

The practical habit is to check the commit against your model's actual config.json before getting excited:

python3 -c "
import json; c=json.load(open('config.json')); t=c.get('text_config',c)
h,kv=t['num_attention_heads'],t['num_key_value_heads']
print('heads',h,'kv',kv,'GQA ratio',h//kv,'head_dim',t.get('head_dim'))"

None of this means "skip the upgrade". I kept 0.32.2 because it is a large correctness release with no regression. It just is not a speedup, and I would rather say so than repeat a changelog.

[2026] LM Studio vs Direct `mlx_lm.server`

I ran the coder through LM Studio for a while, because JIT loading with an idle TTL is genuinely convenient. Then I moved it back to a direct mlx_lm.server. The measured difference on an identical agentic task:

Backend	Same task, end to end
Direct `mlx_lm.server`	63.7s
LM Studio (MLX engine)	65.1s

A wash, and it should be: LM Studio's Apple Silicon engine is mlx-lm underneath. Same model, same quant, same speed. Anyone telling you one is dramatically faster than the other is measuring something else, probably contention (see above).

The real differences are operational:

Direct mlx_lm.server wins on:

Exact version control over mlx and mlx-lm
The wired-limit shim, which is the whole ballgame. LM Studio raises the wired limit and gives you no way to stop it
No cold start, since the model never idles out
Prompt cache that persists across sessions

LM Studio wins on:

JIT load plus idle TTL, so the model leaves RAM when unused. mlx_lm.server has no TTL and holds the model forever
A GUI, per-model load profiles, and reasoningEffort in the API

That last point matters if you migrate: reasoningEffort is an LM Studio API extension. mlx_lm.server ignores it. Set reasoning server-side instead, via --chat-template-args '{"enable_thinking": true}'. Check your model's chat_template.jinja for what it accepts. Qwen3.8 takes reasoning_effort of xhigh, medium or low, and raises on anything else.

My conclusion: use LM Studio to explore models, run production through mlx_lm.server with the shim.

[2026] OpenCode Harness Gotchas

Two behaviors that cost me real debugging time, because both look exactly like the model hanging:

opencode run blocks forever at init waiting on a permission prompt when there is no TTY. It never sends a single request to your server. Pass --auto.
It ignores the process working directory, resolving the project to $HOME and then failing its file picker with "Can not run certain FFF features in a file system root or home directories". Pass --dir <path>.

Both hangs reproduce identically against any backend, so a stall here is never evidence about your model or your server. Check whether a request actually arrived before you blame inference:

grep -c "POST /v1/chat/completions" /tmp/mlx-lm-coder.log

The Model Hunt

(Original 2025 content, preserved. Current recommendation follows.)

Attempt 1: GLM-4.7-Flash-4bit

Failed because general-purpose models lack the precision needed for agentic coding workflows, producing repetitive loops on complex tasks.

Attempt 2: Qwen2.5-72B-Instruct-4bit

Downloaded successfully (~40GB) but processing took 18 minutes for modest context, causing connection crashes. Dense models activate every parameter per token, making them prohibitively slow despite fitting in RAM.

Attempt 3: Devstral-Small-2-24B-8bit

Server warning stated: "Received tools but model does not support tool calling." No available parser existed for this architecture in mlx-lm v0.30.6.

Attempt 4: Qwen3-Coder-30B-A3B-Instruct-8bit (Winner)

Successfully operated with correct tool parsing, strong instruction-following, and adequate speed. Uses approximately 33GB RAM. Mixture of Experts architecture activates only 3B of 30B parameters per token, achieving 3B-model inference speed with 30B-model knowledge.

Attempt 5: Qwen3-Coder-Next-4bit (Winner After Bug Fix)

The crash revealed a critical debugging lesson. The model shipped with "tool_parser_type": "json_tools" in its config, but outputs XML-formatted tool calls. The parser attempted JSON parsing on XML content, causing immediate failure.

The Fix: Manually edit the cached tokenizer_config.json:

find ~/.cache/huggingface/hub/models--mlx-community--Qwen3-Coder-Next-4bit \
  -name "tokenizer_config.json"

Change "json_tools" to "qwen3_coder". After restart, tool calling functions properly.

[2026] Current Daily Driver: Qwen3.8-27B MLX 4-bit

Dense 27B, 256k native context, and it loads text-only under mlx_lm despite shipping a vision config (model_type: qwen3_5, with a text_config block). Roughly 15GB on disk, about 34 tok/s decode on an M3 Ultra.

Yes, dense, after I spent 2025 preaching MoE. The nuance I would add now: MoE still wins decisively on throughput per parameter, but a 27B dense model at 4-bit is fast enough for interactive agentic work while being noticeably more reliable at tool calling and long-horizon edits. For comparison, on the same machine a 26B-A4B MoE chat model runs about 83 tok/s, roughly 2.4x the tokens per second, and it is the right pick for conversation. For coding I will trade tok/s for fewer wrong turns.

Measured in real use: multiple compactions in a single session, prefills up to 146,838 tokens, 324 requests served, zero crashes. That last number is only possible because of the wired-limit shim. The original issue report in mlx#3186 was a panic on a ~173k-token prefill, which is the same territory.

Quantization Note

The community now publishes many quants of the same model at nearly the same size, and they are not equivalent. Measured against the bf16 original by top-1 token agreement, at ~16GB the various 4-bit builds land around 89%, while a 5.00bpw DWQ build at 17.7GB reaches ~91.5%, and 6-bit at 22.8GB reaches ~96.6%. If you have the RAM headroom, going one step up in bits buys more fidelity than switching between same-size 4-bit builds.

Treat vendor-published agreement charts with mild suspicion, though: DWQ methods are trained to reproduce the parent model's outputs, so "agreement with bf16" is a metric they optimize for directly. It is a fair number, not a neutral one.

Model Comparison Table

Model	Type	RAM	Speed	Tool Calls	Verdict
GLM-4.7-Flash-4bit	MoE	~16GB	Fast	Yes	Loops on complex tasks
Qwen2.5-72B-4bit	Dense	~42GB	Very slow	Untested	Too slow for interactive use
Devstral-24B-8bit	Dense	~25GB	Medium	No (no parser)	Dead on arrival
Qwen3-Coder-30B-8bit	MoE	~33GB	Fast	Yes	Good 2025 pick
Qwen3-Coder-Next-4bit	MoE	~50GB	Fast	Yes (after fix)	Best quality (needs 64GB+ RAM)
Qwen3.8-27B-MLX-4bit	Dense	~15GB	~34 tok/s	Yes	[2026] Current daily driver

Tool Call Flow

Complete sequence from user input to response:

User enters query in OpenCode
HTTP POST sends message and available tools to localhost:8080
Server tokenizes message to token IDs
Model runs inference on GPU
Parser (e.g., qwen3_coder) converts XML output to structured JSON
OpenCode receives parsed tool call and executes it
Model receives execution results and generates natural language response

RAM and Model Selection Guidance

RAM Requirements:

16GB: Small models only (7B-4bit), tight margins
32GB: 7B-8bit or 13B-4bit comfortably
64GB: Qwen3-Coder-30B-8bit (optimal)
96GB+: Qwen3-Coder-Next-4bit or multiple models

Keep models under 75% of total RAM for system overhead.

[2026] A correction to my own advice above: "keep models under 75% of RAM" is where the danger actually lives, because ~75% is precisely what set_wired_limit claims by default. The rule is not wrong, but it is not the safety margin I implied. With wiring left enabled you can sit comfortably inside that budget and still panic the kernel, because the problem is unreclaimable memory, not insufficient memory. Disable the wired limit and the same budget behaves the way you expect.

MoE vs Dense Models:
Mixture of Experts architectures activate only a subset of parameters per token, providing speed comparable to smaller dense models while maintaining quality approaching their full parameter count. For local inference, MoE models deliver superior speed-to-quality ratios.

Pre-Download Checklist

Verify mlx-lm has a tool parser for the architecture
Confirm it's a coding-focused model
Prefer MoE over dense architectures for interactive use
Check quantization level against available RAM
Validate tool_parser_type in the model's tokenizer_config.json matches actual output format
[2026] Check num_attention_heads / num_key_value_heads if you care about specific kernel optimizations
[2026] If it ships a vision config, confirm mlx_lm loads it text-only before committing to the download

Debugging Tips

Looping responses: Model may lack agentic coding capability; try alternatives
"Model does not support tool calling": No parser available for this architecture in current mlx-lm version
JSON decode errors on tool calls: Parser mismatch; verify tool_parser_type in config
Very slow responses: Dense model too large; switch to MoE or more quantized variant
Connection failures: Ensure server runs on correct port matching OpenCode configuration
[2026] Machine reboots under load: Not your fault. mlx#3186. Disable set_wired_limit
[2026] Whole Mac lags as context fills: Same root cause. Wired memory cannot be reclaimed
[2026] Config change had no effect: If it is a LaunchAgent, kickstart ignored your plist edit. Use bootout plus bootstrap
[2026] Benchmark numbers swing wildly: Another model is resident. Check before believing anything
[2026] opencode run hangs with no server request: Missing --auto or --dir, not a model problem

Conclusion

Running local AI coding assistants on Apple Silicon is practical and achievable, and eighteen months on I still use this daily instead of a cloud assistant.

What changed is my sense of where the difficulty lives. In 2025 I thought the hard part was model selection and tool parsers. Those are solved problems now. The hard part is everything that only appears once the thing runs continuously: an Apple driver bug nobody has fixed, memory that the OS cannot reclaim, benchmarks that lie when a second model is resident, and service tooling that silently ignores your edits.

None of that is in a quickstart guide, which is why it is in this one.

Resources

MLX on GitHub
mlx-lm on GitHub
OpenCode
MLX Community on HuggingFace
[2026] mlx#3186: the IOGPU kernel panic
[2026] Trigger isolation and launcher shim

Unidirectional Data Flow Architecture in SwiftUI

Bruno Mello — Fri, 26 Dec 2025 00:57:42 +0000

Building scalable SwiftUI applications requires a predictable state management pattern. This article presents a Unidirectional Data Flow (UDF) architecture that enforces a single direction for data: User Action → Event → Action → State Change → View Update.

The Problem

Traditional iOS architectures often suffer from:

Scattered state across multiple @Published properties
Testing challenges due to tight coupling between layers
Unpredictable updates when multiple sources modify the same state
Difficult debugging when state changes come from multiple directions

The Solution: Unidirectional Data Flow

┌─────────────────────────────────────────────────────────────┐
│  View (SwiftUI)                                             │
│  @ObservedObject viewModel                                  │
└─────────────────────────────────────────────────────────────┘
       │ handle(event: ViewEvent)           ↑ @Published viewState
       ↓                                    │
┌─────────────────────────────────────────────────────────────┐
│  ViewModel (CombineViewModel)                               │
│  - EventActionMap: ViewEvent → DomainAction                 │
│  - Reducer: DomainState → ViewState                         │
│  - SINGLE @Published viewState (one source of truth)        │
└─────────────────────────────────────────────────────────────┘
       │ dispatch(action:)                  ↑ statePublisher
       ↓                                    │
┌─────────────────────────────────────────────────────────────┐
│  DomainInteractor (CombineInteractor)                       │
│  - CurrentValueSubject<DomainState, Never>                  │
│  - Business logic + state mutations                         │
└─────────────────────────────────────────────────────────────┘
       │                                    ↑
       ↓                                    │
┌─────────────────────────────────────────────────────────────┐
│  Service Layer (Protocol-based)                             │
│  - Network, Storage, etc.                                   │
└─────────────────────────────────────────────────────────────┘

Four Core Protocols

1. CombineViewModel

The ViewModel protocol enforces a single source of truth with one @Published property:

protocol CombineViewModel: ObservableObject {
    associatedtype ViewState: Equatable
    associatedtype ViewEvent

    var viewState: ViewState { get }
    func handle(event: ViewEvent)
}

Key principle: One @Published viewState property contains ALL UI state. This eliminates scattered state and makes the view's data source predictable.

2. CombineInteractor

The DomainInteractor handles business logic and state management:

protocol CombineInteractor {
    associatedtype DomainState: Equatable
    associatedtype DomainAction

    var statePublisher: AnyPublisher<DomainState, Never> { get }
    func dispatch(action: DomainAction)
}

3. ViewStateReducing

Pure function that transforms domain state to view state:

protocol ViewStateReducing {
    associatedtype DomainState
    associatedtype ViewState

    func reduce(domainState: DomainState) -> ViewState
}

4. DomainEventActionMap

Maps UI events to business actions:

protocol DomainEventActionMap {
    associatedtype ViewEvent
    associatedtype DomainAction

    func map(event: ViewEvent) -> DomainAction?
}

Complete Implementation Example

Let's build a User List feature to demonstrate the architecture.

Step 1: Define View Events

Events represent user interactions:

enum UserListViewEvent: Equatable {
    case onAppear
    case onRefresh
    case onUserTapped(userId: String)
    case onUserNavigated
}

Step 2: Define Domain Actions

Actions represent business operations:

enum UserListDomainAction: Equatable {
    case loadUsers
    case refreshUsers
    case selectUser(userId: String)
    case clearSelectedUser
}

Step 3: Define Domain State

Business state with all relevant data:

struct UserListDomainState: Equatable {
    var users: [User]
    var isLoading: Bool
    var isRefreshing: Bool
    var error: String?
    var hasLoadedInitialData: Bool
    var selectedUser: User?

    static var initial: UserListDomainState {
        UserListDomainState(
            users: [],
            isLoading: false,
            isRefreshing: false,
            error: nil,
            hasLoadedInitialData: false,
            selectedUser: nil
        )
    }
}

Step 4: Define View State

UI-optimized state derived from domain state:

struct UserListViewState: Equatable {
    var users: [UserViewItem]
    var isLoading: Bool
    var isRefreshing: Bool
    var errorMessage: String?
    var showEmptyState: Bool
    var selectedUser: User?

    static var initial: UserListViewState {
        UserListViewState(
            users: [],
            isLoading: false,
            isRefreshing: false,
            errorMessage: nil,
            showEmptyState: false,
            selectedUser: nil
        )
    }
}

struct UserViewItem: Identifiable, Equatable {
    let id: String
    let displayName: String
    let avatarURL: URL?

    init(from user: User) {
        id = user.id
        displayName = "\(user.firstName) \(user.lastName)"
        avatarURL = URL(string: user.avatarUrl)
    }
}

Step 5: Event to Action Mapping

struct UserListEventActionMap: DomainEventActionMap {
    func map(event: UserListViewEvent) -> UserListDomainAction? {
        switch event {
        case .onAppear:
            return .loadUsers
        case .onRefresh:
            return .refreshUsers
        case let .onUserTapped(userId):
            return .selectUser(userId: userId)
        case .onUserNavigated:
            return .clearSelectedUser
        }
    }
}

Step 6: View State Reducer

Pure transformation from domain to view state:

struct UserListViewStateReducer: ViewStateReducing {
    func reduce(domainState: UserListDomainState) -> UserListViewState {
        UserListViewState(
            users: domainState.users.map { UserViewItem(from: $0) },
            isLoading: domainState.isLoading,
            isRefreshing: domainState.isRefreshing,
            errorMessage: domainState.error,
            showEmptyState: !domainState.isLoading
                && !domainState.isRefreshing
                && domainState.users.isEmpty
                && domainState.hasLoadedInitialData,
            selectedUser: domainState.selectedUser
        )
    }
}

Step 7: Domain Interactor

Note the naming convention: *DomainInteractor:

final class UserListDomainInteractor: CombineInteractor {
    typealias DomainState = UserListDomainState
    typealias DomainAction = UserListDomainAction

    private let userService: UserService
    private let stateSubject = CurrentValueSubject<UserListDomainState, Never>(.initial)
    private var cancellables = Set<AnyCancellable>()

    var statePublisher: AnyPublisher<UserListDomainState, Never> {
        stateSubject.eraseToAnyPublisher()
    }

    var currentState: UserListDomainState {
        stateSubject.value
    }

    init(serviceLocator: ServiceLocator) {
        self.userService = try! serviceLocator.retrieve(UserService.self)
    }

    func dispatch(action: UserListDomainAction) {
        switch action {
        case .loadUsers:
            loadUsers()
        case .refreshUsers:
            refreshUsers()
        case let .selectUser(userId):
            selectUser(userId: userId)
        case .clearSelectedUser:
            clearSelectedUser()
        }
    }

    private func loadUsers() {
        guard !currentState.isLoading, !currentState.hasLoadedInitialData else { return }

        updateState { state in
            state.isLoading = true
            state.error = nil
        }

        userService.fetchUsers()
            .sink { [weak self] completion in
                if case let .failure(error) = completion {
                    self?.updateState { state in
                        state.isLoading = false
                        state.error = error.localizedDescription
                    }
                }
            } receiveValue: { [weak self] users in
                self?.updateState { state in
                    state.users = users
                    state.isLoading = false
                    state.hasLoadedInitialData = true
                }
            }
            .store(in: &cancellables)
    }

    private func refreshUsers() {
        updateState { state in
            state.isRefreshing = true
            state.error = nil
        }

        userService.fetchUsers()
            .sink { [weak self] completion in
                if case let .failure(error) = completion {
                    self?.updateState { state in
                        state.isRefreshing = false
                        state.error = error.localizedDescription
                    }
                }
            } receiveValue: { [weak self] users in
                self?.updateState { state in
                    state.users = users
                    state.isRefreshing = false
                }
            }
            .store(in: &cancellables)
    }

    private func selectUser(userId: String) {
        guard let user = currentState.users.first(where: { $0.id == userId }) else { return }
        updateState { state in
            state.selectedUser = user
        }
    }

    private func clearSelectedUser() {
        updateState { state in
            state.selectedUser = nil
        }
    }

    private func updateState(_ transform: (inout UserListDomainState) -> Void) {
        var state = stateSubject.value
        transform(&state)
        stateSubject.send(state)
    }
}

Step 8: ViewModel (Single Source of Truth)

Critical: The ViewModel has a single @Published viewState property:

@MainActor
final class UserListViewModel: CombineViewModel, ObservableObject {
    typealias ViewState = UserListViewState
    typealias ViewEvent = UserListViewEvent

    // SINGLE source of truth - one @Published property
    @Published private(set) var viewState: UserListViewState = .initial

    private let domainInteractor: UserListDomainInteractor
    private let reducer: UserListViewStateReducer
    private let eventMap: UserListEventActionMap
    private var cancellables = Set<AnyCancellable>()

    init(
        serviceLocator: ServiceLocator,
        reducer: UserListViewStateReducer = UserListViewStateReducer(),
        eventMap: UserListEventActionMap = UserListEventActionMap()
    ) {
        self.domainInteractor = UserListDomainInteractor(serviceLocator: serviceLocator)
        self.reducer = reducer
        self.eventMap = eventMap

        setupBindings()
    }

    func handle(event: UserListViewEvent) {
        guard let action = eventMap.map(event: event) else { return }
        domainInteractor.dispatch(action: action)
    }

    private func setupBindings() {
        domainInteractor.statePublisher
            .map { [reducer] state in reducer.reduce(domainState: state) }
            .removeDuplicates()
            .receive(on: DispatchQueue.main)
            .assign(to: &$viewState)
    }
}

Step 9: SwiftUI View

The view reads from the single viewState and emits events:

struct UserListView<R: UserListNavigationRouter>: View {
    private var router: R
    @ObservedObject var viewModel: UserListViewModel

    init(router: R, viewModel: UserListViewModel) {
        self.router = router
        self.viewModel = viewModel
    }

    var body: some View {
        content
            .navigationTitle("Users")
            .refreshable {
                viewModel.handle(event: .onRefresh)
            }
            .onAppear {
                viewModel.handle(event: .onAppear)
            }
            .onChange(of: viewModel.viewState.selectedUser) { _, newValue in
                if let user = newValue {
                    router.route(navigationEvent: .userDetail(user))
                    viewModel.handle(event: .onUserNavigated)
                }
            }
    }

    @ViewBuilder
    private var content: some View {
        if viewModel.viewState.isLoading {
            ProgressView()
        } else if let error = viewModel.viewState.errorMessage {
            ErrorView(message: error) {
                viewModel.handle(event: .onRefresh)
            }
        } else if viewModel.viewState.showEmptyState {
            EmptyStateView(message: "No users found")
        } else {
            userList
        }
    }

    private var userList: some View {
        List(viewModel.viewState.users) { user in
            UserRow(user: user)
                .onTapGesture {
                    viewModel.handle(event: .onUserTapped(userId: user.id))
                }
        }
    }
}

Why Single @Published Matters

Having a single @Published viewState property is crucial:

Before (Anti-pattern)

// DON'T DO THIS - Multiple sources of truth
@Published var users: [User] = []
@Published var isLoading: Bool = false
@Published var error: String? = nil

Problems:

View can receive partial updates
State can become inconsistent
Harder to reason about current state
Testing requires checking multiple properties

After (Correct Pattern)

// DO THIS - Single source of truth
@Published private(set) var viewState: UserListViewState = .initial

Benefits:

Atomic state updates
Consistent state guaranteed
Easy to snapshot and debug
Single property to test

Testing Strategy

Testing the Reducer (Pure Function)

@Test
func testReducerShowsEmptyState() {
    let reducer = UserListViewStateReducer()
    let domainState = UserListDomainState(
        users: [],
        isLoading: false,
        isRefreshing: false,
        error: nil,
        hasLoadedInitialData: true,
        selectedUser: nil
    )

    let viewState = reducer.reduce(domainState: domainState)

    #expect(viewState.showEmptyState == true)
    #expect(viewState.users.isEmpty == true)
}

Testing the Event Map

@Test
func testEventMapMapsOnAppearToLoadUsers() {
    let eventMap = UserListEventActionMap()

    let action = eventMap.map(event: .onAppear)

    #expect(action == .loadUsers)
}

Testing the DomainInteractor

@Test
func testDomainInteractorLoadsUsers() async {
    let mockService = MockUserService()
    mockService.usersToReturn = [User(id: "1", firstName: "John", lastName: "Doe")]

    let serviceLocator = ServiceLocator()
    serviceLocator.register(UserService.self, instance: mockService)

    let interactor = UserListDomainInteractor(serviceLocator: serviceLocator)

    interactor.dispatch(action: .loadUsers)

    // Wait for async operation
    try? await Task.sleep(nanoseconds: 100_000_000)

    #expect(interactor.currentState.users.count == 1)
    #expect(interactor.currentState.hasLoadedInitialData == true)
}

Testing the ViewModel

@Test
@MainActor
func testViewModelUpdatesViewStateOnLoad() async {
    let serviceLocator = ServiceLocator()
    serviceLocator.register(UserService.self, instance: MockUserService())

    let viewModel = UserListViewModel(serviceLocator: serviceLocator)

    viewModel.handle(event: .onAppear)

    // Wait for state propagation
    try? await Task.sleep(nanoseconds: 100_000_000)

    #expect(viewModel.viewState.users.isEmpty == false)
}

File Structure

Organize each feature with this structure:

UserList/
├── API/
│   └── UserService.swift           # Protocol + Live/Mock implementations
├── Domain/
│   ├── UserListDomainInteractor.swift
│   ├── UserListDomainState.swift
│   ├── UserListDomainAction.swift
│   ├── UserListViewStateReducer.swift
│   └── UserListEventActionMap.swift
├── ViewModel/
│   └── UserListViewModel.swift
├── View/
│   └── UserListView.swift
├── ViewEvents/
│   └── UserListViewEvent.swift
├── ViewStates/
│   └── UserListViewState.swift
└── Router/
    └── UserListNavigationRouter.swift

Key Takeaways

Single Source of Truth: One @Published viewState in the ViewModel
Pure Reducers: Transform domain state to view state without side effects
Event Mapping: Separate UI events from business actions
Testability: Each layer can be tested in isolation
Predictable Flow: Data flows in one direction only

Conclusion

This Unidirectional Data Flow architecture provides:

Predictable state management through single source of truth
Easy debugging with clear data flow
Excellent testability at every layer
Scalable patterns that work for small and large features
Clean separation between UI, presentation, and business logic

The pattern requires more initial setup but pays dividends in maintainability, testability, and developer experience as your application grows.

DEV Community: Bruno Mello

Running a Fully-Local AI Agent on a Mac Studio, OpenClaw + Ollama + MLX

TL;DR findings

0. Prerequisites

1. Architecture

2. The two local LLM providers

3. Wiring models into the agent + aliases

4. Two agents from one config (private vs public)

5. Ollama: tuning + keeping the model warm

5a. Service env (LaunchAgent: ~/Library/LaunchAgents/homebrew.mxcl.ollama.plist)

5b. Warm-up script: ~/ollama-warmup.sh

5c. Warm-up LaunchAgent: ~/Library/LaunchAgents/com.ollama.warmup.plist

6. MLX: a persistent OpenAI-compatible server

LaunchAgent: ~/Library/LaunchAgents/com.mlx-lm.server.plist

6b. [2026] The kernel panic that will reboot your Mac

What actually triggers it

The one mitigation that works

The shim

The symptom you are probably already living with

7. Benchmarking: and the gotcha that almost fooled me

7b. [2026] For a chat agent, tok/s is the wrong metric

8. Bonus: fully-local voice (TTS + STT)

TTS: OmniVoice on :17494 (messages.tts)

STT: mlx-whisper on :17495

8b. [2026] Voice delivery: the duplicate-message trap

8c. [2026] Memory search needs an embedding model, and it fails hard

9. Gotchas worth knowing

10. Service map (what autostarts)

Running Local LLMs as Your AI Coding Assistant on Apple Silicon

What We're Building

The Architecture (Three Layers)

Key Concepts

What Is MLX?

[2026] The Bug Nobody Warns You About: Kernel Panics from set_wired_limit

What actually triggers it

The one mitigation that works

Implementing it

[2026] Why Your Mac Gets Laggy as Context Fills

Setting Up

Prerequisites

Installation Steps

[2026] Running It as a Real Service

[2026] The launchd Trap That Cost Me an Hour

[2026] Do Not Trust Your Own Benchmarks

[2026] Reading MLX Releases: Does This Commit Help You?

[2026] LM Studio vs Direct mlx_lm.server

[2026] OpenCode Harness Gotchas

The Model Hunt

Attempt 1: GLM-4.7-Flash-4bit

Attempt 2: Qwen2.5-72B-Instruct-4bit

Attempt 3: Devstral-Small-2-24B-8bit

Attempt 4: Qwen3-Coder-30B-A3B-Instruct-8bit (Winner)

Attempt 5: Qwen3-Coder-Next-4bit (Winner After Bug Fix)

[2026] Current Daily Driver: Qwen3.8-27B MLX 4-bit

Quantization Note

Model Comparison Table

Tool Call Flow

RAM and Model Selection Guidance

Pre-Download Checklist

Debugging Tips

Conclusion

Resources

Unidirectional Data Flow Architecture in SwiftUI

The Problem

The Solution: Unidirectional Data Flow

Four Core Protocols

1. CombineViewModel

2. CombineInteractor

3. ViewStateReducing

4. DomainEventActionMap

Complete Implementation Example

Step 1: Define View Events

Step 2: Define Domain Actions

Step 3: Define Domain State

Step 4: Define View State

Step 5: Event to Action Mapping

Step 6: View State Reducer

Step 7: Domain Interactor

Step 8: ViewModel (Single Source of Truth)

Step 9: SwiftUI View

5a. Service env (LaunchAgent: `~/Library/LaunchAgents/homebrew.mxcl.ollama.plist`)

5b. Warm-up script: `~/ollama-warmup.sh`

5c. Warm-up LaunchAgent: `~/Library/LaunchAgents/com.ollama.warmup.plist`

LaunchAgent: `~/Library/LaunchAgents/com.mlx-lm.server.plist`

TTS: OmniVoice on :17494 (`messages.tts`)

[2026] The Bug Nobody Warns You About: Kernel Panics from `set_wired_limit`

[2026] LM Studio vs Direct `mlx_lm.server`