Running Qwen3.6 Locally on a MacBook: oMLX + opencode, Faster Than I Expected

One 14-inch MacBook Pro. No internet, no API bill. Chat and coding both work, and token output is fast.

Load Qwen3.6-35B-A3B (MLX 4bit) locally with oMLX. It is good enough for everyday chat and coding.

• For chat, just use the built-in web Chat of oMLX. No other client needed.
• For coding, use opencode and point it at the local oMLX server.
• Output is fast: roughly 100 tokens/s, estimated from the token counter in my screen recording (details below).
• Simple tasks and text-heavy tasks (translation, for example) can be handed to it, which can save a lot of API cost.
• There are two costs. First, memory: the model takes close to 20GB, so a 48GB machine runs it comfortably, but watch the headroom. Second, while a task is running the GPU hits 100%, the fans spin up, and there is some noise.

Test Environment

Item	Spec
Machine	MacBook Pro 14" (M5 Pro)
CPU	15 cores (10 performance + 5 super cores)
GPU	16 cores
Memory	48 GB
OS	macOS Tahoe 26.6.1
Inference server	oMLX v0.7.2
Model	Qwen3.6-35B-A3B-MLX-4bit

Why Qwen3.6-35B-A3B

The 35B-A3B in the name is the key. It is a MoE (Mixture of Experts) model with 35B total parameters, but only about 3B are active per token.

• The "knowledge" is that of a 35B model, so quality beats a small dense model with similar compute.
• Each step only computes about 3B parameters, so generation is fast.
• After 4bit quantization the size is manageable, and Apple Silicon's unified memory can hold it.

MLX is Apple's own machine learning framework. It uses Metal and unified memory directly, and is usually easier and faster on a Mac than generic solutions. oMLX is a local inference server built on MLX, with a built-in web Chat.

Chat: Use oMLX's Built-in Web Chat

Once oMLX is running, it comes with a web chat UI. Model selection, conversation history and search all work out of the box.

A few details from the screenshot:

• The left side has the conversation list and search. The bottom right lets you switch models (here Qwen3.6-35B-A3B-MLX-4bit).
• Next to the input box is a Reasoning toggle. Turn it on for deep thinking, and off for daily Q&A to save time.
• The bottom right shows the context used by the current conversation in real time. In the screenshot it is 2,325 tokens, about 7%.

The recording below (about 34 seconds) is one complete conversation:

• video demo

The flow:

1. I first asked "How can I use oMLX Chat?". The model thought for a while (Thinking), then said "oMLX Chat" is not in its training data, and asked whether I meant MLX, Ollama, LM Studio or something similar.
2. I followed up with "Do you have some examples?". After thinking again, the model started streaming continuously. It gave three ways to run models locally: Ollama + Open WebUI, LM Studio and Apple MLX, with full code blocks.

This detour is normal: oMLX is a new tool and not in the model's training data. A local model is good for general Q&A, rewriting and explaining code. For very new tools, you still need to read the docs yourself.

Coding: opencode + Local oMLX

For coding I used opencode (an AI coding assistant for the terminal). After pointing its model at the local oMLX endpoint, the bottom of the UI shows the current mode and model: Build · Qwen3.6-35B-A3B-MLX-4bit · oMLX.

Here is a recording (about 23 seconds):

How to connect it: oMLX exposes an OpenAI-compatible API, so you only need to add a custom provider in the opencode config. The config below is an example. Use the port and model ID that your own oMLX shows.

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "omlx": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "oMLX",
      "options": {
        "baseURL": "http://localhost:<oMLX port>/v1"
      },
      "models": {
        "Qwen3.6-35B-A3B-MLX-4bit": {
          "name": "Qwen3.6-35B-A3B-MLX-4bit"
        }
      }
    }
  }
}

After that, pick this model in opencode, and in Build mode it can read code, edit files and run commands. All inference happens on your machine and the code never leaves it, which is especially useful for company projects.

Speed and Resource Usage

For speed, I made a rough estimate from the token counter in the chat recording above. The counter at the bottom right of the input box reads 1,617 tokens at 25s and 2,224 tokens at 31s. That is over 600 new tokens in about 6 seconds, or roughly 100 tokens/s. This is only an order-of-magnitude estimate from the UI counter. The counting method and refresh timing can introduce error, so it is not a rigorous benchmark.

[Add your precise measurements here, e.g. time to first token, generation tokens/s, coding task duration]

Subjectively the output feels smooth. The gap with cloud models is mainly in reasoning depth on complex tasks, not in waiting time.

For memory, here is Activity Monitor:

Beyond memory, one more thing to know up front: while a task is running, GPU usage hits 100%, the fans start spinning, and there is some noise. This is the unavoidable price of running large models locally. If you work in a quiet office or at night, be prepared. For long batch jobs (like the translation mentioned below), plug in the power and keep the laptop somewhere ventilated.

The screenshot above was taken with a browser, an IDE and other everyday apps open. The omlx-server process alone uses about 18GB. Memory pressure shows Normal, but 84% is used and there is a little swap. So 48GB is a comfortable minimum. With 32GB, close other big apps or switch to a smaller model.

My Take: What to Hand Over to a Local Model

My conclusion after using it: simple tasks and text-heavy tasks can be fully handed to a local model, and only complex tasks that need deep reasoning go to a top cloud model. Not everything needs the most expensive model.

Good for local:

• Translation: lots of text, simple logic, little need for reasoning depth. Ideal for batch jobs.
• Rewriting, polishing, summarizing, extraction: also text-heavy, and a local model handles them fine.
• Simple coding tasks: reading and explaining code, writing scripts, small changes, adding tests.
• Privacy-sensitive content: code and documents never leave the machine.

Still better in the cloud:

• Complex architecture design, large refactors, hard problems that need long chains of reasoning.
• Anything that needs up-to-date knowledge of tools and libraries (models have a knowledge cutoff, as the oMLX example above showed).

A Real Bill: 5,000 Articles, Over 100 USD

I once used Claude to translate about 5,000 articles, and the API cost was over 100 USD. Looking back, this kind of task never needed a flagship model: the source is clear, the goal is clear, and the quality bar is "fluent and accurate", not "deep reasoning".

If I had run it on a local Qwen3.6-35B-A3B, that money would mostly have been saved. The cost would be electricity and machine time. There is also no quota locally, so a batch job can just run in the background without worrying about the bill. My approach now would be:

1. Batch, simple, text-heavy tasks (translation, summaries, formatting) go to the local model.
2. After local processing, only send the few doubtful or especially important parts to a cloud model for a second pass.
3. Truly complex reasoning and design tasks go straight to a top cloud model.

That way most tokens are spent locally, and cloud money goes only where it is worth it.

Summary

oMLX makes running models locally painless: one service provides both web Chat and a compatible API, and together with opencode, both chat and coding work on a single MacBook. A MoE model like Qwen3.6-35B-A3B gives 35B-level knowledge with 3B-level generation speed, and the experience on Apple Silicon is very good. If you have a Mac with around 48GB of memory, it is worth a try. Handing translation, summarization and other simple but token-hungry work to the local model, and saving cloud quota for truly hard problems, is the most cost-effective way I have found so far. Just be ready for the GPU at full load and some fan noise while it runs.

What Do You Think?

Have you run large models locally? What machine, which model, how fast? Which tasks would you hand to a local model, and which would you keep in the cloud?

Feedback and comments are very welcome. Please also point out anything inaccurate in this post, and I will keep updating it.

A Quick Plug

I built a personal product called GoodAI.Life (https://goodai.life).
I’m currently building Good AI Life(opens in a new tab), a next-generation AI directory designed to showcase your product in action.

If you are interested, please take a look, give it a try, and send me feedback.