I Tried Local LLMs for Coding on My M3 MacBook Pro: What Worked and What Didn't

Like many developers, I have been curious about running an LLM locally for some time.

The idea sounds almost perfect: download a model, keep the prompts and code on my own machine, avoid another subscription, and connect the model directly to VS Code. I already had a MacBook Pro with an M3 processor and 36 GB of unified memory, so I wanted to find out how far I could realistically push it.

I was not trying to prove that a laptop could run an LLM. We already know that it can.

My real question was more practical:

Can a local model become my everyday coding assistant for an actual project?

During a week of experiments, I tried different models, compared regular and MLX variants, changed the context size, connected Ollama to VS Code over my home network, and attempted to use it for more than isolated code snippets.

Some parts worked much better than I expected. Other parts reminded me that running a model and having a productive coding workflow are two very different things.

---

TL;DR;

I used Ollama on a 36 GB M3 MacBook Pro and connected it to VS Code, including from a separate Lenovo laptop over my local network. Gemma 4 26B worked well, while gemma4:31b-mlx gave me the best overall local-model experience. qwen3-coder:30b was also worth testing for coding-focused tasks.

For questions, explanations, small changes, and focused code generation, the setup was genuinely useful. For a medium-complexity project with repeated planning, repository exploration, code changes, and validation, it was too slow for the way I wanted to work.

I eventually went back to GitHub Copilot/Codex for daily development. But I did not consider the local experiment a failure. It helped me understand where local models are already useful, where they still struggle, and why the complete coding workflow matters as much as the model itself.

---

The setup I had in mind

These were the two machines in my setup:

Machine	Hardware	Role in the experiment
MacBook Pro	M3 processor, 36 GB unified memory	Ollama server and main host for the larger local models
Lenovo laptop	Intel i9-13900HX, RTX 4060 Laptop GPU with 8 GB VRAM, 32 GB RAM, Windows 11	VS Code client connected to the Mac over Wi-Fi/LAN

The Lenovo was already a powerful development laptop. However, its 8 GB of dedicated GPU memory was not the best fit for the 26B-to-31B-class models I wanted to try. The Mac's larger unified-memory pool made it the more interesting inference host.

The architecture was simple:

VS Code on Lenovo -> local Wi-Fi -> Ollama on MacBook Pro -> local model

Ollama exposes an HTTP API on port 11434. By default it listens only on the local machine, but the bind address can be changed with OLLAMA_HOST when access from another device is needed.

On macOS, the relevant setting looks like this:

launchctl setenv OLLAMA_HOST "0.0.0.0:11434"

After restarting Ollama, I could point the VS Code integration to the MacBook's local IP address.

Before troubleshooting anything inside VS Code, I found it useful to verify the simpler network path first from the Lenovo:

curl http://<macbook-local-ip>:11434/api/tags

If that returned the installed model list, the Mac, Ollama server, port, and local network route were working. Only then did it make sense to debug the editor configuration. This small separation saved me from treating every connection problem as a VS Code problem.

I kept this limited to my trusted home network. Binding to 0.0.0.0 makes the service reachable beyond localhost, so I would not expose port 11434 directly to the public Internet.

The installation was not the difficult part. Ollama made downloading, starting, stopping, and switching models straightforward.

For example:

ollama run gemma4:26b

To see what was still loaded in memory, this command was particularly helpful:

ollama ps

That small command answered an important question during testing: was a model fully loaded for accelerated inference, split between processors, or already unloaded?

Understanding what the model names really mean

Before this experiment, names such as 26b, 30b, and 31b felt like product versions. They are not. The B represents billions of parameters.

A bigger number usually suggests more model capacity, but it does not automatically mean a better experience on a particular laptop. Architecture, quantization, memory use, context length, and the amount of work the model performs for every response all matter.

That became clear when I compared these models:

Model	What I noticed
gemma4:26b	A very good starting balance. It worked well on my machine and was capable enough to make local coding feel practical.
gemma4:26b-mlx	Interesting because MLX is optimized around Apple silicon. It reinforced that the runtime and model format can matter almost as much as the parameter count.
qwen3-coder:30b	A coding-focused option worth trying, especially for targeted implementation questions.
gemma4:31b-mlx	The strongest overall local experience on my MacBook Pro, but still not fast enough for my preferred end-to-end workflow on a medium project.
glm-4.7:cloud	Something I evaluated as a cloud option, but it was not part of the fully local comparison.

There was also an architectural detail behind what I observed. Gemma 4 26B is a mixture-of-experts model with only a subset of its parameters active for a token, while the 31B version is dense. The labels are close, but the work performed during inference is not identical. That is one reason model size alone is an incomplete performance guide.

The first results were genuinely impressive

Once the model appeared inside the VS Code workflow, the experience felt surprisingly normal.

I could ask it to:

• explain unfamiliar code;
• generate a small method;
• suggest a refactoring;
• write a unit-test outline;
• compare two implementation approaches; and
• help think through a class or API design.

For these focused tasks, the local setup was good. There was also something satisfying about watching a reasonably capable model respond from my own laptop without sending the prompt to a hosted inference service.

This is the stage where it is easy to declare success. A prompt goes in, useful code comes out, and the local-LLM demo works.

But that was not the test I cared about.

I wanted to use the model the same way I use a modern coding assistant: give it a goal, let it inspect several files, create a plan, make changes, reconsider a decision, run checks, fix problems, and continue.

That is where the experience changed.

A coding agent multiplies every bit of latency

A single slow answer is manageable. An agentic coding loop is different.

This was basically the difference between my first prompt and a real project.

One feature may require the model to:

1. inspect the repository;
2. read the relevant files;
3. understand the existing design;
4. propose a plan;
5. generate one or more edits;
6. interpret a build or test failure; and
7. repeat part of the cycle.

If every step takes noticeably longer, the delay compounds. I was no longer waiting once for an answer. I was waiting at every turn of the development loop.

On small tasks, the 31B MLX model could produce good results. On a medium-complexity project, the overall flow became too slow. The model could often do the next step, but I had to wait long enough that I would lose momentum.

This was the biggest lesson from my experiment:

Coding productivity depends on the latency of the complete loop, not just the quality of one generated answer.

A model can be impressive in a chat window and still feel impractical as an autonomous or semi-autonomous coding assistant.

Context length was not a free upgrade

Repository-level work also needs context. A model must see enough of the codebase, instructions, conversation, and tool output to make a coherent change.

It is tempting to increase the context window as far as the model supports, but the configured context consumes memory. More context can reduce the resources available for inference and can make an already heavy workflow slower.

Ollama allows the context length to be configured when starting the server:

OLLAMA_CONTEXT_LENGTH=32768 ollama serve

There is an interesting tension in the current Ollama guidance. For systems in the 24–48 GiB memory tier, Ollama defaults to a 32K context, but its documentation recommends at least 64K for agents and coding tools. Both recommendations make sense: the first is cautious about memory, while the second reflects how much context repository-level tools may need.

My 32K starting point was therefore not the model's theoretical limit. It was a practical compromise for a machine with 36 GB shared across the model, context cache, operating system, editor, and other applications.

The best value is not automatically the largest value. On the 36 GB Mac, the model, its context cache, macOS, and any other running applications share the same memory pool.

For me, this became a balancing act:

• A smaller model responded faster but was less reliable on complex work.
• A larger model produced stronger answers but slowed down the development loop.
• More context helped with repository awareness but increased memory pressure.
• Reducing context improved resource use but limited how much of the project the assistant could consider at once.

There was no setting that made all four problems disappear.

MLX helped, but it did not change the final decision

Because I was using Apple silicon, I also explored MLX variants. MLX is designed for machine learning on Apple silicon and can make good use of its unified-memory architecture.

gemma4:31b-mlx ended up being the best-performing local option in my tests from an overall quality perspective. It showed me that my MacBook Pro was more capable than I had initially assumed.

However, optimization does not remove the underlying workload. A 31B-class model still has to generate tokens, reason through the task, work with context, and participate in every step of the agent loop.

MLX improved the experiment. It did not turn the laptop into a cloud inference cluster.

I also briefly explored whether I should leave Ollama and work directly with MLX-LM. It was an interesting direction, but I eventually dropped it from this experiment. My goal was to improve the coding experience inside VS Code, not replace a simple serving layer with another stack that I would also need to configure and maintain. For this test, Ollama's convenience was part of the value.

I also considered adding more hardware

At one point, I wondered whether adding one or more Mac minis would solve the problem.

That idea is attractive: keep VS Code on the development machine, put models on headless Macs, and treat them as local inference servers.

It can be useful if the goal is to separate development work from inference or serve different models from different machines. But multiple computers do not automatically combine their memory into one larger pool for a single Ollama model. Without a distributed inference layer designed for that purpose, each machine remains its own server.

Buying more hardware also changes the economics. A local setup may avoid per-token charges, but hardware, electricity, configuration, maintenance, and my own waiting time are not free.

Before buying another machine, I decided I first needed to prove that local inference would improve my day-to-day workflow. In my case, it had not reached that point yet.

Why I returned to GitHub Copilot

After trying to tune the local setup, I went back to GitHub Copilot for my regular development work.

The reason was not that the local models were bad. They were surprisingly good.

The reason was flow.

For a medium project, I wanted faster iterations, better repository-level assistance, and less time watching each step complete. Copilot gave me a more practical experience on the work I was actually doing.

This comparison also helped me understand that a coding product is more than an LLM. The model matters, but so do context collection, file selection, tool execution, edit application, error recovery, and the user experience around the entire loop.

Running a capable model locally solved only one part of that system.

Where I would still use a local model

I have not abandoned local LLMs. I would still use this setup for:

• learning how models, quantization, context, and inference actually behave;
• focused code explanations and small implementation questions;
• experimenting offline;
• testing prompts without worrying about usage quotas; and
• building prototypes that need a local model API.

I would also keep it as a secondary assistant rather than forcing it to replace every cloud coding tool.

That is probably the most useful mindset I gained from the experiment. The choice does not have to be completely local or completely cloud-based. Different tools can serve different parts of the workflow.

What I would do differently next time

If I repeat this comparison, I will make it more systematic.

I would use the same set of tasks for every model:

1. explain an unfamiliar class;
2. implement a contained feature;
3. add tests to existing code;
4. debug a deliberately introduced failure;
5. refactor across several files; and
6. complete a repository-level task with planning and validation.

For each task, I would record:

• time to first useful response;
• total task-completion time;
• number of corrections needed;
• peak memory pressure;
• whether the model stayed coherent as context grew; and
• whether the final code passed the build and tests.

Ollama's API already returns useful timing fields, including total_duration, load_duration, prompt_eval_count, prompt_eval_duration, eval_count, and eval_duration. That means a future comparison does not need to depend only on a stopwatch or how fast a response feels.

For example, output generation speed can be calculated as:

tokens per second = eval_count / (eval_duration / 1,000,000,000)

I would still keep this as a secondary metric. A model can generate tokens quickly and take the code in the wrong direction. The result that matters is whether it completed the task correctly, how much intervention it needed, and how long the complete loop took.

Tokens per second is interesting, but completed developer work per minute is the metric I actually care about.

Final thoughts

My local LLM experiment started with a simple question: could I replace my cloud coding assistant with models running on my own hardware?

For my current machine and workflow, the honest answer is: not completely.

The 36 GB M3 MacBook Pro could run much more capable models than I expected. Gemma 4 26B worked well, and the 31B MLX variant was genuinely impressive. Ollama also made the operational side simple enough that I could focus on the models rather than spend the entire week configuring infrastructure.

But once I moved from isolated prompts to a medium-complexity project, speed became the limiting factor. The local models could help me code, but they could not yet keep up with the pace I wanted from a daily agentic workflow.

So I returned to GitHub Copilot—with a better understanding of what I was paying for and a much more realistic view of local AI.

Local LLMs are no longer just a novelty. They are useful today. But for me, the best use is currently as a private, flexible second tool—not the only coding assistant on my desk.

---

References

• Ollama: VS Code integration
• Ollama FAQ: context, networking, memory, and model management
• Ollama: context length and model offloading
• Ollama API: generation response and timing fields
• Ollama model library: Gemma 4
• Ollama model library: Qwen3 Coder
• MLX documentation: unified memory on Apple silicon

The observations in this article are based on my own hardware, model choices, prompts, and projects. Performance and output quality will vary with the machine, quantization, context size, integration, and workload.

AI assistance note: I used AI to help organize and edit this article. The setup, experiments, observations, and conclusions are based on my own experience.