One of the quietest objections to AI coding agents is that they send your source code to a third-party API. For proprietary systems, regulated code, or anything under NDA, that objection is a hard stop. OpenCode offers an alternative: run the agent against a local model through Ollama, and your code never leaves the machine.
We set this up for a client project that could not use cloud APIs. The goal was not to match Claude Sonnet. It was to find out whether a local model was useful at all for day-to-day coding tasks.
The Setup
The stack is simple on paper: Ollama serves a local model, OpenCode points at the Ollama endpoint, and the agent runs against http://localhost:11434. We used Qwen 2.5 Coder 14B on an M3 Max with 36 GB of unified memory. Smaller models work, but 14B was the smallest size that produced coherent multi-file edits.
Configuration lives in OpenCode's provider settings. You add an Ollama provider, set the model name, and leave the API key blank. OpenCode then sends prompts to the local endpoint instead of Anthropic or OpenAI.
Context window is the first limitation you hit. A 14B model typically handles 32K tokens at most. That is enough for a single file plus some surrounding context, but not enough for a large refactor that needs to read dozens of files. Plan tasks accordingly.
What Local Mode Handles Well
Three tasks worked reliably:
- Boilerplate generation. Creating a new API endpoint from an existing pattern, writing test stubs, and generating TypeScript types from a JSON sample. The local model followed existing conventions because the examples were in its immediate context.
- Small refactors. Renaming functions, extracting helpers, and updating call sites within a single module. The model made occasional import-path mistakes, but they were easy to catch in the diff.
- Code explanation. Asking "what does this function do?" or "why is this test failing?" produced useful answers because the answer required reasoning over code already loaded into context.
The common thread is that all three tasks fit inside the model's context window and do not require deep architectural reasoning.
Where Local Mode Struggles
The local model fell down on anything that required planning across files. A task like "add pagination to every list endpoint" needs the agent to read route handlers, service functions, and response types across the codebase, then produce a consistent change. The local model either missed files or generated inconsistent implementations.
Speed was also a factor. A single prompt-response cycle against the local 14B model took 15-45 seconds depending on output length. Claude Sonnet over API returned in 3-8 seconds for similar prompts. Local inference is free, but it is not fast.
A Practical Split
The setup that worked best was a split workflow. Use the local model for:
- Writing new files from a clear pattern
- Explaining or summarizing existing code
- Tasks where latency does not matter
Switch to a cloud provider for:
- Multi-file refactors
- Debugging unfamiliar code paths
- Tasks where missing a file is expensive
OpenCode makes that switch easy because the model provider is just a config setting. You can run the same agent against Ollama in the morning and Claude in the afternoon without changing your workflow.
Do not assume local means private by default. If Ollama downloads a model from the internet, the download itself leaks that you are using that model. The actual prompts and code stay local, but network metadata does not. For air-gapped environments, download models separately and transfer them offline.
Hardware Notes
We tested on three machines:
- M3 Max, 36 GB RAM: Qwen 2.5 Coder 14B ran comfortably. 32K context worked without swapping.
- M2 Pro, 16 GB RAM: The same model was usable but slow. Context windows above 16K caused noticeable system slowdown.
- Linux desktop, RTX 4090: Faster than the Macs for the same model, but setup was more involved.
For occasional local use, 16 GB is enough. For daily local use, 32 GB or a dedicated GPU is strongly recommended.
Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.
Top comments (0)