A new write-up from Thoughtworks engineer Birgitta Böckeler on Martin Fowler's blog documents what actually happens when you run small local models for agentic coding — not autocomplete, full agentic tasks with file reads and writes. The verdict is nuanced, honest, and worth reading if you've been tempted by local-first setups.
The short version: not plug-and-play, but not a dead end either.
"It has been a frustrating experience with sometimes confusing results... agentic coding capabilities are definitely very far away from what I've now become used to with bigger models."
What the evaluation covered
The setup used an M3 Max (48GB RAM) and M5 Pro (64GB RAM), tested primarily with Qwen3.6 35B MoE and Gemma 4 models, using OpenCode and Pi as coding harnesses. The evaluation ran through a "viability funnel":
- Does it fit in RAM?
- Does it run at reasonable speed?
- Can it handle tool calling?
- Does it produce functionally correct code?
- Can it sustain a longer conversation / bigger context?
- Can it handle complex multi-step tasks?
Most models cleared the early gates but fell apart at the later ones — inconsistently, and in ways that were hard to reproduce.
The confusing bit: results that don't make sense
One of the most striking findings: the same model on different hardware gave wildly different quality outputs — not just different speeds. Qwen 35B MoE failed a charting task 5/7 times on the M3 Max, then only 1/7 times on the M5 Pro with 64GB RAM. Same model, same settings, different RAM ceiling.
There's also a manual-vs-automated gap: models that succeeded manually in back-and-forth conversation failed 3/3 times when asked to one-shot the same task in an automated eval. This makes benchmarking tricky and most published evals probably understate real-world viability for interactive use.
The actual insight: task shape matters more than model choice
After the structured evals, the author ran Qwen3.6 35B MoE day-to-day and found a clear pattern in what works:
- Bash and Python scripts — often fine
- Small, precisely scoped changes to existing files — often fine
- Tasks pre-planned by a bigger model, then delegated for execution — promising pattern
- Broad code research + multi-file edits — falls apart fast
- Complex logic from scratch — falls apart even faster
The framing that resonated: think of it like the early Copilot multi-file edit days. You have to be more specific, more deliberate, and more engaged. Which — the author notes — is actually kind of good. Less "surrender to the model", more review discipline.
A colleague who uses the same model daily on a real Python/React codebase backs this up, with the caveat that augmenting the harness with code-graph tools (Graphify, Understand Anything) meaningfully improves results.
What to do
- Curious about local models? Start with Qwen3.6 35B MoE at 4BIT quantisation — it's the most viable option right now for 48GB+ Apple Silicon machines.
- Picking tasks? Favour small, well-scoped, single-file changes. Let a cloud model do the planning; use the local model for execution.
- Evaluating? Don't trust automated one-shot benchmarks alone — the interactive experience diverges significantly.
- Setting up the harness? Disable reasoning mode, max out context window. Augment with graph tools if your codebase requires code search.
Full write-up (and the accompanying factors memo) is worth the read if you're seriously exploring this space.
✏️ Drafted with KewBot (AI), edited and approved by Drew.
Top comments (0)