Every "run a local model instead of Claude Code" guide opens the same way: a wall of SWE-bench scores, a triumphant "we're within five points of the frontier," and a shopping list of GPUs. I've read a dozen of them and run most of the setups. They're not wrong, exactly. They're measuring the thing that's easy to measure instead of the thing that decides whether you can actually close your Claude Code tab.
Here's the version nobody puts in the headline, after pointing Claude Code and Codex at local models for real work:
A local LLM writing a correct function is a solved problem. A local LLM being a reliable agent — emitting a clean tool call, ten times in a row, across a multi-file edit, without drifting from the schema — is not. And the second one is the entire job.
So let's do this properly. We'll go through the model families that actually matter in late 2026, what each is genuinely for, the hardware reality, the setup (it's 60 seconds now), the failure modes nobody screenshots, and a cost model so you know whether this saves you a cent. With charts, because a table of numbers you have to squint at is exactly the problem.
First: why this suddenly works at all
Two things changed in 2026, and neither is "the models got louder on Twitter."
1. Mixture-of-Experts broke the hardware wall. A dense 70B model activates all 70B parameters for every token. A MoE model has a huge total parameter count but only fires a sliver per token — Qwen's 35B-A3B activates ~3B. VRAM is set by the file size you load; speed is set by the tiny active slice. So a model that used to need a server now runs on a gaming GPU, and quantized, on a laptop.
2. The agents learned to speak local. This is the part the 2025 posts missed. In January 2026, Ollama shipped native support for the Anthropic Messages API, and LM Studio and llama.cpp followed. That's the unlock: Claude Code no longer needs a proxy shim to talk to localhost. You point one environment variable at it and it works. The plumbing stopped being the hard part — which means the conversation finally has to move past "can it code."
The number everyone quotes — and why it misleads you
SWE-bench Verified is the stat in every headline, and the reported 2026 figures genuinely crowd the frontier.
Read that chart and the story looks finished: a model on a single 24GB card lands at ~77, within a rounding error of the cloud frontier. But here's the catch, and it's the whole article:
SWE-bench measures "given a bug, produce a correct patch." That's a writing task. It says almost nothing about the four things that actually break when the model is driving your editor.
-
Tool-call validity — can it emit a well-formed
edit_file/run_commandcall the harness can parse, reliably, not usually? - Long-horizon consistency — does call #11 still remember the plan from call #2?
- Context under load — does quality hold at 40K tokens of real repo, or collapse long before the advertised window?
- Instruction-following — when you say "don't touch the tests," does it not touch the tests?
None of those four are on the leaderboard. All four are why your local setup feels different from the demo.
⚠️ Treat every number in this post as vendor-reported. Different guides quote the same model at wildly different scores because they quietly swap SWE-bench Verified for Pro or Terminal-Bench and never say so. Verify on the official SWE-bench leaderboard before you buy a GPU on the strength of a percentage.
The 7 families worth knowing (and what each is actually for)
Think in families, not single checkpoints — each ships in several sizes, and the right pick is the one that fits your card and your job.
1. Qwen 3.x — the backbone of local coding
If you run one family, it's this one. Qwen is the open-weight leader on both coding and tool-calling, which is the combination that matters.
- Qwen 3.6 27B (dense) — the default self-hosted coder. ~77% SWE-bench Verified, ~22GB quantized, fits a single 24GB GPU or an M-series Mac. Reliable tool caller.
- Qwen 3.5 35B-A3B (MoE) — 35B total, ~3B active. Runs shockingly light (8GB VRAM at Q4) because only the active slice costs compute. The "frontier on a laptop" headliner.
- Qwen3-Coder (30B-A3B / 480B-A35B / Next-80B-A3B) — code-specialist line. The 30B is a great daily driver; the 480B is the most capable open coder if you have 64GB+ unified memory or multi-GPU.
Best for: your general coding agent, tool-use workflows, and anyone who wants one model that does 80% of the job well. Weak spot: nothing glaring — it's the safe default.
2. GLM-4.x — the long-session agent on a budget
GLM punches above its size on long, tool-heavy agent runs, and the Flash variant is the lightest thing that's genuinely usable.
- GLM-4.7-Flash — ~59% SWE-bench, runs on 5–8GB laptops. The "I only have an integrated-ish GPU" option that isn't a toy.
- GLM-4.6 / 4.x (MIT license) — recommended for long, multi-step agent sessions where consistency over many calls matters more than raw single-shot score.
Best for: laptop-tier hardware, long agentic sessions, permissive-license shops. Weak spot: lower ceiling on hard single-shot coding than Qwen 27B.
3. DeepSeek V3.2 / V4-Pro — the highest open ceiling
DeepSeek posts the top open result — ~80.6% SWE-bench Verified for V4-Pro — but it's "local" only if your desk has a server in it (80GB+).
Best for: pure, hard code tasks where you have server-class hardware or a self-hosted box for a team. Weak spot: not a consumer-GPU model; this is the escalation tier, not the daily driver.
4. Devstral Small 2 — the permissive-license workhorse
~68% SWE-bench Verified, Apache-2.0, fits 12–16GB. The best "I need a clean license and it runs on a mid-range card" option, and a solid agentic coder for its size.
Best for: commercial use with license certainty, mid-range GPUs, agent sub-tasks. Weak spot: multi-file reasoning tops out below the 27B tier.
5. Kimi K2.x — long-horizon frontier (if you have the rack)
Kimi targets long-horizon agentic work and scores well on the harder Pro benchmark, but needs heavy quantization or server hardware to run locally.
Best for: long autonomous runs on serious hardware. Weak spot: not a realistic single-consumer-GPU pick; benchmark figures are on Pro, so don't compare them head-to-head with the Verified numbers above.
6. GPT-OSS (20B / 120B) — OpenAI's open weights, Codex-native
OpenAI's open-weight line slots naturally into Codex CLI via the --oss flag. The 20B runs on consumer hardware; the 120B is server-tier.
Best for: teams already standardized on Codex who want a local fallback with zero tooling friction. Weak spot: Qwen/GLM generally edge it on open tool-calling reliability.
7. Gemma 4 — the lightweight generalist
Gemma 4 (12B / 27B) is the lighter-hardware generalist — great for a 16GB machine when you want a capable all-rounder rather than a code specialist.
Best for: laptops, mixed coding+writing use, low-VRAM boxes. Weak spot: a generalist, not a coding specialist — it'll trail the coder-tuned models on hard SWE tasks.
(Honourable mention: **MiniMax M2.x, trained agent-first and priced aggressively — worth watching if throughput-per-dollar is your constraint.)
The efficiency knee: more hardware ≠ better
Here's the chart that should actually drive your GPU decision — performance against the VRAM it costs to get there.
The curve climbs steeply up to ~24GB and then flattens — the 480B coder even dips below the 27B on Verified while costing 3× the memory. The lesson: a single 24GB consumer GPU running Qwen 3.6 27B is the sweet spot for self-hosting. Everything past it is diminishing returns unless you specifically need the server-tier ceiling.
The wall the leaderboards hide: tool calling
This is the finding that reorganized how I pick a local model. Tool calling is a separate trained skill from writing code. A model can produce a beautiful function and still fail the one thing an agent needs — emitting a valid tool call on the first try, every try.
What the evidence converges on:
- Reliability falls off a cliff below ~4B parameters. Sub-7B models emit malformed tool calls no matter which agent wraps them. The harness can't save a model that can't format the call.
- Native tool-use training beats bolted-on grammar constraints. Models trained with a tool template behave; models with constrained decoding stapled on top drift.
- The model matters more than the harness. Swapping Claude Code → OpenCode → Codex won't fix a model that garbles JSON.
- Multi-step chains are where it dies. One tool call is fine. Five in a row, each depending on the last, is where smaller local models start hallucinating calls — inventing a function that doesn't exist, or re-calling one with drifted arguments — and the agent spirals into a retry loop.
The practical cut: for agentic work, want a 14B–32B model from a family with native tool training. In late 2026 that overwhelmingly means Qwen, with GLM-4.x and Devstral Small 2 as credible alternates. Anything under ~7B is a chat model cosplaying as an agent.
The setup is genuinely 60 seconds
Because the agents speak local now, this is anticlimactic — which is the point.
Claude Code → local model (via Ollama):
# 1. pull a capable coder
ollama pull qwen3-coder:30b
# 2. point Claude Code at localhost + a throwaway token
export ANTHROPIC_BASE_URL="http://localhost:11434"
export ANTHROPIC_AUTH_TOKEN="local-dummy"
# 3. run it
claude
Codex CLI → local model: Codex talks to any OpenAI-compatible endpoint. With Ollama, the --oss flag points it at the local server (:11434); LM Studio serves on :1234/v1. Pin providers/models in config.toml.
Harness pick: Ollama is the terminal-first option (it even has one-command launchers for Claude Code / Codex / OpenCode); LM Studio is the GUI. Either works — remember, the model, not the harness, decides your tool-call reliability.
Two gotchas that each cost me an evening:
- Set the context window explicitly — at least 32K, ideally 64K+. Defaults are often tiny and a coding agent starves instantly. But quality degrades well before the advertised limit, and the KV cache eats the VRAM you wanted for weights. Budget for both.
-
To go back to the cloud:
unset ANTHROPIC_BASE_URL(or remove it fromsettings.json). Obvious until you forget and wonder why "Claude" got dumber.
Hardware tier guide
| Your hardware | Pick | Reality |
|---|---|---|
| 8GB laptop / small GPU | GLM-4.7-Flash, Gemma 4 12B | Usable for single-file help; expect retries on agentic work |
| 12–16GB GPU | Devstral Small 2, Qwen3-Coder 14B-class | Solid daily coder; clean Apache-2.0 option |
| 24GB GPU (the sweet spot) | Qwen 3.6 27B | The one to buy for; reliable tool caller, ~77% SWE-bench |
| 64GB+ unified / multi-GPU | Qwen3-Coder-480B, GPT-OSS 120B | Specialist ceiling; diminishing returns |
| Server 80GB+ | DeepSeek V4-Pro, Kimi K2.x | Team box / escalation tier, not a desk machine |
Where it actually breaks (the honest part)
Run this for a week on real work and the pattern is consistent: a good local coder gives you roughly 70% of cloud Claude for routine single-file tasks, and the gap widens fast as the work gets agentic. The failure modes, in order of how often they bite:
- Schema drift → retry loops. The model emits a slightly malformed edit, the harness rejects it, it drifts differently on the retry. On a hard step this eats a dozen round-trips.
- Multi-file reasoning falls apart. "Rename this across the module and fix the imports" is where cloud is boring and reliable and a local 30B gets lost by file three.
- Latency you feel. A chain cloud finishes in seconds can take minutes locally — not because tokens/sec is bad, but because every retry is another full generation.
- The context cliff is early and sharp. The model gets subtly worse around half the stated window, and "subtly worse at following the plan" is exactly what you can't afford in an agent.
None of this makes local useless. It makes local a different tool, not a cheaper clone.
So does it actually save money?
This is the real reason people try, so let's do the math instead of waving at "free." Local isn't free — it's prepaid. You trade a metered API bill for fixed hardware + electricity + time spent babysitting retries.
- A used 24GB GPU that runs Qwen 3.6 27B comfortably: a few hundred dollars, one time.
- Electricity under sustained load: real but small — single-digit dollars a month for an individual.
- Against that: one heavy Claude Code user can burn hundreds a month, and teams running agent swarms post eye-watering bills (the "$4,660 in a month" screenshots are a genre now).
Honest conclusion: for an individual doing casual coding, local rarely pays back versus just using the cloud carefully — the hardware and fiddling cost more than you'll save. Where it flips hard is high-volume, repetitive, or privacy-bound workloads — especially agent swarms, where you route the cheap 80% of calls to a local model and spend frontier tokens only on the 20% that need them. That routing — not "replace Claude Code" — is the actual 2026 win.
The framework that makes the call
Stop asking "which model is best." Ask "which job is this."
Reach for local when the work is:
- Private or offline — code that can't leave the machine.
- High-volume, cheap-per-task — the thousand little classify/summarize/boilerplate sub-calls in a swarm.
- Narrow and single-file — a function, a test, a regex, a migration.
- A credit-saver — route the easy 80% local, escalate the hard 20%.
Keep cloud Claude/Codex for:
- Multi-file, long-horizon changes where consistency over many steps is the value.
- Large-context work where you need the whole window to actually work.
- "Right the first time, fast" — anything where retry loops cost more than tokens.
The one-liner I've landed on: local is becoming the default, and the frontier model is the escalation path. That's the opposite of how most teams run it today — and it's where the money is.
The real answer
Can a local LLM replace Claude Code? For being your agent on hard, multi-step, large-context work — not yet, and the thing holding it back isn't the SWE-bench score, it's tool-calling reliability over long sessions. For being the cheap, private, always-available workhorse for the 80% of tasks that don't need a frontier model — absolutely, today, on a single 24GB GPU, and it's getting better monthly.
The mistake is treating it as a binary. The people getting real value aren't "replacing" anything. They run two models and know which job goes where.
If this was useful, follow me — I write about AI agents, local models, and the unglamorous engineering around them, and I'm testing more of these setups as the models ship.
Your turn: what's the actual line for you — the specific task where you reach for the cloud and refuse to trust a local model? I'm collecting the real break points, not the benchmark ones. 👇
Sources & verify it yourself
Numbers in this post are vendor-reported and move monthly — don't buy hardware on the strength of a blog percentage. Check them at the primary sources:
- SWE-bench leaderboard — the official coding benchmark; confirm Verified vs Pro before comparing.
- Berkeley Function-Calling Leaderboard — the one that actually measures tool-calling, the metric this whole post argues matters most.
- Ollama — local serving + the native Anthropic/OpenAI-compatible endpoints that make the Claude Code / Codex setup a one-liner.
- Model cards for the real sizes, licenses and quant builds: Qwen, DeepSeek, Mistral / Devstral, GLM (Z.ai).
The cost and tool-calling-tier charts are illustrative, drawn to show the shape of the trade-off — plug in your own API bill and GPU price for the numbers that apply to you.
Top comments (1)
The line you're collecting has a number attached, and it is not a benchmark score — so here is the arithmetic for both halves of your argument.
Your "ten times in a row" bar, in per-call terms. If one call is valid with probability q and the run needs k calls, the run survives with q^k — a lower bound, since your drift observation is exactly the reason a retry does not fully recover:
Read the third column: to finish an 11-call chain 90 % of the time you need q ≥ 0.9905 per call; for 95 % you need 0.9953. That number is not on any of the charts — a tool-call score reports q, almost nothing reports the k at which q^k crosses your acceptance threshold. It is also why "reliable tool caller" and "reliable agent" are different claims: 0.99 and 0.995 are indistinguishable on a single-call chart and differ by a third on a 50-call run (0.61 vs 0.78).
The SWE-bench comparison cannot carry its own headline. For a paired comparison over the 500 Verified problems, the gap a pair of runs can resolve is about 1.96·√d ⁄ 500, where d is the number of problems the two models disagree on. A one-point gap is resolvable only if they disagree on fewer than 7 of the 500. Two models sitting at 76 % and 77 % do not have near-identical failure sets — they disagree on tens of problems — so at d = 30 the comparison resolves nothing below 2.2 points and at d = 50 nothing below 2.8. "Within a rounding error of the frontier" is not a small gap; it is a gap the instrument does not see, and the two readings lead to different actions: a small gap invites you to close it, an unresolvable one says the leaderboard column has stopped being the thing worth tuning. Same arithmetic kills the cross-family comparisons later in the piece — Verified, Pro and Terminal-Bench numbers from different harnesses are not one scale.
So the line, stated as a measurement rather than a task list: per task class, the per-call validity q, and the chain length k at which q^k falls below what you accept. Neither needs a GPU purchase to obtain. Run the same class twice — the rerun is the only control that separates the model from the session, and without it your week of experience is a single sample of your own sampling noise — and count calls, not tasks. The break point you publish then has a denominator attached.
(A small caveat on the second table: d is the quantity nobody publishes. If you have outputs for two local models on the same Verified subset, d is one pass over the per-problem results, and it is worth a line in the post — it converts "looks close" into "close, given these two disagree on N of 500", which is the form a reader can check.)