I pay for three different AI subscriptions and I kept catching myself doing the same dumb thing: opening a second terminal, or a browser tab, just to hand a task to a different model, because Claude Code only ever talks to Claude. Meanwhile the model picker inside Claude Code sits right there, teasing you with a dropdown that only ever has Anthropic’s own lineup in it.
I’d seen a few people online building small gateway plugins to work around this, routing Claude Code’s traffic out to other providers behind the scenes. That got me curious enough to actually try building the plumbing myself instead of trusting a screenshot. What I found is that the trick isn’t exotic. Claude Code already exposes the one environment variable you need, ANTHROPIC_BASE_URL, and the rest is just picking (or writing) a proxy that speaks Claude's API on one side and whatever you actually want to pay for, or not pay for, on the other.
This is the writeup of what I set up, what broke, and the one version of it I’ve kept running.
Why this isn’t as hacky as it sounds
Claude Code, like most CLI coding agents built on top of the Anthropic SDK, doesn’t hardcode api.anthropic.com. It reads ANTHROPIC_BASE_URL (and ANTHROPIC_AUTH_TOKEN for the key) and sends every request there instead. Anthropic ships this specifically so enterprises can point Claude Code at a private gateway, a compliance proxy, or a regional mirror. Nobody designed it as a "bring your own model" escape hatch, but that's exactly what it becomes once you realize the proxy on the other end doesn't have to be Anthropic-shaped at all. It just has to accept requests in the Anthropic Messages format and respond in the same shape Claude Code expects, including the streaming event format and tool-call blocks.
That’s the whole trick: a small HTTP service that translates Anthropic’s request and response schema into whatever the actual backend speaks (usually OpenAI’s chat completion format, since that’s the lingua franca almost every provider, including local ones, has converged on) and translates the response back.
Claude Code --Anthropic Messages format--> local proxy --OpenAI format--> GPT-5 / Grok / Ollama / whatever
Claude Code <--Anthropic Messages format-- local proxy <--OpenAI format-- (response, tool calls, streaming deltas)
Once you see it drawn out like that, the “why doesn’t Claude Code just support other models” question stops mattering. It already does, indirectly, as long as something sits in the middle doing the translation.
The ecosystem is small but it’s real
I expected to find one obvious tool. Instead I found a handful of small, mostly single-maintainer projects, all solving the same translation problem with slightly different scopes. Here’s what I actually tried, not just what showed up in search results.
+---------------------------+--------+------------------------------+------------------------------------------+
| Project | Stack | Backends | Notes |
+---------------------------+--------+------------------------------+------------------------------------------+
| claude-code-proxy | Go | OpenAI, OpenRouter, Ollama | Single binary, pattern-based model |
| (nielspeter) | | | mapping (*opus*/*sonnet*/*haiku*), |
| | | | full streaming + tool call support |
+---------------------------+--------+------------------------------+------------------------------------------+
| claude-code-ollama-proxy | Python | Ollama, OpenAI (fallback), | Built specifically for local-first use, |
| (mattlqx) | / uv | Gemini | uses LiteLLM under the hood for format |
| | | | translation |
+---------------------------+--------+------------------------------+------------------------------------------+
| claude-code-router | Node | OpenAI, Anthropic, Gemini, | The most feature-complete option, ships |
| (musistudio) | | DeepSeek, Kimi, custom | a UI, rule-based routing (background / |
| | | endpoints | think / longContext), retries, failover |
+---------------------------+--------+------------------------------+------------------------------------------+
| LiteLLM proxy | Python | 100+ providers | Not built for Claude Code specifically, |
| (generic) | | | but exposes an Anthropic-compatible route |
| | | | so it works the same way |
+---------------------------+--------+------------------------------+------------------------------------------+
They’re all MIT licensed, all run on localhost by default, and all do the same core job. The difference is mostly how much routing logic they bolt on top of the translation layer. I started with the simplest one so I could actually understand the request/response shape before trusting a router’s rule engine to make decisions for me.
Setting it up with a paid backend first
I wanted to confirm the plumbing worked before I dragged a local model into it, so the first pass used OpenAI as the backend through claude-code-proxy.
# clone and build (Go toolchain required)
git clone https://github.com/nielspeter/claude-code-proxy.git
cd claude-code-proxy
go build -o claude-code-proxy cmd/claude-code-proxy/main.go
Config lives in a plain env file:
# ~/.claude/proxy.env
OPENAI_API_KEY=sk-...
OPENAI_BASE_URL=https://api.openai.com/v1
# map Claude's model tiers to whatever you're actually paying for
ANTHROPIC_DEFAULT_OPUS_MODEL=gpt-5
ANTHROPIC_DEFAULT_SONNET_MODEL=gpt-5
ANTHROPIC_DEFAULT_HAIKU_MODEL=gpt-5-mini
Start the proxy, then point Claude Code at it instead of Anthropic’s own endpoint:
./claude-code-proxy & # listens on localhost:8082 by default
export ANTHROPIC_BASE_URL=http://localhost:8082
claude
That’s it. Claude Code boots up exactly as normal, the interface doesn’t change, but every message now round-trips through GPT-5 instead of Claude. Tool calls, file edits, the whole agentic loop, all worked without me touching Claude Code’s own configuration.
The thing that actually surprised me: this proxy pattern already existed before “model routing plugins” became a thing people were writing blog posts about. It’s the same idea Anthropic’s own enterprise gateway docs describe, just self-hosted and pointed somewhere unofficial.
The part I actually cared about: running it against a free, local model
Paying to run GPT behind Claude Code defeats half the point for me. What I wanted was a way to hand the boring 80% of a session (formatting fixes, rote refactors, “write the obvious test for this function”) to something free and local, and save the paid, higher-quality models for the parts that actually need judgment.
Ollama makes this almost embarrassingly simple, because as of recent releases it exposes an OpenAI-compatible endpoint natively, at /v1, no adapter required.
# install Ollama if you don't have it: https://ollama.com/download
ollama pull qwen2.5-coder:14b
ollama serve
Then point the same proxy at it instead of OpenAI:
# ~/.claude/proxy.env
OPENAI_BASE_URL=http://localhost:11434/v1
# no API key needed for local Ollama, the field just needs to be non-empty
OPENAI_API_KEY=ollama
ANTHROPIC_DEFAULT_SONNET_MODEL=qwen2.5-coder:14b
ANTHROPIC_DEFAULT_HAIKU_MODEL=qwen2.5-coder:7b
./claude-code-proxy &
export ANTHROPIC_BASE_URL=http://localhost:8082
claude
No API key, no billing dashboard, nothing leaves my machine. If you’d rather not install Ollama natively, the Docker version is the same setup in a container:
# docker-compose.yml
services:
ollama:
image: ollama/ollama:latest
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
volumes:
ollama_data:
docker compose up -d
docker exec -it $(docker compose ps -q ollama) ollama pull qwen2.5-coder:14b
Same OPENAI_BASE_URL=http://localhost:11434/v1, nothing else changes.
Where it actually broke
None of this worked perfectly on the first try, and the failure modes are worth listing because they’re the reason “just point it at a proxy” undersells the effort involved.
+-------------------------------------+--------------------------------------------------------------+
| What broke | Why, and what fixed it |
+-------------------------------------+--------------------------------------------------------------+
| Tool calls silently stopped mid-task | Smaller local models (7B/8B) don't reliably emit well-formed |
| | tool-call JSON under Claude Code's system prompt, which is |
| | dense and multi-step. Bumping to a 14B code-tuned model fixed |
| | most of it, not all of it. |
+-------------------------------------+--------------------------------------------------------------+
| Streaming responses arrived as one | The proxy has to translate each OpenAI streaming delta into a |
| big chunk instead of token-by-token | matching Anthropic streaming event. If it buffers instead of |
| | forwarding incrementally, you lose the live-typing feel, it |
| | still works, it just feels laggy. |
+-------------------------------------+--------------------------------------------------------------+
| Claude Code's /model picker didn't | The simple proxies don't hook into the picker UI at all, model |
| show my local model as an option | selection happens entirely through the env file mapping, not |
| | inside Claude Code itself. Only the router-style tools with a |
| | UI (claude-code-router) actually inject entries into /model. |
+-------------------------------------+--------------------------------------------------------------+
| Context got truncated on long | Local models running through Ollama default to a much smaller |
| sessions | context window than the model card advertises unless you |
| | explicitly set `num_ctx` when pulling or running the model. |
+-------------------------------------+--------------------------------------------------------------+
That context window one cost me a confusing afternoon. Ollama will happily load a model advertised as supporting 128k tokens and then silently cap the actual context at 2048 or 4096 unless you override it, which means a long Claude Code session degrades into the model “forgetting” the start of the conversation with no error message at all. Setting it explicitly is not optional if you’re doing anything beyond a quick one-off:
ollama run qwen2.5-coder:14b --keepalive 30m
# or, more durably, in a Modelfile:
# PARAMETER num_ctx 32768
The routing decision that actually matters
Getting the plumbing to work is the easy 20%. The harder question is deciding what should go where, and this is where I stopped treating it as a binary “local vs paid” choice and started thinking about it in terms of task shape.
+----------------------------------+-------------------------------+---------------------------+
| Task type | What I route it to | Why |
+----------------------------------+-------------------------------+---------------------------+
| Mechanical edits, renames, | Local Ollama model | Free, fast enough, doesn't |
| boilerplate, formatting fixes | (qwen2.5-coder:14b) | need real judgment |
+----------------------------------+-------------------------------+---------------------------+
| Everyday feature work, most | Mid-tier hosted model | Good enough reasoning at a |
| refactors, writing tests | (gpt-5-mini class) | fraction of top-tier cost |
+----------------------------------+-------------------------------+---------------------------+
| Architecture decisions, gnarly | Whatever your best model is | This is where model quality |
| bugs, anything security-adjacent | (Claude Opus, GPT-5, etc.) | differences actually show |
+----------------------------------+-------------------------------+---------------------------+
If I only ever route by “is this cheap or free,” I end up asking a 14B local model to do things it genuinely can’t do reliably, and I burn more time debugging its confidently wrong tool calls than I save on inference cost. The proxy setup makes the routing mechanically possible. It doesn’t make the judgment call for you, and honestly that part didn’t get easier the more tools I tried, it just got easier once I stopped expecting a plugin to solve it automatically and started manually swapping the env file based on what I was actually about to ask for.
One thing worth flagging before you copy any of this
Running the proxy on localhost only is the safe default. If you change the bind address to 0.0.0.0 so you can reach it from another machine on your network, which I did briefly to test from a laptop, you're now exposing an unauthenticated endpoint that can spend real money on whatever paid backend you've configured, to anyone on that network. None of the proxies I tried ship auth on by default. If you need remote access, put it behind something that actually checks a credential, don't just open the port.
Where I landed
I kept the simplest option running, claude-code-proxy with two env files I swap between: one pointed at a paid model for anything that needs real reasoning, one pointed at local Ollama for the mechanical stuff. I didn't end up needing the full router-with-a-UI tool, mostly because my actual routing decision is coarse enough (two buckets, not five) that a config file swap is less friction than learning a rules engine.
The bigger takeaway for me wasn’t really about Claude Code specifically. It’s that the “agentic coding tool” and the “model behind it” are more decoupled than the interface makes them look, and once you’ve traced through one translation proxy, that decoupling stops feeling like a hack and starts feeling like the actual architecture underneath most of these tools. Claude Code just happens to be the one I use daily, so it’s the one I bothered to wire up.
If you try this yourself, start with a paid backend first, the same way I did. It’s much easier to tell whether your proxy setup is broken versus whether your model is just too small for the task, if you’re not debugging both variables at once.
Tags: claude-code, ollama, llm-routing, self-hosted, developer-tools, ai-agents, local-llm
Top comments (0)