DEV Community: Shek

Migrating Claude Code to a custom backend in 2 lines (and what to actually watch for)

Shek — Sat, 06 Jun 2026 04:52:03 +0000

TL;DR: Claude Code's ANTHROPIC_BASE_URL env var lets you redirect every request to any compatible backend in one shell line. Almost no one does it, but it unlocks caching, rate-limit pooling, multi-vendor routing, audit logging, and billing routing without rewriting any code. Here's the 2-line setup, the things that quietly break, and the production checklist for actually running it.

The 2-line setup

export ANTHROPIC_BASE_URL=https://your-proxy.example.com
export ANTHROPIC_API_KEY=your-key
claude

That's it. The Anthropic SDK (which Claude Code is built on) doesn't care where it's sending requests, as long as the response comes back in Anthropic Messages format. It'll append /v1/messages automatically, forward all your anthropic-version and anthropic-beta headers, and stream SSE the same way.

Same trick works with Cursor, Cline, Continue, and anything else that builds on the Anthropic SDK.

Why would you do this?

I've found five patterns that justify a custom backend:

1. Caching middleware

Anthropic's prompt cache feature requires manual cache_control markers that most teams skip. A proxy can inject them server-side on every system message. For a Claude Code session with a 20-30K-token system prompt and 50+ turns, this cuts your bill by 60-80%.

2. Rate-limit pooling

Most teams have 3-5 API keys spread across projects. Without a proxy, each is rate-limited independently — you can hit 429 on one while another sits idle. A proxy can round-robin across keys, exposing one virtual key with the combined rate budget.

3. Multi-vendor abstraction

Want to route claude-haiku-4-5 calls to one provider and claude-opus-4-7 calls to another (maybe one offers Haiku throughput discounts)? A proxy inspects the model field and routes accordingly. Your Claude Code config doesn't change.

4. Audit logging

Claude Code makes a lot of requests. Without instrumentation, you have no idea what each session cost. A proxy can log per-request metadata (model, tokens, latency, session id) to a warehouse.

5. Billing routing

Different team members hitting different cost centers? A proxy can map API keys to billing tags and forward usage to finance.

What breaks (and what to watch for)

The 2-line setup works for the happy path. In practice, here's what bites you.

Streaming behavior

Anthropic returns chunked SSE for stream: true. If your proxy buffers responses or doesn't flush each chunk immediately, you'll see massive perceived latency — Claude Code will spin while waiting for the full response.

Fix: pass through SSE chunks without buffering. In Caddy, that means flush_interval -1. In Cloudflare Workers, use TransformStream with explicit .write() per chunk. In Node, set noDelay: true on the response socket.

Header forwarding

Anthropic's SDK sends headers like:

anthropic-version
anthropic-beta (for prompt cache, batches, etc.)
x-api-key (auth)
x-stainless-* (SDK telemetry)

If your proxy strips any of these before forwarding to the real upstream, certain features break silently:

Strip anthropic-version → SDK gets default-version response which may not match its expectations
Strip anthropic-beta → prompt cache headers ignored, you wonder why caching isn't working
Strip x-stainless-* → harmless

Rule of thumb: forward everything except host and authorization (since you're rewriting those).

Tool use iteration loops

Claude Code uses tool calling heavily. A typical session has 30-50 tool call rounds, each one a separate API call with the full conversation history.

If your proxy is rate-limited at 60 RPM per source IP, a single Claude Code session can blow through it in 30 seconds.

Fix: rate-limit per API key, not per source IP. And budget for high burst rates (200+ RPM during heavy tool-call sessions).

Request body inspection

If you want to inject cache_control or rewrite request bodies, you have to parse the JSON, modify, re-serialize. Be careful: large requests (50K+ tokens of context as JSON-encoded strings) take noticeable CPU time to parse.

Minimal injection in Node:

const body = JSON.parse(rawBody);

// Inject cache_control on the first system message
if (body.system && typeof body.system === 'string') {
  body.system = [{
    type: 'text',
    text: body.system,
    cache_control: { type: 'ephemeral' },
  }];
}

const newRawBody = JSON.stringify(body);

For Claude Code's typical traffic this adds <5ms per request. For very large workloads, consider stream-parsing.

Streaming error handling

Anthropic sends error SSE events mid-stream when something fails (rate limit, content filter, etc.). Most naive proxies forward the body as opaque bytes, which works. If your proxy tries to be "smart" and parse the SSE, you have to handle error events without breaking the connection.

The safe path: don't parse SSE in your proxy unless you absolutely have to.

A minimal production checklist

If you're building this proxy yourself:

[ ] TLS termination (Let's Encrypt or Cloudflare front)
[ ] Pass-through SSE streaming (no buffering)
[ ] Per-key rate limiting (not per-IP)
[ ] Forward all headers except host/authorization
[ ] Handle JSON request body modification gracefully
[ ] Log per-request metadata without storing message bodies (privacy)
[ ] Retry on upstream 5xx with exponential backoff
[ ] Health check endpoint that doesn't proxy
[ ] Graceful degradation when upstream is down (return 502 fast, don't hang)
[ ] CORS headers if you want browser clients to hit it directly (TypingMind etc.)

That's not "2 lines" anymore. The 2-line part is the SDK config; the checklist is the actual cost of running it in production.

The side benefit no one talks about

When you put a proxy in front of Claude Code, you get observability for free. You can see:

Every request and its token count
Which sessions are expensive
Which prompts repeat (cache candidates)
Tool-call patterns
Latency distribution per model
Time-of-day spend curves

Anthropic's dashboard tells you the bill. Your proxy tells you why. For any team running Claude Code at scale, that's often more valuable than the cost optimization itself.

When NOT to do this

A few cases where a custom backend is more trouble than it's worth:

Single developer running claude once a day — direct API is fine
Team < 5 people with simple usage patterns
No engineering bandwidth to maintain a proxy
Using Anthropic-specific beta features that might break in a proxy chain

The break-even point is roughly:

You have > 3 API keys to manage, OR
Monthly Claude bill > $500, OR
You need cross-team usage analytics, OR
You're integrating with multiple LLM vendors

Below that, just use the real Anthropic API and pocket the engineering time.

If you don't want to build it

I built MidRelay precisely because this checklist took 6 weeks to get right and I figured other teams shouldn't repeat it. Hosted proxy with both Anthropic and OpenAI surfaces, prompt cache injection on by default, per-key usage logs, CORS enabled for browser clients.

Pointing Claude Code at it is the 2-line setup at the top of this post.

But honestly: the techniques here work whether you use MidRelay, build your own, or pick any of the other gateways. The interesting thing is almost nobody knows you can point Claude Code at a custom backend, even though Anthropic explicitly documented it. If this post does nothing else but make you realize that capability exists, that's worth your reading time.

If you've built something similar, drop a comment with what bit you in production — I'm collecting patterns for a follow-up post on multi-vendor LLM routing.

Why your Claude API bill is 3x what it should be (and how to fix it)

Shek — Fri, 22 May 2026 19:24:53 +0000

TL;DR: I audited a friend's startup that was spending $4,200/month on Claude API. Only $1,300 produced business value. The other $2,900 was waste — split across three patterns that hit most teams using LLM APIs in production. Here's how to find them in your own bill, and the code to fix each one.

The audit

A friend running a B2B doc-summarization product asked me to look at their Claude bill. Q1 was $4,200/month and climbing. We pulled their request logs into a spreadsheet, classified each call by purpose, then estimated what each should have cost. The answer was uncomfortable:

Bucket	Cost	Producing business value?
First-time doc analysis	$890	Yes
User chat turns	$410	Yes
Repeated system prompts (no cache)	$1,810	No
Opus calls that should be Sonnet	$680	No
Serial bulk runs (should be batched)	$410	No

Three problems, $2,900/month of waste. Each one is unsexy and easy to miss, but together they were 70% of the bill.

Culprit #1: prompt caching is off

This is the silent killer. Claude 4.x supports prompt caching: send a 5-minute or 1-hour TTL cache_control block, and Anthropic charges you ~10x less for cached tokens on subsequent requests. Pricing today (per million tokens for Sonnet 4.6):

Fresh input: $3.00
Cache write: $3.75 (one-time, slightly more than fresh)
Cache read: $0.30 — 10x cheaper

The catch: you have to opt in per-request, and most code doesn't. Before/after:

# Before — every call pays for the full system prompt
client.messages.create(
    model="claude-sonnet-4-6",
    system="You are an expert at...[2000 words of rules + examples]",
    messages=[{"role": "user", "content": user_input}],
    max_tokens=1024,
)

# After — system prompt cached for 5 minutes
client.messages.create(
    model="claude-sonnet-4-6",
    system=[
        {
            "type": "text",
            "text": "You are an expert at...[2000 words of rules + examples]",
            "cache_control": {"type": "ephemeral"},
        },
    ],
    messages=[{"role": "user", "content": user_input}],
    max_tokens=1024,
)

One-line change. 90% discount on every subsequent call within the cache TTL.

For my friend: 20K tokens of system prompt × 8 requests/min × 50% cache hit ratio = ~$80/day saved. That alone was $2,400/month — most of the $1,810 leak.

OpenAI SDK calling Claude (via compatible proxies) has equivalent semantics:

client.chat.completions.create(
    model="claude-sonnet-4-6",
    messages=[...],
    prompt_cache_key="user-session-12345",  # Stable across calls = cache hit
)

Action: open your last week of API logs. If you have any repeated system content across requests, you're leaking.

Culprit #2: model overkill

The mental shortcut "Claude = quality, just always use Opus" is expensive. Opus is 4x the cost of Sonnet for inputs, 5x for outputs. For a lot of work, Sonnet or even Haiku is indistinguishable.

I ran 5 tasks across the lineup (1000 samples, scored by judge model + human spot-check):

Task	Opus 4.7	Sonnet 4.6	Haiku 4.5	Best price/quality
JSON extraction from PDFs	99.2%	98.7%	96.4%	Haiku
Code review (real bugs)	74%	74%	61%	Sonnet
Creative copy (blind judged)	51% pref	48% pref	32% pref	Sonnet
Multi-step reasoning chain	89%	76%	54%	Opus
Customer chat	92% sat	89% sat	81% sat	Sonnet

The pattern: Opus wins clearly only on complex multi-step reasoning. For most tasks Sonnet is within margin of error at 1/4 the cost. Haiku trades 2-5% accuracy for 1/13 the cost — fine when you have downstream validation.

My friend was running every doc through Opus by default. Switching to Sonnet for analysis + Haiku for tagging dropped that bucket from $680 to $140. No quality complaints.

Action: pick the 3 most expensive endpoints in your bill, A/B-test them on the next cheapest model for a week, score outputs blind.

Culprit #3: serial calls when you could batch

If your work doesn't need a response in the next 30 seconds, the Anthropic Message Batches API charges half price with a 24-hour SLA. Same models, same quality, half the bill.

Good fits:

Nightly summarization runs
Classifying or tagging large datasets
Embedding generation for indexing
Internal report generation
Training data prep

Bad fits:

Anything user-facing (you'll wait hours)
Anything where input depends on previous output

batch = client.messages.batches.create(
    requests=[
        {
            "custom_id": f"doc-{doc.id}",
            "params": {
                "model": "claude-sonnet-4-6",
                "max_tokens": 1024,
                "messages": [{"role": "user", "content": doc.text}],
            },
        }
        for doc in docs
    ],
)

# Poll until done (or just check tomorrow)
while batch.processing_status != "ended":
    time.sleep(60)
    batch = client.messages.batches.retrieve(batch.id)

My friend had a nightly job re-summarizing all docs from the previous 24h. Moving it from asyncio.gather to batches cut that bucket from $410 to $205, no user-visible impact.

Action: any cron job, weekly report, or async task hitting your LLM API — most can be batched.

The result

After three changes (cache hint, model rebalance, batch the async work), my friend's monthly bill went $4,200 → $1,540. Same product, same quality, no rewrites — just turning on features the API already supports.

If your bill feels high, do the same audit:

Pull last 30 days of API calls
Count distinct system prompts. <10 unique but >10,000 calls = no caching
Look at top 5 model+endpoint combos by spend. Anything simple enough to downshift?
Find your largest single-day spike. Batch job? Use the batches API.

A shortcut, if you don't want to instrument all this

I built a little proxy called MidRelay that handles the first two automatically: it injects a per-key cache hint into every request (even SDK code that doesn't know about cache_control gets the discount), and it exposes both OpenAI and Anthropic surfaces from the same key so you can route model-by-model without rewriting.

It also happens to be 60-80% cheaper than calling Anthropic / OpenAI directly. (Same models, same wire protocol — your existing SDK just changes the base_url.)

$5 of free credit to test it: drop a comment, I'll DM a code. First 100 readers, no signup gate.

But honestly — the techniques above work on any provider. Even if you never touch MidRelay, just turning on cache_control and downshifting one over-spec'd Opus call will cut your bill more than any "AI cost optimization" SaaS will.

Check your logs tonight.