DEV Community: Bo Shen

Claude Fable 5 Went from Free to Offline in 72 Hours — What I Learned About AI Coding Costs

Bo Shen — Mon, 15 Jun 2026 20:30:01 +0000

Last week, Anthropic launched Fable 5 — their most powerful model ever — free for all Pro/Max subscribers through June 22.

Three days later, the US government issued an export control directive. Fable 5 went dark worldwide.

Developers who hardcoded claude-fable-5 in their workflows woke up to broken pipelines. Anthropic received the directive at 5:21pm ET on June 12 and had to comply immediately.

This isn't a post about geopolitics. It's about what this event reveals about the true cost of AI-assisted coding — and why model routing is the most underrated skill in a developer's toolkit right now.

The Real Cost of AI Coding in June 2026

Let's talk numbers that most people aren't tracking:

Model	Input (per 1M tokens)	Output (per 1M tokens)	Typical coding session cost
Claude Fable 5	$10	$50	$5-15 per task
Claude Opus 4.8	$5	$25	$2-8 per task
Claude Sonnet 4	$1.50	$7.50	$0.50-2 per task
GPT-5.5	~$2.50	~$10	$1-3 per task

One Reddit user reported burning $200 in under 60 minutes with Fable 5. Another tracked 35 Claude Code subscriptions that would cost $80K/month at API rates.

The Insight: 80% of Your Coding Tasks Don't Need the Most Powerful Model

I run multiple AI coding agents daily across a portfolio of 10+ apps. Six months ago, my monthly AI coding bill hit $10K.

Today it's around $3K.

The difference wasn't switching to cheaper models across the board. It was routing different task types to the right model:

What Actually Needs Frontier Models (Fable/Opus)

Complex architectural decisions
Multi-file refactoring with subtle dependencies
Novel algorithm implementation
Debugging race conditions or memory leaks

What Works Great with Mid-Tier Models (Sonnet/GPT-5.5)

Boilerplate generation and scaffolding
Unit test writing
Documentation
Simple bug fixes
Code formatting and linting

What Smaller Models Handle Fine

Commit message generation
Simple string transformations
Template filling
Configuration file updates

When I actually tracked which model was doing what, I found that roughly 60-70% of my tokens were going to tasks that a Sonnet-class model would handle equally well.

The Fable 5 Shutdown Proved Something Else

Beyond cost, the overnight shutdown exposed a resilience problem.

If your entire workflow depends on a single model from a single provider, you don't have a workflow — you have a single point of failure.

My setup auto-fell back to Opus 4.8 when Fable went offline. No configuration changes, no manual intervention, no lost work. That's not because I predicted a government export control order. It's because I assumed any model can become unavailable at any time.

This has happened before:

OpenAI rate limits during peak hours
Anthropic's extended outage in March
Google's API deprecation cycle

Building model fallback chains isn't paranoia. It's good engineering.

How to Start Routing Today

You don't need fancy infrastructure. Here's a simple approach:

1. Classify your tasks

Before sending a prompt, tag it: planning, implementation, debugging, testing, documentation, formatting.

2. Create a routing table

planning       → opus/fable (complex reasoning matters)
implementation → sonnet (good enough, 5x cheaper)
debugging      → opus (needs deep understanding)
testing        → sonnet (formulaic, template-driven)
documentation  → sonnet (clarity over intelligence)
formatting     → haiku/small (trivial tasks)

3. Track and iterate

Log which model handled which task, then review: did the cheaper model produce acceptable results? Over time, you'll discover your personal routing table.

The Bigger Picture

The AI coding landscape in June 2026 looks like this:

Models are getting more capable AND more expensive at the top end
The gap between tiers is narrowing for common tasks
Availability is no longer guaranteed (regulatory, rate limits, outages)
Smart routing beats brute-force spending every time

The developers who'll thrive aren't the ones with unlimited API budgets. They're the ones who treat model selection as an engineering problem — matching the right tool to the right task, with fallbacks for when things go wrong.

I'm Bo. I run 10+ AI-powered apps and spend too much time thinking about model costs. Previously cut our team's Claude Code bill from $10K/mo to $3K with task-level routing. Find me @aplomb2 on X.

Claude Fable 5 Costs $50/M Output Tokens — Here's How I Cut My AI Coding Bill by 70%

Bo Shen — Thu, 11 Jun 2026 20:28:16 +0000

Anthropic just dropped Claude Fable 5, and the reactions are split right down the middle:

Developers who tried it: "This is the best coding model I've ever used."
Developers who saw the bill: "Wait, $50 per million output tokens?"

Both reactions are correct. Fable 5 is genuinely a leap forward for complex reasoning tasks. But at 2x the price of Opus 4.8, sending everything through it is financial suicide — especially for teams running multiple concurrent coding agents.

I've been running AI coding agents at scale for the past year. Here's how I keep my monthly bill under $3K while still using frontier models where they actually matter.

The Problem: Not All Coding Tasks Are Equal

Here's what most developers get wrong: they pick one model and use it for everything. Whether they're generating boilerplate, writing tests, debugging a gnarly race condition, or architecting a new service — same model, same cost.

But when I audited my team's actual Claude Code usage over 3 months, the breakdown was shocking:

Task Type	% of Total Tokens	Frontier Model Needed?
Boilerplate & scaffolding	~25%	❌ Haiku handles it fine
Test generation	~20%	❌ Sonnet is perfect
Simple refactors & linting	~15%	❌ Any model works
Feature implementation	~25%	⚠️ Sonnet/Opus depending on complexity
Complex architecture & debugging	~15%	✅ This is where Fable shines

Only ~15% of our token spend actually benefited from frontier-tier models. The other 85% was burning money on tasks where cheaper models produce identical results.

The Math That Changed Everything

Let's do some napkin math with current pricing:

Before routing (everything on Opus 4.8):

My team: ~400K output tokens/day across all agents
400K × 30 days × $25/M = ~$300/day = ~$9,000/month

After task-level routing:

60% of tokens → Haiku ($0.25/M output): $1.80/day
25% of tokens → Sonnet ($3/M output): $9.00/day
15% of tokens → Opus/Fable ($25-50/M output): $67.50/day
Total: ~$78/day = ~$2,340/month

That's a 74% reduction with zero quality loss on the complex work. In fact, the complex work got better because now we can afford to use Fable 5 where it actually matters instead of rationing a mid-tier model across everything.

How Task-Level Routing Actually Works

The concept is simple: classify the task, then pick the cheapest model that can handle it well.

Here's the decision tree I use:

1. Is this a known pattern? (boilerplate, CRUD, test scaffolding)
   → Haiku. Fast, cheap, good enough.

2. Does it require understanding context across multiple files?
   → Sonnet. Great balance of capability and cost.

3. Does it involve:
   - Complex multi-step reasoning?
   - Subtle bug hunting across a large codebase?
   - Architecture decisions with tradeoffs?
   - Novel algorithm design?
   → Opus or Fable 5. Worth every penny.

The key insight: you don't need to be perfect at classification. Even a rough 60/30/10 split saves massive money compared to running everything on a single tier.

What I Learned Running This for 6 Months

1. Cheaper models fail gracefully

When Haiku gets a task that's slightly too complex, it doesn't produce garbage — it produces a reasonable attempt that Sonnet can refine. The cost of occasionally "upgrading" a task is way less than running everything on expensive models.

2. Token count is a poor proxy for complexity

A 200-line test file generation burns lots of tokens but needs zero frontier reasoning. A 5-line debugging insight might need Fable-tier understanding. Route on task type, not token volume.

3. The model landscape is fragmenting fast

This week alone: Microsoft dropped 7 MAI models at Build, MiMo Code is matching Sonnet at a fraction of the cost, and Apple revealed Siri AI routes between multiple models including Gemini.

Even Apple — the world's biggest company — doesn't bet on a single model. They route dynamically based on the task.

4. The cost spread is only getting wider

Fable 5 at $50/M output vs Haiku at $0.25/M means a 200x price difference between the cheapest and most expensive Claude models. That spread makes routing not just nice-to-have — it's table stakes for anyone spending more than a few hundred a month.

Getting Started

You don't need a fancy routing framework to start:

Audit your usage: What percentage of your AI coding tasks actually need frontier reasoning? Most teams overestimate this by 3-5x.
Start with two tiers: Route "simple" tasks to Haiku, everything else to your current model. Even this basic split saves 40-50%.
Iterate: As you get comfortable, add a middle tier (Sonnet) and refine your classification rules based on actual results.
Measure: Track quality metrics alongside cost. You'll likely find that quality stays flat or improves (because you can afford frontier models for the hard stuff).

The Bigger Picture

The AI model market is heading toward massive fragmentation. New models ship weekly. Prices vary 200x between tiers. Every major player is building their own models.

The developers and teams who thrive won't be the ones using the most expensive model for everything. They'll be the ones who match the right model to the right task — automatically, at scale.

Fable 5 is incredible. Use it where it matters. Use something cheaper everywhere else. Your wallet will thank you.

I'm Bo — I've been building AI-powered apps and cutting costs through intelligent model routing. Follow me on X (@aplomb2) for more on making AI coding affordable.

Claude Fable 5: The 7.5x Cost Trap and How to Fix It with Task-Level Routing

Bo Shen — Wed, 10 Jun 2026 09:26:25 +0000

Anthropic dropped Claude Fable 5 yesterday — their most capable model ever. Everyone's talking about the benchmarks. But here's what actually matters for your bill: the same model can cost you 7.5x more depending on one setting.

Let me explain, and then show you exactly how we used this to cut our AI coding costs from $10K/mo to $3K/mo.

The Hidden Cost Lever

Fable 5 introduces 5 thinking effort levels: low, medium, medium-high, high, and max.

Same model. Same intelligence. But wildly different costs:

Thinking Level	Cost per Query	Relative Cost
Low	~$0.10	1x
Medium	~$0.20	2x
Medium-High	~$0.35	3.5x
High	~$0.50	5x
Max	~$0.72	7.5x

Most developers will leave this on default (high/max) and never think about it. That's the trap.

Why This Matters More Than You Think

Fable 5 costs $10/M input and $50/M output — exactly double Opus 4.8. Combined with the thinking effort multiplier, a heavy coding session can burn through budget shockingly fast.

One user on r/ClaudeAI reported burning 2% of their Max 20x plan per minute during a heavy Fable 5 session. At that rate, you'd exhaust a $200/mo plan in under an hour of focused work.

But here's the thing: most coding tasks don't need max thinking.

Renaming variables? Low thinking is fine.
Writing unit tests from existing code? Medium at most.
Fixing a typo or config change? Low.
Complex architecture decisions? Now you want max.

The skill gap in 2026 isn't "which model do I use." It's "how much thinking does this task actually need."

The Three-Layer Routing Approach

Here's what actually worked for us across 10+ products:

Layer 1: Model Selection

Not everything needs Fable 5. We route across three tiers:

Routine tasks (config changes, formatting, boilerplate) → Haiku-class models (~$0.01/query)
Standard reasoning (code review, debugging, feature implementation) → Sonnet/Opus tier (~$0.05-0.15/query)
Frontier-required (architecture decisions, complex multi-step reasoning) → Fable 5

Layer 2: Thinking Effort (NEW with Fable 5)

When a task does need Fable 5, match the thinking effort:

# Pseudocode for thinking effort routing
def get_thinking_effort(task):
    if task.type in ["search", "retrieval", "classification"]:
        return "low"      # ~$0.10
    elif task.type in ["code_review", "debugging", "refactoring"]:
        return "medium"    # ~$0.20
    elif task.type in ["architecture", "security_audit", "migration"]:
        return "max"       # ~$0.72
    else:
        return "medium"    # safe default

Layer 3: Prompt Caching

Fable 5 offers a 90% discount on cached input tokens. If your system prompt and tool definitions are consistent across calls, cached input drops from $10/M to $1/M.

This is massive for agentic workflows where the same context gets sent repeatedly.

Real Numbers: Our Before and After

Before routing (everything on Claude Opus, max settings):

$10,200/mo across 10 products
Average cost per developer task: $0.85

After three-layer routing:

$3,100/mo — a 70% reduction
Average cost per developer task: $0.26

The breakdown of where tasks actually land:

62% of tasks → cheap models (Haiku/Sonnet class)
31% of tasks → mid-tier (Opus 4.8 or Fable 5 low/medium thinking)
7% of tasks → Fable 5 max thinking

That 7% is doing the heavy lifting. The other 93% was burning money for no quality improvement.

The Classification Trick

"But how do you know which tier a task needs?"

A lightweight classifier. We use a Haiku-class model to analyze each task before routing it. The classifier itself costs ~0.1% of what it saves. Here's the approach:

Take the task description/prompt
Ask a cheap model: "Rate this task's complexity: routine/standard/frontier"
Route based on the answer

It's not perfect — maybe 85% accurate. But an 85%-accurate router that saves 70% is vastly better than a 100%-accurate "just send everything to the best model" approach that costs 3x more.

What Doesn't Work

Things we tried that failed:

Static rules per task type: Too rigid. "All debugging goes to Opus" misses that most debugging is simple.
LLM judge picking the model: Recursive cost problem — the judge itself is expensive if you use a good model for it.
Just eating the cost: Works until your CFO sees the bill. Or until Microsoft bans your Claude Code license (yes, this happened last week).

The Industry Is Catching Up

Within 12 hours of Fable 5 launching, every major technical guide led with cost control:

TrueFoundry: "cost control isn't optional"
Spicy Advisory: "reserve it for high-value work, keep cheaper models for routine"
OpenRouter: already listing it with routing support

When every guide about a new model starts with "here's how to NOT use it for everything" — the single-model era is officially over.

Getting Started

If you're running any AI coding workflow (Claude Code, Cursor, Aider, custom agents):

Audit your current usage: What percentage of your API calls actually need frontier-level reasoning?
Start simple: Route "obviously easy" tasks to a cheaper model. Even a basic keyword-based router saves 30-40%.
Add thinking effort: For tasks that do need Fable 5, default to medium instead of max. Upgrade to max only for specific task types.
Measure per-task cost: Not per-API-call cost. A single "refactor this module" can fan out into 30+ sub-agent calls. Track cost per user intent.

The model keeps getting better. The pricing keeps going up. The only sustainable strategy is routing.

We've been building routing tools for AI coding workflows at CodeRouter. If you're spending more than $500/mo on AI coding APIs, task-level routing will pay for itself in the first week.

What Apple's WWDC Just Taught Us About AI Model Routing

Bo Shen — Mon, 08 Jun 2026 19:58:44 +0000

Today at WWDC 2026, Apple dropped what might be the most important signal in AI strategy this year — and it wasn't Siri AI, macOS Golden Gate, or iOS 27.

Apple chose NOT to build their own foundation model.

Siri AI, the ground-up rebuild Apple has been working on for two years, runs on Google's Gemini models. The world's most valuable company decided that model selection and integration matters more than model ownership.

This should change how every developer thinks about AI.

The "One Model" Trap

Most teams today pick a single model and route everything through it. Claude for coding. GPT for chat. Gemini for multimodal. Then they wonder why their bills are astronomical or their output quality is inconsistent.

Microsoft just banned engineers from using Claude Code after realizing it cost more than the humans it was supposed to assist. Multiple Reddit threads this week show developers racking up $120K+ in API tokens, or getting called into HR after 10 days of heavy usage.

The problem isn't the model. It's the assumption that one model fits all tasks.

What Apple Got Right

Apple's approach with Siri AI is essentially a routing layer:

On-device models handle simple queries (privacy-preserving, instant)
Apple's own server-side models handle mid-tier tasks
Gemini handles the complex reasoning and generation

Each query goes to the right-sized model for the job. A timer request doesn't need a trillion-parameter model. A complex research question does.

This is the same architecture pattern that's emerging in AI coding tools:

Task	What You Need	What Most People Use
Planning/Architecture	Strong reasoning (Opus/o3-level)	Opus for everything
Implementation	Fast, accurate code gen (Sonnet/Flash)	Opus for everything
Debugging	Pattern matching + context (Flash/Haiku)	Opus for everything
Tests/Boilerplate	Speed over intelligence (Haiku/Flash)	Opus for everything

See the pattern? Using a frontier model for boilerplate is like using a Ferrari to go grocery shopping. It works, but the economics don't.

The Math That Changed My Workflow

I run 10+ apps and was spending about $10K/month on Claude Code before I started routing tasks by complexity:

Planning phase: Opus (worth the cost for architecture decisions)
Implementation: Sonnet 4 (90% of Opus quality at 20% of the cost)
Test writing: Flash/Haiku (these are pattern-matching tasks)
Debugging: Depends on complexity — start with Flash, escalate if needed

Result: $10K → ~$3K/month with no measurable drop in output quality. The savings came entirely from not using the most expensive model for tasks that didn't need it.

How To Implement This

You don't need special tools. You need discipline:

1. Categorize Before You Prompt

Before sending a task to your AI coding tool, ask: "Does this need frontier intelligence, or am I just generating predictable code?"

Most tasks fall into the "predictable" bucket — CRUD endpoints, test scaffolding, config files, migration scripts.

2. Use Model Tiers

Set up your environment with at least two model tiers:

# .env or tool config
PLANNING_MODEL=claude-opus-4
IMPLEMENTATION_MODEL=claude-sonnet-4
FAST_MODEL=gemini-2.5-flash

Route manually at first. The patterns become obvious quickly.

3. Measure Before You Optimize

Track your usage by task type for one week. Most developers find that 60-70% of their tokens go to tasks that a cheaper model handles equally well.

4. Escalation, Not Default

Start every task at the lowest reasonable tier. If Flash can't handle it, escalate to Sonnet. If Sonnet struggles, bring in Opus. This "escalation ladder" catches most tasks at the cheapest tier.

The Bigger Picture

Apple just validated at the largest scale possible what indie developers have been learning the hard way: the future of AI isn't picking the best model — it's building the routing layer that picks the right model for each task.

Every week brings new models. DeepSeek, MiMo, Gemini, Claude, GPT — the landscape shifts constantly. Betting on one model is a losing strategy. Building an orchestration layer that can swap models as the landscape evolves? That's the moat.

If the world's most valuable company decided that model routing beats model ownership, maybe it's time to rethink your own AI stack.

What's your experience with multi-model workflows? Are you routing tasks to different models, or still using one model for everything? Drop your approach in the comments.

GitHub Copilot's New Credit Pricing: A Token-by-Token Breakdown (And How to Cut Your AI Coding Bill by 70%)

Bo Shen — Mon, 01 Jun 2026 20:37:22 +0000

GitHub just switched Copilot to credit-based pricing and the community is in shock. Users are reporting bills jumping from $38/month to $800+. Here's what's actually happening, why, and what you can do about it.

The Credit Math Nobody Did

1 AI Credit = $0.01. Copilot Pro+ gives you 7,500 credits/month for $39. That's $75 worth of credits for $39 — sounds generous until you realize how fast agent mode burns through them.

A typical agentic coding session uses Claude 4.6 Sonnet at 1,500 credits per million output tokens. A single complex refactoring prompt can easily consume 50K-100K output tokens, costing $0.75-$1.50 per request. Eight requests and you've burned $6-12, or roughly half your monthly allowance.

The math gets worse with Opus: 7,500 credits per million output tokens. One architecture planning session with Opus can eat 20% of your monthly budget in 10 minutes.

The Real Problem: One Pipeline for Every Task

Here's what most people miss. The cost explosion isn't just about token pricing — it's about model selection.

Copilot's agent mode fires the same heavy-reasoning model at every task:

Tab completion? Full reasoning pipeline.
Writing a test? Full context window loaded.
Simple rename refactor? Same model as architecture planning.

When I tracked my actual AI coding usage for a month, the breakdown looked like this:

Task Type	% of Requests	Ideal Model Tier	Cost/1M tokens
Implementation/boilerplate	~60%	Sonnet/4o	$3-5
Tests, linting, docs	~20%	Flash/mini	$0.15-0.30
Architecture/complex debug	~15%	Opus/GPT-5	$15-75
Tab completion	~5%	Flash	$0.075

65% of requests don't need an expensive model. But when everything runs through one pipeline, you pay Opus prices for autocomplete.

What I Actually Pay: $60/month for $200 Worth of Work

I ditched Copilot three months ago and switched to direct API keys. Here's my actual setup:

Planning & Architecture: Claude Opus or GPT-5 via API

Used for: system design, complex debugging, multi-file refactors
~15% of my requests, ~$25/month

Implementation: Claude Sonnet 4.5

Used for: writing new features, code generation, PR descriptions
~60% of my requests, ~$20/month

Tests & Docs: Gemini Flash or GPT-4o-mini

Used for: unit tests, documentation, linting suggestions
~25% of my requests, ~$5/month

Total: ~$50-60/month for the same (often better) output that would cost $400+ on Copilot's new credit system.

The Key Insight: Task Complexity Should Drive Model Selection

Think about it like this. You wouldn't use a chainsaw to cut bread, and you wouldn't use a butter knife to fell a tree. But that's exactly what single-model coding tools do — they give you one tool for everything.

The 70% savings don't come from cheaper API rates. They come from matching model capability to task complexity:

Identify the task type before sending a prompt
Route to the appropriate model tier based on reasoning requirements
Track actual token usage to validate your model selections

This isn't hypothetical. My team went from $10K/month in AI coding costs to $3K by implementing this approach. Same output quality, same development velocity. The only difference was being intentional about which model handles which task.

Practical Steps If You're Leaving Copilot

1. Start tracking your actual usage patterns
Before optimizing, you need data. Log your requests for a week and categorize them. You'll probably find that 60-70% of your interactions don't need the top-tier model.

2. Set up direct API access
Both Anthropic and OpenAI offer straightforward API pricing. Claude Sonnet at $3/$15 per million input/output tokens and Gemini Flash at $0.075/$0.30 are your workhorses.

3. Use different models for different tasks
Tools like Claude Code, Cursor, and Aider all support model switching. Set your default to Sonnet, and manually switch to Opus only for complex reasoning tasks.

4. Monitor your costs weekly
Track spend per model tier. If your Opus usage exceeds 20%, you're probably over-using it. If Flash usage is under 15%, you're probably under-using it.

The Copilot Credit System Might Actually Be a Good Thing

Hot take: transparent token pricing is better than opaque subscriptions. The old flat-rate model hid the true cost of AI coding, which meant nobody optimized their usage. Now that the costs are visible, the developers who learn to match models to tasks will come out way ahead.

The ones who keep blasting everything through one expensive model? They'll pay $800/month and wonder why.

I run a portfolio of AI-powered apps and spend way too much time thinking about model costs. If you want to compare notes on cutting AI coding bills, I'm @aplomb2 on X.

How We Cut Our AI Coding Bill by 65% Without Sacrificing Quality

Bo Shen — Sun, 31 May 2026 03:45:31 +0000

Last month, a post on r/ExperiencedDevs went viral: a company spending $1 million per month on AI API costs. Layoffs wouldn't even make a meaningful dent.

The painful part? They couldn't force teams onto cheaper models because quality genuinely dropped on complex tasks. Sound familiar?

We faced the same wall at $10K/month across our team. Here's how we solved it — and cut costs by 65% without a single developer complaint.

The Problem: All-or-Nothing Model Selection

Most teams pick one model and use it for everything:

Claude Opus for every API call? Powerful but expensive.
Switch everyone to Haiku? Cheap but quality suffers on hard problems.
Let devs choose per-request? Nobody bothers. They default to the best model every time.

This is the "blanket model" trap. You're either overpaying or underperforming.

The Insight: Not All Tasks Are Equal

We audited 30 days of our API usage and discovered something obvious in hindsight:

Task Type	% of Calls	Needs Frontier Model?
Linting & formatting	15%	No
Boilerplate generation	20%	No
Simple completions	25%	No
Test generation	10%	Rarely
Complex debugging	15%	Yes
Architecture decisions	10%	Yes
Code review (nuanced)	5%	Yes

60-70% of calls didn't need a frontier model at all. They ran identically on Haiku, Gemini Flash, or even smaller models.

But the remaining 30%? Those genuinely needed Opus-tier reasoning.

The Solution: Task-Level Routing

Instead of forcing a model choice at the team level, we implemented automatic routing by task complexity:

Classify the request — Is this a simple completion or a complex reasoning task?
Route to the appropriate model — Simple → Haiku/Flash. Complex → Opus/GPT-4.
Make it invisible — Developers don't pick models. The system picks for them.

The key insight: routing should be invisible to developers. If they have to think about which model to use, they'll always pick the most powerful one (just in case). The system needs to make that decision automatically.

The Results

After 30 days of task-level routing:

Cost reduction: ~65% ($10K → $3.5K/month)
Quality complaints: zero — Complex tasks still got Opus
Developer friction: zero — They didn't even notice the switch
Latency improvement: 40% — Smaller models respond faster for simple tasks

The biggest surprise? Quality actually improved on some tasks. Smaller models are less prone to over-thinking simple requests. Ask Opus to format an import statement and it might refactor your entire file. Ask Haiku and it just... formats the import.

How to Implement This

You have a few options:

Option 1: Manual Rules (Free)

Write a classifier that routes based on prompt length, keywords, or context. Crude but effective for simple cases.

def route_model(prompt, context):
    # Simple heuristic routing
    if len(prompt) < 100 and context.get('task') in ['lint', 'format']:
        return 'haiku'
    elif context.get('task') in ['debug', 'architecture', 'review']:
        return 'opus'
    else:
        return 'sonnet'  # middle ground

Option 2: LLM-Based Classification

Use a tiny model to classify the task before routing. Adds ~50ms latency but much more accurate.

Option 3: Specialized Routing Tools

Tools like CodeRouter handle this automatically for coding workflows — they classify by development phase (planning, implementation, testing, debugging) and route accordingly.

Lessons Learned

Start with data. Audit your actual API usage before optimizing. You'll be surprised how many calls are trivial.
Don't trust developers to self-route. They'll always pick the best model "just in case." Make it automatic.
Measure quality, not just cost. Some tasks genuinely need frontier models. Don't cheap out on the 30% that matters.
The biggest savings aren't from switching models — they're from not using expensive models when you don't need to.

TL;DR

60-70% of AI coding calls don't need a frontier model
Task-level routing cuts costs 50-70% with zero quality loss
Make routing invisible to developers
The fix for "$1M/month AI bills" isn't fewer models — it's smarter model selection

I'm Bo, founder building tools for AI-powered development. If you're drowning in API costs, I've been there. Happy to chat in the comments.