Building CostRouter — Route AI requests to the cheapest capable model automatically.

#productivity #startup #webdev #discuss

The Problem

Teams using Claude, GPT, and Gemini APIs are routing every request to their most expensive model by default—even simple text extraction or Q&A that doesn't need GPT-5 or Claude Opus. This wastes 50-70% of AI spend. Most companies either build janky in-house routing logic or manually manage model selection, both approaches requiring engineering time and constant tweaking.

What I'm Thinking of Building

CostRouter is an API gateway that sits between your app and your AI providers. It analyzes each request's complexity (length, keywords, structure) and routes to the cheapest model that can handle it—Llama for simple tasks, Gemini Flash for medium complexity, GPT-5/Claude for reasoning-heavy work. One-line integration: just change your OpenAI base_url. We charge 10% of verified savings, so there's zero upfront cost.

Who It's For

Engineering leads and startup CTOs at companies using OpenAI/Claude/Gemini APIs for production apps (B2B SaaS, AI-powered products, enterprises). Primarily: Series A–C startups and mid-market companies doing 100k–500k API requests/month, where monthly LLM spend is $2k–$10k.

Key Features (Planned)

Automatic complexity scoring: analyzes request length, keywords, and structure to route intelligently
One-line integration: change your OpenAI base_url, no SDK rewrites needed
Multi-model support: Llama 4 Scout, Gemini 3 Flash, GPT-5.2, Claude Opus—add your own
Request logging & audit trail: track which models handled what, replay edge cases
Custom routing rules: override complexity scores for specific use cases or customers
Spend dashboard: real-time savings tracker, per-endpoint breakdown, cost projections

I'm validating this idea before writing a single line of code. If this resonates with you, I'd love your feedback:

If routing to cheaper models automatically could cut your LLM bill by 40–60% with zero code changes, how much would you need to see in savings before it was worth switching from your current setup?

Check out the concept page and let me know what you think.