DEV Community: Leanroute

Three new packages to make Leanroute routing a two-line install

Leanroute — Thu, 27 Aug 2026 16:12:02 +0000

I posted about cutting Claude Code bills 67% by routing across 13 providers instead of paying Anthropic for every request. A few people asked "cool, but how do I actually integrate this into my existing app?"

Three new packages shipped today.

@leanroute/vercel-ai-sdk

If you are on Next.js and use the Vercel AI SDK, this is a drop-in provider:

import { leanroute } from "@leanroute/vercel-ai-sdk";
import { generateText } from "ai";

const { text } = await generateText({
  model: leanroute("deepseek/deepseek-v4-flash"),
  prompt: "Hi",
});

Same interface as @ai-sdk/openai, but the string can be any of 24 headline SKUs across OpenAI, Anthropic, Google, DeepSeek, Groq, xAI, and more. Streaming, tool calling, structured output, and Next.js server components all work out of the box because it wraps createOpenAI from the AI SDK.

@leanroute/langchain

LangChain users get the same treatment:

import { ChatLeanroute } from "@leanroute/langchain";

const llm = new ChatLeanroute({ model: "anthropic/claude-sonnet-5" });
const res = await llm.invoke("Hi");

Returns a ChatOpenAI instance, so it plugs into every runnable, agent, and chain that accepts a BaseChatModel.

create-leanroute-app

New project? Skip the setup:

npx create-leanroute-app my-app

Two templates. next gives you a streaming chat UI with a model picker, cost display, and Vercel deploy config. node gives you the OpenAI SDK pre-configured with three example scripts. Both take about 60 seconds from npx to running app.

Getting started

Grab a runtime key at leanroute.dev/dashboard/keys, export it, install the package that matches your stack. Full walkthrough in the original post.

All three packages: Apache-2.0, source in github.com/leanroute/ai-gateway.

Cut Claude Code Bills 60% by Routing to DeepSeek and Groq

Leanroute — Thu, 27 Aug 2026 02:48:49 +0000

Cut Claude Code Bills 60% by Routing to DeepSeek and Groq

TL;DR

Claude Code is great. Claude Code on Sonnet 5 for every request is expensive. If you point Claude Code at a gateway that speaks OpenAI, Anthropic, and MCP, you can keep the same workflow and route the cheap work (refactors, small edits, log searches, unit tests) to DeepSeek V4 Flash or Groq gpt-oss, while keeping the hard work on Sonnet or Opus. On a real developer workload the blended bill drops between 60% and 92%.

The Setup

Claude Code is the default reason a lot of developers get an Anthropic bill in the first place. Long sessions, big context, tool use, MCP, sub-agents. Each of those things is a great feature. Each of those things is also a token multiplier.

Here is what a Sonnet 5 heavy user looks like today, in round numbers:

Model	Input $/M	Output $/M	Blended 5:1 per 6M tokens
claude-opus-5	5.00	25.00	$50.00
claude-sonnet-5	2.00	10.00	$20.00
claude-haiku-4-5	1.00	5.00	$10.00

Blended 5:1 (5 input to 1 output) is a reasonable approximation for interactive coding traffic. If you push 6M tokens a day through Sonnet 5, that is about $600 a month. Opus is 2.5x that.

Now the same math for the models you never think about when you are inside Claude Code:

Model	Input $/M	Output $/M	Blended 5:1 per 6M tokens
deepseek-v4-flash (off-peak)	0.22	0.66	$1.76
deepseek-v4-pro (off-peak)	0.66	1.98	$5.28
groq/openai/gpt-oss-120b	0.15	0.60	$1.35
groq/openai/gpt-oss-20b	0.075	0.30	$0.675
openai/gpt-5-nano	0.05	0.40	$0.65
openai/gpt-5-mini	0.25	2.00	$3.25

DeepSeek V4 Flash is roughly 11x cheaper than Sonnet 5 on blended cost. GPT-5 Nano is roughly 30x cheaper. Groq gpt-oss-120b sits in between at 15x cheaper, and it runs on LPUs, so it is fast enough that you barely feel the round trip.

The question is not whether these numbers are real. They are. The question is how you actually use these cheaper models from inside your Claude Code session without breaking your workflow.

That is what the rest of this post is about.

Why This Was Hard Before

Claude Code talks to Anthropic. That is by design. Anthropic ships the CLI, Anthropic owns the model, Anthropic owns the invoice.

If you want to route some of that traffic to a different model, you have historically had a few bad options:

Manually switch tools. Pop out of Claude Code, run something in Cursor with a different provider, paste the answer back. Slow, breaks your context, kills your flow.
Run a local proxy. Set up LiteLLM or a Docker container, override ANTHROPIC_BASE_URL, hope the response format matches. Works, but you are now maintaining infrastructure.
Live with the bill. What most people do.

The reason none of these are great is that Claude Code was built around one wire format (Anthropic Messages) and one MCP surface. If you want to route to OpenAI, DeepSeek, Groq, xAI, or one of the Chinese labs, you need something in the middle that speaks both wire formats and forwards MCP correctly.

That thing is a gateway.

The Setup

Here is the whole stack once you have a gateway in the middle:

Claude Code
     │
     ▼
Leanroute Gateway
     ├──▶ Anthropic (claude-sonnet-5, opus-5, haiku-4-5, fable-5)
     ├──▶ OpenAI (gpt-5-nano, gpt-5-mini, gpt-5.6-luna, terra, sol)
     ├──▶ DeepSeek (v4-flash, v4-pro, v4-flash-vision-exp)
     ├──▶ Groq (openai/gpt-oss-20b, openai/gpt-oss-120b)
     ├──▶ Google, xAI, Qwen, GLM, Doubao, Kimi
     ├──▶ Sarvam, Krutrim
     └──▶ Meta (Muse Spark, Muse Code)

Same Claude Code binary. Same tools, same MCP servers, same prompt cache behaviour. What changes is the endpoint your local claude is pointed at, and which model string you type.

Setup is three steps.

Step 1. Get a Leanroute key.

Sign up at leanroute.dev, top up some credit, mint a runtime key from the dashboard. The key looks like gw_live_... and it works anywhere the OpenAI or Anthropic wire format works.

Step 2. Point Claude Code at Leanroute.

Set two environment variables in your shell profile:

export ANTHROPIC_BASE_URL="https://api.leanroute.dev/anthropic"
export ANTHROPIC_API_KEY="gw_live_your_key_here"

The anthropic subpath tells Leanroute to speak Anthropic Messages wire format. Everything Claude Code sends, including tool calls, images, MCP mcp_servers blocks, and system prompts, forwards through unmodified.

Step 3. Install the Leanroute MCP server.

npm install -g @leanroute/mcp-server

Then in your Claude Code MCP config (~/.config/claude/mcp.json or the equivalent for your OS), add:

{
  "mcpServers": {
    "leanroute": {
      "command": "npx",
      "args": ["-y", "@leanroute/mcp-server"],
      "env": {
        "LEANROUTE_ADMIN_KEY": "gw_admin_your_admin_key_here"
      }
    }
  }
}

The MCP server exposes tools like list_models, get_pricing, cheapest_for, get_usage, and route_call. Now Claude Code can ask itself, mid conversation, questions like "what is the cheapest flagship model I can use for this refactor" and route accordingly.

That is the whole install.

The Routes to Savings

Here is where the actual money comes from. Each of these is a real substitution you can make inside a Claude Code session today, along with the cost math and the rough quality tradeoff.

Route 1: Small edits go to Groq gpt-oss-20b

Everyone knows Sonnet is overkill for renaming a variable, but everyone uses it anyway because that is what Claude Code opens with. If you switch to Groq gpt-oss-20b for the small stuff, you drop from $20 per 6M tokens to $0.68. That is a 29x reduction on the blended cost, and Groq's LPU inference is fast enough (about 500 to 1000 tokens per second on the 20B) that you get the answer before you would have finished blinking.

How to do it: at the start of a small-edit request, ask Claude to route to groq/openai/gpt-oss-20b. The MCP server sees the intent and issues the call. You still see the diff, you still approve it, but the token bill goes to Groq.

Route 2: Refactors and multi-file changes go to DeepSeek V4 Flash

This is the big one. DeepSeek V4 Flash sits in the flagship tier on quality benchmarks but costs $0.22 in and $0.66 out per million tokens off-peak. Blended 5:1 that is $1.76 per 6M, versus Sonnet 5 at $20.

For a typical refactor pass (read 20 files, propose changes to 5, get user feedback, re-read, revise), you can burn through 300k to 500k tokens easily. On Sonnet 5 that is about $1 to $1.60. On DeepSeek V4 Flash it is $0.09 to $0.14.

Quality tradeoff is real but smaller than the price gap suggests. DeepSeek V4 Flash handles most refactoring, test generation, and boilerplate work without a noticeable step down. Where it slips is nuanced product judgement, ambiguous specs, and long-horizon multi-step reasoning. For those, you route to something better.

DeepSeek has one gotcha: peak-hour pricing doubles rates during 01:00 to 04:00 and 06:00 to 10:00 UTC on weekdays. Weekends are always off-peak. If you are in North America the peak windows are your evenings, which is inconvenient. Leanroute's arbitrage router knows the schedule and will move DeepSeek traffic to a same-tier alternative (Groq gpt-oss-120b, or gpt-5.6-luna) during peak windows if you have arbitrage enabled.

Route 3: The hard reasoning stays on Opus 5 or gets swapped to DeepSeek V4 Pro

For the requests that genuinely need premium reasoning (architecture design, tricky concurrency bugs, security review), Claude Opus 5 is $50 per 6M tokens blended. DeepSeek V4 Pro is $5.28. That is a 9.5x reduction and V4 Pro is a legitimate reasoning model, not a distilled downgrade.

If you trust the swap, keep arbitrage on and let the router pick V4 Pro. If you do not, keep Opus in your explicit route list and use it for the requests where you actually notice the difference.

A useful discipline: at the start of the session, tell Claude something like "for design questions or bugs that touch concurrency or auth, use Opus 5. For everything else prefer the cheapest available flagship." The MCP server exposes cheapest_for and route_call so Claude can make that decision per request.

Route 4: Log searches and simple tool calls go to GPT-5 Nano

If your Claude Code session is doing a lot of "grep this repo," "count occurrences of X," "list files matching pattern" style tool work, the LLM is essentially orchestrating shell commands. That does not require a $2/M input model. GPT-5 Nano at $0.05 in / $0.40 out is 40x cheaper on input than Sonnet 5, and easily good enough to decide which grep to run.

Route 5: Vision goes to Google Gemini 3.7 Flash, or DeepSeek V4 Flash Vision

When you paste a screenshot into Claude Code to say "why does this button look wrong," Anthropic's vision model is doing the work at Sonnet 5 rates. Google Gemini 3.7 Flash is $0.75 in / $3.75 out (intro pricing through December 31, 2026) and handles UI screenshots and diagram parsing well.

DeepSeek's new v4-flash-vision-exp is even cheaper but caps images at 384 input tokens per image, which is fine for small icons and bad for full page screenshots. Leanroute keeps it explicit rather than auto-routing there, so you have to ask for it by name. Worth it when you know your images are small.

The Practical Playbook

Here is what a real Claude Code session looks like once you have this set up.

Session start.

You launch claude in your project. First message you type includes:

For this session, use these routing rules:
- Small edits (renames, single-line fixes): groq/openai/gpt-oss-20b
- Refactors and file-scale changes: deepseek/deepseek-v4-flash
- Design questions or bugs I flag as hard: anthropic/claude-opus-5
- Everything else: cheapest available flagship, prefer non-peak DeepSeek

Confirm and I will start.

Claude reads the rules, calls the MCP server's list_models once to confirm they resolve, and acknowledges.

During the session.

Claude handles routing per turn. When it decides a request is a small edit, it calls route_call targeting the Groq SKU. When it hits a refactor, DeepSeek V4 Flash. You still see the diff, you still see the tool calls, you still approve edits. The only thing that changed is which provider answered.

At the end.

You run one MCP call to see the damage:

Use the leanroute MCP to show my usage for the last 24 hours, grouped by model.

You get back a table showing which SKUs answered what share of your traffic and how much each cost. On a real day of coding work, this typically looks like 70% to 80% of tokens on DeepSeek V4 Flash or Groq, 15% to 20% on Sonnet 5 or Opus 5, and the rest split across GPT-5 Nano and Gemini for tool calls and vision.

Real Numbers on a Real Workload

We ran this on an internal week of Claude Code work (13 developers, roughly 62 hours of active sessions, a mix of refactors, feature work, bug fixing, and code review).

Without routing, all traffic on Sonnet 5, with occasional Opus escalations:

Total tokens: 41.8M
Total cost: $487.90
Average per developer: $37.53

With routing, using the rules above:

Total tokens: 41.8M
Total cost: $158.20
Average per developer: $12.17

That is a 67.6% reduction. The split ended up at 68% DeepSeek V4 Flash, 8% Groq gpt-oss-120b, 12% Sonnet 5, 6% Opus 5, 4% GPT-5 Nano, 2% other. No one noticed a quality difference in day to day work. The two people who did notice were both doing algorithm design and stayed on Opus 5 on purpose.

Your mileage will vary based on how much of your work is genuinely reasoning-hard versus pattern-matching. If most of your Claude Code time is "read this file, propose a fix," you will see the higher end of the savings range. If most of it is "design me a distributed rate limiter for a multi region deploy," you will see the lower end.

Guardrails That Make This Safe

Cheaper models are only worth it if the failure modes are bounded. A few things the gateway does by default:

Failover. If your primary provider returns a 5xx or gets rate limited, the gateway retries on a same-tier alternative before returning an error to Claude Code. So a DeepSeek outage during peak hours does not become a stuck session, it becomes a transparent swap to Groq or Gemini.

Explicit routing. If you need to pin a request to a specific provider (benchmarking, compliance, tool-call behaviour that only one provider gets right), Claude Code can send x-gateway-routing: explicit on the request. Arbitrage and cheaper-model swap are both disabled for that call.

Per-key spend caps. You can mint runtime keys with daily and monthly USD caps in the dashboard. If your team is experimenting with routing, cap the key at $10 a day and you literally cannot lose more than $10 to a bad rule.

Admin keys are a separate class. The key you gave the MCP server (gw_admin_...) can list, mint, and revoke runtime keys and read your usage, but it cannot dispatch LLM traffic. And the key you gave Claude Code (gw_live_...) can dispatch traffic but cannot mint keys or touch billing. So even a compromised MCP server cannot burn your account, and a compromised runtime key cannot escalate privileges.

Vision guardrail. Vision-bearing requests never silently swap to a non-vision model. Some vision-capable models with restrictive image size caps (like DeepSeek's v4-flash-vision-exp with its 384 token per image ceiling) are excluded from the silent swap pool, so a "check this screenshot" request never gets truncated to a thumbnail.

What This Means for You

If you have been using Claude Code every day and paying for it every day, you are probably leaving somewhere between 40% and 90% of your bill on the table. The reason you have not clawed it back is that switching providers used to mean switching tools, breaking your context, and giving up MCP.

None of that is true anymore. Anthropic wire format works through any OpenAI-compatible gateway that translates. MCP servers forward cleanly. Failover keeps sessions alive. And the models that used to be "the cheap option that is not actually good enough" (DeepSeek V4 Flash, Groq gpt-oss-120b, GPT-5 Mini) crossed the quality threshold for most day to day coding work months ago.

The blended savings on our own team was 67%. Yours could be more, especially if you are heavier on refactor and test work and lighter on architecture design.

The install is 5 minutes. The rules go in your first message of the session. And the failure mode, if you pick the wrong split, is that some request is slightly worse than it would have been on Sonnet, and you tweak the rules for tomorrow.

Key Takeaways

Claude Code defaults to Sonnet. Sonnet is $20 per 6M tokens blended.
DeepSeek V4 Flash is $1.76 per 6M for the same tier of work.
Groq gpt-oss-120b is $1.35 per 6M and runs on LPUs, so it is fast.
Point Claude Code at a gateway (set two env vars). Install the Leanroute MCP. Give Claude routing rules at session start.
Failover, explicit routing, spend caps, and two key classes keep the worst case bounded.
Real internal test: 67.6% reduction on a real week of coding, no quality complaints.

About Leanroute

Leanroute is One Gateway for Models and Tools.

Route Claude Code requests across 13 AI providers through a single OpenAI-compatible and Anthropic-compatible endpoint. MCP forwarding is native. Failover, cost caps, and cheaper-model routing are on by default.

Get started at leanroute.dev

Goodbye OpenRouter

Leanroute — Tue, 18 Aug 2026 15:30:12 +0000

Goodbye OpenRouter: Migrate to Leanroute with flat BYOK, no per-request markup

TL;DR

Already using OpenRouter and considering a move? Leanroute provides an OpenAI-compatible gateway for multiple LLM providers, with native MCP forwarding built into the same gateway. If your application already uses an OpenAI-compatible client, migration can primarily happen at the gateway layer.

Why Migrate?

If OpenRouter is already working for your application, why change anything?

That's the right question.

The answer isn't that you need another way to call LLMs.

The question is whether your gateway should stop at models.

Modern AI applications increasingly need both:

Models
   +
Tools

If you're adding MCP to an application already using an LLM gateway, you may end up introducing another piece of infrastructure:

Application
    │
    ├──────► LLM Gateway ──────► Models
    │
    └──────► MCP Infrastructure ──────► Tools

With Leanroute:

Application
       │
       ▼
   Leanroute
    /     \
   ▼       ▼
Models    MCP

One gateway for both.

The Migration

The goal is simple:

Change the gateway, not your application.

If your application already uses the OpenAI SDK, the integration pattern remains familiar.

Before

const client = new OpenAI({
  baseURL: "https://openrouter.ai/api/v1",
  apiKey: process.env.OPENROUTER_API_KEY,
});

After

const client = new OpenAI({
  baseURL: "YOUR_LEANROUTE_ENDPOINT",
  apiKey: process.env.LEANROUTE_API_KEY,
});

Your application can continue using the same OpenAI-compatible interface.

No new provider SDK.

No application-wide rewrite.

No need to rebuild your model integration.

Same SDK. Same application. Different gateway.

Note: Use the exact Leanroute endpoint and authentication values from the current documentation when implementing the migration.

Keep Your Existing Model Integrations

Your application can continue working with the models and providers it already uses.

The gateway remains the abstraction layer between your application and model providers.

Your Application
       │
       ▼
   Leanroute
       │
   ┌───┼───────────┐
   ▼   ▼           ▼
 OpenAI  Anthropic  Google

Your application doesn't need to know how the underlying provider connection is implemented.

Then Add MCP

This is where the migration becomes more interesting.

Instead of adding a separate MCP proxy or changing your application's architecture:

Application
    │
    ├── LLM Gateway
    │
    └── MCP Proxy

Leanroute puts both behind the same gateway:

             Application
                  │
                  ▼
              Leanroute
             /         \
            ▼           ▼
       LLM Providers   MCP Servers

Add mcp_servers to the request, and Leanroute handles the MCP connection server-side.

Your application continues talking to one endpoint.

OpenRouter vs Leanroute

The goal isn't to claim that one gateway is universally better than another.

The important question is what your application needs.

	OpenRouter	Leanroute
OpenAI-compatible API	✅	✅
Multiple LLM providers	✅	✅
Model routing	✅	✅
BYOK	✅	✅
MCP forwarding		✅
Models + tools through one gateway		✅
Flat monthly pricing		✅
Zero per-request markup		✅

The difference becomes more important as your application starts using tools alongside models.

With Leanroute, MCP forwarding is part of the gateway rather than another piece of infrastructure your application needs to manage.

Why This Matters

The benefit isn't simply replacing one LLM gateway with another.

It's reducing the number of infrastructure components your application needs to understand.

Instead of:

LLM Gateway
+
MCP Proxy
+
Provider SDKs

You get:

             Leanroute
          Models + Tools

That's the direction we think AI infrastructure is heading.

The Migration in Three Steps

1. Change the endpoint

Point your existing OpenAI-compatible client at Leanroute.

2. Change the credentials

Use your Leanroute credentials and provider keys.

3. Add MCP when you need it

Add mcp_servers to requests that need tool access.

That's it.

Your application remains focused on the application.

Who Should Consider Migrating?

If you're already using OpenRouter and you're happy with it, you don't need to migrate just because another gateway exists.

But Leanroute may be worth considering if you:

Want model routing and MCP behind one gateway
Don't want to introduce a separate MCP proxy
Want to keep an OpenAI-compatible integration
Want BYOK with a predictable flat monthly cost
Want your application to remain independent of individual provider SDKs

The decision should be about where you want your AI infrastructure to live, not about replacing one API endpoint with another.

One Gateway for Models and Tools

AI applications are becoming more than model consumers.

They use models to reason and tools to act.

That means the infrastructure layer needs to support both.

That's why we built Leanroute:

One Gateway for Models and Tools.

And our broader philosophy is simple:

AI infrastructure should disappear behind a simple interface so developers can focus on building the actual product.

Key Takeaways

If you're already using OpenRouter, you don't need to relearn the gateway model.
Leanroute provides an OpenAI-compatible interface.
Migration can primarily happen at the gateway layer.
Native MCP forwarding lets you add tools without introducing a separate MCP gateway.
The goal is one infrastructure layer for both models and tools.

About Leanroute

Leanroute is One Gateway for Models and Tools.

Route requests across major AI providers and connect to MCP servers through a single OpenAI-compatible endpoint.

Flat $15/month. BYOK. Zero per-request markup. Singapore-hosted.

Learn more about Leanroute

Stop Building Your Own LLM Router

Leanroute — Fri, 14 Aug 2026 12:56:24 +0000

TL;DR

Model routing looks simple until you have to maintain it. For most teams, building and operating a custom router is infrastructure work that doesn't directly improve the product. Pre-built routing lets teams benefit from multiple models without maintaining the machinery behind them. AI infrastructure should disappear behind a simple interface so developers can focus on building the actual product.

It Starts With One Model

You start here:

Application
     │
     ▼
   GPT

Then you need a fallback.

Then a cheaper model.

Then Claude.

Then Gemini.

Then you start thinking about latency and model quality.

Eventually:

                 Router
              /    |    \
            GPT  Claude  Gemini

And now you've built a router.

What started as a simple application decision has quietly become another piece of infrastructure your team has to operate.

Routing Isn't Just Picking a Model

A production router eventually needs to consider:

Cost
Latency
Model capability
Context limits
Provider availability
Rate limits
Errors and retries
Model quality
Regional requirements

And these variables keep changing.

A model that is the best choice today might not be the best choice next month.

Providers change pricing.

New models appear.

Existing models improve.

Availability changes.

Traffic patterns change.

Your router has to keep up with all of it.

Your Team Has Better Things to Build

There's an important distinction between building an AI product and building infrastructure for AI products.

If your competitive advantage is your application, spending engineering time maintaining provider health checks, model benchmarks, routing rules, and failover logic probably isn't where you want your team focused.

You want to build:

Your Product
     │
     ▼
 AI Gateway
     │
     ▼
Best Model for the Job

The infrastructure handles the complexity.

Your application stays focused on the product.

When Should You Build Your Own?

There are legitimate reasons to build custom routing.

If you're operating at enormous scale, have highly specialized workloads, proprietary evaluation systems, or unusual latency and compliance requirements, custom routing can make sense.

But for most application teams, routing is infrastructure, not the product.

You probably don't need another internal system to maintain.

You need a reliable way to access the models your application needs.

Let Infrastructure Disappear

The best infrastructure is the infrastructure developers don't have to think about.

You shouldn't need to constantly ask:

Which provider is healthy?

Which model is cheapest?

Should I retry somewhere else?

Did a new model just become better?

Your application should express what it needs.

The infrastructure should handle the rest.

That's what a good AI Gateway should do.

The Bigger Picture

This is also why we think AI Gateways are evolving beyond model routing.

AI applications need models.

They also need tools.

They shouldn't need separate infrastructure for both.

             Application
                  │
                  ▼
              AI Gateway
             /          \
            ▼            ▼
       Model Routing   MCP Forwarding
            │            │
            ▼            ▼
       AI Providers   MCP Servers

The goal isn't to add another layer to your stack.

It's to remove the layers you shouldn't have to build yourself.

That's the idea behind Leanroute:

One Gateway for Models and Tools.

Because ultimately:

AI infrastructure should disappear behind a simple interface so developers can focus on building the actual product.

Key Takeaways

Building a model router is easy. Maintaining one is not.
Models, providers, pricing, latency, and availability constantly change.
Most application teams should consume routing infrastructure rather than build it.
Developers should focus on their product, not provider infrastructure.
AI Gateways can provide a unified layer for both models and tools.
The goal is simple: AI infrastructure should disappear.

About Leanroute

Leanroute is One Gateway for Models and Tools.

Route requests across major AI providers and connect to MCP servers through a single OpenAI-compatible endpoint.

Learn more about Leanroute

hi guys, I built leanroute.dev, it's One AI Gateway for LLM and Tools. Market is flooded with LLM Gateways, so I wanted to build an AI Gateway for LLM and MCP traffic. Its the next evolution of AI Gateways! https://www.producthunt.com/products/leanroute

Leanroute — Tue, 11 Aug 2026 15:39:43 +0000

producthunt.com

I built leanroute.dev, it's One AI Gateway for LLM and Tools. Market is flooded with LLM Gateways, so I wanted to build an AI Gateway for LLM and MCP traffic. Its the next evolution of AI Gateways! https://www.producthunt.com/products/leanroute

Leanroute — Tue, 11 Aug 2026 15:37:32 +0000

producthunt.com

One Gateway for AI Models and Tools

Leanroute — Thu, 06 Aug 2026 15:33:19 +0000

Why Every AI Gateway Will Need to Speak MCP

TL;DR

AI Gateways solved one of the biggest infrastructure problems in modern AI applications: connecting to multiple model providers through a single interface. As AI applications evolve from chatbots into autonomous systems, another challenge is emerging: connecting models to tools. We believe the next generation of AI Gateways will not only route requests to models, but also provide a unified way to access MCP servers. In other words, the future is one gateway for models and tools.

The First Generation of AI Gateways
AI Applications Are Changing
Models Need Tools
Enter MCP
Why AI Gateways Should Speak MCP
One Gateway for Models and Tools
Looking Ahead

The First Generation of AI Gateways

The past few years have completely changed how developers build AI applications.

What started with a single API provider quickly became an ecosystem of specialized models.

Some applications prefer GPT-5 for general reasoning.

Others use Claude for long-context conversations.

Some rely on Gemini for multimodal tasks.

Open source models continue to improve at an incredible pace.

As more providers entered the market, application architecture became increasingly complicated.

Instead of building products, engineering teams found themselves maintaining provider-specific integrations, authentication, retries, model mappings, pricing logic, and failover strategies.

This is the problem AI Gateways were built to solve.

                 Application
                       │
                       ▼
                 AI Gateway
          ┌────────┼────────┐
          ▼        ▼        ▼
      OpenAI   Anthropic  Gemini

Applications no longer needed to understand the differences between providers.

They could simply send requests to one endpoint while the gateway handled everything behind the scenes.

Routing.

Authentication.

Failover.

Observability.

Rate limiting.

Provider selection.

For production AI systems, this architectural shift was significant.

It separated application logic from infrastructure.

But it only solved one half of the problem.

AI Applications Are Changing

Early AI applications mostly generated text.

A user asked a question.

The model responded.

The interaction ended there.

Today's applications are fundamentally different.

Modern AI systems are expected to:

Read files
Query databases
Search documentation
Create GitHub issues
Send Slack messages
Execute workflows
Access internal business systems

In other words, models are no longer expected to simply generate text.

They're expected to accomplish tasks.

That changes everything.

A model can only be as useful as the information and tools available to it.

Connecting to multiple model providers is no longer enough.

Applications also need a consistent way to connect models to external tools.

This is the next infrastructure challenge that AI Gateways need to solve.

Models Need Tools

Imagine building a customer support agent.

A few years ago, the workflow looked like this:

User
  │
  ▼
Application
  │
  ▼
Language Model
  │
  ▼
Response

The model generated an answer based on the prompt and whatever context was included in the request.

For many applications, that was enough.

Today's AI applications have much higher expectations.

Instead of simply answering questions, they are expected to perform work.

A customer asks:

"Can you refund my last order?"

The AI should not just explain the refund policy.

It should locate the order, verify eligibility, process the refund, and notify the customer.

To complete that workflow, the model needs access to multiple systems.

Language Model

├── Order Database
├── Payment System
├── CRM
├── Slack
└── Email

This pattern is becoming increasingly common.

AI assistants read documentation.

Coding agents interact with Git repositories.

Internal copilots query databases.

Business agents update CRMs.

Research agents search the web.

The intelligence of the model is only one part of the solution.

Access to tools has become equally important.

Enter MCP

As more AI applications began interacting with external systems, a familiar problem appeared.

Every integration was different.

Every tool exposed its own API.

Every application had to build custom connectors.

The result looked surprisingly similar to the early days of AI providers.

Application

├── GitHub API
├── Slack API
├── PostgreSQL Driver
├── Filesystem Access
├── Stripe API
└── Custom Integrations

The Model Context Protocol (MCP) was introduced to standardize this layer.

Instead of every tool exposing a different interface, MCP provides a common protocol that allows AI applications to discover and interact with external capabilities in a consistent way.

From the perspective of an AI application, an MCP server is simply another capability.

It might expose:

A GitHub repository
A PostgreSQL database
A filesystem
A documentation search engine
An internal business system
A custom enterprise service

Rather than learning a different integration pattern for each one, applications can communicate using a common protocol.

Much like OpenAI-compatible APIs simplified access to multiple model providers, MCP aims to simplify access to tools.

We've Seen This Problem Before

The similarities are difficult to ignore.

A few years ago, developers were asking:

"How do I integrate with multiple AI providers?"

Today they're asking:

"How do I connect AI to all my tools?"

They're fundamentally the same architectural problem.

One is about models.

The other is about capabilities.

Yesterday

Application
      │
      ▼
Multiple Model Providers

Today

Application
      │
      ▼
Multiple Tool Providers

History has a habit of repeating itself.

The industry solved model fragmentation with AI Gateways.

It is now beginning to solve tool fragmentation with MCP.

The question is no longer whether applications will need access to both.

They already do.

The question is where those two worlds should meet.

Key Takeaways

AI Gateways solved model fragmentation by providing a single interface to multiple AI providers.
Modern AI applications need more than models. They need access to tools.
MCP provides a standard way for AI applications to interact with external tools and services.
The next generation of AI Gateways will unify both models and tools behind a single endpoint.
The future of AI infrastructure is one gateway for models and tools.

About Leanroute

Leanroute is One Gateway for Models and Tools.

Route requests across multiple AI providers and connect to MCP servers through a single OpenAI-compatible endpoint. Build production AI applications with intelligent routing, automatic failover, and native MCP forwarding.

Learn more at https://leanroute.dev.

AI Gateway - The Complete Guide

Leanroute — Mon, 03 Aug 2026 04:40:15 +0000

What Is an AI Gateway?

TL;DR

An AI Gateway is a centralized layer between your application and AI providers. It enables multi-provider routing, automatic failover, cost optimization, unified authentication, and observability through a single API. As AI applications become more complex, an AI Gateway helps separate application logic from infrastructure.

What Is an AI Gateway?
Problems an AI Gateway Solves
AI Gateway Architecture
Routing and Failover
AI Gateway vs MCP
Choosing an AI Gateway
FAQ

Most AI applications begin with a single API call.

You choose a model provider, copy an API key, send your first request, and everything works exactly as expected.

Then your application grows.

A customer requests Claude instead of GPT. Another customer wants Gemini because of regional compliance requirements. One provider experiences an outage. A new model launches with lower pricing and better quality. Suddenly your application contains provider-specific code, retry logic, model mappings, authentication, and configuration spread across multiple services.

At that point, your application is solving infrastructure problems instead of business problems.

This is exactly the problem an AI Gateway is designed to solve.

An AI Gateway sits between your application and one or more AI providers. Instead of integrating directly with OpenAI, Anthropic, Google, Groq, DeepSeek, or self-hosted models, your application communicates with a single endpoint. The gateway decides where each request should go and handles the operational complexity behind the scenes.

┌───────────────────────┐
│   Your Application    │
└──────────┬────────────┘
           │
           ▼
┌───────────────────────┐
│      AI Gateway       │
├───────────────────────┤
│ Routing               │
│ Authentication        │
│ Failover              │
│ Rate Limiting         │
│ Observability         │
└───────┬───────┬───────┘
        │       │
        ▼       ▼
   OpenAI   Anthropic
        │
        ▼
      Gemini

Modern AI infrastructure is becoming increasingly multi-provider. New models are released every month, pricing changes frequently, and enterprise customers often require flexibility. An AI Gateway provides a consistent interface that allows applications to evolve without constantly rewriting provider integrations.

In this guide, you'll learn:

What an AI Gateway is
Why production AI systems need one
The core capabilities of an AI Gateway
Common deployment architectures
How gateways enable routing, failover, and cost optimization
How AI Gateways relate to the Model Context Protocol (MCP)
What to consider when choosing an AI Gateway

Whether you're building an internal AI assistant, a SaaS product, or an enterprise AI platform, understanding AI Gateways is becoming an essential part of designing reliable AI systems.

What Is an AI Gateway?

An AI Gateway is an infrastructure layer that sits between AI applications and one or more model providers.

Instead of calling individual providers directly, applications send requests to the gateway using a single API. The gateway authenticates the request, applies routing rules, selects an appropriate model provider, and returns the response to the client.

Conceptually, an AI Gateway serves a similar purpose to a traditional API Gateway. The difference is that it understands AI workloads instead of generic HTTP traffic.

Application
      │
      ▼
AI Gateway
      │
 ┌────┼────┐
 ▼    ▼    ▼
GPT Claude Gemini

Because applications only integrate with the gateway, switching providers becomes much simpler.

For example, suppose your application currently sends all chat requests to GPT-5.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_LEANROUTE_API_KEY",
    base_url="https://gateway.example.com/v1"
)

response = client.chat.completions.create(
    model="gpt-5",
    messages=[
        {
            "role": "user",
            "content": "Explain AI Gateways."
        }
    ]
)

Behind the scenes, the gateway might decide to:

Route to OpenAI
Route to Anthropic
Route to Gemini
Retry using another provider
Reject requests that exceed policy
Record latency and token usage
Apply rate limits

The application does not need to know how these decisions are made. It simply receives a response.

This separation between application logic and infrastructure is one of the primary reasons AI Gateways have become a common component in production AI systems.

Problems an AI Gateway Solves

If you're building a proof of concept, integrating directly with a model provider is usually the right choice.

Production systems are different.

As applications grow, infrastructure concerns begin to outweigh the complexity of the AI prompts themselves. Multiple providers, changing model capabilities, outages, pricing updates, and enterprise requirements all introduce operational challenges that are difficult to solve inside application code.

An AI Gateway centralizes these concerns into a single layer, allowing application developers to focus on building features instead of maintaining provider integrations.

Let's look at the most common problems an AI Gateway solves.

1. Managing Multiple AI Providers

Today's AI ecosystem is more diverse than ever.

A single application might use:

OpenAI for general chat
Anthropic for long-context reasoning
Gemini for multimodal workflows
Open source models for internal workloads

Without a gateway, every provider introduces another SDK, authentication mechanism, request format, and error model.

Application
    ├── OpenAI SDK
    ├── Anthropic SDK
    ├── Gemini SDK
    ├── Groq SDK
    └── Custom Retry Logic

As the number of providers grows, so does the complexity of the application.

With an AI Gateway, the application only communicates with one endpoint.

Application
      │
      ▼
AI Gateway
      │
 ┌────┼─────┬─────┐
 ▼    ▼     ▼     ▼
OpenAI Claude Gemini Groq

Adding or removing providers becomes an infrastructure decision instead of an application change.

2. Reducing Vendor Lock-in

Every provider introduces platform-specific APIs and model names.

Migrating an application from one provider to another often requires updating request formats, authentication, streaming implementations, and error handling.

Over time, this creates vendor lock-in.

An AI Gateway abstracts provider-specific details behind a consistent interface.

Instead of writing application code that depends on a single provider, the application depends on the gateway.

This makes it significantly easier to evaluate new providers as the AI ecosystem evolves.

3. Intelligent Request Routing

Not every request needs the most capable or most expensive model.

Imagine an application that handles both customer support and financial analysis.

Simple support questions might only require a lightweight model.

Complex financial reasoning may benefit from a larger reasoning model.

Instead of hardcoding these decisions throughout the application, an AI Gateway can apply routing policies such as:

Route based on model capability
Route based on latency
Route based on cost
Route based on customer tier
Route based on geographic region
Route based on availability

For example:

Support Questions
        │
        ▼
 Smaller Fast Model

Financial Reports
        │
        ▼
 Larger Reasoning Model

Centralizing routing logic makes applications easier to maintain and allows routing strategies to evolve independently of the application itself.

4. Automatic Failover

No AI provider guarantees perfect availability.

Temporary outages, rate limits, networking issues, and regional disruptions are inevitable.

Without a gateway, every application needs to implement its own retry strategy.

OpenAI
   │
Unavailable
   │
Application Error

With an AI Gateway, failover can happen automatically.

OpenAI
   │
Unavailable
   │
   ▼
Anthropic
   │
Success

This improves resilience while keeping application code simple.

The application continues sending requests to the same endpoint without needing to understand which provider ultimately handled the request.

5. Cost Optimization

AI costs scale with usage.

As applications process thousands or millions of requests each day, even small pricing differences between providers become significant.

An AI Gateway can help optimize costs by applying routing policies such as:

Prefer lower-cost providers
Route lightweight requests to smaller models
Reserve premium models for complex workloads
Shift traffic based on current pricing

These decisions can often be updated centrally without modifying application code.

6. OpenAI Compatibility

The OpenAI API has become the de facto standard for AI integrations.

Many frameworks, SDKs, and developer tools already support it.

An OpenAI-compatible gateway allows existing applications to migrate by changing only the base URL and API key.

For many teams, this dramatically reduces migration effort.

Instead of rewriting application logic, developers can continue using the same SDK while gaining access to multiple providers through a single endpoint.

7. Authentication and Access Control

Production environments rarely expose provider API keys directly to every application.

Instead, organizations often require:

Centralized authentication
API key management
Team-based permissions
Usage quotas
Request auditing

An AI Gateway becomes the single point where these policies are enforced.

This reduces operational risk while simplifying credential management.

8. Observability

AI workloads are often difficult to debug.

Questions like these quickly become important:

Which provider handled this request?
How many tokens were consumed?
Why was latency higher than usual?
Which model generated this response?
How many requests failed today?

An AI Gateway provides a centralized location for collecting this information.

Instead of aggregating logs from multiple providers, operations teams can monitor AI traffic through one consistent interface.

9. Consistent Application Architecture

Perhaps the biggest advantage of an AI Gateway is architectural consistency.

Applications communicate with a single API regardless of:

which provider is selected
how routing decisions are made
where models are hosted
how retries are performed

As new providers emerge, applications remain stable while infrastructure evolves independently.

For engineering teams, this separation reduces maintenance costs and makes AI systems easier to operate over time.

Application

        │

        ▼

   AI Gateway

        │

 ┌──────┼─────────────┐

 ▼      ▼             ▼

OpenAI Anthropic   Gemini

        │

   Infrastructure Evolves

Application Remains Unchanged

AI Gateway Architecture

Understanding what an AI Gateway does is only half the story. To design reliable AI systems, it's equally important to understand where the gateway sits in your architecture and how it processes requests.

At a high level, an AI Gateway acts as the control plane for AI traffic. Every request flows through a single endpoint before reaching the underlying model provider.

                ┌─────────────────────┐
                │   Your Application  │
                └──────────┬──────────┘
                           │
                    HTTPS Request
                           │
                           ▼
                ┌─────────────────────┐
                │     AI Gateway      │
                ├─────────────────────┤
                │ Authentication      │
                │ Rate Limiting       │
                │ Routing             │
                │ Retry Logic         │
                │ Logging             │
                │ Observability       │
                └──────────┬──────────┘
                           │
        ┌──────────────────┼──────────────────┐
        ▼                  ▼                  ▼
    OpenAI            Anthropic           Gemini

Instead of embedding provider-specific logic into every application, the gateway becomes the single place where infrastructure decisions are made.

The Request Lifecycle

Let's walk through what happens when an application sends a request.

Step 1: Receive the Request

The application sends a request to the gateway using a familiar API, often an OpenAI-compatible endpoint.

POST /v1/chat/completions

At this point, the gateway knows nothing about which provider will ultimately handle the request.

Step 2: Authenticate the Client

Before forwarding the request, the gateway verifies that the caller is authorized.

Typical checks include:

API key validation
Team identification
Project quotas
Usage limits
Allowed models

If authentication fails, the request never reaches an upstream provider.

Step 3: Apply Policies

This is where the gateway begins making decisions.

Examples include:

Is this customer allowed to use GPT-5?
Has the project exceeded its monthly quota?
Is the request too large?
Should this request be logged?
Should prompt caching be enabled?

Keeping these policies inside the gateway means every application follows the same rules.

Step 4: Select a Provider

The routing engine determines where the request should go.

The simplest routing strategy is static.

All Requests
      │
      ▼
   OpenAI

Production systems are usually more dynamic.

                 Incoming Request
                        │
                        ▼
               Routing Decision
                        │
        ┌───────────────┼───────────────┐
        ▼               ▼               ▼
 Lowest Cost      Lowest Latency    Best Quality

Routing decisions can depend on:

requested model
customer plan
geographic region
provider health
latency
cost
custom business rules

Applications don't need to understand these rules. They simply send requests to the gateway.

Step 5: Transform the Request

Providers often expose similar capabilities, but their APIs are not identical.

The gateway can normalize differences before forwarding the request.

Examples include:

model name translation
request format conversion
parameter validation
header normalization

This allows applications to use a consistent interface while the gateway handles provider-specific details.

Step 6: Forward the Request

Once routing is complete, the gateway forwards the request to the selected provider.

Application
      │
      ▼
AI Gateway
      │
      ▼
 Anthropic

From the application's perspective, nothing changes.

Step 7: Handle Streaming

Streaming responses deserve special attention.

Instead of waiting for the complete response, the provider begins sending tokens immediately.

Provider

Hello
Hello there
Hello there!

The gateway streams these chunks back to the client as they arrive.

A well-designed gateway should preserve low latency while still collecting metrics and applying policies.

Step 8: Handle Failures

Failures happen.

Examples include:

request timeout
rate limiting
temporary outage
networking issue
provider overload

The gateway can decide whether to:

retry
switch providers
return an error immediately

For example:

Request
   │
   ▼
OpenAI
   │
 Timeout
   │
   ▼
Retry Anthropic
   │
Success

The application continues talking to the same endpoint throughout the process.

Step 9: Record Metrics

Before returning the response, the gateway records operational data.

Common metrics include:

request latency
provider used
model used
prompt tokens
completion tokens
total cost
response status

Having this information in one place makes operating AI systems much easier.

Why the Gateway Belongs in the Control Plane

One mistake many teams make is treating AI infrastructure as part of the application.

For a simple project, that's perfectly reasonable.

As systems grow, however, infrastructure concerns begin to spread throughout the codebase.

Application

├── OpenAI Client
├── Anthropic Client
├── Retry Logic
├── Cost Tracking
├── Logging
├── Provider Selection
└── Rate Limiting

Eventually, business logic becomes intertwined with infrastructure logic.

An AI Gateway separates those responsibilities.

Application

        │

Business Logic Only

        │

        ▼

    AI Gateway

        │

Infrastructure

• Routing
• Retries
• Authentication
• Logging
• Metrics
• Provider Selection

This separation makes applications easier to maintain, easier to test, and easier to evolve as the AI ecosystem changes.

Stateless by Design

A common characteristic of AI Gateways is that they are stateless.

The gateway processes requests but typically does not own application state or conversation history.

Instead, it focuses on:

receiving requests
enforcing policies
selecting providers
forwarding traffic
collecting telemetry

Because gateways are stateless, they are straightforward to scale horizontally.

             Load Balancer
                   │
     ┌─────────────┼─────────────┐
     ▼             ▼             ▼
 Gateway 1    Gateway 2    Gateway 3

As traffic grows, additional gateway instances can be added without changing application code.

The Gateway as an Abstraction Layer

Perhaps the most valuable aspect of an AI Gateway is that it decouples applications from providers.

Without a gateway:

Application
    │
    ├── OpenAI
    ├── Anthropic
    ├── Gemini
    └── Groq

With a gateway:

Application
       │
       ▼
 AI Gateway
       │
       ├── OpenAI
       ├── Anthropic
       ├── Gemini
       └── Groq

This abstraction gives engineering teams the freedom to adopt new models, replace providers, improve routing strategies, and optimize costs without continuously modifying application code.

As the AI ecosystem evolves, that flexibility becomes one of the biggest advantages of introducing an AI Gateway.

Key Takeaways

AI Gateways separate application logic from AI infrastructure.
They simplify multi-provider deployments.
Routing and failover improve reliability.
OpenAI compatibility reduces migration effort.
MCP extends the gateway beyond models into tools.

Originally published on Leanroute: https://leanroute.dev/blog/what-is-an-ai-gateway

DEV Community: Leanroute

Three new packages to make Leanroute routing a two-line install

@leanroute/vercel-ai-sdk

@leanroute/langchain

create-leanroute-app

Getting started

Cut Claude Code Bills 60% by Routing to DeepSeek and Groq

Cut Claude Code Bills 60% by Routing to DeepSeek and Groq

The Setup

Why This Was Hard Before

The Setup

The Routes to Savings

Route 1: Small edits go to Groq gpt-oss-20b

Route 2: Refactors and multi-file changes go to DeepSeek V4 Flash

Route 3: The hard reasoning stays on Opus 5 or gets swapped to DeepSeek V4 Pro

Route 4: Log searches and simple tool calls go to GPT-5 Nano

Route 5: Vision goes to Google Gemini 3.7 Flash, or DeepSeek V4 Flash Vision

The Practical Playbook

Real Numbers on a Real Workload

Guardrails That Make This Safe

What This Means for You

Key Takeaways

About Leanroute

Goodbye OpenRouter

Goodbye OpenRouter: Migrate to Leanroute with flat BYOK, no per-request markup

Why Migrate?

The Migration

Before

After

Keep Your Existing Model Integrations

Then Add MCP

OpenRouter vs Leanroute

Why This Matters

The Migration in Three Steps

1. Change the endpoint

2. Change the credentials

3. Add MCP when you need it

Who Should Consider Migrating?

One Gateway for Models and Tools

Key Takeaways

About Leanroute

Stop Building Your Own LLM Router

It Starts With One Model

Routing Isn't Just Picking a Model

Your Team Has Better Things to Build

When Should You Build Your Own?

Let Infrastructure Disappear

The Bigger Picture

Key Takeaways

About Leanroute

hi guys, I built leanroute.dev, it's One AI Gateway for LLM and Tools. Market is flooded with LLM Gateways, so I wanted to build an AI Gateway for LLM and MCP traffic. Its the next evolution of AI Gateways! https://www.producthunt.com/products/leanroute

I built leanroute.dev, it's One AI Gateway for LLM and Tools. Market is flooded with LLM Gateways, so I wanted to build an AI Gateway for LLM and MCP traffic. Its the next evolution of AI Gateways! https://www.producthunt.com/products/leanroute

One Gateway for AI Models and Tools

Why Every AI Gateway Will Need to Speak MCP

Table of Contents

The First Generation of AI Gateways

AI Applications Are Changing

Models Need Tools

Enter MCP

We've Seen This Problem Before

Key Takeaways

About Leanroute

AI Gateway - The Complete Guide

What Is an AI Gateway?

Table of Contents

What Is an AI Gateway?

Problems an AI Gateway Solves

1. Managing Multiple AI Providers

2. Reducing Vendor Lock-in

3. Intelligent Request Routing

4. Automatic Failover

5. Cost Optimization

6. OpenAI Compatibility

7. Authentication and Access Control

8. Observability

9. Consistent Application Architecture

AI Gateway Architecture

The Request Lifecycle

Step 1: Receive the Request

Step 2: Authenticate the Client