DEV Community: Leanroute

Announcing opa-sidecar-a2a: A Reference Implementation for Chain-Aware Agent Authorization

Leanroute — Tue, 15 Sep 2026 14:26:03 +0000

Announcing opa-sidecar-a2a: A Reference Implementation for Chain-Aware Agent Authorization

TL;DR

We just open-sourced opa-sidecar-a2a, a reference implementation of chain-aware authorization for agent-to-agent (A2A) traffic. It runs as a single Go binary alongside your agent or gateway, evaluates delegation chains against Rego policies, and returns allow/deny with obligations. MIT licensed, no product to buy, no vendor lock-in. This post explains why we built it, what it actually does, and how it fits into a broader story about the emerging A2A authorization gap.

The problem we kept seeing

Every A2A implementation shipping to production today ends up in one of two spots on authorization, both wrong.

Spot one: API-key trust. Agent B trusts agent A because agent A holds a valid API key. This tells B nothing about whether A is authorized to call B on behalf of the specific user or workflow that originated the request. A compromised or misbehaving agent can call anywhere its key allows, and the callee has no way to know whether the request actually originated with a user who granted this specific action.

Spot two: bearer-token pass-through. Agent A forwards the user's original bearer token to agent B. B checks it, and if it's valid, allows. This assumes agent A is trusted enough to hold the user's credentials and eliminates the ability to distinguish "the user allowed A to do X" from "A decided to do X on the user's behalf without telling them."

Neither model survives contact with a real delegation chain. Think about how a typical planner-executor pattern looks: user delegates to planner, planner delegates to executor, executor invokes a tool, tool calls a downstream API. Four hops, four different trust relationships. API-key trust collapses them all into "does the caller hold a valid key." Bearer-token pass-through collapses them all into "does the caller have the user's token."

What actually happens in a real delegation is more like: user grants planner permission to book_flight. Planner grants executor permission to invoke_tool:flight_booking_v1. Executor calls the tool. At each hop the grant is narrower than the previous one, and the callee should be able to verify the whole chain before allowing.

That's chain-aware authorization. And nothing in the OSS ecosystem was implementing it in a way we could point at.

What we built

opa-sidecar-a2a is a single Go binary that runs alongside an agent or a gateway. It exposes one production endpoint (POST /authorize), takes a delegation chain and a proposed action as input, and returns allow/deny plus obligations.

The moving parts:

Signature verification. Every grant in the chain carries an ed25519 signature from its issuer. The sidecar verifies each signature against a trust dir of public keys before the policy sees the input. If any signature fails, the grant is marked sig_verified: false and the reference policy denies.

Chain linking. Adjacent hops must link: grant N+1's issuer must equal grant N's subject. The user grants to planner, planner grants to executor, executor is the caller. Any break in that chain is a deny.

Scope matching. The chain must cover the action being attempted. If the user granted book_flight but the caller attempts send_email, the chain doesn't cover it, deny.

Time bounds. Every grant carries an exp. If any grant in the chain is expired, deny.

Policy evaluation. Once the chain is verified, the Rego policy makes the final call. This is where deployment-specific rules land: rate limits, sensitive-resource restrictions, per-user allowlists, obligations to attach to the response.

Obligations. A policy can return allow WITH obligations. Common shapes: "MUST redact PII from response," "MUST log to audit stream X," "MUST cap response bytes." The caller is responsible for honoring them.

Why a reference implementation and not a product

We could have built this into Leanroute directly and marketed it as a feature. We chose not to, for three reasons.

First, chain-aware authorization is a standard that needs to exist, not a differentiator we should keep to ourselves. If we bury the pattern inside our proxy nobody else can look at it, learn from it, or push it forward. As a standalone reference implementation, anyone building an A2A stack (with or without us) can adopt the shape.

Second, we do not currently proxy A2A traffic through Leanroute. Building the authorization layer without the transport is architecturally weird if we called it a Leanroute feature. As a reference implementation it makes sense: publish the "correct" pattern first, plug our own proxy into it when that ships.

Third, and honestly: we are a small team. Product surface area is expensive. Standalone reference implementations get to be minimal and understandable in a way that in-product features never do. This repo is meant to be read in an afternoon by anyone who wants to understand the pattern.

What's in the repo

Compact but complete for v0.1:

Single Go binary (cmd/opa-sidecar) with SIGHUP hot-reload.
Reference Rego policies covering the common cases: baseline chain verification, scoped tool invocation, time-bound restrictions, obligations.
Worked planner-executor demo (two Go agents plus a shell script) that exercises three scenarios: happy path, scope-creep attack, tampered chain.
Full protocol docs, policy-authoring guide, architecture notes.
CI covering build, test, gofmt, vet, golangci-lint, and OPA policy syntax.
MIT licensed.

How it fits into the broader picture

Two things are converging that make chain-aware authorization urgent, not academic.

Regulated agent deployments. EU AI Act Article 12 and DORA Article 11 (both in force through 2026) require auditable evidence trails for agent actions in regulated industries. That means when a bank's customer service agent invokes a tool that transfers funds, the audit log needs to show not just "agent X did Y" but "user U delegated to agent X, agent X delegated to agent Y, agent Y invoked tool Z, and here's the signed chain proving it." OPA sidecar produces the enforcement side of that story. Its natural complement is a transparency log for the evidence side (see MCP Transparency Log, which we're prototyping next).

MCP tool invocation at scale. As MCP becomes the standard way for agents to invoke external tools, "who authorized this tool call" becomes the question. Chain-aware authorization gives MCP servers a way to answer it that doesn't require them to trust every agent that talks to them.

What comes next

v0.1 is the reference. v0.2 will add revocation, OpenTelemetry, and Envoy ext_authz gRPC support so it works cleanly in service-mesh deployments. v0.3 will add W3C Verifiable Credentials 2.0 compatibility for the credential format.

If you're building A2A workflows today and hitting the authorization gap, please try it and open issues. Especially if:

You're on the Google Agent2Agent SDK and want to see what a compatible authorization sidecar looks like.
You're running MCP tool servers and need to authorize incoming tool calls by more than "does the caller have a valid session."
You're in a regulated industry and need to produce audit evidence that current API-key models can't.

Repo: github.com/leanroute/opa-sidecar-a2a. Everything is MIT, no signup needed.

Sources:

Kong AI Gateway Alternative (2026): When a Managed BYOK Gateway Wins

Leanroute — Mon, 14 Sep 2026 01:49:11 +0000

Kong AI Gateway Alternative (2026): When a Managed BYOK Gateway Wins

TL;DR

Kong AI Gateway is the enterprise-procurement-safe choice. It's the AI layer bolted onto Kong's existing API gateway product, which lots of enterprises already run. Kong Enterprise contracts start around $30-50K annually and scale into six figures. The Kong OSS core is free but missing the management UI, advanced analytics, OIDC, and the AI-specific plugins that make Kong AI Gateway compelling. This post is written for teams that got Kong AI Gateway put in front of them by their platform team or their procurement process and want to sanity-check the alternatives. It compares Kong against Leanroute, LiteLLM, Bifrost, and Portkey, and (because we're the Leanroute team writing this) is honest about the cases where Kong is genuinely the right call.

Who Actually Googles "Kong AI Gateway Alternative"

Three audiences:

Startups whose platform team said "we already run Kong for services, let's use Kong AI Gateway." That's a rational-sounding sentence that hides a $30-50K/year enterprise contract. If your monthly LLM spend is $2K, spending $2.5K/mo on the gateway to route it is upside-down math. These teams want confirmation that "we already run Kong" is not, on its own, a good enough reason.

Mid-market teams comparing Kong to lighter managed alternatives during procurement. Legitimate. Kong's brand is safe; the pricing is not. They want to know if a $15-$25/mo BYOK gateway can do the same job for their traffic shape.

Enterprise procurement teams checking whether the AI category has an incumbent yet. Kong is one of the "safe" names in this bucket, alongside Datadog and Splunk. They want to know whether picking Kong for the AI-gateway line item is a defensible choice or a habit.

If you're one of those three, read on.

What Kong AI Gateway Gets Right

Genuine, so the pushback lands honestly:

Procurement safety. Kong is an established API gateway vendor with existing enterprise contracts, DPAs, SOC 2, and legal reviews already done at a lot of companies. Adding an AI gateway line to an existing Kong contract is easier than onboarding a net-new vendor. This is a real, non-technical advantage.

Plugin ecosystem. Kong's plugin architecture is mature. If you need a custom auth flow, an unusual transformation, or an integration with an obscure system, someone has probably already written the Lua plugin.

Consistent operational model. If your platform team already runs Kong for non-AI services, running the AI gateway is marginal cost, with the same admin plane, same routes, and same runbook.

Enterprise features that actually matter for regulated buyers. OIDC, mTLS, WAF, RBAC, dev portal. Not all of these are AI-specific, but if you need them for your API surface, having them consistent between the AI gateway and everything else is worth something.

Self-hosted-only story. For buyers whose compliance rules require the gateway to live inside their infrastructure, Kong is a defensible answer.

For a Fortune-500 with an existing Kong footprint and a security-first buying committee, Kong AI Gateway is a defensible pick. We're not going to pretend otherwise.

Where Kong Overshoots for Most Teams

The line-items that look small in an enterprise deck and huge in a startup budget:

License cost. Kong Enterprise contracts start around $30-50K annually for small deployments. For a team doing $5K/mo of LLM traffic, the gateway costs more than the traffic, so the ratio is inverted. Even at $50K/mo LLM traffic, spending $50K/year on the gateway is a 8% overhead line item; managed alternatives run 0.5-2% at that scale.

Setup surface. Kong's core is a real API gateway with Nginx underneath. The AI plugins layer on top of that. If your team is not already running Kong, standing it up for AI gateway alone is a project measured in days to weeks, not hours. Someone on Reddit deploying Bifrost got it working in 30 seconds; standing up Kong takes longer.

Kong OSS is not Kong AI Gateway. The OSS release is missing Kong Manager (the UI), advanced analytics, OIDC, and the enterprise AI plugins. "We'll just use the OSS one" is often "we'll use a fraction of what made Kong appealing."

Operational familiarity gate. If your platform team already knows Kong, adding the AI layer is easy. If they don't, they're learning a full API gateway stack to solve a much narrower problem (route requests to LLM providers). That's like learning Kubernetes to run one binary.

Roadmap lag. Kong's AI-specific features (MCP passthrough, semantic caching, LLM-specific guardrails) landed later than the pure-play LLM gateways. As of 2026 they're catching up, but "later than the specialists" is the honest read.

None of this makes Kong bad. It makes "we already run Kong" not, on its own, a good enough reason to also run Kong AI Gateway.

The Case Comparison

Dimension	Kong AI Gateway	Leanroute (managed)	LiteLLM (self-host)	Bifrost (self-host)	Portkey (managed)
Pricing entry point	$30-50K+/year	$15 / $25 monthly	Free + ops	Free + ops	$49/mo + $9/100K logs
Hosting	Self-host	Managed	Self-host	Self-host	Managed
Setup time	Days to weeks	Minutes	Hours	Under 1 minute	Minutes
BYOK	Yes	Yes, 14 providers	Yes	Yes	Yes
Providers	Depends on plugins	14 curated	100+	12+	40+
MCP passthrough	Via plugins	Native	Via plugins	Native	Roadmap
Guardrails at edge	Via plugins	7 built-in	Via plugins	Via config	Yes (paid tier)
Governance UI	Kong Manager (paid)	Included	Enterprise tier	Yes (self-host)	Yes
Ops burden	Medium-high	None	High	Medium	None
Enterprise procurement fit	Highest	Growing	Low	Medium	Post-PANW
OpenAI-compatible wire	Yes	Yes	Yes	Yes	Yes

When Kong Is Actually the Right Call

Being straight, so you don't switch by default when Kong is genuinely the fit:

You already run Kong for your non-AI services. Adding the AI gateway is marginal effort. Even at $30-50K/year, the platform team's operational familiarity offsets a lot of that.
Your procurement requires a Gartner-quadrant-listed vendor. Kong is on it. Most of us aren't. If your buying committee has a "must be on the MQ" line, some alternatives are eliminated before the technical evaluation even starts.
You need mTLS, OIDC, WAF, and RBAC consistent between your AI gateway and your non-AI API surface. Bolt-on gateways for AI-only can't match Kong's breadth here.
Your traffic is 100M+ requests/month and you have SRE headcount already. At that scale the license cost amortizes and the operational familiarity of "same runbook as everything else" pays back.
You're building a regulated agent deployment (financial services, healthcare, defense). Kong's enterprise-security surface is a shorter compliance conversation than most alternatives.

When a Managed BYOK Gateway Wins

For most teams outside those five cases:

You're a startup or mid-market team. $30-50K/year on the gateway is real money that could fund your product roadmap. Managed BYOK at $15-25/mo does the same routing job for 1-2% of the cost.
You don't already run Kong. The setup surface is real, and "we might use Kong later for services" is not a good enough reason to eat the AI-gateway setup cost now.
Your team doesn't have SRE capacity. Kong is a real API gateway with real operational needs. If your team is 5 engineers, "run and monitor Kong" is not what you want them doing.
Your compliance answer is satisfied by a SOC 2 Type II managed vendor. A lot of buyers assume they need self-hosting for compliance and don't actually. A managed vendor with the right paperwork clears most standards.
You want to be on the routing layer in a week, not a quarter. Managed BYOK is a same-day setup. Kong AI Gateway with procurement, POC, and platform-team setup is a quarter minimum.

How the Migration Works

For teams moving from Kong AI Gateway to managed BYOK:

Sign up for Leanroute. BYOK Starter is $15/mo, 14-day free trial.
Paste your provider keys into the Providers page. OpenAI, Anthropic, Google, xAI, DeepSeek, Bedrock, etc. Keys are AES-256-GCM encrypted at rest, so a database dump reveals nothing usable.
Change your base URL from your Kong route to https://api.leanroute.dev/v1. Wire is OpenAI-compatible on both sides; no application code changes.
Migrate any Kong-specific plugin behaviour to Leanroute equivalents. Rate-limit plugin → Leanroute per-key caps. Auth plugin → Leanroute API keys. Guardrail plugin → Leanroute's 7 built-in rules. This is the actual work.
Run 10% of traffic through Leanroute for two weeks. Compare logs, latency, cost, and error rates.
Flip the ratio when the numbers agree. Keep Kong for your non-AI services if you're still using it there; drop the AI-gateway line from the contract at renewal.

The migration is intentionally boring because both are OpenAI-wire. The one place it gets less boring is if you had a lot of custom Lua plugins in front of your LLM traffic; you'll need to decide whether to replicate them in application code, use Leanroute's built-ins, or keep Kong specifically for the paths that need those plugins.

The Broader Point

The API gateway market has always had two camps: general-purpose gateways with an AI plugin (Kong AI Gateway, Apigee AI extension, Tyk AI) and AI-specialist gateways (Leanroute, Portkey, LiteLLM, Bifrost). The general-purpose camp wins on procurement safety and consistency with your existing API surface. The specialist camp wins on price, setup speed, and depth of AI-specific features.

The wrong move is picking a general-purpose gateway because "we already run Kong" and paying $30-50K/year for what a specialist gateway does for $15-25/mo. The other wrong move is picking a specialist gateway when your enterprise procurement process really does require a Gartner-listed vendor with an existing MSA.

Match the gateway camp to your buying committee. If it's a platform-and-security team at a Fortune-500, Kong probably wins. If it's an AI team at a startup or mid-market SaaS, a specialist gateway probably wins.

If you're in the second camp, we're at leanroute.dev. If you're in the first camp, Kong AI Gateway is a defensible pick and we won't try to talk you out of it.

Sources:

FinOps for LLM Spend: Three Reasons Your Bill Changed

Leanroute — Fri, 11 Sep 2026 04:11:43 +0000

FinOps for LLM Spend: Three Reasons Your Bill Changed

TL;DR

When your LLM bill moves month to month, one of three separate things caused it. Your usage changed (more or fewer tokens). Your pricing changed (a discount kicked in, a markup went away, cache-hit rates got billed differently). Or the way your app talks to the model changed (routing, caching, model choice). A dashboard that only shows "spend down 40%" without splitting those three apart is how teams end up thanking the wrong lever, and repeating the wrong fix next quarter. Here is a simple framework for pulling them apart before you claim you saved money.

Why this matters

Cloud FinOps figured this out a decade ago. Everyone in the AWS world knows that a lower bill can come from three unrelated things. Fewer instance hours. A Savings Plan kicking in. A rightsizing project. Confusing them is bad practice, because each one points to a different next step.

LLM spend is going through the same learning curve, from scratch, with teams that were not around for the cloud version. And it is harder here, because more moving parts sit between "user typed a prompt" and "line item on the invoice." Model choice, cache hit rate, provider markup, batch discounts, per-org rate limits, and gateway routing all pull on the number.

The framework below names the three axes. In practice, most real cost changes are a mix of movement on some or all of them. The trap is treating a combined change as if one lever caused it.

Axis 1: Usage change

This is what most dashboards actually show. It is the change in how much work your app is asking the LLM to do, before any pricing or architecture decisions.

What causes it

More or fewer end users
More or fewer requests per user
Longer or shorter prompts (context growth, bigger RAG documents)
Longer or shorter completions (chattier or terser system prompts, JSON-mode differences)
New features that add or remove LLM calls

How to measure it on its own

Hold everything else fixed. Compare input_tokens + output_tokens month to month, not dollars. If your token count changed but your cost per token did not, the whole delta is usage.

Common mistake

Crediting a lower bill to "we optimized" when the real story is that fewer users showed up. That is a churn signal, not an efficiency signal. Congratulations, you saved money on the LLM bill and lost a third of your customers.

The FinOps question worth asking every month

What would this bill have been at last month's cost per token? If the answer is close to this month, usage moved. If it is far off, usage is flat and something else is doing the work.

Axis 2: Pricing change

Pricing change is what changed about how you buy the tokens, not what tokens you bought. Most engineering teams never see this axis because it lives in contracts, procurement, and BD conversations.

What causes it

A committed-use discount kicked in (OpenAI, Anthropic, and Google all offer volume tiers)
Provisioned throughput on Bedrock replaced on-demand rates
A gateway or aggregator markup got added or removed (5 to 10 percent on OpenRouter, 20 to 40 percent on some enterprise wrappers, zero on flat-fee BYOK gateways)
Cache-hit rates got billed differently (Anthropic charges 10 percent of the base for cached input, OpenAI dropped cached-input rates in mid-2026, DeepSeek moved to a peak and off-peak model on 2026-08-16)
Batch API discounts (50 percent off if you can wait)
Switching between BYOK and managed-invoice paths through the same infrastructure

How to measure it on its own

Compute the effective dollars per million input tokens and dollars per million output tokens for the same model, at the same request pattern, on last month and this month. If usage is flat and the effective rate changed, the whole delta is pricing.

Common mistake

Giving engineering credit for a bill drop that was really the finance team negotiating a committed-use tier. The engineers who worked on caching feel like the cache saved money. The finance team knows the commit saved money. Both are right, and both are underselling. Without splitting them, one team's story becomes the official story, and the real driver gets missed.

The FinOps question worth asking every month

If usage had been identical to last month, what would we have paid at this month's contract terms? That number is your pricing-only impact.

Axis 3: Architecture change

Architecture change is what the gateway, routing, and caching layer changed in how the workload got served, before it ever reached the model.

What causes it

Cache hit rate moved (prompt cache, semantic cache, RAG retrieval cache)
Cheaper-model routing (a prompt that used to hit Sonnet 5 now hits DeepSeek V4 Flash on the same-tier arbitrage rule)
Model choice changed (dropping GPT-5.6 Sol for GPT-5.6 Terra on a workload that did not need premium reasoning)
Retry policy tightened (fewer retries after upstream 5xx errors)
Failover happened (primary provider was down, secondary provider costs different)
Batch consolidation (multiple small requests merged into one)
Guardrail overhead (prompt-injection detection adds one moderation API call per request)
MCP tool-use loops resolved in one round trip instead of three

How to measure it on its own

For a fixed workload and fixed contract terms, compare the effective cost per request. If the token bill dropped but the request count and unit rates are flat, the architecture in front of the model changed. Your gateway logs are the source of truth here. You need per-request records of model chosen, cache hit or miss, retry count, and failover events.

Common mistake

Treating a cache-hit-rate improvement as "we saved money" when the customer's prompt pattern happened to shift toward more repetitive requests. That is usage pretending to be architecture. Real architectural wins move the hit rate on a stable prompt pattern.

The FinOps question worth asking every month

If we had served this month's workload with last month's routing rules, cache configuration, and model selection, what would we have paid at this month's contract terms? That number is your architecture-only impact.

A concrete example

Two teams. Both send 100 million Sonnet 5 tokens per month.

Team A

On-demand direct to Anthropic
Retail pricing: $2 per 1M input, $10 per 1M output
5-to-1 input-to-output ratio, so blended cost is $20 per 6M tokens
100M tokens per month works out to about $2,000 per month

Team B

Has a 10 percent committed-use discount with Anthropic
Routes through a flat-fee BYOK gateway ($25 per month)
Prompt cache hit rate is 30 percent
Same 5-to-1 ratio, same 100M tokens

Team B math:

Full-price tokens: 70 percent of ($333 input + $1,667 output) minus 10 percent commit = about $1,260
Cache-hit tokens (30 percent, at 10 percent of the input rate): 30 percent of $333 times 0.1 = about $10
Gateway fee: $25
Total: about $1,295 per month

Same model. Same workload. 35 percent unit cost difference. Now the questions.

How much of the gap is usage? Zero. Both teams sent identical traffic.
How much is pricing? About $200. The 10 percent commit on 90 percent of the traffic.
How much is architecture? About $500. The 30 percent cache hit rate and the routing.
How much is the "gateway is expensive" tax? Negative $475. The $25 flat fee is tiny compared to the savings the routing enables.

If Team B's dashboard just says "we spend 35 percent less on LLMs than Team A does," they cannot answer any of those four questions. If Team A ever tries to close the gap, they will guess wrong about which lever to pull.

The discipline

Every monthly LLM cost review should answer the same three questions, in this order.

What did usage do? Token count month to month. Request count month to month.
What did pricing do? Effective dollars per token, by model, month to month.
What did architecture do? Effective cost per request, at fixed usage and fixed contract terms, month to month.

The three numbers should add up to the total dollar change. If they do not, you have unattributed variance. That usually means a fourth thing is quietly moving. Someone changed the default model. A new feature launched that no one told finance about. A retry storm during a provider outage bloated the bill.

Teams that do this monthly develop something the teams that skip it lack. An accurate story about why the LLM bill did what it did. Which is what makes future budgets defensible and future optimizations credible.

What this means for gateway choice

This post is on a gateway vendor's blog, so it is worth being honest about where the framework lands on that question.

How you route affects the architecture axis heavily, and the pricing axis in some paths. A flat-fee BYOK gateway is really two levers bundled together. It caps the "we take a percent of your tokens" markup at zero (pricing) and lets you route across model choice, cache, and failover (architecture). Whether that combination beats a direct-to-provider setup depends entirely on your cache hit rate, your model diversity, and whether you already have committed-use pricing negotiated.

But that decision has to be made with the numbers split apart. Not with a "we saved money" claim on either side. The gateway is a lever. Whether pulling it is worth it is a FinOps question, and it needs the three-axes framework to answer honestly.

The bigger point

Cloud FinOps became a discipline because too many teams confused usage growth with rising unit cost, or Savings Plan kicking in with a rightsizing win. The result was that when the AWS bill spiked, nobody could tell whether it was a scaling problem, a pricing problem, or a wasted-resource problem. And the fix was always a guess.

LLM spend is going through the same curve. Faster. With fewer reflexes in place. Teams that build the reflex early (split before you attribute, name the axis before you name the win) will make better product bets in year two of their AI investment. Teams that skip it will keep pulling levers and never quite know which one moved the number.

If you are building the LLM cost story at your company, the three axes are the first tool. The dashboards, the gateway choice, the vendor contracts all follow from having the vocabulary to talk about which axis moved.

Sources:

OpenRouter Alternative: When BYOK Beats Per-Request Markup

Leanroute — Mon, 07 Sep 2026 11:39:34 +0000

OpenRouter Alternative: When BYOK Beats Per-Request Markup

TL;DR

OpenRouter's model — one API key, hundreds of models, they hold the provider keys, they mark up per token — is excellent for hobby projects and model exploration. It becomes expensive around the point where your monthly LLM spend crosses roughly $200-500 and stays there, and it becomes limiting the moment you need spend caps enforced at gateway edge, provider commitments, or BYOK for compliance reasons. This post walks through the actual arithmetic on when to switch to a BYOK-based alternative, what you gain, what you give up, and how the migration works.

Who Actually Googles "OpenRouter Alternative"

There are two audiences searching for this:

Developers whose app grew up. Started on OpenRouter because it was the fastest way to try five models in an afternoon. Now the app has real users, real traffic, and a real monthly bill. The small per-request markup that was invisible at $50/mo becomes real money at $2,000/mo — a low-four-figure yearly tax that a different gateway model would erase.

Teams that want BYOK for control reasons unrelated to price. Their finance team wants the OpenAI invoice to come from OpenAI, not through a reseller. Or they have OpenAI enterprise pricing they cannot use through OpenRouter. Or they want to keep their provider rate-limit relationship (spinning up a new key on OpenRouter pulls from OpenRouter's shared pool, not yours). Or their security team has already vetted OpenAI's and Anthropic's DPAs and does not want to re-do it for a reseller.

If you are one of those two, the rest of this post is written for you.

The Economics: When Does the Markup Start to Matter

OpenRouter's take is not hidden. They publish per-model pricing and it is transparently a small markup above provider pricing. The question is not "is the markup fair" (it is) — the question is "at what point do I actually care."

Rough thresholds. Using a 5% markup as the floor and 10% as the ceiling:

Monthly LLM spend	5% markup cost	10% markup cost	Alternative flat cost
$50	$2.50	$5.00	$15-25
$200	$10	$20	$15-25
$500	$25	$50	$15-25
$2,000	$100	$200	$15-25
$10,000	$500	$1,000	$15-25

Break-even lives around the $200-500/month spend point. Below that, OpenRouter's markup is less than a flat $15-25/mo alternative and you should stay put. Above $500/mo the arithmetic flips hard: at $2K/mo of usage you are paying $100-200/mo in markup vs $25 flat, a $75-175/mo saving. At $10K/mo of usage the saving is $500-1000 a month.

The other axis is direction of travel. If your monthly spend is heading up as your product grows, the crossover happens once. It does not un-happen. The right time to switch is the month before you cross the threshold, not the month after.

What BYOK Actually Changes

The obvious change is billing: OpenAI bills you, Anthropic bills you, Google bills you, and the gateway charges only its flat subscription.

The less obvious changes matter more:

You own your provider rate limits. On OpenRouter your requests go through OpenRouter's pool. If OpenRouter as a whole gets rate-limited or their key gets deprioritized during a provider incident, your app feels it. With BYOK your app hits your provider account, whose rate limits you control by contacting the provider directly.

You get provider-native features you did not know you were missing. OpenAI's tier-based rate limits, Anthropic's provisioned throughput on Bedrock, Google's project-level quotas, xAI's beta features gated to specific accounts — all of these depend on the request originating from your account, not a shared one.

You can negotiate. At $10K+/mo of spend you can get committed-use discounts from OpenAI (typically 5-15% off), enterprise pricing from Anthropic, or provisioned throughput on Bedrock. None of that is available if the invoice is aggregated through a reseller.

You keep the data in the provider relationships you have already vetted. Some finance and security teams have already gone through the review of OpenAI's DPA and Anthropic's SOC 2, and re-doing that for a reseller is friction.

Spend caps are enforced at the edge, not after the fact. OpenRouter can alert you when you cross a budget, but by then the money is spent. A BYOK gateway with per-org and per-key caps will refuse the request before it hits the provider — so a runaway loop in your code costs you at most one over-limit request, not a weekend of billing.

What OpenRouter Gets Right That BYOK Alternatives Don't

Honest counter-side:

Zero setup for model exploration. Adding a 15th provider on OpenRouter is a dropdown change. Adding a 15th provider on any BYOK gateway involves signing up for that provider, getting a key, adding it to the gateway. If your app switches providers weekly for experimentation, this friction is real.

Aggregation of small usage. If you use ten providers at $5/mo each, opening ten accounts and managing ten keys is annoying. OpenRouter collapses that to one invoice.

Some regional coverage is easier through their pool. OpenRouter has business relationships with providers that individual developers cannot always match, especially for a subset of China-based providers.

If your usage looks like "many providers, low total spend, exploratory," OpenRouter is right and you should stay. If your usage looks like "few providers, real spend, production," BYOK usually wins.

Alternatives Compared

	OpenRouter	Leanroute	Portkey	LiteLLM (self-host)
Pricing	Per-token markup	Flat $15 / $25 monthly	Per-request tiers	Free + ops cost
BYOK	No (they hold keys)	Yes — 14 providers	Yes	Yes
Providers reachable	100+	14 curated	40+	100+
MCP passthrough	No	Native	Roadmap	Via plugins
Guardrails at edge	No	7 built-in	Yes (paid tier)	Via plugins
Hard spend caps	Alerts only	Enforced at edge	Yes	Yes (config)
Ops burden	None	None	None	High
OpenAI-compat wire	Yes	Yes	Yes	Yes

Leanroute deliberately curates 14 providers rather than chasing 100+. The rationale: adding a provider means we test the wire format, verify SigV4 / OAuth / whatever auth the provider uses, keep the pricing table current, run the canary against a live account every deploy, and take pages when it breaks. Providers we do not run in production we do not add. Portkey and LiteLLM have wider provider reach; Leanroute has narrower with tighter operational commitment. Pick the trade-off that matches your workload.

Migration: How Long It Takes

Assuming your app already uses the OpenAI SDK (most do), moving from OpenRouter to a BYOK gateway is a five-step change:

Sign up for the BYOK alternative, mint a runtime key.
Add your provider keys (OpenAI, Anthropic, DeepSeek, etc.) — for most providers this is paste-your-key; for AWS Bedrock this is an AWS IAM credential blob.
Change your OPENAI_BASE_URL from https://openrouter.ai/api/v1 to the new endpoint.
Change the API key from your OpenRouter key to your new gateway key.
Optional: transition model IDs. On OpenRouter they are provider/model (e.g., openai/gpt-5-mini). Most BYOK gateways including Leanroute use the same convention, so this is usually a no-op.

Send one canary request, check the response header for the routing hop, then cut traffic over. For a typical Leanroute-from-OpenRouter migration, an afternoon.

For a fuller walkthrough with code, see Migrating from OpenRouter to Leanroute.

Two Concrete Scenarios

Scenario A: side project at $80/mo. You are running a small productivity app for a few hundred users, mostly hitting openai/gpt-5.6-luna and anthropic/claude-haiku-4-5. Monthly OpenRouter bill: $80 + ~$5 markup. Switching to Leanroute costs $15-25/mo flat and saves nothing — the $10 markup is less than the flat fee. Verdict: stay on OpenRouter.

Scenario B: growing SaaS at $1,800/mo. You have real customers, real traffic, and the app is chewing through openai/gpt-5.6-terra for the main workflow and deepseek/deepseek-v4-pro for the batch summarization job. Monthly OpenRouter bill: $1,800 + ~$90-180 markup. Switching to Leanroute at $25/mo saves $65-155/mo and gets you hard spend caps, MCP passthrough, and guardrails. Verdict: switch.

Scenario C: enterprise with $15K/mo of Claude traffic. You already have an Anthropic enterprise agreement with committed-use pricing. OpenRouter cannot honor your committed-use pricing because the requests are not originating from your Anthropic account. Every request is being billed at retail plus markup. Verdict: switch immediately — you are paying twice.

When to NOT Switch

Signals OpenRouter is still the right answer for you:

Monthly spend under ~$200 and staying there.
You experiment across 20+ models regularly and want them all one dropdown away.
Your team has no interest in managing provider accounts.
You need models Leanroute or other BYOK gateways do not carry (some China-hosted models, some niche fine-tunes).

None of those are things to be embarrassed about. Pick the tool that fits.

The Meta Point

The OpenRouter model — aggregator plus markup — is a genuinely good fit for a specific band of users. The BYOK model — provider relationships stay with you, gateway is flat priced — is a genuinely good fit for a different band. The transition point is roughly the same point where a startup transitions from "moving fast, model shopping" to "we have real users, our LLM bill is a real line item, we want the provider relationships in our name."

If you want help thinking through your specific numbers, email us. Bring your last three months of OpenRouter invoices and we will tell you honestly whether switching saves you enough to bother.

Sources:

LiteLLM Alternatives in 2026: Honest Comparison of 5 LLM Gateways

Leanroute — Sun, 06 Sep 2026 14:37:11 +0000

LiteLLM Alternatives in 2026: Honest Comparison of 5 LLM Gateways

TL;DR

LiteLLM was the default OSS LLM gateway from 2024 through mid-2026. The recent CVE-2026-35029 privilege-escalation issue plus the working-month ops overhead of running a self-hosted gateway have pushed a lot of teams to reevaluate. This post compares five real alternatives — Leanroute, Portkey, Kong AI Gateway, Helicone, and OpenRouter — on the dimensions that actually matter: hosting model, wire format, BYOK, MCP support, guardrails, and honest verdicts about who each one is for. It is written by the Leanroute team but includes the cases where you should NOT switch to us.

Why People Are Searching for LiteLLM Alternatives

Three overlapping reasons show up in the search data and in our sales inbox:

The recent CVE. CVE-2026-35029 (privilege escalation from a read-only viewer role to config-modifier) landed a few days ago. If you already run LiteLLM the patch is fast, but the CVE prompted teams to ask a bigger question: "do we have someone on-call to catch the next one?"

The ops burden. LiteLLM is free to install and expensive to run. Postgres, Redis, monitoring, patching, secret rotation, on-call. Our working estimate for a mature self-hosted deployment carrying real production traffic is 15-25 engineer-hours per week — a real number that most teams do not budget for when they choose "free."

The category moved. LiteLLM was designed in 2023-2024 when the problem was "translate between provider APIs." In 2026 the problem is that plus MCP passthrough, agent-to-agent auth, spend caps that actually work at the gateway edge, and a security model that treats the gateway as a first-class trust boundary rather than an internal utility.

None of that means LiteLLM is bad. The maintainers have been diligent about patching, the OSS model is legit, and the community is real. It means the space has widened and the right answer depends on what you actually need.

Quick Decision Tree

Before the detailed comparison, the shortest useful version:

You want to keep self-hosting and just need MORE than LiteLLM (auth, guardrails, MCP) → Kong AI Gateway or Portkey OSS.
You want to STOP self-hosting → managed Leanroute, managed Portkey, or Helicone.
You want observability only (logs, cost tracking) and already have a routing layer → Helicone.
You want to give hobbyist devs multi-provider access without dealing with keys → OpenRouter.
You want a single OpenAI-compatible endpoint that speaks BYOK to 14 providers, adds MCP passthrough, and is priced flat per month regardless of traffic → Leanroute.

Now the details.

The Five Alternatives

1. Portkey

Portkey ships both an OSS gateway you can self-host and a managed cloud. The OSS gateway is credible and actively maintained; the managed cloud adds observability, prompt management, and a UI. Wire format is OpenAI-compatible. Guardrails and semantic caching are in the managed tier. Their focus is enterprise sales, which shows up in the product depth and in the pricing conversations.

Best for: teams that want either self-host-and-stay-in-control OR a managed alternative with a mature enterprise-shaped feature set. Migration from LiteLLM is straightforward because both speak OpenAI.

Not for: teams that want flat monthly pricing and zero per-request markup. Portkey's managed tier is priced on request volume.

2. Kong AI Gateway

Kong is the incumbent web API gateway with an AI gateway layer bolted on. If your team already runs Kong for other services, adding the AI gateway is genuinely marginal cost — same admin plane, same routes, same operational model. Fully self-hosted.

Best for: enterprises with an existing Kong footprint and an SRE team that already knows the operational model. This is the "safe procurement" pick — we saw it beat LiteLLM at a real enterprise account recently, and it won on being an established vendor, not on being a better AI gateway.

Not for: startups without a platform team. The setup surface is real, and the AI-specific features (spend caps, model routing, guardrails) are less mature than the web-API-gateway core.

3. Helicone

Helicone is primarily observability — logs, cost tracking, request replay — with proxy-mode routing as a secondary function. Simple integration: change your base URL and you get a dashboard. Open-source with a hosted tier.

Best for: teams that want visibility into what their app is spending on LLM calls without owning the routing layer. You keep talking to OpenAI or Anthropic directly (via the proxy), Helicone logs everything, and you get charts.

Not for: teams that need routing decisions, failover, BYOK to multiple providers behind a single endpoint, or MCP passthrough.

4. OpenRouter

OpenRouter is a managed multi-provider endpoint. One API key gets you access to hundreds of models across all major providers. They handle the provider keys, they handle billing, you send requests. Pricing is per-token with a small markup on top of provider pricing.

Best for: individual developers exploring models, hobbyists, small apps where "one credit card, many models" is the whole win. Great for prototyping.

Not for: production workloads where the markup adds up, teams that already have provider commitments and want BYOK, or teams that want spend caps enforced at gateway edge (OpenRouter alerts you after the spend, not before). If you fall into that bucket, see our OpenRouter alternative post for the fuller argument.

5. Leanroute (us)

We built Leanroute for the middle: teams that have outgrown OpenRouter's markups and hobbyist framing, but do not want the ops burden of LiteLLM or Kong. Flat $15 / $25 per month, BYOK to 14 providers (OpenAI, Anthropic, Google, xAI, DeepSeek, Groq, AWS Bedrock, Z.AI, Kimi, Doubao, Sarvam, Krutrim, plus two more), MCP passthrough on the same endpoint, a 7-rule guardrail starter library at the gateway edge, hard spend caps enforced before your provider invoice grows.

Best for: startup and mid-market teams running production LLM traffic who want managed but with their own provider relationships. Flat pricing means the gateway cost is predictable as you scale. BYOK means you keep provider rate limits, discounts, and commitments.

Not for:

Teams with data-residency requirements no managed vendor can meet — you should self-host (LiteLLM, Portkey OSS, or Kong).
Enterprises whose procurement requires a Gartner MQ-listed vendor. We are not on it, we are honest about it, come back to us when you are ready.
Teams that need observability only, without routing — Helicone is a cleaner fit for that shape.

Comparison Table

Dimension	LiteLLM (self-host)	Leanroute	Portkey	Kong AI Gateway	Helicone	OpenRouter
Hosting model	Self-host	Managed (SG)	Both	Self-host	Both	Managed
Wire format	OpenAI-compat	OpenAI-compat	OpenAI-compat	OpenAI-compat + Kong routes	OpenAI / Anthropic proxy	OpenAI-compat
BYOK to your provider account	Yes	Yes (14 providers)	Yes	Yes	Yes	No (they hold keys)
MCP passthrough	Via plugins	Native	Roadmap	Roadmap	No	No
Guardrails at edge	Via plugins	7 built-in	Built-in (managed tier)	Via plugins	No	No
Hard spend caps	Yes (config-heavy)	Yes (per-org, per-key)	Yes	Yes	Alerts only	Alerts only
Pricing model	Free + ops cost	Flat monthly	Per-request tiers	Enterprise contract	Freemium / tiered	Per-token markup
Ops burden	High	None	Medium / None	High	None	None

Migration Path from LiteLLM

If you decide to move off LiteLLM, the migration to any OpenAI-compatible alternative is mostly a base URL change:

Sign up for the alternative and set up your provider keys (either transferred as BYOK or added fresh).
Change OPENAI_BASE_URL in your app from your LiteLLM instance to the new endpoint.
Verify the first canary request. Most gateways expose a response header (Leanroute uses x-gateway-provider) that confirms the routing hop.
Cut over the rest of your traffic when the canary is clean.

For a typical Leanroute migration that takes an afternoon; for teams with heavy per-org config, custom guardrails, or unusual auth budget one or two days. Full docs at leanroute.dev/docs. If you want a hand, sales@leanroute.dev will reply personally.

When to Stay on LiteLLM

Two cases where the honest answer is "keep what you have":

You are one of the teams doing LiteLLM well. If you have a platform person who already owns the deployment, patches within hours of CVE announcements, has secret rotation set up, and monitors upstream provider health — you are running it correctly. Switching gains you nothing.

You need customization at the gateway layer that only a codebase you control gives you. Some teams add custom auth plugins, custom routing rules, or custom logging pipelines that would be awkward to reproduce on a managed gateway. If you have written more than a few hundred lines of LiteLLM extensions, you are probably in this camp.

For everyone else, the question worth asking is: does your team actually have the hours to keep owning the operational surface? If the answer is no, that ownership gap ships CVEs faster than the fixes.

The Meta Point

An LLM gateway is a trust boundary. It holds your provider keys, sees every prompt, and often has admin access to spend caps and routing rules. The right gateway is the one your team can actually own — which is a mix of feature fit and operational fit. Do not pick on features alone; ops overhead is real and compounds.

The teams that get hurt by CVEs like the LiteLLM one are the teams that installed the gateway eighteen months ago, moved on, and never revisited whether they had the capacity to keep it patched. That is true of any middleware, not just LLM middleware. Revisit the choice once a year. The stakes are quietly getting higher on both sides.

If you want to talk through your specific setup, email us. We will tell you honestly if we are a fit and point you elsewhere if we are not.

Sources:

LiteLLM Alternative - Self-hosting an LLM Gateway: when it's right and when it's not

Leanroute — Thu, 03 Sep 2026 02:29:30 +0000

Self-Hosting an LLM Gateway: When It's Right, When It's Not

TL;DR

An LLM gateway sits on your critical path with your provider keys, your prompts, and often admin access to spend caps and routing rules. When you self-host it, you own the security posture, the patch cadence, and the full ops stack: Postgres, Redis, monitoring, on-call. The CVE-2026-35029 privilege escalation in LiteLLM is a reminder that middleware is not something you can install once and forget. This post is about what self-hosting actually costs on a working month, when it is still the right call, and when a managed gateway makes more sense. It is not an argument that managed is better than self-hosted. It is an argument that the choice deserves an honest cost model instead of a default.

What Just Happened

A few days ago a privilege-escalation issue landed in LiteLLM (CVE-2026-35029, along with a few related chains). In short: a read-only viewer account could reach a config path that let it modify configuration, block keys or endpoints, and in some chains escalate further inside the gateway. The maintainers patched it. LiteLLM users who apply the update quickly are fine.

That is not a LiteLLM-specific problem. Every popular OSS middleware has shipped security fixes in the last two years. Envoy, Kong, Traefik, Consul, even Nginx itself. Complex middleware with an admin surface and a lot of moving pieces is a place where privilege boundaries slip. It always has been.

The reason this one caught attention is that LLM gateways are a newer category. Many teams installed one during 2024-2025 because they needed cost tracking and provider failover, and then never revisited the operational side. The CVE is a good moment to revisit it.

Why LLM Middleware Is a Higher-Value Target Than It Looks

Three reasons LLM gateways are worth attacking:

1. The gateway holds every provider key in your stack.

OpenAI, Anthropic, Google, xAI, DeepSeek, whatever else. If those keys have no per-model spend cap on the upstream side (most do not have granular caps), stealing them is worth real money to the attacker. A compromised OpenAI key can be used to burn thousands of dollars in a few hours before rate limits kick in.

2. The gateway sees every prompt.

Prompts routinely contain data the sending team did not intend to expose: customer PII pasted into a system prompt, secrets embedded in a code snippet, internal document content in a RAG retrieval, drafted emails that reveal deal terms. The gateway is a natural exfiltration point because it has all of it in one place.

3. Gateways often ship with weaker default auth than the app calling them.

The gateway was often built as an internal tool: it started life on a private network with basic-auth admin access, and the auth surface never got hardened when the tool became critical infrastructure. Many self-hosted deployments still have their admin UI reachable from the public internet on the same origin as the API, protected only by a shared bearer token stored in an env var.

Web API gateways went through the same maturation curve between 2017 and 2020. Kong, Envoy, and Traefik all had multiple privilege-escalation CVEs in that window. What is happening to LLM middleware now is that same curve, compressed into a shorter window because the category matured faster.

The Real Cost of Self-Hosting: An Honest Checklist

Here is what you actually take on when you decide to self-host an LLM gateway. Not the marketing version, the working-month version.

Patch cadence. Someone on your team needs to be watching the upstream repo for CVE announcements, evaluating whether they apply to your deployment, testing patches in staging, and deploying them fast. Estimate: 2-4 hours per month in a quiet month, 10+ hours in a month with a critical CVE.

Secret handling. BYOK keys for every provider, an admin key, database passwords, Redis credentials, JWT signing keys if you added auth. Rotation policy, storage (Vault? sealed secrets? env vars?), audit trail on who accessed what. Estimate: one-time setup of 8-16 hours, ongoing 1-2 hours per month for rotation.

Database operations. Postgres for request logs, usage aggregations, per-org config. You are now responsible for backups, restore drills, connection pooling, index maintenance, and cost tracking as the log table grows. Estimate: 2-4 hours per month once stable.

Redis operations. Cache, rate limits, spend counters. Persistence config, memory limits, monitoring for eviction pressure. Estimate: 1-2 hours per month.

Monitoring. Uptime, latency, error rates, upstream provider health, cache hit rates, spend against per-org caps. Someone needs to configure alerting thresholds and respond to pages. Estimate: 4-8 hours one-time setup, 1-3 hours per month tuning.

On-call. LLM outages happen. OpenAI has multi-hour incidents roughly monthly. Anthropic has intermittent 5xx storms. When those happen, your gateway is retrying, failing over, hopefully caching, and generally under load in ways it is not under load the rest of the time. Someone needs to be reachable. Estimate: rotating burden across 2+ engineers, or the founder is on-call 24/7.

Config changes without downtime. Adding a new provider, changing routing rules, updating guardrail policies, adjusting spend caps. All these need to happen without a restart if you have real traffic. Estimate: implicit in the platform team's ongoing time, hard to break out.

Adds up to: For a mature self-hosted LLM gateway serving real production traffic, budget 15-25 engineer-hours per week across your team. Not per month. Per week.

At a rough fully-loaded engineer cost of $150k/year in the US or $75k in APAC, that is $22-56k per year of engineering time going into keeping the gateway working. That is the number to compare against managed pricing, not the "$0 for OSS" number.

When Self-Hosting IS the Right Call

None of the above means managed always wins. There are real cases where self-hosting is correct:

You have data residency requirements no managed vendor can meet. Some regulated verticals (regional banking, defense, government) require that inference metadata never leave a specific jurisdiction, and no managed gateway currently has infrastructure in that jurisdiction. Self-host in-region.

You already have a platform team that runs this class of software. If you have SREs who already operate Envoy or Kong for other services, adding an LLM gateway to their scope is marginal cost. The delta is smaller than for a team that has never run middleware before.

You need deep compliance controls that require end-to-end audit. Some SOC 2 or ISO 27001 audits require that you can prove which humans touched which config knobs and when. A managed vendor gives you their audit trail, not yours. If your auditor wants your audit trail, you host.

You are at a volume where the managed markup exceeds a dedicated engineer. If you are pushing tens of billions of tokens per month, a 5-10% markup on inference is real money. At some threshold (often around $500k/year in LLM spend), a dedicated platform engineer to run the gateway becomes cheaper than the vendor's take. Do the math.

When Managed IS the Right Call

The other side, also honestly:

Small team, under 10 engineers. You do not have a dedicated platform person, and the founder-engineers who would end up owning the gateway have higher-leverage work to do. Managed is the right call by a wide margin.

You do not already run OSS middleware in production. If nobody on your team currently gets paged for Envoy or Kong, adding an LLM gateway is not marginal. It is a new class of on-call that will crowd out other work.

You do not want to be paged when OpenAI goes down. Managed gateways handle upstream provider outages for you: they retry, fail over, degrade gracefully. When you self-host, you get to watch your own retries.

The vendor is on-call for their infrastructure and you are on-call for your app. That split is often the right one at your stage.

If You Are Currently on LiteLLM and Considering a Switch

For a Leanroute-specific answer: the migration path is short because Leanroute exposes an OpenAI-compatible endpoint. In most cases the change is:

Sign up, mint a runtime key, add your provider keys via BYOK.
Change your OPENAI_BASE_URL from your LiteLLM instance to https://api.leanroute.dev/v1.
Keep your existing model IDs (openai/gpt-5.6-luna, anthropic/claude-sonnet-5, etc.) — they work as-is.
Verify with a canary request. Check the x-gateway-provider response header to prove routing.
Cut over the rest of your traffic when the canary is clean.

For a typical deployment that takes an afternoon. For a deployment with heavy per-org config, custom guardrails, or unusual auth, budget a day or two. Full docs at leanroute.dev/docs. If you want a hand, sales@leanroute.dev will reply personally.

The Meta Point

An LLM gateway is a trust boundary. When you own the gateway, you own the trust: your team's patching, your team's secret handling, your team's audit trail. When a vendor owns it, they own the trust, and their business exists on being trusted. Both models work. Both models fail when the owner is not paying attention.

The wrong answer is not making an active choice. The teams that get hurt by CVEs like the LiteLLM one are the teams that installed the gateway eighteen months ago, moved on, and never revisited whether they had the capacity to keep it patched. That is true of any middleware, not just LLM middleware.

If you are self-hosting, make sure someone on your team owns the operational surface, and give them the time to actually own it. If you are on managed, make sure the vendor's security posture and status page match your risk tolerance. Either way, revisit the choice once a year. The stakes are quietly getting higher on both sides.

If you are on the fence between the two, the honest question to ask is: do we have the team, and are we willing to give that team the hours? If either answer is no, managed is the right call.

Sources:

Chain-aware Authorization: OPA + MCP for the A2A Auth Problem

Leanroute — Sat, 29 Aug 2026 05:09:35 +0000

Chain-aware Authorization: OPA + MCP for the A2A Auth Problem

TL;DR

AI agents are nothing but APIs on cron with autonomy and reasoning. That means authorization needs to happen at every tool call, not just at session start. And the policy needs to see the full call chain to make sane decisions, otherwise a "read PII" and a "send email" both scoped correctly still add up to a data leak. The pattern I think will become standard is OPA as a sidecar to every MCP server, evaluating each tool call against a Rego policy that treats the call chain as input. I am calling this chain-aware authorization because nobody has named it yet and every agent stack is going to need it.

The Problem That Keeps Coming Up

I keep seeing this in LinkedIn posts. Someone builds an agent, gives it a bunch of tool scopes, and then discovers that the composition of tools is where all the actual risk lives.

Here is the shape of it. Say your agent has read:crm and send:email. Both scopes are legit on their own. Reading a customer record, fine. Sending an email, fine. But an agent that reads a customer's PII and then sends an outbound email based on what it read? That is the shape of a data exfiltration. Neither scope caught it because they were granted separately, at session start, and never re-checked against each other.

The moment agents can chain tool calls, session-scope auth is done. It sees each call in isolation. It cannot look at what came before. And what came before is often the entire story.

This is the A2A auth problem. And in the projects I have been part of over the last few years, I have not seen anyone solve it cleanly. Everyone hand-rolls something.

Why the Usual Answers Are Bad

Three patterns dominate today, and all three fail the composition test.

One key per session, static scopes. What every framework does out of the box. Grant read:crm and send:email at session start, agent uses them the whole loop. Zero re-authorization between calls. This is the version of "auth" that says "the door was locked when you walked in" and then just gives you the run of the house. Easy to build. Easy to break.

One key per tool, narrower scopes. Better for blast radius, worse for ergonomics. The agent is now juggling 15 credentials. And composition is still unrestricted, because if it needs both tools, it holds both keys. The keys just look nicer.

Framework-native ACLs. LangGraph guards, CrewAI validators, custom middleware in every agent stack. All hand-rolled, all different, all buried in application code that nobody security-reviews. When your compliance team asks "prove which tool calls this agent was allowed to make last quarter," nobody has a real answer. They have a guess and a Slack thread.

None of these treat auth as what it actually is. Auth is a pipeline concern. Not an application concern. The moment you make it the application's job, it stops getting done.

The Lesson From Web APIs (Which We Already Learned)

We solved this exact problem a decade ago for web services. Two patterns won:

Envoy + OPA. Envoy sits as a reverse proxy in front of every service. OPA sits as a sidecar next to Envoy. Every request from outside hits Envoy, which asks OPA "is this allowed?" before forwarding to the service. The service does not know OPA exists. Auth is a network-layer concern.

Istio auth policies. Same idea at the service mesh. Every east-west call gets policy-checked. The mesh does not care what the service is.

The lesson from both patterns is the same. Authorization belongs in a layer that sees every call and knows nothing about what the calls are guarding. If the service has to opt in, it does not get done.

AI agents in 2026 are exactly where web services were in 2016. Every framework rolls its own auth in application code. Every framework's auth is bad. The pattern that fixed it before is going to fix it again.

The Pattern: Chain-aware Authorization

Here is what the stack looks like.

      Agent (LangGraph, CrewAI, Claude Code, whatever)
                        │
                        ▼
                  MCP Server
                        │
                  ┌─────┴─────┐
                  │           │
                  ▼           ▼
              OPA Sidecar  Tool Registry
                  │           │
                  ▼           ▼
             Allow / Deny  Actual Tool
                            (DB, API, filesystem)

Every tool call goes through the MCP server. Before the server dispatches to the actual tool, it calls out to an OPA sidecar with a payload that includes:

The tool name being invoked
The arguments the agent wants to pass
The identity of the agent and the human it acts for
The full chain of tool calls made so far in this session
Any data returned from previous tool calls the policy might care about (redact or hash if needed)

OPA runs a Rego policy against this input. It returns one of three things:

Allow. MCP forwards the call to the tool as normal.
Deny. MCP returns an error to the agent, ideally with a policy reason so the agent can adapt.
Transform. MCP forwards a modified version. This is the underrated one. The policy can redact fields from the response before the agent sees them, add a filter to a DB query, or rewrite the target of an outbound call.

Now, when I first thought about this, I said "OPA is stateless, we need to make it stateful." I was wrong about the framing. OPA policies are pure functions of their input, and that is actually the whole reason this works. You do not need OPA to become stateful. You need to pass the state as input. The chain is input. The policy stays pure. The state lives one layer up, in the MCP server that appends to the chain as tool calls happen.

That distinction matters because pure policies stay testable, deterministic, and auditable. The moment you try to bolt hidden state onto OPA, you lose all three.

A Real Rego Example

Here is a policy that blocks the composition case I opened with. Read PII, then try to send an email? Denied.

package agent.authz

default allow := false

# Track which sensitive data types the agent has read this session.
# `input.chain` is the array of prior tool calls, oldest first.
sensitive_data_read contains kind if {
    some call in input.chain
    call.tool == "crm.read_customer"
    kind := "pii"
}

sensitive_data_read contains kind if {
    some call in input.chain
    call.tool == "vault.read_secret"
    kind := "credential"
}

# Outbound network tools that could exfiltrate data.
outbound_tool if {
    input.tool in {"email.send", "http.post", "webhook.dispatch"}
}

# The rule. Allow unless the agent has read sensitive data AND is now
# trying to make an outbound call. Neither the read nor the send is
# blocked in isolation. Only the composition.
deny_reason := "read_then_send_composition" if {
    outbound_tool
    count(sensitive_data_read) > 0
}

allow if {
    not deny_reason
}

The agent's call chain gets passed in as input.chain. The current call is input.tool. That is the whole policy. A handful of rules. Testing it is one Rego test file. Auditing it is a five minute code review.

Compare that to "add a check in the LangGraph node that calls email.send," which is what most stacks do today. That check lives in one framework, in one node, in one repo, invisible to security review. If a new dev adds a second email node next month, they will forget the check.

What This Actually Gives You

Interoperable. Any MCP server + any OPA sidecar + any Rego policy. Swap the agent framework, keep the policy. Swap the MCP server, keep the policy. Swap OPA for a home-grown evaluator, keep the same policy interface. Nothing else in the stack cares.

Auditable. Every allow, deny, and transform is a log line with the full input and the policy version. When someone asks "prove this agent could not exfiltrate customer data," you point at the OPA logs. Rego has good tooling here already because enterprise Kubernetes has been running this pattern at scale for years.

Context-aware. The policy sees the whole story. Rules like "no more than three writes to the same table per hour," "no email send after any finance.* read," "no LLM call over $0.10 unless the human approved this session," all of these become one-liners.

Testable. Rego has a first-class test framework. Every rule ships with test cases that fake input.chain. You catch the "oops, this rule breaks the happy path" case before it goes live.

Portable. OPA is a CNCF graduated project. Rego is not going anywhere. Policies you write today will still evaluate in five years. Your bespoke LangGraph guard, on the other hand, is at the mercy of the framework maintainer.

What Is Still Hard

I want to be honest about the tradeoffs, because I hate reading blog posts that pretend the recommended pattern is free.

Latency. Every tool call now includes a round trip to OPA. OPA is fast, usually sub-millisecond for well-written policies, but for agent loops that fire hundreds of tool calls per session, it adds up. Run OPA as a true sidecar in the same pod so the network hop is loopback. Keep policies small. Cache policy decisions where the input is stable.

Policy authoring is a real skill. Rego has a learning curve, and small teams may not have anyone who wants to own it. My workaround is to treat policy authoring as a security-team activity and start from an open-source policy library that you adapt, rather than writing from scratch.

The chain has to be trusted. If the agent lies about its chain (says "no previous calls" when there were 20), the policy is worthless. This is why the MCP server has to be the one appending to the chain and the enforcement point. The agent cannot forge what it does not write.

Cross-agent policies are unsolved. When agent A calls agent B, whose chain is the input? Both. The right answer is probably "merge the callee's chain with the caller's chain, policy sees the union." But this is not a settled pattern yet. If you are shipping A2A today, be honest with yourself that the auth story is provisional.

Where This Sits at Leanroute

We are implementing chain-aware authorization for LLM routing decisions internally. Every LLM call through the gateway is a policy decision. Which provider, at what price, with what scope. The current implementation uses hand-rolled rules and per-key spend caps. We are migrating toward OPA-backed policy so customers can bring their own rules without patching our code.

If you want to try the pattern today, the @leanroute/mcp-server package on npm is a good starting point. It has the MCP hooks you need to insert an OPA call between the tool dispatch and the actual tool. I plan to publish a reference implementation of the full sidecar setup in the next few weeks. If you want the code as soon as it drops, follow the Leanroute GitHub.

Key Takeaways

AI agents are nothing but APIs on cron with autonomy and reasoning. Everything we learned about API authorization still applies.
Session-scope auth cannot handle composition attacks. Chain-aware auth can.
The right primitive is OPA, running as a sidecar to your MCP server. Every tool call gets policy-checked against the full call chain.
Rego policies stay stateless, pure, testable, and auditable. The chain is input, not state.
The pattern is called chain-aware authorization. Nobody named it. So I did.
Interoperable. Portable. Boring in the way infrastructure should be.

About Leanroute

Leanroute is One Gateway for Models and Tools.

Route LLM calls across 13 providers, forward MCP tools, enforce policy at the gateway. One integration point for cost, routing, and authorization.

Learn more at leanroute.dev

Three new packages to make Leanroute routing a two-line install

Leanroute — Thu, 27 Aug 2026 16:12:02 +0000

I posted about cutting Claude Code bills 67% by routing across 13 providers instead of paying Anthropic for every request. A few people asked "cool, but how do I actually integrate this into my existing app?"

Three new packages shipped today.

@leanroute/vercel-ai-sdk

If you are on Next.js and use the Vercel AI SDK, this is a drop-in provider:

import { leanroute } from "@leanroute/vercel-ai-sdk";
import { generateText } from "ai";

const { text } = await generateText({
  model: leanroute("deepseek/deepseek-v4-flash"),
  prompt: "Hi",
});

Same interface as @ai-sdk/openai, but the string can be any of 24 headline SKUs across OpenAI, Anthropic, Google, DeepSeek, Groq, xAI, and more. Streaming, tool calling, structured output, and Next.js server components all work out of the box because it wraps createOpenAI from the AI SDK.

@leanroute/langchain

LangChain users get the same treatment:

import { ChatLeanroute } from "@leanroute/langchain";

const llm = new ChatLeanroute({ model: "anthropic/claude-sonnet-5" });
const res = await llm.invoke("Hi");

Returns a ChatOpenAI instance, so it plugs into every runnable, agent, and chain that accepts a BaseChatModel.

create-leanroute-app

New project? Skip the setup:

npx create-leanroute-app my-app

Two templates. next gives you a streaming chat UI with a model picker, cost display, and Vercel deploy config. node gives you the OpenAI SDK pre-configured with three example scripts. Both take about 60 seconds from npx to running app.

Getting started

Grab a runtime key at leanroute.dev/dashboard/keys, export it, install the package that matches your stack. Full walkthrough in the original post.

All three packages: Apache-2.0, source in github.com/leanroute/ai-gateway.

Cut Claude Code Bills 60% by Routing to DeepSeek and Groq

Leanroute — Thu, 27 Aug 2026 02:48:49 +0000

Cut Claude Code Bills 60% by Routing to DeepSeek and Groq

TL;DR

Claude Code is great. Claude Code on Sonnet 5 for every request is expensive. If you point Claude Code at a gateway that speaks OpenAI, Anthropic, and MCP, you can keep the same workflow and route the cheap work (refactors, small edits, log searches, unit tests) to DeepSeek V4 Flash or Groq gpt-oss, while keeping the hard work on Sonnet or Opus. On a real developer workload the blended bill drops between 60% and 92%.

The Setup

Claude Code is the default reason a lot of developers get an Anthropic bill in the first place. Long sessions, big context, tool use, MCP, sub-agents. Each of those things is a great feature. Each of those things is also a token multiplier.

Here is what a Sonnet 5 heavy user looks like today, in round numbers:

Model	Input $/M	Output $/M	Blended 5:1 per 6M tokens
claude-opus-5	5.00	25.00	$50.00
claude-sonnet-5	2.00	10.00	$20.00
claude-haiku-4-5	1.00	5.00	$10.00

Blended 5:1 (5 input to 1 output) is a reasonable approximation for interactive coding traffic. If you push 6M tokens a day through Sonnet 5, that is about $600 a month. Opus is 2.5x that.

Now the same math for the models you never think about when you are inside Claude Code:

Model	Input $/M	Output $/M	Blended 5:1 per 6M tokens
deepseek-v4-flash (off-peak)	0.22	0.66	$1.76
deepseek-v4-pro (off-peak)	0.66	1.98	$5.28
groq/openai/gpt-oss-120b	0.15	0.60	$1.35
groq/openai/gpt-oss-20b	0.075	0.30	$0.675
openai/gpt-5-nano	0.05	0.40	$0.65
openai/gpt-5-mini	0.25	2.00	$3.25

DeepSeek V4 Flash is roughly 11x cheaper than Sonnet 5 on blended cost. GPT-5 Nano is roughly 30x cheaper. Groq gpt-oss-120b sits in between at 15x cheaper, and it runs on LPUs, so it is fast enough that you barely feel the round trip.

The question is not whether these numbers are real. They are. The question is how you actually use these cheaper models from inside your Claude Code session without breaking your workflow.

That is what the rest of this post is about.

Why This Was Hard Before

Claude Code talks to Anthropic. That is by design. Anthropic ships the CLI, Anthropic owns the model, Anthropic owns the invoice.

If you want to route some of that traffic to a different model, you have historically had a few bad options:

Manually switch tools. Pop out of Claude Code, run something in Cursor with a different provider, paste the answer back. Slow, breaks your context, kills your flow.
Run a local proxy. Set up LiteLLM or a Docker container, override ANTHROPIC_BASE_URL, hope the response format matches. Works, but you are now maintaining infrastructure.
Live with the bill. What most people do.

The reason none of these are great is that Claude Code was built around one wire format (Anthropic Messages) and one MCP surface. If you want to route to OpenAI, DeepSeek, Groq, xAI, or one of the Chinese labs, you need something in the middle that speaks both wire formats and forwards MCP correctly.

That thing is a gateway.

The Setup

Here is the whole stack once you have a gateway in the middle:

Claude Code
     │
     ▼
Leanroute Gateway
     ├──▶ Anthropic (claude-sonnet-5, opus-5, haiku-4-5, fable-5)
     ├──▶ OpenAI (gpt-5-nano, gpt-5-mini, gpt-5.6-luna, terra, sol)
     ├──▶ DeepSeek (v4-flash, v4-pro, v4-flash-vision-exp)
     ├──▶ Groq (openai/gpt-oss-20b, openai/gpt-oss-120b)
     ├──▶ Google, xAI, Qwen, GLM, Doubao, Kimi
     ├──▶ Sarvam, Krutrim
     └──▶ Meta (Muse Spark, Muse Code)

Same Claude Code binary. Same tools, same MCP servers, same prompt cache behaviour. What changes is the endpoint your local claude is pointed at, and which model string you type.

Setup is three steps.

Step 1. Get a Leanroute key.

Sign up at leanroute.dev, top up some credit, mint a runtime key from the dashboard. The key looks like gw_live_... and it works anywhere the OpenAI or Anthropic wire format works.

Step 2. Point Claude Code at Leanroute.

Set two environment variables in your shell profile:

export ANTHROPIC_BASE_URL="https://api.leanroute.dev/anthropic"
export ANTHROPIC_API_KEY="gw_live_your_key_here"

The anthropic subpath tells Leanroute to speak Anthropic Messages wire format. Everything Claude Code sends, including tool calls, images, MCP mcp_servers blocks, and system prompts, forwards through unmodified.

Step 3. Install the Leanroute MCP server.

npm install -g @leanroute/mcp-server

Then in your Claude Code MCP config (~/.config/claude/mcp.json or the equivalent for your OS), add:

{
  "mcpServers": {
    "leanroute": {
      "command": "npx",
      "args": ["-y", "@leanroute/mcp-server"],
      "env": {
        "LEANROUTE_ADMIN_KEY": "gw_admin_your_admin_key_here"
      }
    }
  }
}

The MCP server exposes tools like list_models, get_pricing, cheapest_for, get_usage, and route_call. Now Claude Code can ask itself, mid conversation, questions like "what is the cheapest flagship model I can use for this refactor" and route accordingly.

That is the whole install.

The Routes to Savings

Here is where the actual money comes from. Each of these is a real substitution you can make inside a Claude Code session today, along with the cost math and the rough quality tradeoff.

Route 1: Small edits go to Groq gpt-oss-20b

Everyone knows Sonnet is overkill for renaming a variable, but everyone uses it anyway because that is what Claude Code opens with. If you switch to Groq gpt-oss-20b for the small stuff, you drop from $20 per 6M tokens to $0.68. That is a 29x reduction on the blended cost, and Groq's LPU inference is fast enough (about 500 to 1000 tokens per second on the 20B) that you get the answer before you would have finished blinking.

How to do it: at the start of a small-edit request, ask Claude to route to groq/openai/gpt-oss-20b. The MCP server sees the intent and issues the call. You still see the diff, you still approve it, but the token bill goes to Groq.

Route 2: Refactors and multi-file changes go to DeepSeek V4 Flash

This is the big one. DeepSeek V4 Flash sits in the flagship tier on quality benchmarks but costs $0.22 in and $0.66 out per million tokens off-peak. Blended 5:1 that is $1.76 per 6M, versus Sonnet 5 at $20.

For a typical refactor pass (read 20 files, propose changes to 5, get user feedback, re-read, revise), you can burn through 300k to 500k tokens easily. On Sonnet 5 that is about $1 to $1.60. On DeepSeek V4 Flash it is $0.09 to $0.14.

Quality tradeoff is real but smaller than the price gap suggests. DeepSeek V4 Flash handles most refactoring, test generation, and boilerplate work without a noticeable step down. Where it slips is nuanced product judgement, ambiguous specs, and long-horizon multi-step reasoning. For those, you route to something better.

DeepSeek has one gotcha: peak-hour pricing doubles rates during 01:00 to 04:00 and 06:00 to 10:00 UTC on weekdays. Weekends are always off-peak. If you are in North America the peak windows are your evenings, which is inconvenient. Leanroute's arbitrage router knows the schedule and will move DeepSeek traffic to a same-tier alternative (Groq gpt-oss-120b, or gpt-5.6-luna) during peak windows if you have arbitrage enabled.

Route 3: The hard reasoning stays on Opus 5 or gets swapped to DeepSeek V4 Pro

For the requests that genuinely need premium reasoning (architecture design, tricky concurrency bugs, security review), Claude Opus 5 is $50 per 6M tokens blended. DeepSeek V4 Pro is $5.28. That is a 9.5x reduction and V4 Pro is a legitimate reasoning model, not a distilled downgrade.

If you trust the swap, keep arbitrage on and let the router pick V4 Pro. If you do not, keep Opus in your explicit route list and use it for the requests where you actually notice the difference.

A useful discipline: at the start of the session, tell Claude something like "for design questions or bugs that touch concurrency or auth, use Opus 5. For everything else prefer the cheapest available flagship." The MCP server exposes cheapest_for and route_call so Claude can make that decision per request.

Route 4: Log searches and simple tool calls go to GPT-5 Nano

If your Claude Code session is doing a lot of "grep this repo," "count occurrences of X," "list files matching pattern" style tool work, the LLM is essentially orchestrating shell commands. That does not require a $2/M input model. GPT-5 Nano at $0.05 in / $0.40 out is 40x cheaper on input than Sonnet 5, and easily good enough to decide which grep to run.

Route 5: Vision goes to Google Gemini 3.7 Flash, or DeepSeek V4 Flash Vision

When you paste a screenshot into Claude Code to say "why does this button look wrong," Anthropic's vision model is doing the work at Sonnet 5 rates. Google Gemini 3.7 Flash is $0.75 in / $3.75 out (intro pricing through December 31, 2026) and handles UI screenshots and diagram parsing well.

DeepSeek's new v4-flash-vision-exp is even cheaper but caps images at 384 input tokens per image, which is fine for small icons and bad for full page screenshots. Leanroute keeps it explicit rather than auto-routing there, so you have to ask for it by name. Worth it when you know your images are small.

The Practical Playbook

Here is what a real Claude Code session looks like once you have this set up.

Session start.

You launch claude in your project. First message you type includes:

For this session, use these routing rules:
- Small edits (renames, single-line fixes): groq/openai/gpt-oss-20b
- Refactors and file-scale changes: deepseek/deepseek-v4-flash
- Design questions or bugs I flag as hard: anthropic/claude-opus-5
- Everything else: cheapest available flagship, prefer non-peak DeepSeek

Confirm and I will start.

Claude reads the rules, calls the MCP server's list_models once to confirm they resolve, and acknowledges.

During the session.

Claude handles routing per turn. When it decides a request is a small edit, it calls route_call targeting the Groq SKU. When it hits a refactor, DeepSeek V4 Flash. You still see the diff, you still see the tool calls, you still approve edits. The only thing that changed is which provider answered.

At the end.

You run one MCP call to see the damage:

Use the leanroute MCP to show my usage for the last 24 hours, grouped by model.

You get back a table showing which SKUs answered what share of your traffic and how much each cost. On a real day of coding work, this typically looks like 70% to 80% of tokens on DeepSeek V4 Flash or Groq, 15% to 20% on Sonnet 5 or Opus 5, and the rest split across GPT-5 Nano and Gemini for tool calls and vision.

Real Numbers on a Real Workload

We ran this on an internal week of Claude Code work (13 developers, roughly 62 hours of active sessions, a mix of refactors, feature work, bug fixing, and code review).

Without routing, all traffic on Sonnet 5, with occasional Opus escalations:

Total tokens: 41.8M
Total cost: $487.90
Average per developer: $37.53

With routing, using the rules above:

Total tokens: 41.8M
Total cost: $158.20
Average per developer: $12.17

That is a 67.6% reduction. The split ended up at 68% DeepSeek V4 Flash, 8% Groq gpt-oss-120b, 12% Sonnet 5, 6% Opus 5, 4% GPT-5 Nano, 2% other. No one noticed a quality difference in day to day work. The two people who did notice were both doing algorithm design and stayed on Opus 5 on purpose.

Your mileage will vary based on how much of your work is genuinely reasoning-hard versus pattern-matching. If most of your Claude Code time is "read this file, propose a fix," you will see the higher end of the savings range. If most of it is "design me a distributed rate limiter for a multi region deploy," you will see the lower end.

Guardrails That Make This Safe

Cheaper models are only worth it if the failure modes are bounded. A few things the gateway does by default:

Failover. If your primary provider returns a 5xx or gets rate limited, the gateway retries on a same-tier alternative before returning an error to Claude Code. So a DeepSeek outage during peak hours does not become a stuck session, it becomes a transparent swap to Groq or Gemini.

Explicit routing. If you need to pin a request to a specific provider (benchmarking, compliance, tool-call behaviour that only one provider gets right), Claude Code can send x-gateway-routing: explicit on the request. Arbitrage and cheaper-model swap are both disabled for that call.

Per-key spend caps. You can mint runtime keys with daily and monthly USD caps in the dashboard. If your team is experimenting with routing, cap the key at $10 a day and you literally cannot lose more than $10 to a bad rule.

Admin keys are a separate class. The key you gave the MCP server (gw_admin_...) can list, mint, and revoke runtime keys and read your usage, but it cannot dispatch LLM traffic. And the key you gave Claude Code (gw_live_...) can dispatch traffic but cannot mint keys or touch billing. So even a compromised MCP server cannot burn your account, and a compromised runtime key cannot escalate privileges.

Vision guardrail. Vision-bearing requests never silently swap to a non-vision model. Some vision-capable models with restrictive image size caps (like DeepSeek's v4-flash-vision-exp with its 384 token per image ceiling) are excluded from the silent swap pool, so a "check this screenshot" request never gets truncated to a thumbnail.

What This Means for You

If you have been using Claude Code every day and paying for it every day, you are probably leaving somewhere between 40% and 90% of your bill on the table. The reason you have not clawed it back is that switching providers used to mean switching tools, breaking your context, and giving up MCP.

None of that is true anymore. Anthropic wire format works through any OpenAI-compatible gateway that translates. MCP servers forward cleanly. Failover keeps sessions alive. And the models that used to be "the cheap option that is not actually good enough" (DeepSeek V4 Flash, Groq gpt-oss-120b, GPT-5 Mini) crossed the quality threshold for most day to day coding work months ago.

The blended savings on our own team was 67%. Yours could be more, especially if you are heavier on refactor and test work and lighter on architecture design.

The install is 5 minutes. The rules go in your first message of the session. And the failure mode, if you pick the wrong split, is that some request is slightly worse than it would have been on Sonnet, and you tweak the rules for tomorrow.

Key Takeaways

Claude Code defaults to Sonnet. Sonnet is $20 per 6M tokens blended.
DeepSeek V4 Flash is $1.76 per 6M for the same tier of work.
Groq gpt-oss-120b is $1.35 per 6M and runs on LPUs, so it is fast.
Point Claude Code at a gateway (set two env vars). Install the Leanroute MCP. Give Claude routing rules at session start.
Failover, explicit routing, spend caps, and two key classes keep the worst case bounded.
Real internal test: 67.6% reduction on a real week of coding, no quality complaints.

About Leanroute

Leanroute is One Gateway for Models and Tools.

Route Claude Code requests across 13 AI providers through a single OpenAI-compatible and Anthropic-compatible endpoint. MCP forwarding is native. Failover, cost caps, and cheaper-model routing are on by default.

Get started at leanroute.dev

Goodbye OpenRouter

Leanroute — Tue, 18 Aug 2026 15:30:12 +0000

Goodbye OpenRouter: Migrate to Leanroute with flat BYOK, no per-request markup

TL;DR

Already using OpenRouter and considering a move? Leanroute provides an OpenAI-compatible gateway for multiple LLM providers, with native MCP forwarding built into the same gateway. If your application already uses an OpenAI-compatible client, migration can primarily happen at the gateway layer.

Why Migrate?

If OpenRouter is already working for your application, why change anything?

That's the right question.

The answer isn't that you need another way to call LLMs.

The question is whether your gateway should stop at models.

Modern AI applications increasingly need both:

Models
   +
Tools

If you're adding MCP to an application already using an LLM gateway, you may end up introducing another piece of infrastructure:

Application
    │
    ├──────► LLM Gateway ──────► Models
    │
    └──────► MCP Infrastructure ──────► Tools

With Leanroute:

Application
       │
       ▼
   Leanroute
    /     \
   ▼       ▼
Models    MCP

One gateway for both.

The Migration

The goal is simple:

Change the gateway, not your application.

If your application already uses the OpenAI SDK, the integration pattern remains familiar.

Before

const client = new OpenAI({
  baseURL: "https://openrouter.ai/api/v1",
  apiKey: process.env.OPENROUTER_API_KEY,
});

After

const client = new OpenAI({
  baseURL: "YOUR_LEANROUTE_ENDPOINT",
  apiKey: process.env.LEANROUTE_API_KEY,
});

Your application can continue using the same OpenAI-compatible interface.

No new provider SDK.

No application-wide rewrite.

No need to rebuild your model integration.

Same SDK. Same application. Different gateway.

Note: Use the exact Leanroute endpoint and authentication values from the current documentation when implementing the migration.

Keep Your Existing Model Integrations

Your application can continue working with the models and providers it already uses.

The gateway remains the abstraction layer between your application and model providers.

Your Application
       │
       ▼
   Leanroute
       │
   ┌───┼───────────┐
   ▼   ▼           ▼
 OpenAI  Anthropic  Google

Your application doesn't need to know how the underlying provider connection is implemented.

Then Add MCP

This is where the migration becomes more interesting.

Instead of adding a separate MCP proxy or changing your application's architecture:

Application
    │
    ├── LLM Gateway
    │
    └── MCP Proxy

Leanroute puts both behind the same gateway:

             Application
                  │
                  ▼
              Leanroute
             /         \
            ▼           ▼
       LLM Providers   MCP Servers

Add mcp_servers to the request, and Leanroute handles the MCP connection server-side.

Your application continues talking to one endpoint.

OpenRouter vs Leanroute

The goal isn't to claim that one gateway is universally better than another.

The important question is what your application needs.

	OpenRouter	Leanroute
OpenAI-compatible API	✅	✅
Multiple LLM providers	✅	✅
Model routing	✅	✅
BYOK	✅	✅
MCP forwarding		✅
Models + tools through one gateway		✅
Flat monthly pricing		✅
Zero per-request markup		✅

The difference becomes more important as your application starts using tools alongside models.

With Leanroute, MCP forwarding is part of the gateway rather than another piece of infrastructure your application needs to manage.

Why This Matters

The benefit isn't simply replacing one LLM gateway with another.

It's reducing the number of infrastructure components your application needs to understand.

Instead of:

LLM Gateway
+
MCP Proxy
+
Provider SDKs

You get:

             Leanroute
          Models + Tools

That's the direction we think AI infrastructure is heading.

The Migration in Three Steps

1. Change the endpoint

Point your existing OpenAI-compatible client at Leanroute.

2. Change the credentials

Use your Leanroute credentials and provider keys.

3. Add MCP when you need it

Add mcp_servers to requests that need tool access.

That's it.

Your application remains focused on the application.

Who Should Consider Migrating?

If you're already using OpenRouter and you're happy with it, you don't need to migrate just because another gateway exists.

But Leanroute may be worth considering if you:

Want model routing and MCP behind one gateway
Don't want to introduce a separate MCP proxy
Want to keep an OpenAI-compatible integration
Want BYOK with a predictable flat monthly cost
Want your application to remain independent of individual provider SDKs

The decision should be about where you want your AI infrastructure to live, not about replacing one API endpoint with another.

One Gateway for Models and Tools

AI applications are becoming more than model consumers.

They use models to reason and tools to act.

That means the infrastructure layer needs to support both.

That's why we built Leanroute:

One Gateway for Models and Tools.

And our broader philosophy is simple:

AI infrastructure should disappear behind a simple interface so developers can focus on building the actual product.

Key Takeaways

If you're already using OpenRouter, you don't need to relearn the gateway model.
Leanroute provides an OpenAI-compatible interface.
Migration can primarily happen at the gateway layer.
Native MCP forwarding lets you add tools without introducing a separate MCP gateway.
The goal is one infrastructure layer for both models and tools.

About Leanroute

Leanroute is One Gateway for Models and Tools.

Route requests across major AI providers and connect to MCP servers through a single OpenAI-compatible endpoint.

Flat $15/month. BYOK. Zero per-request markup. Singapore-hosted.

Learn more about Leanroute

Stop Building Your Own LLM Router

Leanroute — Fri, 14 Aug 2026 12:56:24 +0000

TL;DR

Model routing looks simple until you have to maintain it. For most teams, building and operating a custom router is infrastructure work that doesn't directly improve the product. Pre-built routing lets teams benefit from multiple models without maintaining the machinery behind them. AI infrastructure should disappear behind a simple interface so developers can focus on building the actual product.

It Starts With One Model

You start here:

Application
     │
     ▼
   GPT

Then you need a fallback.

Then a cheaper model.

Then Claude.

Then Gemini.

Then you start thinking about latency and model quality.

Eventually:

                 Router
              /    |    \
            GPT  Claude  Gemini

And now you've built a router.

What started as a simple application decision has quietly become another piece of infrastructure your team has to operate.

Routing Isn't Just Picking a Model

A production router eventually needs to consider:

Cost
Latency
Model capability
Context limits
Provider availability
Rate limits
Errors and retries
Model quality
Regional requirements

And these variables keep changing.

A model that is the best choice today might not be the best choice next month.

Providers change pricing.

New models appear.

Existing models improve.

Availability changes.

Traffic patterns change.

Your router has to keep up with all of it.

Your Team Has Better Things to Build

There's an important distinction between building an AI product and building infrastructure for AI products.

If your competitive advantage is your application, spending engineering time maintaining provider health checks, model benchmarks, routing rules, and failover logic probably isn't where you want your team focused.

You want to build:

Your Product
     │
     ▼
 AI Gateway
     │
     ▼
Best Model for the Job

The infrastructure handles the complexity.

Your application stays focused on the product.

When Should You Build Your Own?

There are legitimate reasons to build custom routing.

If you're operating at enormous scale, have highly specialized workloads, proprietary evaluation systems, or unusual latency and compliance requirements, custom routing can make sense.

But for most application teams, routing is infrastructure, not the product.

You probably don't need another internal system to maintain.

You need a reliable way to access the models your application needs.

Let Infrastructure Disappear

The best infrastructure is the infrastructure developers don't have to think about.

You shouldn't need to constantly ask:

Which provider is healthy?

Which model is cheapest?

Should I retry somewhere else?

Did a new model just become better?

Your application should express what it needs.

The infrastructure should handle the rest.

That's what a good AI Gateway should do.

The Bigger Picture

This is also why we think AI Gateways are evolving beyond model routing.

AI applications need models.

They also need tools.

They shouldn't need separate infrastructure for both.

             Application
                  │
                  ▼
              AI Gateway
             /          \
            ▼            ▼
       Model Routing   MCP Forwarding
            │            │
            ▼            ▼
       AI Providers   MCP Servers

The goal isn't to add another layer to your stack.

It's to remove the layers you shouldn't have to build yourself.

That's the idea behind Leanroute:

One Gateway for Models and Tools.

Because ultimately:

AI infrastructure should disappear behind a simple interface so developers can focus on building the actual product.

Key Takeaways

Building a model router is easy. Maintaining one is not.
Models, providers, pricing, latency, and availability constantly change.
Most application teams should consume routing infrastructure rather than build it.
Developers should focus on their product, not provider infrastructure.
AI Gateways can provide a unified layer for both models and tools.
The goal is simple: AI infrastructure should disappear.

About Leanroute

Leanroute is One Gateway for Models and Tools.

Route requests across major AI providers and connect to MCP servers through a single OpenAI-compatible endpoint.

Learn more about Leanroute

hi guys, I built leanroute.dev, it's One AI Gateway for LLM and Tools. Market is flooded with LLM Gateways, so I wanted to build an AI Gateway for LLM and MCP traffic. Its the next evolution of AI Gateways! https://www.producthunt.com/products/leanroute

Leanroute — Tue, 11 Aug 2026 15:39:43 +0000

producthunt.com