LiteLLM Alternatives in 2026: Honest Comparison of 5 LLM Gateways
TL;DR
LiteLLM was the default OSS LLM gateway from 2024 through mid-2026. The recent CVE-2026-35029 privilege-escalation issue plus the working-month ops overhead of running a self-hosted gateway have pushed a lot of teams to reevaluate. This post compares five real alternatives — Leanroute, Portkey, Kong AI Gateway, Helicone, and OpenRouter — on the dimensions that actually matter: hosting model, wire format, BYOK, MCP support, guardrails, and honest verdicts about who each one is for. It is written by the Leanroute team but includes the cases where you should NOT switch to us.
Why People Are Searching for LiteLLM Alternatives
Three overlapping reasons show up in the search data and in our sales inbox:
The recent CVE. CVE-2026-35029 (privilege escalation from a read-only viewer role to config-modifier) landed a few days ago. If you already run LiteLLM the patch is fast, but the CVE prompted teams to ask a bigger question: "do we have someone on-call to catch the next one?"
The ops burden. LiteLLM is free to install and expensive to run. Postgres, Redis, monitoring, patching, secret rotation, on-call. Our working estimate for a mature self-hosted deployment carrying real production traffic is 15-25 engineer-hours per week — a real number that most teams do not budget for when they choose "free."
The category moved. LiteLLM was designed in 2023-2024 when the problem was "translate between provider APIs." In 2026 the problem is that plus MCP passthrough, agent-to-agent auth, spend caps that actually work at the gateway edge, and a security model that treats the gateway as a first-class trust boundary rather than an internal utility.
None of that means LiteLLM is bad. The maintainers have been diligent about patching, the OSS model is legit, and the community is real. It means the space has widened and the right answer depends on what you actually need.
Quick Decision Tree
Before the detailed comparison, the shortest useful version:
- You want to keep self-hosting and just need MORE than LiteLLM (auth, guardrails, MCP) → Kong AI Gateway or Portkey OSS.
- You want to STOP self-hosting → managed Leanroute, managed Portkey, or Helicone.
- You want observability only (logs, cost tracking) and already have a routing layer → Helicone.
- You want to give hobbyist devs multi-provider access without dealing with keys → OpenRouter.
- You want a single OpenAI-compatible endpoint that speaks BYOK to 14 providers, adds MCP passthrough, and is priced flat per month regardless of traffic → Leanroute.
Now the details.
The Five Alternatives
1. Portkey
Portkey ships both an OSS gateway you can self-host and a managed cloud. The OSS gateway is credible and actively maintained; the managed cloud adds observability, prompt management, and a UI. Wire format is OpenAI-compatible. Guardrails and semantic caching are in the managed tier. Their focus is enterprise sales, which shows up in the product depth and in the pricing conversations.
Best for: teams that want either self-host-and-stay-in-control OR a managed alternative with a mature enterprise-shaped feature set. Migration from LiteLLM is straightforward because both speak OpenAI.
Not for: teams that want flat monthly pricing and zero per-request markup. Portkey's managed tier is priced on request volume.
2. Kong AI Gateway
Kong is the incumbent web API gateway with an AI gateway layer bolted on. If your team already runs Kong for other services, adding the AI gateway is genuinely marginal cost — same admin plane, same routes, same operational model. Fully self-hosted.
Best for: enterprises with an existing Kong footprint and an SRE team that already knows the operational model. This is the "safe procurement" pick — we saw it beat LiteLLM at a real enterprise account recently, and it won on being an established vendor, not on being a better AI gateway.
Not for: startups without a platform team. The setup surface is real, and the AI-specific features (spend caps, model routing, guardrails) are less mature than the web-API-gateway core.
3. Helicone
Helicone is primarily observability — logs, cost tracking, request replay — with proxy-mode routing as a secondary function. Simple integration: change your base URL and you get a dashboard. Open-source with a hosted tier.
Best for: teams that want visibility into what their app is spending on LLM calls without owning the routing layer. You keep talking to OpenAI or Anthropic directly (via the proxy), Helicone logs everything, and you get charts.
Not for: teams that need routing decisions, failover, BYOK to multiple providers behind a single endpoint, or MCP passthrough.
4. OpenRouter
OpenRouter is a managed multi-provider endpoint. One API key gets you access to hundreds of models across all major providers. They handle the provider keys, they handle billing, you send requests. Pricing is per-token with a small markup on top of provider pricing.
Best for: individual developers exploring models, hobbyists, small apps where "one credit card, many models" is the whole win. Great for prototyping.
Not for: production workloads where the markup adds up, teams that already have provider commitments and want BYOK, or teams that want spend caps enforced at gateway edge (OpenRouter alerts you after the spend, not before). If you fall into that bucket, see our OpenRouter alternative post for the fuller argument.
5. Leanroute (us)
We built Leanroute for the middle: teams that have outgrown OpenRouter's markups and hobbyist framing, but do not want the ops burden of LiteLLM or Kong. Flat $15 / $25 per month, BYOK to 14 providers (OpenAI, Anthropic, Google, xAI, DeepSeek, Groq, AWS Bedrock, Z.AI, Kimi, Doubao, Sarvam, Krutrim, plus two more), MCP passthrough on the same endpoint, a 7-rule guardrail starter library at the gateway edge, hard spend caps enforced before your provider invoice grows.
Best for: startup and mid-market teams running production LLM traffic who want managed but with their own provider relationships. Flat pricing means the gateway cost is predictable as you scale. BYOK means you keep provider rate limits, discounts, and commitments.
Not for:
- Teams with data-residency requirements no managed vendor can meet — you should self-host (LiteLLM, Portkey OSS, or Kong).
- Enterprises whose procurement requires a Gartner MQ-listed vendor. We are not on it, we are honest about it, come back to us when you are ready.
- Teams that need observability only, without routing — Helicone is a cleaner fit for that shape.
Comparison Table
| Dimension | LiteLLM (self-host) | Leanroute | Portkey | Kong AI Gateway | Helicone | OpenRouter |
|---|---|---|---|---|---|---|
| Hosting model | Self-host | Managed (SG) | Both | Self-host | Both | Managed |
| Wire format | OpenAI-compat | OpenAI-compat | OpenAI-compat | OpenAI-compat + Kong routes | OpenAI / Anthropic proxy | OpenAI-compat |
| BYOK to your provider account | Yes | Yes (14 providers) | Yes | Yes | Yes | No (they hold keys) |
| MCP passthrough | Via plugins | Native | Roadmap | Roadmap | No | No |
| Guardrails at edge | Via plugins | 7 built-in | Built-in (managed tier) | Via plugins | No | No |
| Hard spend caps | Yes (config-heavy) | Yes (per-org, per-key) | Yes | Yes | Alerts only | Alerts only |
| Pricing model | Free + ops cost | Flat monthly | Per-request tiers | Enterprise contract | Freemium / tiered | Per-token markup |
| Ops burden | High | None | Medium / None | High | None | None |
Migration Path from LiteLLM
If you decide to move off LiteLLM, the migration to any OpenAI-compatible alternative is mostly a base URL change:
- Sign up for the alternative and set up your provider keys (either transferred as BYOK or added fresh).
- Change
OPENAI_BASE_URLin your app from your LiteLLM instance to the new endpoint. - Verify the first canary request. Most gateways expose a response header (Leanroute uses
x-gateway-provider) that confirms the routing hop. - Cut over the rest of your traffic when the canary is clean.
For a typical Leanroute migration that takes an afternoon; for teams with heavy per-org config, custom guardrails, or unusual auth budget one or two days. Full docs at leanroute.dev/docs. If you want a hand, sales@leanroute.dev will reply personally.
When to Stay on LiteLLM
Two cases where the honest answer is "keep what you have":
You are one of the teams doing LiteLLM well. If you have a platform person who already owns the deployment, patches within hours of CVE announcements, has secret rotation set up, and monitors upstream provider health — you are running it correctly. Switching gains you nothing.
You need customization at the gateway layer that only a codebase you control gives you. Some teams add custom auth plugins, custom routing rules, or custom logging pipelines that would be awkward to reproduce on a managed gateway. If you have written more than a few hundred lines of LiteLLM extensions, you are probably in this camp.
For everyone else, the question worth asking is: does your team actually have the hours to keep owning the operational surface? If the answer is no, that ownership gap ships CVEs faster than the fixes.
The Meta Point
An LLM gateway is a trust boundary. It holds your provider keys, sees every prompt, and often has admin access to spend caps and routing rules. The right gateway is the one your team can actually own — which is a mix of feature fit and operational fit. Do not pick on features alone; ops overhead is real and compounds.
The teams that get hurt by CVEs like the LiteLLM one are the teams that installed the gateway eighteen months ago, moved on, and never revisited whether they had the capacity to keep it patched. That is true of any middleware, not just LLM middleware. Revisit the choice once a year. The stakes are quietly getting higher on both sides.
If you want to talk through your specific setup, email us. We will tell you honestly if we are a fit and point you elsewhere if we are not.
Sources:
Top comments (0)