On March 24, 2026, two things happened in the LLM infrastructure world that had nothing to do with each other and everything to do with the same question: what do you actually trust with the traffic between your app and a model provider?
That morning, Portkey announced it had made its entire AI gateway fully open source under Apache 2.0 — governance, observability instrumentation, authentication, cost controls, and a new MCP Gateway for policing what AI agents are allowed to touch, all of it, no license key, no "upgrade to unlock" wall. The same day, LiteLLM — the open-source proxy that had spent three years building its reputation on exactly that kind of openness — disclosed that two of its PyPI packages, litellm==1.82.7 and litellm==1.82.8, had been compromised via a poisoned dependency in its own CI/CD security-scanning pipeline and were live on the public registry for about 40 minutes before being pulled.
Neither event caused the other. But together they reframe the decision every team building on multiple LLM providers is currently making, whether they've noticed it or not: do you route your traffic through a marketplace, a proxy you run yourself, or a vendor's control plane — and which of those three failure modes are you actually prepared to own?
This isn't a hypothetical. If you're calling more than one model provider — which by mid-2026 is most teams doing anything serious with LLMs, if only for fallback when a provider has a bad day — you're already making this choice, usually by whichever tool a blog post or a coworker mentioned first. It's worth making on purpose, and it's worth making now for a few concrete reasons: the pace of new model releases means hardcoding a single provider's SDK is a maintenance liability, not a simplification; agentic workloads have made tool-calling and multi-step model chains common enough that a single provider outage can now take down an entire pipeline rather than one feature; and enough teams have been burned by an unannounced provider price change or rate-limit tightening that "abstract the provider behind a gateway" has gone from a nice-to-have to a default architectural assumption in a lot of 2026 stacks. This piece compares the three tools actually competing for that traffic: OpenRouter, LiteLLM, and Portkey.
Three different bets on what a "gateway" should be
All three solve the same surface problem — you want one API that can call OpenAI, Anthropic, Google, Mistral, a dozen open-weight models on various inference providers, and switch between them without rewriting your integration every time a provider has an outage or a better/cheaper model ships. Where they diverge is in what they think you should own.
OpenRouter is a marketplace. You send it an API key, it fronts 500+ models from 80+ providers behind an OpenAI-compatible endpoint, and it bills you either through its own credit system or by proxying your own provider keys (BYOK). There's no infrastructure to run. You don't pick a region, you don't provision anything, you don't think about uptime for the gateway itself because there isn't one you manage — the router load-balances across the top providers for a given model automatically when you don't pin a specific one, which is also its default approach to keeping any single provider's downtime from becoming your downtime.
LiteLLM is a proxy you deploy. It's a Python project (from BerriAI, a Y Combinator W23 company) that gives you an OpenAI-compatible interface to 100+ LLMs, but you run the container. That means Postgres for spend tracking, Redis if you want caching, your own scaling story, and your own patch cadence — in exchange for the traffic never leaving infrastructure you control, and zero markup on top of whatever provider rates you're already paying.
Portkey is a managed control plane that, as of this year, you can also self-host for free. That's the part the March announcement changed. Historically Portkey's pitch was: don't run infrastructure, get an edge-deployed gateway with a 99.99% uptime SLA, and pay for the observability and guardrails layered on top. As of March 24, the gateway itself — routing, fallbacks, circuit breakers, the new MCP Gateway for agent tool governance, the full model catalog — is Apache 2.0 and self-hostable with no feature gate. What Portkey kept behind the paywall is the part that's genuinely expensive to run well: the log storage and dashboard, semantic caching at scale, multi-team RBAC, compliance certifications, and support with an SLA attached.
What "one API" actually looks like in practice
All three converge on the same surface-level promise: an OpenAI-compatible endpoint so you don't have to maintain three different SDKs for three different providers. The differences show up in how you point at a specific model and how you configure what happens when that model or its provider is unavailable.
With OpenRouter, model selection happens in the request body — you're calling one hosted endpoint and passing a model string that includes the provider prefix:
import requests
response = requests.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": "Bearer YOUR_OPENROUTER_KEY"},
json={
"model": "anthropic/claude-opus-5",
"messages": [{"role": "user", "content": "Summarize this incident report."}],
"route": "fallback",
},
)
There's no deployment step. The route and provider-preference fields are how you influence OpenRouter's load-balancing behavior across upstream providers for a given model, without you having to define the fallback chain yourself.
With LiteLLM, you're standing up a proxy and defining the routing behavior in a config file you own:
model_list:
- model_name: primary-model
litellm_params:
model: anthropic/claude-opus-5
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: primary-model
litellm_params:
model: openai/gpt-5.1
api_key: os.environ/OPENAI_API_KEY
router_settings:
routing_strategy: latency-based-routing
fallbacks: [{"primary-model": ["primary-model"]}]
That YAML block is also the whole point of self-hosting: the fallback logic, the budget caps, and the routing strategy live in a file your team controls and can audit line by line, rather than in a vendor's black box.
With Portkey, the equivalent configuration is a "config object" you either attach per-request or manage centrally through the gateway, and — since March — you can run that gateway yourself:
{
"strategy": { "mode": "fallback" },
"targets": [
{ "provider": "anthropic", "override_params": { "model": "claude-opus-5" } },
{ "provider": "openai", "override_params": { "model": "gpt-5.1" } }
]
}
The practical difference isn't the shape of the JSON — it's that Portkey's config objects are designed to be managed centrally and versioned through its dashboard (or your own if self-hosted), with circuit breakers that can trip a target out of rotation automatically after repeated failures, which is the kind of operational behavior LiteLLM's YAML gives you the primitives to build but doesn't ship pre-wired.
What changed, concretely
Before March 2026, the decision between LiteLLM and Portkey was easy to state even if it wasn't easy to make: LiteLLM if you wanted free and open, Portkey if you wanted managed and polished. That framing is now wrong, or at least incomplete. Portkey's core gateway is free and open too. The axis that actually separates them now is architecture and operational philosophy, not license.
LiteLLM is a Python proxy built around a callback/hook system — you wire in your own guardrail checks, your own logging destinations, your own budget logic via its plugin architecture. It's flexible in the way that "you write the glue code" is always flexible. Portkey's open-sourced gateway is TypeScript, built for edge deployment, and ships more of the agent-era governance story out of the box — the MCP Gateway specifically handles OAuth 2.1 for controlling what tools and data sources an AI agent is allowed to reach, which is a problem LiteLLM's proxy model wasn't originally designed around because it predates agents being the dominant workload.
So the honest post-March comparison isn't "open vs. closed." It's: do you want a Python-based proxy with a plugin model and years of production hardening around raw request routing, or a TypeScript-based edge gateway with agent-tool governance built in and a managed tier you can graduate into later without switching gateways? Both are now free to self-host. Neither is free to operate well.
That last sentence is where LiteLLM's March 24 incident actually matters — not because it makes LiteLLM unsafe (the compromised packages were live for roughly 40 minutes before PyPI quarantined them, and users running the official Docker image, which most production deployments do, were not affected), but because it's a live demonstration of the tradeoff self-hosting makes you own. When you run the proxy yourself, a supply-chain failure in your dependency chain — even one that originates in the project's own CI tooling, as this one reportedly did, via a compromised Trivy dependency — is your incident to detect, contain, and communicate about. LiteLLM did that within the same day. Not every team running a fork of an open-source proxy in a private VPC will notice as fast.
Why this decision has real weight
Cost. All three pass through provider token rates without a markup on the tokens themselves — that part of the pricing story is now table stakes across the category, not a differentiator. Where the money actually goes: OpenRouter takes roughly a 5.5% fee on credit purchases, or lets you bring your own provider key with the fee waived for the first 1M requests a month and a smaller fee after that — meaning at meaningful scale, BYOK is close to free beyond your own provider bills. LiteLLM's proxy is free to self-host, but the honest cost is the infrastructure and the engineering time to run it — the database, the observability you'll inevitably need to build yourself, the on-call rotation — plus an Enterprise tier (SSO, RBAC, audit logs, SLA support) that runs roughly $250/month at the entry point and towards $30,000/year for the tier with dedicated support. Portkey's pricing is unusual: it's not metered on raw requests but on "recorded logs" — a request that gets captured, stored, and surfaced in the observability dashboard — with a free developer tier, Production starting at $49/month, and custom Enterprise pricing above that. If you self-host Portkey's now-open gateway and skip its managed dashboard, you avoid that metering entirely, at the cost of building your own observability.
Latency. OpenRouter and LiteLLM both add routing overhead but no dedicated hop unless you deploy one; Portkey's managed edge deployment quotes 20–40ms of latency overhead from the gateway itself, which is the price of the SLA and the global edge network — worth it if that's genuinely faster than your own multi-region deployment would be, worth checking if it isn't.
DX. OpenRouter wins on time-to-first-request by a wide margin — one API key, no deployment, and a catalog broad enough that "is this new model available yet" is rarely a question you have to ask, because both OpenRouter and LiteLLM tend to ship support for major model launches the same day they drop, which is core to what both are selling. Portkey's new-model support tends to land within the week rather than same-day, which is the tradeoff for shipping a more curated, governed catalog rather than a maximalist one.
Lock-in. This is where the March changes matter most. Before, choosing LiteLLM was partly a hedge against vendor lock-in — you owned the code, full stop. Now that hedge is available with Portkey too, if you're willing to run it yourself and give up the managed dashboard. OpenRouter is the one option here where you're depending on a vendor's marketplace economics rather than a project you could fork; that's a fair trade for the convenience, but it's the one to be clear-eyed about if a two-person team is choosing infrastructure a fifty-person team will inherit.
Security. A hosted marketplace (OpenRouter) and a hosted control plane (Portkey's managed tier) both centralize risk into a vendor's infrastructure — a single compromise there has a wide blast radius, but you're also trusting an organization whose entire job is hardening that one system, and using OpenRouter's BYOK mode limits what OpenRouter itself ever holds. Self-hosting (LiteLLM, or Portkey's self-hosted gateway) keeps that blast radius inside your own perimeter, but transfers the job of noticing a compromise to your own team — which is exactly the scenario LiteLLM's March incident put under a spotlight, credibly and without any evidence it caused downstream harm to users on the standard deployment path.
Maintainability. Self-hosting either open gateway means someone on your team owns upgrade cadence, dependency hygiene, and scaling the proxy layer indefinitely. That's a real, ongoing cost that "free and open source" doesn't erase — it just moves the invoice from a vendor's bill into headcount and incident response time.
Where each one actually fits
OpenRouter is the right call for prototyping, side projects, and any team that wants to A/B test models without committing to a provider relationship — pull whichever of 500+ models fits a given task, pay per token, done. Concretely: a solo developer building a coding-assistant side project who wants to compare five different models on the same prompt without opening five provider accounts; a startup whose product routes different request types to different models (a cheap, fast model for classification, an expensive one for generation) and doesn't want to negotiate contracts with each provider to do it; a hackathon team that needs it working in the next twenty minutes. It's also a reasonable production choice for teams whose core competency isn't infrastructure and who are fine trading a small fee for never thinking about gateway uptime.
LiteLLM fits teams with platform engineers already comfortable running production services, who want the traffic to never leave infrastructure they control — often for compliance reasons, like a healthcare or fintech product where prompts and completions can't transit a third party's servers even briefly. Concretely: a regulated-industry team that needs an audit trail proving no vendor ever saw a customer's data; a platform team standardizing a dozen internal services on one internal LLM gateway with per-team budget caps enforced centrally; a company that's already invested in its own observability stack (Datadog, Grafana) and doesn't want to pay for a second one bundled into the gateway. It's also the more natural fit if your team is already Python-heavy and wants the callback/plugin model for custom routing logic, and it's a reasonable default when the team is prepared to be its own incident response for the proxy layer, including its supply chain — which is a real, ongoing job, not a one-time setup cost.
Portkey, in its managed form, fits teams that want governance and guardrails (PII checks, prompt-injection detection, per-team budget enforcement) without building an observability stack from scratch, and are willing to pay for the dashboard that makes that usable day to day. Concretely: a mid-size SaaS company with several product teams sharing one LLM budget who need a single pane of glass for who's spending what, without assigning an engineer to build that pane; a team shipping a customer-facing chatbot that needs prompt-injection detection today, not after a quarter of building it in-house. Its newly self-hosted form is now a legitimate LiteLLM alternative for teams that want the TypeScript/edge architecture and, increasingly relevant in 2026, built-in governance for what AI agents are allowed to call via MCP — worth a serious look for any team building agents that need to call internal tools and want that access policy enforced at the gateway rather than scattered across each agent's code.
What the marketing conveniently leaves out
OpenRouter's pitch undersells that you're still exposed to whatever reliability problems the underlying provider has — the router mitigates this by load-balancing across providers when you don't pin one, but it can't fully insulate you from a bad model day, and enterprise-grade SLAs, SSO, and priority support are a negotiated add-on, not the default plan.
LiteLLM's "free and open" framing undersells the fact that the free part is the software, not the operation of it — the Enterprise tier exists precisely because SSO, audit logs, and per-project budget isolation are the features teams need once they're past the prototype stage, and those aren't free. The March incident is also a useful corrective to any pitch that frames self-hosting as inherently safer than a managed vendor; it's differently risky, not risk-free.
Portkey's open-source announcement, read as a press release, sounds like Portkey gave away the business. Read as a pricing structure, what it gave away is the part that was never the moat — routing and governance logic are increasingly commoditized across this category. What it kept — the storage, indexing, and dashboard for observability at scale, plus compliance certifications — is the part that's actually expensive to replicate, which is exactly why it stayed paid.
The observability gap nobody's marketing page states plainly
There's a detail buried in Portkey's pricing model that's worth surfacing on its own: billing by "recorded logs" rather than raw requests means the meter is tied to how much you actually observe, not how much traffic you push through the gateway. That's a meaningfully different cost curve than OpenRouter's per-credit fee or LiteLLM's flat infrastructure cost — it means a team that turns down logging verbosity to save money is also turning down the visibility that's supposedly the point of paying for a managed layer in the first place. It's a reasonable pricing model, but it's worth reading the fine print on what counts as a "recorded log" before assuming the $49/month tier covers your actual traffic volume once you're capturing full request/response bodies for debugging.
LiteLLM doesn't have an equivalent built-in observability product to meter in the first place — the proxy exposes hooks and callbacks that let you ship logs to whatever destination you already run (a Datadog integration, a Postgres table, a custom webhook), and building the dashboard on top is your job. That's not a knock; it's consistent with the rest of LiteLLM's design, which optimizes for "give you the primitives" over "give you the finished product." The tradeoff is that the finished product being your job means the actual cost of observability at LiteLLM shows up as engineering time rather than a monthly bill, and engineering time doesn't show up on a pricing page.
OpenRouter sits in between: it ships a usable request-level dashboard for spend and model usage since that's core to a marketplace where users are comparing model costs, but it isn't positioning itself as a compliance or guardrails product — there's no PII-redaction or prompt-injection-detection layer built into the core offering the way there is with Portkey's managed guardrails, which check for issues like prompt injections and PII leaks across more than sixty distinct checks and attach detailed metadata to every logged request. If your team needs that kind of policy enforcement at the gateway layer specifically, rather than built into application code, that's currently more Portkey's territory than the other two — LiteLLM can get there through third-party guardrail integrations wired in as callbacks, but it isn't a catalog you get by flipping a setting.
Head-to-head
| Dimension | OpenRouter | LiteLLM | Portkey |
|---|---|---|---|
| Model | Hosted marketplace | Self-hosted proxy (open source) | Managed gateway, now also self-hostable (open source) |
| Core cost | ~5.5% fee on credit purchases; BYOK free up to 1M req/mo | Free to self-host; Enterprise ~$250/mo to ~$30k/yr | Free dev tier; Production $49/mo; billed on recorded logs, not requests |
| Token markup | None | None | None |
| Model catalog | 500+ models, 80+ providers | 100+ LLMs | Broad catalog; new models land within a week |
| New-model speed | Same-day for major launches | Same-day for major launches | Within the week |
| Infrastructure you run | None | Full proxy stack (DB, cache, scaling) | None (managed) or full gateway (self-hosted) |
| Added latency | Provider-dependent | Provider-dependent, plus your own hop | 20–40ms (managed edge deployment) |
| Guardrails/PII checks | Not a core feature | Build your own via hooks/plugins | 60+ built-in checks (managed); open-source in self-hosted gateway |
| Agent/tool governance | Not a focus | Not a built-in focus | MCP Gateway with OAuth 2.1 |
| Enterprise features | SSO/SLA via negotiation | SSO, RBAC, audit logs, budget isolation (paid tier) | RBAC, compliance certs, SLA support (managed tier) |
| Lock-in | Vendor marketplace | None — you own the deployment | None if self-hosted; managed dashboard if not |
| 2026 security event | — | PyPI supply-chain compromise, ~40 min exposure, contained same day | — |
The independent read
None of these three is wrong, and none of them is finished shipping — this is a category where the ground moved meaningfully in a single day this year, and it will move again. The most useful thing that happened in March wasn't that Portkey open-sourced a gateway; it's that doing so collapsed a distinction — "open-source proxy" vs. "managed vendor" — that a lot of architecture decisions were quietly leaning on. What's left after that collapse is a more honest set of questions: who operates this, who's accountable when it breaks, and what do you actually need built in versus built by you. LiteLLM's incident, handled competently and quickly, is a reasonable data point in that risk calculus, not a verdict against self-hosting — but it's the kind of data point that only shows up once you're running the thing yourself, which is worth remembering before treating "it's open source" as a synonym for "it's safe."
Which reader picks which
If you're shipping a prototype or a small product and want the largest catalog with zero operational overhead, pick OpenRouter and revisit the decision once your token spend or compliance requirements change. If you have platform engineers, a compliance mandate that traffic stay in your own infrastructure, and the appetite to own your dependency hygiene as seriously as LiteLLM had to on March 24, self-hosted LiteLLM is a legitimate, well-trodden choice. If you want governance and observability without building it, and the $49-and-up managed tier is a rounding error next to what your team would spend building the same dashboard, Portkey's managed offering is the fastest path to production polish. And if you wanted LiteLLM's ownership model but Portkey's agent-governance features, that combination — self-hosted Portkey gateway — didn't really exist as a mainstream option before this March, and now it does.
What's your team actually running today, and would you make the same call again knowing what changed this year — especially if you're the one who'd get paged for either a provider outage or a supply-chain alert at 2am?
Sources:
- LiteLLM vs Portkey vs OpenRouter vs Requesty: Best LLM Gateway and Router in 2026
- LiteLLM Pricing 2026: Open-Source & Enterprise Cost Breakdown
- Understanding Portkey AI Gateway Pricing For 2026
- Portkey's Gateway is Now Fully Open Source, Processing over 1 Trillion Tokens Every Day
- Security Update: Suspected Supply Chain Incident | liteLLM
- BYOK | Use Your Own Provider Keys with OpenRouter
- OpenRouter Free Tier 2026: Rate Limits, Models, BYOK
- Provider Routing - Smart Multi-Provider Request Management
Top comments (0)