A gateway sits between your app and the model providers: one API, key management, caching, failover, cost control — and for routers, automatic model selection per request. Here is a neutral index by type (aggregator / proxy / router / API gateway), hosting, and license. No prices.
The matrix
| Tool | Type | Hosting | License | Best for |
|---|---|---|---|---|
| OpenRouter | Aggregator / unified API | Managed | Proprietary | One API and billing across hundreds of models, pay-per-use |
| LiteLLM | Proxy gateway | Both (self-host or cloud) | Open-source | Self-hosted, OpenAI-compatible gateway + spend tracking across 100+ providers |
| Portkey | Proxy gateway + router | Both | Open-source core | Production gateway with routing, caching, guardrails, and observability |
| Cloudflare AI Gateway | Proxy gateway (edge) | Managed | Proprietary | Edge caching, rate-limiting, and analytics across providers with zero infra |
| Helicone | Proxy gateway + observability | Both | Open-source | Drop-in proxy logging, caching, and routing with minimal code |
| Kong AI Gateway | API gateway | Both | Open-source | Teams already on Kong or needing enterprise API governance |
| Vercel AI Gateway | Proxy gateway | Managed | Proprietary | Vercel / Next.js apps — one endpoint with failover and observability |
| Martian | Smart router | Managed | Proprietary | Dynamic routing to the best or cheapest model per request |
| Not Diamond | Smart router | Managed | Proprietary | Routing each prompt to the best-performing model automatically |
| Unify | Router + aggregator | Managed | Proprietary | Benchmarking and routing across providers on quality/cost/latency |
| Eden AI | Aggregator / unified API | Managed | Proprietary | One API across many AI services, not just LLMs |
| Apache APISIX AI Gateway | API gateway | Both | Open-source | Open-source API gateway with AI proxy and governance plugins |
Quick picks
- You want one API and billing across many models → OpenRouter
- You want a self-hosted, OpenAI-compatible gateway → LiteLLM
- You want routing + caching + guardrails in production → Portkey
- You want edge caching/analytics with zero infra → Cloudflare AI Gateway
- You want the cheapest/best model picked per request → Martian or Not Diamond
- You already run Kong or need enterprise API governance → Kong AI Gateway
- You're building on Vercel → Vercel AI Gateway
- You want drop-in proxy logging that doubles as a gateway → Helicone
📚 More from The 2026 AI Stack Index: Automation Tools · Agent Frameworks · Vector Databases · LLM Observability · LLM Gateways
This is a neutral, no-affiliate reference — no prices (they go stale), no rankings-for-pay. The full, always-updated interactive version with FAQs and the rest of the AI-stack indexes lives at aiprosol.com/llm-gateways. Disclosure: I run Aiprosol, an automation consultancy — the index doesn't favour anyone.
Top comments (0)