Vercel AI Gateway: One Endpoint, Zero Token Markup, and Where It Stops

TL;DR

• It's a single HTTP endpoint in front of hundreds of models, referenced as creator/model-name strings, with automatic failover to the same model on a different provider and zero markup on tokens, even with your own provider keys.
• It's built for teams already on Vercel and the AI SDK - switching models is a string change, not a new client - but it's a routing and spend-visibility layer, not a governance one: no per-team RBAC, no budget enforcement, no MCP access control.
• One independent benchmark found the native Anthropic SDK about 15-20% faster than the gateway on small prompts, with the gap nearly disappearing at large context - worth testing on your own workload rather than assuming a fixed tax either way.

What it actually is

The gateway is one endpoint, https://ai-gateway.vercel.sh/v1, sitting in front of models from every major provider. You reference a model as creator/model-name - anthropic/claude-opus-4.8, openai/gpt-5.5 - and in the AI SDK, passing that string as the model automatically routes the call through the gateway. There's no separate client to install for the gateway itself; if you're already using the AI SDK, this is largely a config change.

Concretely, what you get: embeddings alongside chat completions, spend monitoring across every provider you've configured, automatic retries, load balancing across targets, and Bring Your Own Key if you'd rather use your own provider accounts than Vercel's. The one I'd call out specifically: automatic failover to the same model on a different provider when one degrades - not just falling back to a cheaper model, but keeping the exact model constant and switching which infrastructure serves it, so output doesn't change even though the request just got rerouted.

Zero data retention is available across OpenAI, Anthropic, and Google, though it's worth knowing this isn't free by default - team-wide Zero Data Retention costs $0.10 per 1,000 successful requests on Pro and Enterprise plans, and there are similar per-request add-on costs for a team-wide Provider Allowlist or Custom Reporting.

Pricing, briefly

Every Vercel team account gets $5 a month in included AI Gateway credits from first use. Beyond that: no markup on tokens - you pay the provider's listed rate, including when you bring your own key, which is a genuinely different model from gateways that take a percentage on top. You can buy more credits any time with no obligation to keep renewing them. See Vercel's own pricing page for current numbers rather than trusting a snapshot here.

The latency question, honestly

One benchmark I found compared the gateway directly against the native Anthropic SDK: at small prompts (around 10 tokens), the native SDK ran about 15-20% faster, but at large context (120K tokens), the gap between the two nearly disappeared. That's one person's benchmark, not a guarantee about your workload, but it's a useful shape to expect - the routing overhead matters proportionally more on tiny, latency-sensitive calls than on large ones where model inference time dominates anyway.

Where it stops being the right tool

This is a routing and spend-visibility layer, and it's worth being precise about what that doesn't include. There's no per-team or per-user RBAC over which models different groups can call. There's no budget enforcement - spend monitoring tells you what happened, it doesn't block a request before it blows past a limit the way a hard budget cap does. There's no MCP server governance, and no built-in guardrails for PII or prompt injection. None of that is a knock on the product - it's simply solving a different problem (one API, one bill, automatic failover) than the one a dedicated AI gateway with RBAC, budgets, and guardrails solves.

If you're a small team shipping on Vercel and want model routing without adding infrastructure, this is a genuinely good fit. If you need company-wide governance across many teams and many apps - not just visibility into one Vercel project's spend - that's a different category of tool.

This is the part where TrueFoundry's AI Gateway is what I'd reach for once "spend monitoring" isn't enough and you actually need budgets and rate limits enforced per team, not just visible after the fact, plus PII and prompt-injection guardrails applied at the gateway rather than in application code. The two aren't really competing for the same buyer: Vercel AI Gateway is the right call if your traffic is Vercel-shipped apps and the AI SDK is already your interface; TrueFoundry's gateway is built to sit in front of everything - Vercel-deployed apps included - once you have more than one team or more than one deployment target to govern consistently. Worth saying plainly since I'm not a neutral source on this comparison.

If you're running Vercel AI Gateway in production, has the automatic same-model failover actually saved you during a real provider outage, or has it mostly stayed dormant? Curious how often that specific feature actually fires versus just being there as insurance.