Headline: The Vercel AI Gateway is a single endpoint that routes AI SDK calls to any model provider through a plain
"provider/model"string. I removed@ai-sdk/anthropic,@ai-sdk/openai, and three separate provider API keys, and now switch models by editing one string instead of swapping SDK packages.
Key takeaways
- The Vercel AI Gateway is a unified API, GA since August 2025, that sits between your app and multiple LLM providers so one credential and one code path reach any model.
- With the AI SDK you select a model by a
"provider/model"string —'anthropic/claude-sonnet-5','openai/gpt-5'— instead of importing a provider-specific package like@ai-sdk/anthropic. - Model fallbacks let you declare an ordered list of models; if the primary is rate-limited or erroring, the gateway retries the next on the same request without your code branching.
- The gateway adds observability (per-request cost, latency, tokens) and supports zero data retention, so switching providers no longer means re-instrumenting logging.
- The gateway replaces per-provider SDK wiring and key management — not the AI SDK itself; you still call
generateText,streamText, and the same tool-calling API.
What is the Vercel AI Gateway and what problem does it solve?
The Vercel AI Gateway is a hosted proxy that exposes a single OpenAI-compatible endpoint and forwards requests to whichever provider a model string names. Instead of holding an Anthropic key, an OpenAI key, and a Google key — each with its own SDK and rate-limit behavior — you hold one gateway credential and name the model inline. The problem it solves is provider lock-in at the code level: when a call is anthropic('claude-sonnet-5'), trying a different provider means a new import, a new client, and often a new logging shape. When the call is the string 'anthropic/claude-sonnet-5', trying 'openai/gpt-5' is a one-line diff.
How do I switch from a provider SDK to the gateway?
Drop the provider import and pass a "provider/model" string to the AI SDK's model field. The AI SDK resolves any bare string through the gateway by default.
// Before: provider SDK + provider-specific key
import { anthropic } from '@ai-sdk/anthropic';
import { generateText } from 'ai';
const { text } = await generateText({
model: anthropic('claude-sonnet-5'),
prompt: 'Summarize this support ticket in two sentences.',
});
// After: gateway string routing, one AI_GATEWAY_API_KEY
import { generateText } from 'ai';
const { text } = await generateText({
model: 'anthropic/claude-sonnet-5',
prompt: 'Summarize this support ticket in two sentences.',
});
Everything downstream — streamText, generateObject, tool calling — stays identical. The model becomes configuration, not code.
When should I use model fallbacks?
Use fallbacks when a request must succeed even if your preferred provider is rate-limited or down. The gateway tries the primary, and on a provider error retries the next model transparently within the same call.
import { generateText } from 'ai';
import { gateway } from '@ai-sdk/gateway';
const { text } = await generateText({
model: gateway('anthropic/claude-sonnet-5', {
fallbacks: ['openai/gpt-5', 'google/gemini-2.5-pro'],
}),
prompt: 'Draft a release note from this git diff.',
});
I do not use fallbacks on background jobs where I want deterministic behavior — a nightly batch should fail loudly and retry the same model, not silently answer with a different one whose output I never evaluated. Fallbacks trade consistency for availability; spend that trade only where availability matters more.
What does the gateway replace, and what stays?
| Concern | Before (per-provider SDK) | With the AI Gateway |
|---|---|---|
| Model selection |
anthropic('...'), openai('...') imports |
'provider/model' string |
| Credentials | One API key per provider | One gateway key |
| Switching provider | New import + client + key | Edit the model string |
| Fallback on outage | Hand-written try/catch | Ordered fallbacks list |
| Cost + latency | Per-provider dashboards | Unified per-request metrics |
| SDK call shape |
generateText/streamText
|
Unchanged |
What about cost, data retention, and observability?
The gateway reports cost, latency, and token counts per request in one place, so comparing two models on the same prompt no longer means stitching together two providers' billing pages. It also supports a zero data retention mode, meaning request and response bodies are not stored by the gateway — important when the prompt carries customer data and you need a clean data-flow answer for a security review. The practical win is measurement: when model choice is a string and metrics are unified, a real comparison — same prompt, three models, look at cost and latency — is a config change plus a dashboard read, not an engineering project.
What I'd watch before going all-in
One indirection layer means one more dependency in the request path — if the gateway is down, every model call is down, so I still keep the fallback list plus sane timeouts. And a bare string is only provider-agnostic until you rely on a provider-specific feature (a tool format, a cache-control header); then you're back to provider options, just passed through the gateway. Neither is a dealbreaker, but both are worth knowing before you delete the provider packages like I did.
FAQ
Q: Do I still need the AI SDK if I use the Vercel AI Gateway?
A: Yes. The gateway routes model requests; the AI SDK is still what you call in code (generateText, streamText, generateObject). It replaces provider-specific packages like @ai-sdk/anthropic, not the core ai package.
Q: How do I select a model through the gateway?
A: Pass a "provider/model" string to the model field, e.g. 'anthropic/claude-sonnet-5' or 'openai/gpt-5'. Bare model strings resolve through the gateway by default.
Q: What happens if my primary model is rate-limited?
A: With an ordered fallbacks list, the gateway retries the next model on the same request. Without fallbacks, the call returns the provider error for you to handle.
Q: Does the gateway store my prompts?
A: It supports a zero data retention mode where request and response bodies are not retained. Verify the setting for your account before sending sensitive data.
Q: Should I use provider-specific SDKs at all in 2026?
A: Default to gateway string routing for portability. Reach for a direct provider SDK only when you need a provider-specific capability the gateway doesn't surface.
Originally published on devya.dev. Also on eng-ahmed.com. Built by Devya Solutions.
Top comments (0)