My App Got Rate-Limited at 11PM. Here's How the OpenRouter AI SDK Saved My Weekend

It was 11:47 PM on a Friday when my side project's error logs started filling up with 429s. A model I'd been calling directly through its provider's SDK had just gotten aggressively rate-limited — apparently a lot of other people had the same idea that week. No warning email, no grace period. Just a wall of failed requests and a handful of users messaging me asking why the app stopped responding.

I didn't have a fallback. I'd built the whole thing against one provider's SDK, one auth flow, one request format. Switching to a different model meant rewriting the integration layer, not changing a config value.

That night is the reason I don't call any LLM API directly anymore.

What "no fallback" actually costs you

When you build against a single provider's official SDK, you're not just choosing a model — you're coupling your entire request/response handling to that provider's specific shape. Their SDK, their auth headers, their error format, their rate-limit behavior. It works great until something changes on their end: a price hike, a capacity crunch, a policy change, or just a temporary outage.

That Friday night, my options were: wait it out and lose users, or spend the next few hours rewriting how my app talked to an AI model — under pressure, at midnight, with people already annoyed.

I did the second one. It took about three hours. It should've taken ten minutes.

Rebuilding with an OpenAI-compatible gateway

After that weekend, I rebuilt the integration layer around the OpenRouter AI SDK instead of a single provider's SDK. To be precise about what changed, not oversell it: OpenRouter exposes one OpenAI compatible API surface in front of a long list of underlying models, so the request format stays identical no matter which model you're actually hitting.

The part that mattered most for my situation specifically:

If one model is rate-limited or degraded, switching to another is a model string change, not a rewrite
One auth setup, one API key, instead of a separate credential per provider
Error handling is consistent across models, so a fallback path doesn't need provider-specific logic

Here's roughly what my fallback logic looks like now:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://openrouter.ai/api/v1",
  apiKey: process.env.OPENROUTER_API_KEY,
});

const FALLBACK_MODELS = [
  "openai/gpt-4o-mini",
  "deepseek/deepseek-chat",
  "qwen/qwen-2.5-72b-instruct",
];

async function askWithFallback(prompt) {
  for (const model of FALLBACK_MODELS) {
    try {
      const response = await client.chat.completions.create({
        model,
        messages: [{ role: "user", content: prompt }],
      });
      return response.choices[0].message.content;
    } catch (err) {
      console.warn(`${model} failed, trying next:`, err.message);
    }
  }
  throw new Error("All fallback models failed");
}

Because every model in the list speaks the same request format, this loop just works — no per-provider branching, no separate SDKs to import. If gpt-4o-mini gets rate-limited at midnight again, the app quietly falls through to the next model instead of returning 429s to users.

What this setup doesn't guarantee

I want to be honest about the limits here. A fallback list doesn't mean every model performs identically — output quality and latency still vary per model, and pricing is per-model, not per-gateway. And you're still routing everything through one gateway, which is its own single point of dependency, just a different one than before.

That last point is why I don't rely on just one gateway anymore either. I've been running RouteAI alongside OpenRouter — also an OpenAI compatible API gateway, fronting models like DeepSeek, Qwen, GLM, and Kimi. Since both speak the same OpenAI-compatible request shape, I added it as one more entry in the same fallback list above, not a separate integration. I can't make strong claims about which gateway is more reliable long-term — I haven't run either at scale long enough — but having two independent paths into the same set of models means one gateway having a bad night doesn't take my app down with it.

The actual lesson

The rewrite I did at midnight wasn't really about finding "the best model." It was about removing a single point of failure I hadn't noticed I'd built. If your app depends on one AI API called through one vendor's SDK with no fallback path, that's a design decision worth revisiting before a rate limit forces the question at an inconvenient hour.

TL;DR: Building against a single provider's SDK with no fallback means one rate limit or outage can take your app down. An OpenAI-compatible SDK like OpenRouter's lets you keep a list of fallback models behind one consistent request format, so switching models is a config change, not a midnight rewrite.

Worth exploring if this is relevant to your stack: www.fastrouteai.com