I Built a Fallback System for OpenRouter's Free API After It Silently Broke

The Script That Failed Without Warning

My nightly batch job had been running fine for months on OpenRouter's free API. Then it started throwing 429 errors, and nothing in my code had changed. The actual cause: the specific :free model I'd hardcoded had quietly stopped being offered for free. No warning, no deprecation notice — it just started failing.

Here's the fallback system I built afterward, and the mechanics of OpenRouter's free tier that would've saved me the debugging session if I'd understood them going in.

How the Free Tier Actually Works

Free models on OpenRouter carry a :free suffix and cost $0 per token, but the catalog rotates — providers add and pull free variants regularly, and a model that's free today isn't guaranteed to stay that way. Rate limits apply on top of that: 20 requests per minute on free models, and a daily cap that depends on your account history — 50 requests a day if you've never purchased credits, or 1,000 a day once you've bought at least $10 in credits at any point (that higher limit is permanent even if your balance drops back to zero later).

None of that is hidden — it's in OpenRouter's own API documentation — but it's easy to build against a single hardcoded model ID and forget the catalog isn't static.

Getting an OpenRouter API Key

pip install openai python-dotenv

Sign up at openrouter.ai, no card required, and generate a key from the dashboard. The OpenRouter API is OpenAI-compatible, so your existing client library works with just a different base_url:

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

client = OpenAI(
    api_key=os.getenv("OPENROUTER_API_KEY"),
    base_url="https://openrouter.ai/api/v1"
)

The Broken Version

def get_summary(text):
    response = client.chat.completions.create(
        model="some-provider/some-model:free",
        messages=[{"role": "user", "content": f"Summarize: {text}"}],
    )
    return response.choices[0].message.content

This works until the hardcoded model stops being free or gets removed from the catalog — at which point every call fails with no fallback, exactly what happened to me.

The Fallback Version

import time

FREE_MODEL_FALLBACKS = [
    "provider-a/model-x:free",
    "provider-b/model-y:free",
    "provider-c/model-z:free",
]

def get_summary(text, models=FREE_MODEL_FALLBACKS, max_retries_per_model=1):
    last_error = None
    for model in models:
        for attempt in range(max_retries_per_model):
            try:
                response = client.chat.completions.create(
                    model=model,
                    messages=[{"role": "user", "content": f"Summarize: {text}"}],
                )
                return response.choices[0].message.content
            except Exception as e:
                last_error = e
                if "429" in str(e):
                    time.sleep(2)  # brief pause on rate limit before moving on
                continue
    raise Exception(f"All fallback models failed. Last error: {last_error}")

If the first model in the list fails — rate limited, deprecated, or removed from the free catalog entirely — it moves to the next before giving up. This is a five-line change from the broken version, not a rewrite, and it turned a script that failed silently into one that degrades gracefully.

Tracking Rate Limits Across Multiple Scripts

The rate limit that actually caught me wasn't the daily cap — it was the 20-requests-per-minute limit, hit because I had two separate scripts calling the same key around the same time without either one tracking the other's usage. A simple shared counter fixes this if you're running more than one process against the same key:

import time
from collections import deque

request_timestamps = deque()

def rate_limited_call(model, messages, max_per_minute=18):  # slight buffer under 20
    now = time.time()
    while request_timestamps and now - request_timestamps[0] > 60:
        request_timestamps.popleft()
    if len(request_timestamps) >= max_per_minute:
        sleep_time = 60 - (now - request_timestamps[0])
        time.sleep(max(sleep_time, 0))
    request_timestamps.append(time.time())
    return client.chat.completions.create(model=model, messages=messages)

What I'd Check Before Building on This
Don't hardcode a single free model ID for anything you plan to run long-term — check OpenRouter's current free-model list before shipping, since the roster shifts month to month
Build the fallback list in from the start, not after your first silent failure — it's a small amount of code either way
Track rate limits across every process sharing a key, not just within a single script — this is what actually broke my setup, not the daily cap
Where This Left Me

Once I had the fallback logic working reliably, I also tested the same workload through RouteAI, mainly to compare pricing and model availability against what I'd gotten used to on OpenRouter's free tier — that's a decision worth making on your own project's volume and budget, not something either option deserves credit for by default. The fallback pattern above is useful regardless of which gateway or provider you're calling.

TL;DR: OpenRouter's free API models rotate and carry real rate limits (20/min, 50-1000/day depending on account history) — hardcoding a single :free model ID will eventually break silently. Fallback code and rate-limit tracking above turn that into a graceful degradation instead of a 2am debugging session.

Worth exploring if this is relevant to your stack: www.fastrouteai.com