If you call a retired model ID directly, you get a loud 404 and you fix it. The failure mode worth knowing about is quieter.
The whole point of LiteLLM's fallbacks is to absorb failures. So when the first entry in a chain is a model ID that no longer resolves, every request fails over to your second choice — and nothing crashes. Your app keeps answering. You just quietly lose the provider you chose for speed, eat an extra failed round-trip of latency on every single call, and shift 100% of traffic onto the fallback and its rate limits, with nothing in your logs unless you went looking.
This is easy to hit right now because provider catalogs are rotating fast. Groq is the clearest example: llama-3.3-70b-versatile and llama-3.1-8b-instant are still listed in Groq's model docs, but both now carry an Enterprise badge with "Contact Sales" where the price and rate limits used to be, and neither appears on the Free Plan rate-limit table at all. A config written against last year's tutorial points at a model your key can't reach.
Two minutes of maintenance:
curl -s https://api.groq.com/openai/v1/models -H "Authorization: Bearer $GROQ_API_KEY"
Diff that against the model IDs in your config. Then, if you run fallback chains, alert on fallback-hit rate — it is the only place this class of rot ever surfaces.
How the LiteLLM free-tier setup fits together: https://toolfreebie.com/litellm-free-llm-gateway/
Top comments (0)