Half of Free LLM APIs Were Dead After 2 Months

📝 Originally published (in Japanese) at forge.workstyle.tech.

In July, I wrote an article titled "Summary of Free LLM API Providers That Don't Require a Credit Card." I listed eight providers and explained that by bundling multiple providers and implementing fallbacks, you could achieve real-world operation using only the free tiers.

Two months later, I inspected the pipeline running on this setup and found that the success rate had dropped to 60%.

Upon investigating the cause, I discovered that several of the eight providers I had listed were no longer functioning for various reasons, and none of the fallbacks were working. This article documents the full investigation. All figures are based on actual API calls made on 2026-09-07.

The Issue: A 6-Provider Setup Running on Essentially One Provider

I sent the same request to the router (LiteLLM) ten times, and here's what happened:

Success: 6 / Failure: 4

Here are the details of the failures:

litellm.APIError: CerebrasException - Payment required to access this resource.
  Received Model Group=free
  Available Model Group Fallbacks=None

litellm.NotFoundError: GeminiException - models/gemini-2.0-flash is no longer available

The line Available Model Group Fallbacks=None says it all. Despite bundling six providers, no fallbacks were occurring.

I then individually tested each provider's API to check their status.

Provider	Result
Groq	✗ 401 Invalid API Key
Cerebras	✗ 402 Payment required
OpenRouter	✗ 401 API key expired
Gemini	✗ 404 model no longer available
Mistral	✗ 429 Rate limit exceeded
Cohere	○ 200

Only one provider was still functioning. The reason for the 6 out of 10 successes was that LiteLLM's num_retries: 5 setting kept retrying until it landed on Cohere. While it appeared to have redundancy, in reality, all the load was on a single provider.

The Decay Wasn't Uniform

This is where things get interesting. I thought it would be a simple case of "free tiers ending," but each provider had a different reason for failing. Here’s the classification after testing 13 candidates:

1. Free Tier Itself Became Paid — Cerebras

402 Payment required to access this resource. Visit your billing tab.

The key is valid, authentication passes, but payment is required. In July, there was a "1M tokens/day" free tier. This is the only case where the information in my article was clearly incorrect.

2. Service Terminated — GitHub Models

GET https://models.github.ai/catalog/models
→ HTTP 410 Gone

A 410 status code means "permanently gone." While I had considered it a candidate, it’s no longer an option.

3. Models Completely Replaced — Groq

After reissuing the key, the error changed from 401 to 404.

404 The model `llama-3.3-70b-versatile` does not exist or you do not have access to it.

The free tier is still available, but the models are gone. The current catalog includes 9 models, none of which are Llama-based.

allam-2-7b / canopylabs/orpheus-* / groq/compound / groq/compound-mini
openai/gpt-oss-120b / openai/gpt-oss-20b / qwen/qwen3.6-27b / qwen/qwen3.8-27b

4. Only Model Names Expired — Gemini

404 NOT_FOUND: This model models/gemini-2.0-flash is no longer available.
    Please update your code to use models/gemini-3.6-flash

The free tier is still available here too. It was simply a matter of not keeping up with the model name changes.

As a fix, I switched to using the gemini-flash-latest alias. This works in practice and will automatically adapt to future name changes. It’s a measure to avoid repeating the same mistake.

5. Keys Had Expiration Dates — OpenRouter

401 API key expired.

Checking the dashboard, it shows Expires: Expired / Last Used: Never / Key limit: unlimited. The key expired due to its expiration date, not usage. When creating keys, you can choose "No expiration," which should always be selected for persistent use cases.

6. "No Card Required" but Balance Needed — Z.ai

429 Insufficient balance or no resource package. Please recharge.

While no card is required to issue a key, you can't make a single request without recharging. It’s often introduced as a "no-card-required free tier," but the reality is different.

7. Catalog Doesn't Reflect Reality — NVIDIA NIM

The model list API returns 81 entries. I tested all of them.

Success: 13
HTTP 404: 55   ← Models listed in the catalog but don't exist
Timeout/Exceptions: 7
503 / 500 / 400: 6

Of the 13 functioning models, 3 were for content safety classification, 2 were dedicated to translation, and 1 required image input. Only 5 were suitable for general chat use.

If you take the explanation "81 models available for free" at face value, you’ll only realize the issue after implementation.

8. Almost Defunct — SambaNova

Here are the results after testing all 7 models:

DeepSeek-V3.1 / V3.2 / Meta-Llama-3.3-70B / gpt-oss-120b → 402 A payment method is required
MiniMax-M2.7 / M3                                        → 429 high demand
gemma-4-31B-it                                           → ○ 200 (but took 12,021ms)

Only one model remains free, and it takes 12 seconds. Compared to Groq's 220ms, it’s 55 times slower. While you could say "free tier available," its practicality is questionable.

9. Card Required — Nebius / Scaleway

Nebius's registration page requires billing details, including name, address, and card information (with $0 authorization for card verification). Scaleway's official documentation states that "a card and KYC are required to receive normal rate limits."

Neither meets the "no credit card required" condition.

10. Token Permission Pitfalls — HuggingFace

403 This authentication method does not have sufficient permissions

Checking the token, it only had repo.content.read permissions. For inference, inference.serverless.write (displayed as "Make calls to Inference Providers" in the UI) is required.

After re-enabling the permission, it worked. Issuing a key and the key being usable are two different things, but I didn’t realize this until seeing the error message.

The Real Reason Fallbacks Didn't Work

This was the biggest lesson learned.

Here’s how LiteLLM was configured:

router_settings:
  routing_strategy: simple-shuffle
  num_retries: 5
  allowed_fails: 1
  cooldown_time: 60

At first glance, it seems robust with 5 retries. However, in practice, it failed immediately upon hitting Cerebras.

402 (payment required) and 404 (model not found) are not subject to retries. Retries and fallbacks are designed for issues like 429 (rate limits) or timeouts, which can be resolved by waiting. Since 402 and 404 are permanent failures, they are returned immediately.

In other words,

Leaving dead providers in the pool will continue to generate failures at the expected rate.

If one out of six providers is permanently dead, simple math tells us that 1/6 of requests will fail. The assumption that "having many providers ensures safety" only holds if all providers are functioning.

In fact, just removing Cerebras from the pool improved the success rate as follows:

Just fixing Groq and Gemini model names: 6/12 (50%) ← All failures were Cerebras' 402
Excluding Cerebras: 15/15 (100%)

Removing the dead provider was more effective than fixing model names.

Two Types of Inference Model Traps

In my July article, I mentioned that inference models should be avoided for agent use cases. This time, I discovered an even more troublesome issue.

(a) Multi-Turn Conversations Break (Known)

Inference model responses include reasoning_content. Agents accumulate conversation history and send it to the next turn, inadvertently sending this unfamiliar field.

400 property 'messages.*.assistant.reasoning_content' is unsupported

This is a problem specific to agents (multi-turn conversations). Single-turn generation doesn’t send history, so this issue doesn’t occur. This was already mentioned in July.

(b) Even Single-Turn Generation Returns Empty Content (New Discovery)

The real problem is this. Here are the results of asking several inference models, "What is 2+2? Answer with just the number."

Model	content
qwen/qwen3.8-27b	"4" ← Non-inference
qwen/qwen3.6-27b	"" ← Empty
openai/gpt-oss-20b	"" ← Empty
liquid/lfm-2.5-2.6b:free	"" ← Empty
nvidia/nemotron-3-super-120b-a12b	"User asks 2+2..." ← Exposes reasoning

The answer ends up in reasoning_content, leaving content empty. Code that only reads choices[0].message.content receives an empty string without error.

This happens even in single-turn generation. So, it’s not enough to "just be careful with agents when using inference models." Without defenses like empty response retries, it will silently break.

Note that even within the same Qwen series, 3.6 outputs reasoning while 3.8 does not. You can’t judge based on model name alone, so it’s best to test each model once before adding it to the pool to check for the presence of reasoning_content.

Comparison with July Version

Provider	July Description	September Actual
Google Gemini	Free tier available	○ Available ( Model name update required )
Groq	Free tier available	○ Available ( All Llama models gone → Replaced with Qwen, etc. )
Cerebras	1M tokens/day	✗ 402 Payment required
OpenRouter	:free 20RPM	○ Available ( Key expiration, all models are inference type )
NVIDIA NIM	Numerous models	△ Only 13 out of 81 responded
Cohere	Trial tier	○ Available (The only one functioning consistently this time)
SambaNova	Permanent free tier	✗ 6 out of 7 models returned 402
Mistral	Experiment tier	△ Frequent 429 errors
—	(Not mentioned)	○ Cloudflare Workers AI emerged as a strong contender
—	(Not mentioned)	○ HuggingFace works if permission settings are correct

Only 3 out of the original 8 providers remained valid.

New Addition: Cloudflare Workers AI

The most straightforward provider to get working was Cloudflare Workers AI.

• 10,000 Neurons/day, no card required
• API token requires only Account > Workers AI > Read permission for inference ( Edit not needed)
• @cf/meta/llama-3.3-70b-instruct-fp8-fast responds in 570ms, non-inference

One caution: the permission names are confusing. AI Gateway permissions do not work.

A token with only AI Gateway permission returns 401 with error code 10000.

The gap left by Groq removing Llama models was filled by this provider.

Post-Recovery Setup

After removing dead providers, updating model names, and adding three new providers, here’s the result:

Provider	Model	Measured Speed
Groq	qwen/qwen3.8-27b	220ms
Cloudflare	@cf/meta/llama-3.3-70b-instruct-fp8-fast	570ms
Cohere	command-a-03-2025	748ms
HuggingFace	meta-llama/Llama-3.3-70B-Instruct	937ms
OpenRouter	google/gemma-4-26b-a4b-it:free	957ms
NVIDIA NIM	google/diffusiongemma-26b-a4b-it	1,070ms
Gemini	gemini-flash-latest	—
Mistral	mistral-small-latest	Frequent 429 errors

Success rate trend: 6/10 (60%) → 15/15 → 20/20 → 24/24 → 28/28

Conclusion: Summary Articles Go Stale in Two Months

Since half of my own article became unusable in two months, this is also a reminder to myself. Here are three practical conclusions for readers:

1. Implement Health Checks First.
It’s more effective to detect and remove dead providers than to add more providers. In this case, removing one dead provider contributed more to success rate improvement than fixing model names. A simple script sending one request to each provider is sufficient.

2. Fallbacks Are Not a Panacea.
They work for 429 errors and timeouts. They don’t work for 402 (payment required), 404 (model gone), or 401 (key expired). These are permanent failures, so they’ll keep generating errors until manually removed.

3. Don’t Fix Model Names; Verify Before Adding.
Use latest aliases if available. If not, test each model once before adding it to the pool to ensure a response is returned and reasoning_content is not included. Behavior can vary even within the same series.

---

Note: All figures in this article are based on actual measurements as of 2026-09-07. Free tier conditions, available models, and rate limits can change within months, as seen here. Always check each provider’s official documentation for the latest information before implementation. This article, too, will likely be half incorrect in two months.