I pinged 55 LLMs every 8 hours for a week. What stayed up, what broke, what vanished (Sep 15–21)

OnomeoAI — Mon, 21 Sep 2026 15:52:05 +0000

I run onomeo, a small site that gives out free daily credits for 55 models behind one OpenAI-compatible key. Every 8 hours the server sends each model the same prompt, "Reply with the single word: ok", and logs whether it answered and how long it took. This is week one of those logs: Sep 15 11:09 UTC to Sep 21 14:33 UTC, 692 tests.

The big models: 16 of 16 answered every test

GPT-6 Astra, Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Gemini 3.1 Pro, GPT-5.6 Sol, Terra and Luna, Grok 4.6, Kimi K3, GLM-5.3, o3, DeepSeek V4 Pro, Llama 4 Maverick, Mistral Large 3 and Nemotron 3 Ultra joined on Sep 20 and passed all 3 tests since then. Three tests is a small sample; next week's report covers a full week for them. On onomeo they run on the same free daily credits, counted at 1x to 20x depending on the model.

14 free models passed all 19 tests

Model	Passed	Avg response
@cf/qwen/qwen2.5-coder-32b-instruct	19/19	0.35 s
@cf/meta/llama-4-scout-17b-16e-instruct	19/19	0.36 s
ministral-3b-latest	19/19	0.37 s
ministral-8b-latest	19/19	0.48 s
@cf/openai/gpt-oss-120b	19/19	1.11 s
Qwen/Qwen3.8-Flash-Next	19/19	1.61 s
mistral-Nemo-Instruct-2407	19/19	1.93 s
codestral-latest	19/19	1.97 s
gemini-3-flash-preview	19/19	2.15 s
gemini-3.1-flash-lite	19/19	2.24 s
deepseek-v4-flash	19/19	2.35 s
stepfun/step-3.7-flash:free	19/19	2.68 s
Shanghai_AI_Laboratory/Intern-S2-Preview	19/19	3.17 s
deepseek-v4.1-flash	19/19	39.85 s

Cloudflare's hosted models and Mistral's small ones were the fastest. The slowest in that table, deepseek-v4.1-flash, averaged 39.9 s, because it thinks before answering even a one-word prompt.

The flaky ones

Model	Passed
mistralai/mistral-nemotron	9/19
poolside/laguna-s-2.1:free	11/19
z-ai/glm-5.3-flash-free	12/19
tencent/hy3-free	12/19
nvidia/nemotron-3.5-lightning-30b-a3b	15/19

New and gone

New free models this week: google/gemma-4-26b-a4b-it:free (0/2 so far), inclusionai/ling-3.0-flash-vl:free (1/2 so far), nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free (0/2 so far).

Gone: ModelScope stopped serving three models. Qwen3-235B-A22B-Thinking-2507 and Qwen3-Coder-30B-A3B-Instruct return "has no provider supported", and GLM-4.7-Flash returns an empty answer.

Caveats

A one-word prompt shows whether a model is up and how fast it starts. It says nothing about how smart it is.
Models served through OpenRouter are tested once a day, not every 8 hours.
A few SiliconFlow failures on Sep 15 were our fault: our account balance had dropped below zero. Fixed on Sep 16.

The live board updates every 8 hours: https://onomeo.com/free-models?w=0922