What we tell users when 9 of our 13 models are down

Update (2026-10-28, 21:44 UTC+8). The upstream account behind these models recovered within hours: GET /api/health/models now reports 11 of 13 healthy. The numbers below (four healthy, one account at minus three cents) are the state at the time of writing and are kept as the reason this contract exists — not as the current status. For the live number, always call the endpoint.
A gateway fails in two ways. The loud way is a 500. The quiet way is a model that
has been returning 503 for six hours while every dashboard says "active".

We shipped the quiet way twice this month. This is what we changed, and the
contract you can now check before you build against us.

The mistake

We had a health checker that did this, more or less:

for model in catalogue:
    if upstream_returns_2xx():   mark_healthy(model)
    else:                        mark_unhealthy(model)
        if model.unhealthy_for_2_checks(): deactivate(model)

Two assumptions in four lines, both wrong.

One: "unhealthy" is not always the model's fault. When the upstream account
runs out of balance, every model behind it fails at the same instant, with the
same error. My checker saw eight failures and dutifully deactivated eight models.
The models were fine. The account was empty. Deactivating them made the outage
longer, because now they would not even be retried.

The fix was a distinction the code did not previously have:

if all_checks_failed_with_account_level_error():
    log("SKIP auto-isolate: account-level failure (balance) -- do NOT deactivate")
else:
    isolate_the_single_bad_model()

Now that line exists in the log, and it looks like this:

[auto-isolate] SKIP kimi-k3: account-level failure (balance) -- do NOT deactivate the model

Two: "2xx from upstream" does not mean "works for this model name". Some
suppliers route by model name and return a perfectly valid 200 with an error body,
or accept the name in /models and reject it in /chat/completions. A health
check that only looks at status codes will mark those models healthy forever.

The contract we publish now

If a model is unavailable, the response is not a mystery. It is this, with
HTTP 503:

{
  "error": {
    "message": "Service temporarily unavailable: upstream model capacity is exhausted for the requested model. Please retry later or contact [e-mail].",
    "type": "service_unavailable",
    "code": "upstream_capacity_exhausted",
    "hint": "Other models may still be available right now: https://flintapi.ai/api/health/models",
    "available_models": ["deepseek-v4-pro", "deepseek-v4.1-flash", "glm-5.3", "glm-5.3-flash"]
  }
}

Three properties matter here:

1. A machine-readable code (upstream_capacity_exhausted), so your retry and failover logic does not have to parse English.
2. The list of models that are working right now, so a 503 is actionable — you can fall back without a human.
3. Retry-After, so a naive client does not hammer us in a loop.

And the single source of truth, which is public and needs no auth:

GET https://flintapi.ai/api/health/models

It returns total and healthy counts joined against the catalogue that we
actually serve. If that endpoint says four models are healthy, then four models
are healthy — we do not keep a second, prettier number anywhere else.

Why we do not "just multiply the model list"

It is tempting to advertise thirty models and route whatever works. That converts
our availability problem into your correctness problem: your temperature, your
token budget, your structured-output assumptions silently run on a different model
than the one you named. We would rather return a loud 503 for a named model than a
quiet 200 from a substitute.

If you do want substitution, it is opt-in: send model: "flint" and the router
decomposes the prompt and picks per sub-task, and it says which model answered.

The numbers, dated

As of 2026-10-28: 13 models in the catalogue, 4 healthy, because one upstream
account is at a negative balance of about three cents. This post exists partly
because that is an embarrassing ratio and partly because the interesting part is
not the ratio — it is that the ratio is published, checkable, and per-model
rather than hidden behind a green dashboard.

If you are running something similar, two checks are worth adding today:

• Distinguish account-level failures from model-level failures before any automated isolation runs. Otherwise one billing problem takes out your whole catalogue and looks like thirteen separate bugs.
• Return the currently-available list inside your error body. It costs one join and turns every 503 into a self-healing instruction.

Related: What Chinese LLM APIs actually cost in 2026 and
Why your health check said "up" while the model was gone.