14,000 production calls later: image and video APIs fail in two opposite ways

Media generation endpoints fail in two ways that look identical in your logs and need opposite handling. One is a content filter refusing the prompt, which no amount of retrying will fix. The other is a saturated GPU pool, which almost always clears within seconds. If you lump them into one catch block, you will both burn money retrying rejected prompts and give up on requests that would have succeeded.

I pulled the numbers to see how the split actually falls, across 14,069 image and video generations over 30 days. The ratio is not close, and it inverts completely between the two media types. Disclosure up front: the data is from apimodels.app, which is the API platform I work on, so treat it as one operator's dataset rather than an industry benchmark. Everything below is written so you can run the same measurement against whatever provider you use.

The split

The two tiers and the video model below are the same three models the code at the end calls. For image generation, content rejections were 90% of all failures. For video generation on the same platform in the same window, they were 4%.

                        content  capacity  timeout  upstream  bad input
image model, fast tier      823        17       29        20         24
image model, detail tier    232        69       44        16         14
video model                   2        31        3          9         6

Turned into rates against total calls, the picture is sharper:

model                calls   success   infra failures   p50      p90
image, fast tier    10,149     91.0%            0.65%   40.1s    71.2s
image, detail tier   3,630     89.7%            3.55%   52.0s   135.9s
video                  290     82.4%           14.83%   36.5s   104.1s

The fast image tier looks unreliable at 91% success and is not. Strip out the prompts a filter refused and actual infrastructure failure is 66 calls in 10,149. Meanwhile the video model, which looks worse at 82.4%, has essentially no content problem at all: it failed 43 times because capacity was not there.

That inversion has a boring physical cause. Image inference finishes in tens of seconds and the filters sit on a text prompt, which users write freely and sometimes carelessly. Video inference holds a GPU for a minute or more, so pools saturate under load and "come back later" is a normal operating state rather than an incident. Any provider running the same hardware economics will show some version of this shape.

One honest caveat on my own numbers: these models are served through more than one upstream provider, so the capacity column partly reflects routing between them rather than a single vendor's raw availability. Your numbers against a single-vendor API will look different in magnitude. The two-class split is the part that transfers.

Measure it on your own provider

You need failures bucketed into "the user can fix this" and "time can fix this", and most APIs will not hand you that distinction. They return a 4xx or 5xx and a message string.

The mapping that has held up for us is three buckets, not two, because the third one is the one that wakes people up at night:

function classify(status, message) {
  const m = (message || "").toLowerCase();

  // The user can fix this. Never retry. Show them the provider's own wording.
  if (/safety|content policy|moderation|nsfw|prohibited|violat/.test(m)) return "CONTENT";
  if (status === 400 || /invalid|unsupported|too large|must be/.test(m)) return "BAD_INPUT";

  // Time can fix this. Retry with backoff.
  if (status === 429 || /busy|capacity|overload|rate limit|try again/.test(m)) return "CAPACITY";
  if (status >= 500 || /timeout|timed out|econnreset|gateway/.test(m)) return "TRANSIENT";

  // Nobody knows yet. This is the bucket you alert on.
  return "UNKNOWN";
}

Alert on the size of UNKNOWN, not on your overall error rate. A rising UNKNOWN means the provider changed their error wording and your retry logic has quietly stopped working. A rising CONTENT means your users changed, or someone shipped a prompt template that trips a filter, and no amount of infrastructure work will help.

I am not going to pretend this regex table is elegant. It is hand-maintained, it drifts, and every provider that collapses both classes into a generic 500 Internal Error makes it worse. It is still better than one retry policy for everything.

The retry loop the split implies

Retry capacity and transient failures with exponential backoff, and fail content rejections immediately with the message attached.

const RETRYABLE = new Set(["CAPACITY", "TRANSIENT"]);

async function generate(body, tries = 3) {
  for (let i = 0; i < tries; i++) {
    const res = await submitAndPoll(body);
    if (res.ok) return res;

    const bucket = classify(res.status, res.message);
    if (!RETRYABLE.has(bucket)) throw new UserFacingError(bucket, res.message);
    await sleep(2 ** i * 2000); // 2s, 4s, 8s
  }
  throw new Error("still failing after retries");
}

Three attempts at 2/4/8 seconds covers the capacity failures we see, because a saturated pool usually has a slot within ten seconds. Do not push this to ten attempts; if a pool is still full after fifteen seconds it is having a real incident and you want to fail visibly rather than pile on.

Surface content rejections immediately, with the provider's own wording. It is the one failure class a human can act on, and hiding it behind a spinner that retries three times means the user waits thirty seconds to find out their prompt was the problem.

The pipeline these numbers came from

The workload was a two-call pipeline: generate a still frame, then hand that frame to a video model as its first frame. Almost every hosted media API uses the same asynchronous shape, so the structure ports even though the field names will not.

Runtime for everything below: Node 24.19 with the built-in fetch, no SDK. Measurements taken 2026-09-20 over the preceding 30 days.

const BASE = process.env.MEDIA_API_BASE;   // swap for your provider
const HEAD = {
  Authorization: `Bearer ${process.env.API_KEY}`,
  "Content-Type": "application/json",
};

// 1. still frame
const img = await post(`${BASE}/images/generations`, {
  model: "gpt-image-2.5-flare",
  prompt: "a minimalist oak desk with a laptop showing a blue analytics dashboard, soft morning window light",
  aspect_ratio: "16:9",
  resolution: "1K",
  quality: "low",
});
const frame = await poll(`${BASE}/images/generations`, img.data.taskId);

// 2. animate it
const vid = await post(`${BASE}/video/generations`, {
  model: "grok-imagine-video-1.5",
  images: [frame],          // array, even for a single reference
  prompt: "slow push-in on the laptop screen, dust motes in the light",
  duration: 5,
  resolution: "720p",
});

You POST, get a task id, and poll until the state is terminal. A synchronous-looking wrapper around a 40-second median will time out somewhere in your stack, usually at a proxy you forgot about.

That still frame cost $0.008 and took 37 seconds.

The delivered size was 1672x941, not the 1920x1080 a 16:9 request implies. Image models snap dimensions to whatever grid their tiler uses; if a downstream step needs exact pixels, resize after generation rather than trusting the request.

The economics push one way: iterate on the still image, not on the clip. A second of 720p video cost more than six times the entire still frame, so refining motion when the real problem is the composition in frame one is the expensive mistake.

Three things that cost me time

The poll parameter was task_id while the create response returned taskId. A poll loop that returns an empty object for eight minutes rather than erroring is a genuinely miserable debugging session, and snake case versus camel case across create and read is common enough to check first.

Result files expire. Seven days on this platform, and every hosted media API has some version of it. Download to your own storage in the job that created the file, not in a nightly batch that will eventually run against dead URLs.

Billing lands on success. That is good for the bill and confusing for reconciliation, because a task that burned real GPU time upstream and then failed shows as free to you. Do not reconcile spend against request counts.

When not to do this at all

Do not generate when you need the same output twice. Seeds help and do not survive model version changes, so if your product promises a user that what they saved last month still looks the same, generate once and store the file. Treat the model as a source of assets, not a renderer you can call again.

Skip generation entirely for exact text inside an image. Every model in this class still garbles multi-word text at small sizes, and composing type in SVG over a generated background is more reliable than rolling dice on a headline.

And if your failure volume is low enough that you would never notice a 3% difference, skip the classifier too. It earns its keep somewhere north of a few thousand calls a month; below that, a single retry and a clear error message is the correct amount of engineering.

What I would like to know

If you run generation in production, how are you separating these two classes? I am specifically curious about providers that return a bare 500 for both, because the regex table above is the part of our stack I would most like to delete.