DEV Community

hedging8563
hedging8563

Posted on • Originally published at tokenlab.sh

Designing an Async Image API Client That Does Not Lie About Completion

#ai #api #typescript #devops

Image generation is where a seemingly simple API client starts to accumulate production bugs. A request may finish inline for one model, return a task for another, or take a longer path when the input includes edits and uploaded files. Treating every successful HTTP response as a completed image is the fastest way to ship broken retry logic and incorrect user-facing status.

This post adapts the TokenLab article TokenLab Async Image Generation Tasks for Production Apps. The canonical article contains the full implementation discussion; this version focuses on the contract decisions that matter when building an integration.

The response is a delivery decision, not just a payload

An image endpoint can return either a completed representation or an asynchronous task. The client should inspect the response envelope and normalize the delivery mode before it touches application state:

type Delivery =
  | { mode: "sync"; terminal: true }
  | { mode: "async"; task_id: string; status: string; terminal: false };
Enter fullscreen mode Exit fullscreen mode

The important invariant is that mode and terminal state come from the API contract. Do not infer completion from a missing progress field, a truthy data property, or a fast response time. Progress is useful when present, but it is not the completion signal.

Poll by task identity, not by the original request

When the server returns an async task, persist the task ID and the provider-neutral status. A worker can then poll the task endpoint with bounded backoff:

async function waitForTask(id: string) {
  for (let attempt = 0; attempt < 60; attempt += 1) {
    const task = await getTaskStatus(id);
    if (task.status === "succeeded") return task.result;
    if (["failed", "cancelled", "expired"].includes(task.status)) {
      throw new Error(`Media task ${id} ended as ${task.status}`);
    }
    await sleep(Math.min(1000 * 2 ** Math.min(attempt, 5), 30_000));
  }
  throw new Error(`Media task ${id} exceeded the polling budget`);
}
Enter fullscreen mode Exit fullscreen mode

This makes retries explicit. A network timeout after task creation is not proof that the task failed, so blindly replaying the generation request can create duplicate work or duplicate billing. The safer recovery is to reconcile the task ID first.

Keep native request shapes visible

The integration should preserve image prompts, edit inputs, response formats, model IDs, and file references instead of flattening them into a generic chat prompt. That gives callers a stable place to express image-specific behavior and lets the same client support OpenAI-compatible and native endpoint families without pretending they are identical.

For production systems, the practical checklist is:

  • record the request ID and task ID separately;
  • persist the last observed status and terminal timestamp;
  • treat succeeded, failed, cancelled, and expired as terminal states;
  • make the poller idempotent and bounded;
  • keep binary delivery separate from JSON task metadata;
  • show the user whether the result is ready, queued, or failed.

The full TokenLab implementation notes, current model examples, and image/edit endpoint references are in the canonical blog post. TokenLab's public image API documentation is the source of truth for request and response fields.

Top comments (0)