Building Reliable AI Generation Workflows: The API Call Is the Easy Part

Over the past month, I have been building an AI generation product with asynchronous video tasks.

I originally expected model integration to be the difficult part.

It was not.

Sending a request to an AI provider and receiving a task ID is relatively straightforward. The harder engineering work begins when the provider is slow, the network fails at the wrong moment, the user refreshes the page, or money has already changed hands.

This article covers the architecture lessons I learned while turning an AI API into a workflow that can recover from uncertainty.

The basic workflow

At a high level, an asynchronous generation request looks simple:

Client
  → Generation API
  → Local task record
  → AI provider
  → Polling or webhook
  → Result reconciliation
  → Persistent result

The happy path is easy:

1. Validate the request.
2. Calculate the cost.
3. Deduct credits.
4. Submit the provider task.
5. Poll until it succeeds.
6. Store the result.

Production systems rarely stay on the happy path.

The interesting problems appear between those steps.

1. Treat submission as a state machine

A generation task is not simply pending, success, or failed.

There is another important state:

submission_unknown

Imagine sending a POST request to a provider. The provider accepts it and starts the generation, but the response is lost because a gateway times out.

From the application's perspective, two realities are possible:

• the provider never accepted the request;
• the provider accepted it, but the application did not receive confirmation.

Automatically retrying the request may create a second paid generation.

Treating it as an ordinary failure may refund the user while an expensive provider task continues running.

The system therefore needs to distinguish a confirmed rejection from an uncertain submission.

A conceptual state model might look like this:

type GenerationState =
  | 'created'
  | 'submitting'
  | 'pending'
  | 'succeeded'
  | 'failed'
  | 'submission_unknown';

The exact names are not important. The important part is preserving uncertainty instead of forcing every outcome into success or failure.

2. Give every submission a stable idempotency key

Users double-click buttons. Browsers retry requests. Mobile connections disconnect. UI state gets restored after a refresh.

The generation endpoint must assume that the same logical submission can arrive more than once.

Each client submission should carry a stable idempotency key:

type GenerateRequest = {
  prompt: string;
  model: string;
  idempotencyKey: string;
};

On the server, that key should resolve to one local task for one user:

const existingTask = await findTaskByIdempotencyKey({
  userId,
  idempotencyKey,
});

if (existingTask) {
  return existingTask;
}

Database uniqueness is still necessary because two requests may pass the initial lookup concurrently.

A safe flow is:

1. Check for an existing task.
2. Attempt to create the task with a unique key.
3. If the insert loses a race, load and return the winning task.
4. Never deduct credits twice for the same logical submission.

Idempotency is not only protection against double-clicks. It is what makes recovery possible across the entire workflow.

3. Do not confuse an observation timeout with a task timeout

Polling is an observation mechanism. It is not the task itself.

A browser may stop polling after thirty seconds, but the provider task can still be running correctly.

This distinction matters:

Polling window ended ≠ Generation task failed

If the client treats every polling timeout as a terminal failure, it may submit another task when the user tries again.

A better approach is to preserve the original task ID and resume observation later:

async function resumeTask(taskId: string) {
  const localTask = await loadTask(taskId);

  if (isTerminal(localTask.status)) {
    return localTask;
  }

  return observeExistingTask(taskId);
}

Refreshing the page, reopening the application, or returning from another device should continue tracking the same task whenever possible.

The UI can say that observation has paused without claiming that generation has failed.

4. Make reconciliation shared and idempotent

A task may reach a terminal result through several paths:

• foreground polling;
• a provider webhook;
• scheduled recovery;
• an admin repair operation.

These paths should not contain separate versions of the completion logic.

They should all call the same reconciliation function:

async function reconcileTask(input: {
  taskId: string;
  providerStatus: ProviderStatus;
  source: 'poll' | 'webhook' | 'recovery';
}) {
  // Load the current task.
  // Ignore an already-finalized task.
  // Persist the result or failure once.
  // Refund credits at most once.
  // Return the canonical local state.
}

Without shared reconciliation, race conditions become likely.

A webhook may complete the task while foreground polling is still active. Both paths may attempt to save the result or issue a refund.

Terminal transitions must therefore be idempotent:

if (task.finalizedAt) {
  return task;
}

In practice, a simple application-level check may not be enough. Database constraints or transactional updates should enforce the invariant.

The rule is:

Many observers may discover the outcome, but only one transition should finalize it.

5. Calculate billing on the server

The frontend needs to display the expected credit cost, but it must not decide the final charge.

Generation cost may depend on:

• model;
• duration;
• resolution;
• output count;
• generation mode;
• optional capabilities.

The server should parse and validate the real request parameters, then calculate the authoritative cost:

const options = parseGenerationOptions(request.body);

const creditCost = calculateCreditCost({
  model: options.model,
  duration: options.duration,
  resolution: options.resolution,
  outputCount: options.outputCount,
});

The frontend should consume the same shared pricing definitions when possible, but the server must still recompute the charge.

Otherwise, stale clients or modified requests can create a gap between displayed pricing and actual billing.

Refunds need the same discipline.

A refund should be tied to a unique task and transaction reason so that repeated recovery attempts cannot refund the same task twice.

6. Persist enough information to recover the user experience

Task recovery is not only a backend problem.

After a refresh, the frontend needs enough information to reconstruct what the user was doing:

• the original prompt;
• selected model;
• input assets;
• generation settings;
• local task ID;
• current status;
• result metadata.

Persisting only the provider task ID is not enough.

The provider knows how to generate the output, but it does not necessarily know how your product should present the user's intent.

A local task record should remain the application's source of truth. Provider state is external evidence used to update that record.

This separation also prevents internal provider identifiers and implementation details from leaking into the user interface.

What changed in my thinking

Before building this workflow, I thought the central engineering problem was model integration.

Now I see three different layers:

1. Provider transport — submitting and querying external tasks.
2. Product workflow — task identity, recovery, billing, and reconciliation.
3. User experience — explaining progress, uncertainty, failure, and recovery clearly.

The provider API only solves the first layer.

The product is built in the second and third.

AI tools can make implementation faster, but they also make it easier to add features before the underlying state model is reliable.

The most valuable work was not adding another model. It was defining what must remain true when several things fail at once.

Final takeaway

For asynchronous AI products, reliability comes from preserving identity and uncertainty:

• one logical submission maps to one task;
• an unknown outcome is not silently treated as failure;
• polling timeouts do not recreate work;
• every completion path shares one reconciliation contract;
• billing is authoritative on the server;
• refunds and terminal transitions are idempotent;
• the UI resumes existing work instead of submitting again.

The API call may be the visible feature.

The recovery model is the real product.

---

Disclosure: This article is based on my own implementation experience and was reviewed against the system's actual behavior. AI was used to help organize the structure and edit the English draft.