Building a Reliable Photo-to-Video Workflow in Next.js

AI media generation looks simple in a product demo: upload an image, enter a prompt, and wait for a video. The backend workflow is less simple. A generation request can take minutes, files are much larger than ordinary JSON payloads, and model providers can fail after accepting a job.

Treating this as a normal synchronous API request creates timeouts, duplicate jobs, and a frustrating user experience. A more reliable design treats generation as a stateful, asynchronous workflow.

This article walks through that architecture using Next.js and TypeScript.

The core workflow

A production-friendly flow has six steps:

1. The browser requests permission to upload a source image.
2. The browser uploads the image directly to object storage.
3. The application creates a generation job in its database.
4. A worker submits that job to the model provider.
5. The browser polls a lightweight status endpoint.
6. When the job finishes, the result is stored and returned through a signed URL.

The important boundary is step three. The HTTP request that creates the job should return quickly; it should not remain open while the video is generated.

Define an explicit job state machine

Start with a small set of states:

type GenerationStatus =
  | "queued"
  | "processing"
  | "succeeded"
  | "failed";

interface GenerationJob {
  id: string;
  userId: string;
  sourceObjectKey: string;
  prompt: string;
  status: GenerationStatus;
  providerJobId: string | null;
  resultObjectKey: string | null;
  errorCode: string | null;
  createdAt: Date;
  updatedAt: Date;
}

Keeping the states explicit makes both frontend rendering and retry behavior predictable. Avoid a generic message field as the source of truth. Human-readable messages can change; state transitions should not.

A valid transition graph might be:

queued -> processing -> succeeded
                     -> failed

If retries are supported, create a new attempt record or move failed back to queued in one controlled server-side operation. Do not let the client set job states directly.

Upload directly to object storage

Sending a large image through a Next.js server route consumes application bandwidth and can hit platform request-size limits. Instead, generate a short-lived signed upload URL on the server:

// app/api/uploads/route.ts
import { NextResponse } from "next/server";

export async function POST(request: Request) {
  const user = await requireUser(request);
  const { contentType, size } = await request.json();

  if (!ALLOWED_IMAGE_TYPES.has(contentType)) {
    return NextResponse.json({ error: "Unsupported image type" }, { status: 400 });
  }

  if (size > 10 * 1024 * 1024) {
    return NextResponse.json({ error: "Image is too large" }, { status: 400 });
  }

  const objectKey = `uploads/${user.id}/${crypto.randomUUID()}`;
  const uploadUrl = await createSignedUploadUrl(objectKey, contentType);

  return NextResponse.json({ objectKey, uploadUrl });
}

The client can then upload the bytes without routing them through your application server:

await fetch(uploadUrl, {
  method: "PUT",
  headers: { "Content-Type": file.type },
  body: file,
});

Validate the file again before processing it. Browser-provided MIME types and file extensions are not trustworthy. A worker should inspect the actual file signature, decode the image, and enforce pixel-dimension limits.

Make job creation idempotent

Users double-click buttons. Mobile connections retry requests. A page can refresh at exactly the wrong moment. Without idempotency, one action may create multiple paid generation jobs.

Have the client create an idempotency key and send it when creating the job:

const idempotencyKey = crypto.randomUUID();

await fetch("/api/generations", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Idempotency-Key": idempotencyKey,
  },
  body: JSON.stringify({ objectKey, prompt }),
});

Store that key with a unique database constraint scoped to the user. If the same request arrives again, return the existing job instead of creating another one.

Credits should be reserved in the same database transaction that creates the job. Otherwise, concurrent requests can spend the same balance twice.

Move provider calls into a worker

The API route should enqueue work and return 202 Accepted:

return NextResponse.json(
  { jobId: job.id, status: job.status },
  { status: 202 }
);

A background worker can then:

1. Claim a queued job.
2. Validate and download the source asset.
3. Submit it to the model provider.
4. Store the provider's job ID.
5. Receive a webhook or poll the provider.
6. Copy the completed video to your own storage.
7. Mark the job as succeeded.

Copying the result to storage you control is important. Provider URLs are often temporary, and exposing them can leak implementation details or stop old projects from working later.

Poll without overwhelming the server

For many products, simple polling is easier to operate than WebSockets. Poll quickly at first, then back off:

const delays = [2000, 3000, 5000, 8000, 12000];

async function waitForGeneration(jobId: string) {
  let attempt = 0;

  while (true) {
    const response = await fetch(`/api/generations/${jobId}`, {
      cache: "no-store",
    });

    if (!response.ok) throw new Error("Could not read generation status");

    const job = await response.json();
    if (job.status === "succeeded" || job.status === "failed") return job;

    const delay = delays[Math.min(attempt, delays.length - 1)];
    await new Promise((resolve) => setTimeout(resolve, delay));
    attempt += 1;
  }
}

Stop polling when the tab is hidden if immediate updates are unnecessary, and resume when it becomes visible. Also ensure that the status endpoint returns only jobs owned by the authenticated user.

Design failure handling before launch

At minimum, handle these cases:

• The uploaded object does not exist.
• The image cannot be decoded.
• The provider rejects the prompt or image.
• The provider accepts the job but never completes it.
• A webhook arrives more than once.
• The result download fails after generation succeeds.
• The worker crashes between an external API call and a database update.

Webhook handlers should be idempotent, and every processing job should have a deadline. A scheduled recovery task can inspect jobs stuck in processing, ask the provider for their current state, and either finish or fail them deterministically.

Use stable error codes such as INVALID_IMAGE, PROVIDER_TIMEOUT, and CONTENT_REJECTED. Log detailed internal errors, but return short, actionable messages to users.

Product UX is part of the architecture

The interface should reflect the real workflow rather than pretending the request is instant. Show upload progress separately from generation progress, preserve the user's prompt, and make it safe to leave and return to the page.

For a concrete example of how this workflow can be presented as a focused user experience, see Animate Photo AI. The useful product lesson is the separation between input controls and the result preview: users should always know what they submitted and where the output will appear.

Disclosure: Animate Photo AI is my project.

A practical launch checklist

Before shipping, verify that:

• Upload URLs expire quickly and are scoped to one object key.
• File type, byte size, and image dimensions are validated server-side.
• Job creation and credit reservation are transactional.
• Duplicate API requests and webhooks are harmless.
• Users cannot read another user's job or media.
• Provider credentials never reach the browser.
• Queued and processing jobs have timeouts and recovery logic.
• Source files and generated videos have an explicit retention policy.
• Logs include your job ID and the provider job ID.

The main idea is simple: AI generation is not a long API request. It is a durable workflow with observable states. Once that model is reflected in the database, worker, and UI, the system becomes easier to retry, secure, and explain to users.