DEV Community

Saviel Yamani
Saviel Yamani

Posted on

Building an Asynchronous Pipeline: Connecting AI Text-to-Art and Video Generation APIs

Text to art

If you have ever experimented with generative media, you are likely familiar with the manual workflow: you write a prompt in an image generator, download the resulting file, upload it into a video generator, and wait for the render.

While this manual process is fine for sporadic experimentation, it becomes a bottleneck when you try to scale content production, build user-facing products, or run programmatic video generation pipelines.

To automate this, we need to connect these steps in code. However, stitching together an AI text to art generator and an AI video maker from prompt via their APIs presents a major engineering challenge: network timeouts. Video generation models can take anywhere from 10 to 60 seconds (or more) to render a single clip. A synchronous HTTP request will almost certainly time out.

In this guide, we will walk through the architecture of an asynchronous pipeline designed to handle these long-running tasks gracefully using Node.js, queues, and webhooks.

The Pipeline Architecture

Instead of blocking the main thread while waiting for a generation to finish, we want to design a multi-stage decoupled pipeline.

[User Prompt] ──> [Task Queue] ──> [AI Text-to-Art API]
                                            │
                                      (Webhook Callback)
                                            v
[AI Video Maker from Prompt API] <── [Verify & Store Image]
            │
      (Webhook Callback)
            v
   [Final .mp4 Delivery]
Enter fullscreen mode Exit fullscreen mode

Why two stages?

While some platforms offer direct text-to-video endpoints, generating a keyframe first via an AI text to art model and then feeding that image into an image-to-video engine usually results in far better structural consistency, higher aesthetic quality, and more predictable camera physics.

Step 1: Setting up the Job Queue

To handle rate limits and retries, we will use a message queue. In Node.js, BullMQ (backed by Redis) is a reliable choice for managing asynchronous jobs.

// queue.ts
import { Queue, Worker, Job } from 'bullmq';
import IORedis from 'ioredis';

const connection = new IORedis(process.env.REDIS_URL || 'redis://127.0.0.1:6379');

export const videoPipelineQueue = new Queue('video-pipeline', { connection });
Enter fullscreen mode Exit fullscreen mode

Step 2: Triggering the AI Text-to-Art API

When a user submits a prompt, we push a job to the queue. The worker then calls our image generation endpoint (e.g., using Fal.ai, Replicate, or a custom Stable Diffusion/Flux instance) and specifies a webhook URL for the callback.

// worker.ts
import { Worker, Job } from 'bullmq';
import axios from 'axios';

const worker = new Worker('video-pipeline', async (job: Job) => {
  const { prompt, jobId } = job.data;

  // 1. Trigger the AI Text-to-Art generation
  const response = await axios.post(
    'https://api.generator.example/v1/images/text-to-art',
    {
      prompt: prompt,
      aspect_ratio: '16:9',
      webhook_url: `https://yourdomain.com/api/webhooks/image-completed?jobId=${jobId}`
    },
    {
      headers: { Authorization: `Bearer ${process.env.ART_API_KEY}` }
    }
  );

  // We don't wait for the generation here. 
  // We just confirm that the API accepted the job.
  return { status: 'image_generation_initiated', externalId: response.data.id };
}, { connection });
Enter fullscreen mode Exit fullscreen mode

Step 3: Handling the Image Webhook & Animating the Art

Once the image generation is complete, the API provider sends a POST request to our webhook. We verify the payload, grab the image URL, and forward it to the video engine.

At this stage, we pass the image along with motion prompts to our AI video maker from prompt API (e.g., Runway Gen-3/Gen-4, Luma Dream Machine, or Kling).

// webhookRouter.ts
import express from 'express';
import axios from 'axios';

const router = express.Router();

router.post('/api/webhooks/image-completed', async (req, res) => {
  const { jobId } = req.query;
  const { status, output_url } = req.body; // Structure depends on your API provider

  if (status !== 'success') {
    // Handle generation failure
    return res.status(400).send('Image generation failed');
  }

  try {
    // 2. Trigger the AI video maker from prompt (Image-to-Video workflow)
    await axios.post(
      'https://api.video.example/v1/videos/generate',
      {
        image_url: output_url,
        motion_prompt: 'Slow panning shot, cinematic lighting, 4k',
        duration: 5,
        webhook_url: `https://yourdomain.com/api/webhooks/video-completed?jobId=${jobId}`
      },
      {
        headers: { Authorization: `Bearer ${process.env.VIDEO_API_KEY}` }
      }
    );

    return res.status(200).send('Video generation initiated');
  } catch (error) {
    console.error('Failed to chain video generation:', error);
    return res.status(500).send('Internal Server Error');
  }
});
Enter fullscreen mode Exit fullscreen mode

Handling Real-World Production Challenges

While the code above provides the basic skeleton, deploying this to production requires addressing several real-world edge cases:

1. Webhook Packet Loss (The Silent Failure)

Webhooks are inherently unreliable; networks drop packets, and servers restart. If your server is down when the image provider sends the webhook, the pipeline breaks permanently.

  • Mitigation: Implement a polling fallback. For every job in the queue, if you do not receive a webhook callback within 5 minutes, query the API status endpoint directly to check if the asset is ready.

2. Rate Limits and Exponential Backoff

Video API providers enforce strict rate limits (e.g., maximum 5 concurrent generations). Sending too many requests simultaneously will result in 429 Too Many Requests.

  • Mitigation: Configure your queue worker to limit concurrency. When a 429 is encountered, let the queue engine automatically retry the job using exponential backoff (e.g., waiting 5 seconds, then 10, then 20 before trying again).

Conclusion

By breaking down the generation workflow into distinct asynchronous stages, you can build a robust, scalable media pipeline that handles timeout issues and scales gracefully.

Connecting AI text to art and AI video maker from prompt endpoints allows developers to bypass manual, UI-heavy workflows and programmatically explore the potential of automated video generation.

Top comments (0)