Shayan Nadeem

Posted on Jul 8

How I Built a Bulk Video Processing Pipeline with Next.js, BullMQ, and FFmpeg

#nextjs #node #showdev #tutorial

I've been building BatchEdits — a tool that lets content creators upload 50 videos at once and get them back edited simultaneously with silence removed, captions added, and cropped for every platform.

The hardest engineering problem was making 50 videos process truly in parallel without the server falling over.

Here's exactly how I built it.

The Problem With Sequential Processing

The naive approach is obvious:

for (const video of videos) {
  await processVideo(video);
}

For 50 videos this means:

Video 1 finishes → Video 2 starts
Total time = sum of all processing times
One failure stops everything
User waits the entire duration

For a 5-minute video that takes
2 minutes to process — 50 videos
would take 100 minutes sequentially.

We needed parallel processing.

The Architecture

Browser → Next.js API → BullMQ Queue 
                              ↓
                     Worker Pool (N workers)
                     ├── Worker 1 → FFmpeg
                     ├── Worker 2 → FFmpeg  
                     ├── Worker 3 → Whisper
                     └── Worker 4 → FFmpeg
                              ↓
                     Cloudflare R2 (output)
                              ↓
                     Webhook → User notified

The key insight: BullMQ handles the queue and worker orchestration. FFmpeg handles the actual video processing. Whisper handles transcription. They run concurrently.

Direct Upload to R2 (Not Through Server)

The first mistake most people make is routing video uploads through their Next.js server:

Browser → Next.js → R2

This is slow, wastes server bandwidth, and hits Next.js body size limits.

Instead use presigned URLs for direct browser → R2 upload:

// Generate presigned URL server-side
const url = await getSignedUrl(
  s3Client,
  new PutObjectCommand({
    Bucket: process.env.R2_BUCKET_NAME,
    Key: `uploads/${uuid}-${filename}`,
    ContentType: file.type,
  }),
  { expiresIn: 3600 }
);

// Return URL to browser
return { url, key };

// Browser uploads directly to R2
// Using XHR for progress tracking
function uploadWithProgress(
  url: string,
  file: File,
  onProgress: (pct: number) => void
): Promise<void> {
  return new Promise((resolve, reject) => {
    const xhr = new XMLHttpRequest();
    xhr.open("PUT", url);
    xhr.setRequestHeader("Content-Type", file.type);

    xhr.upload.onprogress = (e) => {
      if (e.lengthComputable) {
        onProgress(Math.round(e.loaded / e.total * 100));
      }
    };

    xhr.onload = () => 
      xhr.status === 200 ? resolve() : reject();
    xhr.send(file);
  });
}

Upload 50 files in parallel:

// p-limit prevents overwhelming 
// the connection on slow networks
import pLimit from "p-limit";
const limit = pLimit(5);

await Promise.all(
  files.map((file, i) =>
    limit(() => uploadWithProgress(
      presignedUrls[i].url,
      file,
      (pct) => updateProgress(i, pct)
    ))
  )
);

BullMQ For Parallel Processing

After upload, queue all videos:

// Add all jobs to queue at once
await Promise.all(
  uploadedFiles.map((file) =>
    videoQueue.add("process-video", {
      userId,
      videoId: file.id,
      key: file.r2Key,
      settings: presetSettings,
    })
  )
);

The worker processes each job independently:

// workers/videoProcessor.ts
const worker = new Worker(
  "video-processing",
  async (job) => {
    const { videoId, key, settings } = job.data;

    const startTime = Date.now();

    // Download from R2
    const videoBuffer = await downloadFromR2(key);

    // Transcribe with Whisper
    const transcription = await transcribeAudio(
      videoBuffer
    );

    // Remove silence using FFmpeg
    const silenceRemoved = await removeSilence(
      videoBuffer,
      transcription.sentences,
      settings.silenceLevel
    );

    // Burn captions
    const withCaptions = await burnCaptions(
      silenceRemoved,
      transcription.sentences,
      settings.captionStyle
    );

    // Apply zoom
    const withZoom = await applyZoom(
      withCaptions,
      transcription.sentences,
      settings.zoomLevel
    );

    // Crop for platform
    const output = await cropVideo(
      withZoom,
      settings.cropFormat
    );

    // Upload output to R2
    const outputKey = await uploadToR2(output);

    // Store processing time for analytics
    const processingTime = 
      (Date.now() - startTime) / 1000;

    await db.update(processedVideos)
      .set({
        status: "completed",
        outputKey,
        totalProcessingTimeSeconds: 
          Math.round(processingTime),
      })
      .where(eq(processedVideos.id, videoId));
  },
  {
    connection: redisConnection,
    concurrency: 10, // 10 videos simultaneously
  }
);

concurrency: 10 means 10 videos process at the same time per worker instance. You can run multiple worker instances for more parallelism.

Silence Detection With FFmpeg

The core silence removal uses FFmpeg's silencedetect filter:

async function detectSilence(
  inputPath: string,
  threshold: number = -30, // dB
  minDuration: number = 1.5 // seconds
): Promise<{ start: number; end: number }[]> {

  return new Promise((resolve, reject) => {
    const silences: { start: number; end: number }[] = [];
    let currentStart: number | null = null;

    const ffmpeg = spawn("ffmpeg", [
      "-i", inputPath,
      "-af", `silencedetect=noise=${threshold}dB:d=${minDuration}`,
      "-f", "null",
      "-",
    ]);

    ffmpeg.stderr.on("data", (data: Buffer) => {
      const output = data.toString();

      const startMatch = output.match(
        /silence_start: ([\d.]+)/
      );
      const endMatch = output.match(
        /silence_end: ([\d.]+)/
      );

      if (startMatch) {
        currentStart = parseFloat(startMatch[1]);
      }
      if (endMatch && currentStart !== null) {
        silences.push({
          start: currentStart,
          end: parseFloat(endMatch[1]),
        });
        currentStart = null;
      }
    });

    ffmpeg.on("close", () => resolve(silences));
    ffmpeg.on("error", reject);
  });
}

Then cut the detected silences:

async function removeSilences(
  inputPath: string,
  silences: { start: number; end: number }[],
  outputPath: string
): Promise<void> {

  // Build FFmpeg filter to keep 
  // everything except silence segments
  const keepSegments = getKeepSegments(
    silences, 
    videoDuration
  );

  const filterParts = keepSegments.map(
    (seg, i) => 
      `[0:v]trim=${seg.start}:${seg.end},` +
      `setpts=PTS-STARTPTS[v${i}];` +
      `[0:a]atrim=${seg.start}:${seg.end},` +
      `asetpts=PTS-STARTPTS[a${i}]`
  );

  const concatInputs = keepSegments
    .map((_, i) => `[v${i}][a${i}]`)
    .join("");

  const filter = [
    ...filterParts,
    `${concatInputs}concat=n=${keepSegments.length}` +
    `:v=1:a=1[outv][outa]`
  ].join(";");

  await runFFmpeg([
    "-i", inputPath,
    "-filter_complex", filter,
    "-map", "[outv]",
    "-map", "[outa]",
    outputPath,
  ]);
}

Caption Burning With Word-Level Timestamps

Whisper returns word-level timestamps which enable word-by-word caption highlighting:

const transcription = await openai.audio
  .transcriptions.create({
    file: audioFile,
    model: "whisper-1",
    response_format: "verbose_json",
    timestamp_granularities: ["word"],
  });

// transcription.words contains:
// [{ word: "Hello", start: 0.0, end: 0.3 }]

Burn captions using FFmpeg drawtext:

function buildCaptionFilter(
  words: { word: string; start: number; end: number }[],
  style: CaptionStyle
): string {
  return words.map((word) => {
    const escaped = word.word
      .replace(/'/g, "\\'")
      .replace(/:/g, "\\:");

    return (
      `drawtext=text='${escaped}':` +
      `fontsize=${style.fontSize}:` +
      `fontcolor=${style.color}:` +
      `x=(w-text_w)/2:` +
      `y=(h-text_h)/2:` +
      `enable='between(t,${word.start},${word.end})'`
    );
  }).join(",");
}

What I Learned

1. BullMQ concurrency is the key lever
Start with concurrency: 5 and increase based on your server's RAM. FFmpeg is CPU and memory intensive — too many concurrent jobs will crash your server.

2. Presigned URLs are non-negotiable
Routing video files through Next.js is a bottleneck and a body size limit problem. Direct browser-to-R2 upload is the only sane approach at scale.

3. Whisper accuracy degrades with noise
Clear audio gets 95%+ accuracy. Background music, multiple speakers, or poor microphones drop accuracy fast. Set user expectations accordingly.

4. FFmpeg filter_complex gets complicated fast
Start simple. Get one operation working perfectly before chaining operations. I spent a week debugging a filter_complex string that silently produced corrupted output.

5. Store processing time per stage
Knowing whether transcription or rendering is your bottleneck is
essential for optimization. Store transcriptionTimeSeconds and renderTimeSeconds separately.

The Result

BatchEdits processes 50 videos simultaneously with this architecture. A batch of 12 talking-head clips that would take hours manually finishes in minutes.

If you want to connect this to your
AI agent, the MCP server is open source:
batchedits-mcp-server

Try it free at batchedits.com — first 3 videos, no credit card.

Built by Shayan, solo founder of BatchEdits.
Follow along on X: @shayannadeem123

DEV Community