DEV Community: Bill King

Transparent video exports: why one MP4 URL isn't enough

Bill King — Tue, 08 Sep 2026 05:35:06 +0000

A background-removal API can finish successfully while the exported file is still wrong for the next step. A video that plays on a black background might have lost its transparency—or the player might simply be displaying it against black. Those are different problems.

I work on unbg.video, a video background remover and editor. The project has export paths for WebM, ProRes 4444 MOV, and separate H.264 color and alpha files. Here is a practical way to reason about those outputs, with a small FFmpeg experiment you can run without an AI service.

1. Start with what the recipient needs

“Download video” hides several different jobs:

Next step	Output to consider	What to verify
Display transparent footage on a website	WebM with alpha	Actual transparency in the target browser and device
Continue compositing in an editor	ProRes 4444 MOV with alpha	Import behavior and alpha interpretation in that editor
Transfer color and a separate matte	H.264 color + grayscale alpha files	Matching dimensions, timestamps, duration, and mask convention
Publish a finished scene with its new background	Flattened H.264 MP4	The final composition; transparency is no longer needed

The extension alone does not describe the whole contract. A MOV file does not automatically contain alpha. An ordinary H.264 MP4 export is not a portable RGBA deliverable. In the two-file workflow, the second MP4 stores a grayscale matte as picture data; it is not an embedded alpha channel in the color file.

2. A matte is data, not a black background

For straight alpha, the basic compositing relationship is:

output = alpha * foreground + (1 - alpha) * background

Here alpha is normalized from 0 to 1: zero reveals the background, one keeps the foreground, and intermediate values produce partial coverage. For physically meaningful blending, color-space handling also matters; this equation is a conceptual starting point.

A useful consequence: you cannot safely recover transparency by deleting every black pixel. Black clothing and shadows are legitimate foreground content. Keep the matte.

Also establish whether color is straight or premultiplied. With premultiplied color, RGB already includes multiplication by alpha. Multiplying again can darken the edges. If the silhouette looks right but has a dark fringe, inspect this convention before blaming the segmentation model.

3. Reproduce the export step without a model

This synthetic example creates one second of red video and a horizontal grayscale ramp, combines them, and writes ProRes 4444. It needs an FFmpeg build with prores_ks.

ffmpeg \
  -f lavfi -i 'color=c=red:s=64x64:r=10:d=1,format=rgb24' \
  -f lavfi -i 'nullsrc=s=64x64:r=10:d=1,format=gray,geq=lum=255*X/W' \
  -filter_complex '[0:v][1:v]alphamerge,format=yuva444p10le[v]' \
  -map '[v]' -an \
  -c:v prores_ks -profile:v 4 -pix_fmt yuva444p10le \
  foreground.mov

Then extract a frame of the stored alpha:

ffmpeg -i foreground.mov \
  -vf 'alphaextract,format=gray' -frames:v 1 -update 1 mask.png

The expected mask is a horizontal dark-to-light ramp, not a solid white frame. The red image should become progressively more opaque when composited over a background.

This example was run locally with FFmpeg 8.0.1 for this article: the output decoded as ProRes 4444 with an alpha-capable pixel format, and the extracted 8-bit alpha ramp ranged from 0 to 251 (the last column is 63/64 of full scale). It is an export smoke test, not evidence of browser compatibility or segmentation quality. Audio is deliberately omitted.

FFmpeg documents alpha extraction and merging and the ProRes encoder options. When replacing the synthetic inputs with real color and matte files, first align their dimensions and timeline. Resetting a timestamp does not repair missing or mismatched frames.

4. Represent the result as a bundle

The project stores multiple video assets with color/alpha variants and can package the H.264 pair into a ZIP. That matters: a single videoUrl cannot tell a consumer whether it has a finished composition or only half of a compositing input.

For a new API, a small discriminated union can make that distinction explicit. This is an illustrative design, not a copy of the project's implementation:

type VideoExport =
  | { kind: 'transparent'; url: string; format: 'webm' | 'prores4444' }
  | { kind: 'matte-pair'; colorUrl: string; alphaUrl: string }
  | { kind: 'flattened'; url: string; format: 'mp4' };

Only mark a pair ready when both assets are available. Give files explicit names such as clip_color.mp4 and clip_alpha.mp4, and document which mask values mean opaque. Otherwise the ZIP merely moves the ambiguity to the user.

5. Test the handoff, not just the download

My suggested acceptance checklist is short:

Composite the result over black, white, and a saturated background. Each reveals different edge problems.
Scrub moving edges near the beginning and end. A still frame will not reveal matte drift.
Open the exported artifact in the intended destination, not only the app's own preview.
For paired assets, verify both files and their timing before declaring success.
Keep a transparent export test separate from a flattened MP4 test. They promise different things.

The useful product question is: Can the recipient use this file for their next step without guessing? That question should shape the export format, the API response, and the download label.

Building a Reliable AI Transcription Pipeline on Cloudflare Workers

Bill King — Thu, 27 Aug 2026 05:26:21 +0000

I recently shipped HiTranscript, a web app that turns public video URLs and local media uploads into searchable transcripts and subtitle files.

The transcription model was not the hardest part.

The hard part was building a pipeline that stays correct when uploads are large, requests are retried, callbacks arrive twice, a batch partially fails, or a deployment needs to be rolled back.

This post covers the architecture patterns that made the system more reliable: explicit job states, durable media handoffs, idempotent callbacks, item-level batch tracking, and a single normalized timeline for every output format.

The architecture at a glance

The application uses TanStack Start and TypeScript for the web layer, PostgreSQL for durable task state, and Cloudflare Workers, Queues, and R2 for orchestration and media storage.

Browser
  |
  v
TanStack Start API
  |
  +--> PostgreSQL task record
  |
  +--> private R2 object
  |
  v
Cloudflare Queue
  |
  v
media preparation / transcription worker
  |
  v
signed callback
  |
  v
normalized word timeline
  |
  +--> readable transcript
  +--> subtitle cues
  +--> TXT / DOCX / PDF
  +--> SRT / WebVTT

The important design decision is that the HTTP request does not try to finish the transcription. It only validates the request, persists the intent, and creates a durable handoff.

1. Treat job states as a public contract

A boolean such as isProcessing is not enough for a media pipeline.

A real task can be waiting for media, preparing media, queued for transcription, actively processing, completed, or failed. Each state has different retry and UI behavior.

A simplified state model looks like this:

type TranscriptStatus =
  | "awaiting_media"
  | "media_preparing"
  | "queued"
  | "processing"
  | "completed"
  | "failed";

The value of explicit states is not the union type itself. The value is being able to define valid transitions.

For example:

awaiting_media can move to media_preparing or failed
media_preparing can move to queued or failed
queued can move to processing or failed
completed and failed are terminal

When every write checks the previous state, stale workers cannot move a finished task backward.

2. Use durable handoffs for large media

Passing large audio or video bodies through several HTTP requests creates unnecessary failure points.

The safer pattern is:

Create a task record.
Upload the media to private object storage.
Put a small message on the queue containing the task ID and object key.
Let the worker fetch the object when capacity is available.
Delete or retain the object according to an explicit retention policy.

The queue message should describe work, not carry the work itself.

This also keeps the web process responsive. The browser can display upload progress, while the backend independently reports preparation and transcription progress.

3. Design every callback for duplicate delivery

Retries are normal in distributed systems. A callback may arrive twice because a worker timed out after completing the request, a queue retried the message, or the provider repeated a webhook.

The callback handler therefore has to be idempotent.

A simplified version of the rule is:

async function completeTask(input: CompletionPayload) {
  verifySignature(input);

  const task = await findTask(input.taskId);

  if (task.status === "completed") {
    return task;
  }

  return database.transaction(async (tx) => {
    const updated = await tx.updateTaskWhereStatus({
      taskId: input.taskId,
      expected: ["queued", "processing"],
      next: "completed",
      result: normalizeResult(input.result),
    });

    if (!updated) {
      return findTask(input.taskId);
    }

    await settleBillingOnce(tx, input.taskId);
    return updated;
  });
}

The database transition, result persistence, and billing settlement belong to one consistency boundary. A duplicate callback should return the existing result instead of charging twice or creating a second output.

Signed callbacks are equally important. Idempotency prevents accidental duplication; signature verification prevents unauthorized state changes.

4. A batch is not just a bigger loop

It is tempting to model a batch as one task containing an array of URLs. That becomes painful when one item fails and the other 49 succeed.

A more useful model is:

one batch record for ownership and aggregate status
one item record per source
independent state and error information for every item
an aggregate state derived from the items

This makes partially_completed a first-class outcome rather than an exception.

It also improves fairness. A scheduler can take a capacity snapshot and dispatch work across batches instead of letting one large batch block every single-item request.

5. Preserve word timing as primary data

A transcript paragraph, an SRT file, and a short-form caption layout are different views of the same timing data.

Instead of storing only a large text blob, the pipeline keeps a normalized word timeline:

type TimedWord = {
  text: string;
  startMs: number;
  endMs: number;
  speakerId?: string;
};

From that timeline, the application can derive:

readable paragraphs
speaker-aware sections
standard subtitle cues
shorter social caption cues
search matches
TXT, document, and timed subtitle exports

This avoids running transcription again when the user changes an output option. It also keeps every view aligned to the same source data.

6. Failure handling is part of the product

The internal error may say that a decoder failed, a queue exhausted its retries, or an upstream service rejected a media file. That detail is useful in logs but often harmful in the UI.

The public contract should expose stable, actionable categories such as:

unsupported source
inaccessible or private media
no audio track
file too large
transcription failed
processing timed out

Internally, retain the detailed diagnostic code and attempt history. Externally, show a message the user can act on.

This separation also lets you change providers without changing the product's error language.

7. Deploy the exact version you tested

For this kind of pipeline, deployment safety matters as much as code correctness.

My preferred release flow is:

Build and upload an immutable Worker version with zero production traffic.
Validate the changed routes against that exact version.
Record the source commit and Worker version ID together.
Promote the same version without rebuilding.
Run focused production smoke checks.
Roll back to the previous Worker version if the code is unhealthy.

Rebuilding between validation and promotion breaks the evidence chain. The artifact that reaches users should be the artifact that was tested.

What I would improve next

The next reliability gains are less about adding more providers and more about strengthening the boundaries:

provider-neutral result validation
better timestamp-density checks
queue age and stuck-task observability
automated retention verification
multilingual and long-audio evaluation sets
cost tracking at the task level

A successful provider response is not the same as a usable transcript. Output shape, timestamp coverage, language behavior, and retry semantics all need validation.

Final takeaway

An AI transcription product is a distributed media system before it is an AI demo.

The durable design comes from treating state transitions, storage handoffs, callbacks, billing, and deployment artifacts as explicit contracts. Once those boundaries are reliable, switching models or adding output formats becomes much less risky.

If you are building a similar workflow, I would start with the state machine and idempotency rules before optimizing model latency. Those two decisions will shape almost every failure you have to handle later.

What has been the hardest reliability problem in your own asynchronous pipeline?