DEV Community

Captionly
Captionly

Posted on

Free Video subtitles in seconds — no signup, no watermark, no upload

#whisper #webml #ai

TL;DR — I built Captionly, a
subtitle generator that runs Whisper inside the browser using
@huggingface/transformers (WebGPU when available, WASM as fallback).
Files never leave the device. Here's how the pieces fit together and
the gotchas that took me longest to figure out.

Why in-browser at all?

Every other subtitle tool ships your video to their servers. That means:

  • Upload time gates the whole experience (a 200 MB MP4 is 30+ s before anything starts).
  • Privacy: your raw footage sits in someone else's S3.
  • Cost: someone is paying for GPU minutes — usually you, behind a freemium paywall.

Browsers got the missing pieces in the last 18 months:

  • WebGPU — actual GPU compute inside the tab (Chrome 113+, Safari 18).
  • WebCodecs — hardware-accelerated video decode/encode without a <video> element.
  • Transformers.js 3+ — ports Whisper, Llama, etc. to WebGPU with ONNX Runtime Web.

Put them together and you get a "real" video tool that runs entirely on
the client.

The architecture 

Drop file → WebCodecs decode → resample to 16 kHz mono
        → Whisper (WebGPU/WASM) → timestamped words
        → user edits inline → WebCodecs encode + burned-in subs
        → download
Enter fullscreen mode Exit fullscreen mode

No server in that flow. The only network call is the first transcription,
which pulls the Whisper weights (~60 MB for Xenova/whisper-small) from the
Hugging Face Hub. Subsequent runs are served from the browser's local cache
(handled by transformers.js — no network round-trip).

Gotchas

1. WebGPU is great, except when it's not

whisper-small runs at ~3× real-time on an M2 with WebGPU. The same model
on the WASM backend is ~0.8× real-time. So WebGPU is mandatory for a good
UX. But:

  • Safari 18 ships WebGPU, but transformers.js needs Safari ≥18.4 for the f16 ops Whisper uses.
  • Linux Chrome has WebGPU behind a flag in some distros.
  • Some Android devices report WebGPU but crash on the first kernel.

I detect, attempt, fall back to WASM, and tell the user when it happens.
No silent degradation — a 10× perf drop is too big to hide.

2. Long videos blow up Whisper

Whisper natively handles 30 s chunks. For longer audio you have to chunk
manually with overlap and stitch the outputs. transformers.js exposes
return_timestamps: 'word' and handles the stitching, but you have to
pre-resample to 16 kHz mono first — Whisper's tokenizer expects exactly
that and silently produces garbage if you feed 44.1 kHz stereo.

I use the WebAudio OfflineAudioContext for resampling. ~50 ms for a
10-minute file on the main thread. Then move it to a Web Worker so the UI
doesn't jank during inference.

3. Burning subtitles back into MP4 without a server

This was the trickiest part. The standard tools (ffmpeg.wasm) are heavy
and slow. I went with WebCodecs:

  1. Decode the source video frame-by-frame with VideoDecoder.
  2. Draw each frame onto an OffscreenCanvas.
  3. Composite the active subtitle line on top (one of 12 TikTok-style presets — rendered with the Canvas 2D API).
  4. Re-encode with VideoEncoder.
  5. Mux back into MP4 via mp4-muxer.

Result: a short 1080p clip re-encodes in seconds on a recent laptop (timing
scales with hardware + the WebCodecs encoder), with zero server cost. The
full muxing dance fits in ~400 lines.

What I'd do differently

  • Ship whisper-large-v3-turbo (still ~800 MB) behind an opt-in for users who want lab-quality accuracy. The cache hit on second visit makes this viable even at that size.
  • Add a worker-pooled encoder so longer videos can fan out across cores.

Try it

Captionly — free, no signup, no
watermark. Repo coming soon (I want to clean up a few rough edges first).

Happy to answer architecture questions in the comments.

Top comments (0)