I built a free Whisper transcription site that runs in the browser: 5 things that broke

Taha — Sun, 01 Nov 2026 17:12:47 +0000

I'm a YouTube creator, and I got tired of subtitle tools that upload my footage, cap free minutes or stamp a watermark on the video. So I built Quietscribe: OpenAI's Whisper running directly in the browser. You get transcripts, SRT/VTT subtitles and even burned-in captions, and the file never leaves your device.

It's a static site on Cloudflare Pages: no backend for the default mode and zero hosting cost. Here are the five things that broke along the way, in case you're building something similar.

The stack

Transformers.js v3, running Xenova/whisper-tiny / base / small in a module Web Worker
WebGPU when it's trustworthy, otherwise multi-threaded WASM
COOP: same-origin + COEP: credentialless headers (via Cloudflare Pages _headers) so crossOriginIsolated is true and WASM can use threads
Canvas + MediaRecorder + WebAudio for burning captions into the video (MP4 where the browser supports H.264)

1. "Auto-detect" was silently English

My first Urdu test came back as confident English gibberish. It turned out that when you don't pass a language to a multilingual Whisper model, the library falls back to English with only a console warning:

// TODO: Implement language detection
console.warn('No language specified - defaulting to English (en).');

The fix is to do what Whisper does internally. Take the first ~30 seconds of speech, run a single decoding step after <|startoftranscript|>, and pick the language token with the highest score:

let scores = null;
const capture = (ids, logits) => { if (!scores) scores = logits.data.slice(); return logits; };
await asr.model.generate({
  ...(await asr.processor(clip)),
  decoder_input_ids: [cfg.decoder_start_token_id],
  max_new_tokens: 1,
  logits_processor: [capture],
});
// then argmax over cfg.lang_to_id

2. iPhone Safari just... reloads the page

On iPhone, the model download reached 100% and then the whole page reloaded. There was no error. Safari kills tabs that use too much memory. What fixed it:

Phones use one WASM thread (each thread duplicates memory) and default to whisper-tiny
Before starting, I write the chosen model to sessionStorage. If the page loads and that flag is still there, the last run crashed, so the site steps down a model size and tells the user why.

3. Small models loop

tiny and base sometimes get stuck repeating one word forever, especially in lower-resource languages. Two layers helped:

Detect a loop (4+ identical words in a row, or a very low unique-word ratio) and re-run that piece with no_repeat_ngram_size: 4, repetition_penalty: 1.15
Collapse any 1–4-word phrase repeated 3+ times as a final clean-up step

4. Some languages need a bigger model than a phone can hold

For Urdu, the honest answer is that small on-device models aren't good enough, and the large ones don't fit on a phone. So I added an opt-in Cloud mode. The browser decodes the audio to 16 kHz mono, cuts it into ~60-second pieces at the quietest point, and sends each piece to a Cloudflare Worker running @cf/openai/whisper-large-v3-turbo on Workers AI. Nothing is stored. The free tier covers roughly 200 audio minutes a day.

One gotcha: whisper-large-v3-turbo can't translate. task: 'translate' just returns the original language. So the Worker transcribes first, then translates each segment with @cf/meta/m2m100-1.2b.

Being honest about this mattered. The site says "private by default". It used to say "never uploaded", which stopped being true the day Cloud mode shipped.

5. Burned-in captions and MP4

MediaRecorder with plain video/mp4 produced VP9-in-MP4 in some Chromium builds, which many players reject. Requesting explicit avc1 codec strings first, and falling back to WebM, fixed it. Long captions also need to shrink to fit (down to ~65% size) before wrapping, otherwise you get single orphan words on a second line.

Bonus: share target on Android

With a share_target in the web manifest and a tiny service worker that stores the shared file in Cache Storage and redirects to /?shared=1, Android users can tap Share → Quietscribe from their gallery or WhatsApp. The file still never leaves the device.

It's free, with no sign-up. There's a one-tap sample if you want to see it run without uploading anything: quietscribe.pages.dev. I'd love feedback on speed on your hardware and accuracy in your language.

Written by the AI agent that helped build Quietscribe, on behalf of its maker.

DEV Community: Taha