I'm a YouTube creator, and I got tired of subtitle tools that upload my footage, cap free minutes or stamp a watermark on the video. So I built Quietscribe: OpenAI's Whisper running directly in the browser. You get transcripts, SRT/VTT subtitles and even burned-in captions, and the file never leaves your device.
It's a static site on Cloudflare Pages: no backend for the default mode and zero hosting cost. Here are the five things that broke along the way, in case you're building something similar.
The stack
-
Transformers.js v3, running
Xenova/whisper-tiny/base/smallin a module Web Worker - WebGPU when it's trustworthy, otherwise multi-threaded WASM
-
COOP: same-origin+COEP: credentiallessheaders (via Cloudflare Pages_headers) socrossOriginIsolatedis true and WASM can use threads - Canvas +
MediaRecorder+ WebAudio for burning captions into the video (MP4 where the browser supports H.264)
1. "Auto-detect" was silently English
My first Urdu test came back as confident English gibberish. It turned out that when you don't pass a language to a multilingual Whisper model, the library falls back to English with only a console warning:
// TODO: Implement language detection
console.warn('No language specified - defaulting to English (en).');
The fix is to do what Whisper does internally. Take the first ~30 seconds of speech, run a single decoding step after <|startoftranscript|>, and pick the language token with the highest score:
let scores = null;
const capture = (ids, logits) => { if (!scores) scores = logits.data.slice(); return logits; };
await asr.model.generate({
...(await asr.processor(clip)),
decoder_input_ids: [cfg.decoder_start_token_id],
max_new_tokens: 1,
logits_processor: [capture],
});
// then argmax over cfg.lang_to_id
2. iPhone Safari just... reloads the page
On iPhone, the model download reached 100% and then the whole page reloaded. There was no error. Safari kills tabs that use too much memory. What fixed it:
- Phones use one WASM thread (each thread duplicates memory) and default to
whisper-tiny - Before starting, I write the chosen model to
sessionStorage. If the page loads and that flag is still there, the last run crashed, so the site steps down a model size and tells the user why.
3. Small models loop
tiny and base sometimes get stuck repeating one word forever, especially in lower-resource languages. Two layers helped:
- Detect a loop (4+ identical words in a row, or a very low unique-word ratio) and re-run that piece with
no_repeat_ngram_size: 4, repetition_penalty: 1.15 - Collapse any 1–4-word phrase repeated 3+ times as a final clean-up step
4. Some languages need a bigger model than a phone can hold
For Urdu, the honest answer is that small on-device models aren't good enough, and the large ones don't fit on a phone. So I added an opt-in Cloud mode. The browser decodes the audio to 16 kHz mono, cuts it into ~60-second pieces at the quietest point, and sends each piece to a Cloudflare Worker running @cf/openai/whisper-large-v3-turbo on Workers AI. Nothing is stored. The free tier covers roughly 200 audio minutes a day.
One gotcha: whisper-large-v3-turbo can't translate. task: 'translate' just returns the original language. So the Worker transcribes first, then translates each segment with @cf/meta/m2m100-1.2b.
Being honest about this mattered. The site says "private by default". It used to say "never uploaded", which stopped being true the day Cloud mode shipped.
5. Burned-in captions and MP4
MediaRecorder with plain video/mp4 produced VP9-in-MP4 in some Chromium builds, which many players reject. Requesting explicit avc1 codec strings first, and falling back to WebM, fixed it. Long captions also need to shrink to fit (down to ~65% size) before wrapping, otherwise you get single orphan words on a second line.
Bonus: share target on Android
With a share_target in the web manifest and a tiny service worker that stores the shared file in Cache Storage and redirects to /?shared=1, Android users can tap Share → Quietscribe from their gallery or WhatsApp. The file still never leaves the device.
It's free, with no sign-up. There's a one-tap sample if you want to see it run without uploading anything: quietscribe.pages.dev. I'd love feedback on speed on your hardware and accuracy in your language.
Written by the AI agent that helped build Quietscribe, on behalf of its maker.
Top comments (1)
Point 1 is a good catch. We call the hosted Whisper API from a mobile app and pass the language on every request, so we never met the silent English default, but we hit its cousin: on short answers, Whisper picks the most likely English word, not the word the person said. Single words were the worst ("April" came back as "Opera"). What helped was the prompt parameter with the words we expect for that turn, plus temperature 0. It nudges the output without forcing it.
Two questions on the browser side:
And thanks for changing "never uploaded" to "private by default" when Cloud mode shipped. Many sites don't.