I spent a few weekends building something I've wanted for years: a way to watch a film in a language I don't speak and just hear it in English, spoken aloud, while it plays. Not subtitles. An actual dub. And I wanted the whole thing to run on my own PC, offline, with nothing uploaded anywhere.
It works now. I call it LiveDub. Here's what I learned building it.
The pipeline is boring. The timing is not.
On paper the flow is simple:
- Capture the audio coming out of the browser tab.
- Transcribe it with Whisper.
- Translate the text with a local LLM.
- Speak the English back with a neural voice, and duck the original soundtrack under it.
Each of those pieces already exists and runs locally. Whisper does speech-to-text. LM Studio will happily serve a small local model over an OpenAI-style API for the translation step. Kokoro gives you a neural voice that doesn't sound like a 1998 GPS. The hard part isn't any single stage — it's that a movie doesn't wait for you.
If you transcribe every 3 seconds on a fixed clock, you slice sentences in half and Whisper hallucinates on the fragments. So instead of a timer, I watch the volume envelope of the tab audio and wait for a natural pause — a gap of roughly 450ms — before I treat what came before it as a finished thought. That one change did more for quality than any model upgrade. You translate whole sentences, not shrapnel.
The part I'm actually proud of: picking a voice by pitch
Early on, every character — man, woman, child — came out in the same voice. It felt like a bored narrator reading a script, and it wrecked the illusion instantly.
I didn't want speaker diarization. That's heavy, it needs enrollment, and it's overkill for "is this roughly a male or female voice." So I cheated with physics. Before each line gets sent off to be spoken, I measure the fundamental pitch of the original dialogue. Below about 155Hz I use a male voice, above about 190Hz a female one, and the band in between I treat as unclear and just keep whoever spoke last.
[M] 118Hz am_adam | We don't have much time.
[F] 213Hz af_bella | Then we run.
It's crude and I'll defend it anyway. It's instant, it needs no training, and it's right most of the time. Where it falls down is exactly where you'd expect: whispering, shouting, children, and singing all shift pitch enough to fool it. I exposed the two thresholds as sliders so you can widen the dead band for fewer wrong switches or narrow it to be more responsive. Good enough beat clever here.
Ducking, and the fullscreen trap
When the dub speaks, the original track drops to about 8% volume, then comes back up when the line finishes. Simple crossfade. The annoying discovery was that capturing a tab's audio interferes with that same page's own fullscreen button — so the player's fullscreen control goes dead once you start dubbing. The fix that stuck was just using the browser's own F11 fullscreen instead, or going fullscreen before you hit start.
Why local, and why it's slower on purpose
Everything runs on the machine. No account, no credits, no monthly anything, and nothing you watch ever leaves your computer. That mattered to me more than raw speed.
And there is a cost: the dub always runs a sentence or two behind the picture. It has to — you can't translate a line until the person finishes saying it. Every dubbing tool has this delay, the paid ones included; they just don't tell you. I'd rather be honest that there's a beat of latency than pretend a local pipeline is magic.
What's under the hood
- Browser extension (Brave/Chrome/Edge) captures tab audio, detects the pauses, and plays the dub.
- Whisper running locally does the transcription.
- LM Studio with any small instruct model handles translation — an 8B model is noticeably more idiomatic than a 4B if your machine can spare it.
- Kokoro neural voices do the speaking, with the male/female switch above.
No GPU required, though one makes Whisper snappier. Windows for now.
If you want to try it instead of building your own, I put it up here: https://symshah.gumroad.com/l/livedub — it's five dollars, one time. But honestly, even if you never buy it, the pitch-as-a-voice-selector trick is worth stealing for your own projects.
Top comments (0)