Somewhere between a loaded mail server and a Zoom invite, I wound up building a tool that dubs foreign films in my browser. Not because I set out to. Because I got tired of a movie I wanted to watch being in a language I only half understand, and the cloud dubbing options wanted my video and my credit card.
Real-time video translation sounds like a flex when you say it fast. In practice it's a pipe with three stages, and most people assume it's one big black box. It isn't.
What real-time video translation actually is
Here's the honest definition I use when people ask. It's hearing a line of dialogue in one language and hearing that same line spoken back to you in yours, a couple seconds later, while the video keeps playing. Not reading subtitles — hearing speech you understand. No saved files, no waiting for a render to finish. It just happens while you watch.
The pipeline, stage by stage
Stage one is grabbing the sound. My browser extension captures the tab's audio the moment you press play. No upload, nothing routed through my own network — the audio never leaves the machine.
Stage two is Whisper turning that audio into text as it streams. I feed it chunks, not the whole file, so the transcript lags the picture by only a few seconds.
Stage three is the translation, which runs on a local LLM through LM Studio. Free, and this is the bit people don't believe: it doesn't need a training-grade rig. Python 3.10+, LM Studio, one small model around 2GB. A GPU helps but isn't required.
Then Kokoro speaks the translated line out loud, and it does one thing that quietly makes the whole thing usable: it watches the pitch of whoever's talking and swaps between male and female voices to match. A two-person scene doesn't turn into a wall of one synthetic narrator. The original dialogue ducks under the dub so you still hear the film, just quieter underneath.
The bottleneck nobody mentions
It can't translate until the speaker finishes the sentence. That's physics, not a flaw. Whisper has to hear a line before the LLM can translate it, and the LLM has to produce an answer before Kokoro can speak. Every tool in this space works this way, cloud or local, and any one that pretends otherwise is dubbing from a script someone already gave it. The honest latency is a sentence or two. Fine by me — it beats subtitles for following a conversation on screen.
The part I actually care about is that nothing about this touches a server I don't control. The movie I'm watching, the words on screen, my listening habits — all local, offline once it's set up. I built it as a small Windows app plus a browser extension, and I priced it at a flat $19 one-time instead of a subscription, because running your own translation shouldn't bill you monthly. If that's the trade you're after, it's at https://symshah.gumroad.com/l/livedub.
FAQ
Does it need a powerful computer?
No. A normal laptop runs it; a GPU speeds it up but isn't required.
Does it upload my video anywhere?
No. Capturing, transcribing, translating and speaking all happen on your machine.
Why does the dub lag behind the video?
The translator can't start until the speaker finishes a sentence. That delay is a sentence or two and applies to every dubbing tool, local or cloud.
Top comments (0)