I run AI Video Translator as a solo dev. This week I shipped three features that all came from the same user complaint: "I only find out the translation is wrong after the dub is done."
The problem: errors surfaced at the most expensive step
The original pipeline ran end to end in one pass:
STT → translation → voice cloning / TTS → time-stretch + remix → output
Users heard the result, then noticed a name translated literally, a tone that didn't fit the video, or a term that came out differently in episode 3. The only fix was rerunning the whole dub.
The cost asymmetry was obvious once I looked at it. Editing a line of text is almost free. Regenerating speech is not. So the goal became simple: let users fix mistakes at the cheapest point in the pipeline.
The new pipeline
STT
|
v
Translation (glossary applied)
|
v
[NEW] Human review: line-by-line edit
|
v
Voice cloning / TTS (per speaker)
|
v
Time-stretch + remix
|
v
Dubbed video + SRT (burn-in optional)
1. A review step before synthesis
After translation, the script is shown line by line next to the source. Whatever the user edits becomes the input to TTS.
The main UX constraint is line length. Translated speech already tends to run longer than the source. If a user expands a line with extra explanation, it still has to fit the same time slot, so the voice speeds up. The rule I push in the UI: change the meaning if needed, keep the length close.
2. A glossary applied at translation time
Review fixes one video. Without a glossary, the same mistake comes back in the next one and users fix the same word again and again.
So users can now register source terms with a fixed target translation, applied during the translation step. It's the same idea as the glossary layer I built for character names in my manga translator (write-up here), except in video it's product names, technical terms, and channel-specific phrases. CSV import and export make it reusable across a series and shareable across a team.
The two features feed each other: any word you fix twice during review is a glossary candidate. The first episode takes real review time. By episode ten, review is a quick scan.
3. SRT as a separate output
Previously the only output was a video with subtitles burned in. Fixing one caption typo meant re-rendering the whole video.
Now every dub ships with a separate SRT file, and burn-in is optional. The SRT is built from the reviewed script, so edits land in both the audio and the captions. Creators can upload a clean video to YouTube and add the SRT in YouTube Studio, or upload captions separately to a course platform.
This sits on top of the multi-speaker detection I shipped a day earlier, where each speaker in the video keeps their own voice.
What I got wrong at the start
I originally pitched "fully automatic, from translation to dub" as the main selling point. But users dubbing long series and course content didn't want less control. They wanted a place to intervene. The automation stays; I just added a handle at the point where human input is cheapest.
I wrote a fuller, user-facing walkthrough of the workflow (including a training and course video use case) here: How to Review and Edit AI Dubbing Translations (and Keep Terminology Consistent)
Feedback welcome
- Is line-by-line the right review granularity, or would grouping by paragraph or speaker be better?
- For per-series glossaries, is CSV good enough, or is there a better format for teams?
Thanks to everyone who asked for these.
Top comments (1)
Catching a wrong name before the voice is generated makes so much sense! Nobody wants to redo a whole dub over one word. The glossary is a nice addition too. Fixing the same name in every episode would get old fast 🙂