Only Complete Sentences Should Be Translated — But a Single Counter Betrayed That Design

#ai #claudecode

When watching English videos with LiveTR, the subtitle translations would occasionally get jumbled.

Lines from different people would merge into a single sentence. The first utterance after silence would be translated missing its beginning. Occasionally, nonsensical Japanese would appear.

A dim desk, a mood shot of English video subtitle overlays being converted to Japanese in real-time

You Need Complete Sentences to Translate

If you feed incomplete sentences to a translator, you get incomplete translations. It's obvious.

What comes into play here is how the audio is captured. To process in real-time, LiveTR mechanically splits the incoming audio every five seconds and feeds it to speech recognition. This five-second segmentation naturally ignores where people actually stop speaking. It ruthlessly cuts off sentences right in the middle.

So the English sentences coming out of speech recognition are frequently cut off. Fragments like "I think that we should…" are severed by the five-second wall. If you throw these directly into translation, you'll only get choppy translations back.

So LiveTR doesn't pass these fragments directly to translation. Instead, it holds onto them, and if the continuation comes in the next five seconds, it concatenates them to form a complete sentence before translating. Only complete sentences are sent for translation—this is the core of translation accuracy, and I wrote about its importance in the previous article introducing LiveTR.

The problem is when the continuation never comes. The conversation might pause or change topics, and the held fragment may never get its continuation. In that case, the system decides "waiting longer is futile" and sends the fragment as a sentence on its own. That cutoff mechanism was properly implemented.

That Cutoff Was Pointing in the Wrong Direction

The subtitle confusion happened because this cutoff wasn't working.

A single counter that measures waiting time—the one that decides "send without waiting anymore"—was wired backwards. It triggered when there was no need to wait, and failed to trigger when it should have cut off. So the held fragment remained in limbo, and when someone else started speaking in the next five seconds, it attached to the beginning of their speech.

For example, a five-second segment ends in the middle of "we really need to". Then the conversation flows in a different direction, and in the next five seconds someone says "Yeah anyway the budget is fine.". Normally the previous fragment would be sent alone, but instead it gets attached to form "we really need to anyway the budget is fine."—a sentence no one actually spoke. Because this is faithfully translated, the subtitle becomes something like "We really need to anyway, the budget is fine."—a line mixing two speakers' dialogue.

When the cutoff counter is reversed (top), the previous fragment attaches to the next speaker; with the correct direction (bottom), they are finalized as separate utterances

What I fixed was just this one thing: the direction of the cutoff. The design was correct.

The Translation Engine Was Excellent from the Beginning

LiveTR sends translations to cloud engines (DeepL / Google / Azure / Amazon, selectable in settings). They faithfully translate whatever English text they receive. If you feed them patchwork, they translate patchwork. They have no way of knowing that the English they receive is a collage of two people's lines.

Once I fixed the cutoff direction, complete sentences started reaching the engine. When I played English audio on the actual device, the translation was completely different. I hadn't touched a single line of the translation engine.

Feed it complete sentences, and the same engine returns a good translation. It was that simple.

Download

LiveTR is available for purchase on BOOTH. It can real-time Japanese-ify English videos and streams on Windows + NVIDIA GPU environments. It comes with subtitle overlays and Japanese text-to-speech that reflects the speaker's voice quality.