Aligning two subtitle files sentence by sentence for bilingual captions

Bilingual subtitles, with the original line and its translation shown together, are one of the most effective tools for language learners watching films. The data problem behind them: the two subtitle files almost never share the same line breaks. One English line can map to two Vietnamese lines, or the other way round. Matching by line number breaks after the first mismatch.

Match by time overlap, not by index

Each subtitle cue has a start and an end time. For each line in the primary language, collect every secondary-language cue that overlaps it by a meaningful share of the secondary cue's own duration.

function parseSrt(text) {
  return text.trim().split(/\r?\n\r?\n/).map(block => {
    const lines = block.split(/\r?\n/);
    const [a, b] = lines[1].split(' --> ').map(t => {
      const [h, m, rest] = t.split(':');
      const [s, ms] = rest.split(',');
      return ((+h * 60 + +m) * 60 + +s) * 1000 + +ms;
    });
    return { start: a, end: b, text: lines.slice(2).join(' ') };
  });
}

const overlap = (x, y) => Math.max(0, Math.min(x.end, y.end) - Math.max(x.start, y.start));

function align(primary, secondary) {
  return primary.map(p => {
    const matches = secondary.filter(s => overlap(p, s) >= 0.3 * (s.end - s.start));
    return { ...p, translation: matches.map(s => s.text).join(' ') };
  });
}

With this English file

00:00:03,600 --> 00:00:06,000  Don't worry about me.

and a Vietnamese file split into two cues (Đừng lo and cho tôi.), align returns a single row: Don't worry about me. | Đừng lo cho tôi.

Why 30 percent

A small threshold glues in neighbouring cues that touch the edge of the line for a few milliseconds. A large threshold drops short translated cues that sit inside a long original line. Around a third of the secondary cue's duration is a reasonable starting point for film subtitles; check the result on a few minutes of each new film and adjust.

Things that still need a human

• Translations that merge two original sentences into one long cue will appear next to both rows.
• Subtitles with a constant offset (a different release of the film) must be shifted first. Find the offset from one clearly matching line and add it to every cue.

For an example of sentence-by-sentence bilingual subtitles in use, see Tu Vung 123, a language-learning site for Vietnamese speakers.