In the post-production and broadcast industry videos are processed frame by frame.
Subtitles follow the same rule : in and out timecodes are frame accurate (ex: 00:00:23:22
-> 23 seconds and 22 images).
Browsers process videos with millisecond timestamps, that's why subtitles have to be converted based on video framerate: for video at 25 images per second the previous timecode would look like 00:00:23.880
in milliseconds format (1000/25 * 22 = 880
).
Frame accuracy is really important : subtitles need to disappear right before the next cut and appear right after the previous one.
Problem is that default HTML5 video's refresh rate is too low: subtitles appear and disappear too late creating a poor viewer's experience (and a non-broadcast compliant one).
Example below higlights the issue. Subtitles in the fixed video disappear right before the change of plan:
It might seem marginal, but the post production industry needs frame accuracy.
Due to browser limitations, timeupdate
(event fired when the playing position of a video has changed) is fired every 150-250 milliseconds. It is not enough for frame-accuracy: 25fps means an update every 40ms.
We need to compute which subtitle has to be displayed every frame (instead of doing it every 4-5 frames by default). Video.js subtitles engine does the computation each time the text track's attribute activeCues
getter is called.
text-track.js extract from Video.js source code:
Object.defineProperty(tt, 'activeCues', {
get() {
if (!this.loaded_) {
return null;
}
// nothing to do
if (this.cues.length === 0) {
return activeCues;
}
const ct = this.tech_.currentTime();
const active = [];
for (let i = 0, l = this.cues.length; i < l; i++) {
const cue = this.cues[i];
if (cue.startTime <= ct && cue.endTime >= ct) {
active.push(cue);
} else if (cue.startTime === cue.endTime &&
cue.startTime <= ct &&
cue.startTime + 0.5 >= ct) {
active.push(cue);
}
}
changed = false;
if (active.length !== this.activeCues_.length) {
changed = true;
} else {
for (let i = 0; i < active.length; i++) {
if (this.activeCues_.indexOf(active[i]) === -1) {
changed = true;
}
}
}
this.activeCues_ = active;
activeCues.setCues_(this.activeCues_);
return activeCues;
},
set() {}
});
Then you need to call trigger('cuechange')
on the text track to make sure the video display is up to date:
player.textTracks()[0].activeCues; // computes the current subtitle based on current time
player.textTracks()[0].trigger('cuechange'); // updates the display
requestAnimationFrame
is optimized for animations and has a lot less delay than setInterval
or setTimeout
, so we are going to use it for our time sensitive loop (Frame rate control source here).
Here is the complete source code:
var fps = 25;
var now;
var then = Date.now();
var interval = 1000/fps;
var delta;
function reloadCues() {
requestAnimationFrame(reloadCues);
now = Date.now();
delta = now - then;
if (delta > interval) {
then = now - (delta % interval);
if(videojs.players.player.textTracks().length == 1) {
videojs.players.player.textTracks()[0].activeCues;
videojs.players.player.textTracks()[0].trigger('cuechange')
}
}
}
reloadCues();
Demo project is available here.
Originally published on my personal blog.
Top comments (0)