DEV Community

Mason K
Mason K

Posted on

Ship multi-language audio in HLS: author the manifest, wire the hls.js switcher

#video #webdev #tutorial #javascript

📦 Code: github.com/USER/hls-multi-audio - replace before publishing

TL;DR

We'll add a working language picker to an HLS player. The hard part isn't the dropdown, it's the manifest. We'll author alternate audio with EXT-X-MEDIA audio groups, package it correctly, debug the classic "zero audio tracks" bug, and wire a switcher on hls.js v1.7.

Adaptive video, captions, the whole pipeline already works. Now someone wants an English/Spanish audio toggle. In HLS, "which audio can the viewer pick" is decided at packaging time and written into the master playlist. The player just displays it. Let's build it in that order.

1. Understand the structure (audio groups)

HLS decouples video variants from audio renditions:

  • Each audio rendition is an #EXT-X-MEDIA:TYPE=AUDIO entry pointing at its own media playlist.
  • Renditions are bundled into a named audio group via GROUP-ID.
  • Each video variant (#EXT-X-STREAM-INF) references a group with AUDIO="...".

A correct master playlist:

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud",NAME="English",LANGUAGE="en",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2",URI="audio/en.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud",NAME="Espanol",LANGUAGE="es",DEFAULT=NO,AUTOSELECT=YES,CHANNELS="2",URI="audio/es.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=2128000,CODECS="avc1.640028,mp4a.40.2",AUDIO="aud"
video/720p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1128000,CODECS="avc1.640020,mp4a.40.2",AUDIO="aud"
video/480p.m3u8
Enter fullscreen mode Exit fullscreen mode

Every attribute earns its place:

  • LANGUAGE - BCP-47 code, used for the label.
  • DEFAULT - plays when the viewer has no preference.
  • AUTOSELECT - may be auto-picked from the OS language.
  • CHANNELS - needed so the player can reason about stereo vs surround.
  • BANDWIDTH on each video variant must include the audio group's bitrate, or your ABR logic works from a wrong total.

2. Author the renditions with FFmpeg

Extract/encode each language's audio, then package. First, encode video-only and audio-only renditions:

# video only (no audio), two ladder rungs
ffmpeg -y -i master_en.mov -an -c:v libx264 -preset veryfast -b:v 2000k -vf scale=-2:720 video_720.mp4
ffmpeg -y -i master_en.mov -an -c:v libx264 -preset veryfast -b:v 1000k -vf scale=-2:480 video_480.mp4

# audio only, per language (AAC stereo, aligned settings)
ffmpeg -y -i master_en.mov -vn -c:a aac -b:a 128k -ac 2 audio_en.m4a
ffmpeg -y -i dub_es.mov    -vn -c:a aac -b:a 128k -ac 2 audio_es.m4a
Enter fullscreen mode Exit fullscreen mode

⚠️ Note: keep the same segment duration across video and every audio rendition. Misaligned boundaries cause gaps and slow desync that won't show up in a 10-second test.

Then segment with a packager that emits grouped audio. Bento4's mp4-dash/mp4hls or Shaka Packager handle this cleanly. Example with Shaka Packager:

packager \
  in=video_720.mp4,stream=video,init_segment=v720/init.mp4,segment_template=v720/$Number$.m4s,playlist_name=video/720p.m3u8 \
  in=video_480.mp4,stream=video,init_segment=v480/init.mp4,segment_template=v480/$Number$.m4s,playlist_name=video/480p.m3u8 \
  in=audio_en.m4a,stream=audio,hls_group_id=aud,hls_name=English,language=en,init_segment=aen/init.mp4,segment_template=aen/$Number$.m4s,playlist_name=audio/en.m3u8 \
  in=audio_es.m4a,stream=audio,hls_group_id=aud,hls_name=Espanol,language=es,init_segment=aes/init.mp4,segment_template=aes/$Number$.m4s,playlist_name=audio/es.m3u8 \
  --hls_master_playlist_output master.m3u8 \
  --segment_duration 4
Enter fullscreen mode Exit fullscreen mode

3. Debug the "zero audio tracks" bug

The most common failure: player loads, video plays with sound, but the switcher is empty.

hls.audioTracks → []   # 😞
Enter fullscreen mode Exit fullscreen mode

This is almost always the manifest, not the player. Open master.m3u8 and check, in this order:

  1. Are there #EXT-X-MEDIA:TYPE=AUDIO lines at all? (Packager misconfig drops them.)
  2. Does every #EXT-X-STREAM-INF have AUDIO="aud"?
  3. Do the GROUP-ID and the variant's AUDIO value match exactly, including case?
  4. Is the version sane (#EXT-X-VERSION:6 for grouped audio)?

Fix the manifest, and the tracks appear. You almost never fix this in JavaScript.

4. Wire the switcher (hls.js v1.7) 

// player.js - hls.js 1.7.x
import Hls from "hls.js";

const video = document.querySelector("#video");
const select = document.querySelector("#audio-picker");

if (Hls.isSupported()) {
  const hls = new Hls();
  hls.loadSource("/stream/master.m3u8");
  hls.attachMedia(video);

  hls.on(Hls.Events.AUDIO_TRACKS_UPDATED, (_evt, data) => {
    select.innerHTML = "";
    data.audioTracks.forEach((track, i) => {
      const opt = document.createElement("option");
      opt.value = String(i);
      opt.textContent = track.name || track.lang || `Track ${i}`;
      if (track.default) opt.selected = true;
      select.appendChild(opt);
    });
  });

  hls.on(Hls.Events.AUDIO_TRACK_SWITCHED, (_evt, data) => {
    console.log("now playing audio track", data.id);
  });

  select.addEventListener("change", (e) => {
    hls.audioTrack = Number(e.target.value); // triggers the switch
  });
} else if (video.canPlayType("application/vnd.apple.mpegurl")) {
  // Safari / native HLS: the OS audio menu is driven by EXT-X-MEDIA. No JS needed.
  video.src = "/stream/master.m3u8";
}
Enter fullscreen mode Exit fullscreen mode

That's the whole player side. On Safari you write zero switching code, the native menu reads your EXT-X-MEDIA tags directly.

Gotchas checklist

  • ✅ Same segment duration across video and all audio renditions.
  • ✅ One codec per audio group. Mixing AAC + EC-3? Use separate groups + matching variants.
  • ✅ Video BANDWIDTH includes audio bitrate.
  • ✅ LANGUAGE is BCP-47 (en, es, pt-BR), not freeform.
  • ✅ Exactly one DEFAULT=YES per group.

5. Validate the manifest in CI

The "zero audio tracks" bug ships when a packaging change silently drops a group reference and no human reads the manifest. Catch it with a tiny parser in CI so a bad master playlist fails the build instead of the player.

// validate-audio-groups.mjs - node 20+
import { readFileSync } from "node:fs";

const master = readFileSync(process.argv[2], "utf8");
const lines = master.split(/\r?\n/);

const groups = new Set();
for (const l of lines) {
  if (l.startsWith("#EXT-X-MEDIA:") && l.includes("TYPE=AUDIO")) {
    const m = l.match(/GROUP-ID="([^"]+)"/);
    if (m) groups.add(m[1]);
  }
}

const errors = [];
for (let i = 0; i < lines.length; i++) {
  if (lines[i].startsWith("#EXT-X-STREAM-INF:")) {
    const m = lines[i].match(/AUDIO="([^"]+)"/);
    if (!m) errors.push(`variant missing AUDIO=: ${lines[i]}`);
    else if (!groups.has(m[1])) errors.push(`AUDIO="${m[1]}" has no matching group`);
  }
}

if (errors.length) {
  console.error("Audio group validation failed:\n" + errors.join("\n"));
  process.exit(1);
}
console.log(`OK: ${groups.size} audio group(s), all variants reference a real group`);
Enter fullscreen mode Exit fullscreen mode 
node validate-audio-groups.mjs dist/master.m3u8
# OK: 1 audio group(s), all variants reference a real group
Enter fullscreen mode Exit fullscreen mode

Run it on every packaged output. It is twenty lines and it prevents the single most common multi-audio outage.

6. fMP4 vs TS, and default-track behavior

Two things that trip people once the basics work:

  • Container: prefer fMP4 (CMAF) audio segments over legacy MPEG-TS. fMP4 plays cleaner across MSE browsers, is required for some codecs, and lets you share segments with a DASH manifest later. The Shaka command above already emits fMP4.
  • Default selection across browsers: DEFAULT=YES plus AUTOSELECT=YES is what most players honor, but Safari weighs the OS language against AUTOSELECT tracks first. If a Spanish-locale device keeps starting on Spanish even when you expect English, that is AUTOSELECT doing its job. Set AUTOSELECT=NO on tracks you never want auto-picked.

💡 Tip: persist the viewer's last chosen track (in app state, not the manifest) and re-apply it on the next load by setting hls.audioTrack after MANIFEST_PARSED. HLS has no concept of "remember my language"; that is your job.

7. Troubleshooting table

Symptom Likely cause Fix
audioTracks is empty Variant missing AUDIO= or group-id mismatch Re-read master playlist; run the CI validator
Audio drifts out of sync after minutes Audio/video segment durations differ Re-segment all renditions with one --segment_duration
Switch works but audio cuts out briefly Init segment reload on switch Update to hls.js 1.7+ (smoother audio-track switching)
Track shows but won't play in one browser Mixed codecs in one group Split codecs into separate groups + matching variants
Wrong default language on some devices AUTOSELECT=YES + OS locale Set AUTOSELECT=NO on non-default tracks

What's next

  • Add a surround track as a second group (hls_group_id=aud-surround, CHANNELS="6") and reference it from higher-bitrate variants.
  • Validate manifests in CI with a parser so a packaging change can't ship an empty audio list.
  • Layer WebVTT subtitle groups the same way (TYPE=SUBTITLES), the grouping model is identical.
  • Bump #EXT-X-VERSION to at least 6 once you use fMP4 segments; older version numbers with modern features are a common source of "plays in Safari, breaks in hls.js" reports.
  • If you serve the same content as DASH too, generate the audio AdaptationSets from the same fMP4 segments so you maintain one set of media files for both manifests.

Top comments (0)