DEV Community

Cover image for # 🎥 Web Media Handling — A Complete Frontend Guide (Video, Audio, Streaming & Recording)
Vishwark
Vishwark

Posted on

# 🎥 Web Media Handling — A Complete Frontend Guide (Video, Audio, Streaming & Recording)

#webdev #javascript #react #frontend

Modern web apps deal with media every day — from playing videos to recording streams and building custom players.
This post walks you through everything you need to play, stream, record, and capture video/audio in the browser — step by step.

🧭 What You’ll Learn

By the end of this guide, you’ll understand:

  • 🎬 How to play static or linked video/audio files
  • 🎨 How to create custom controls and show subtitles
  • 📡 How streaming works (HLS/DASH, bitrates, manifests)
  • 🎧 How to manage multiple audio/subtitle tracks
  • 🖥️ How to capture and record media from camera, mic, or screen
  • 🧠 Which file formats and codecs to use and why
  • 🖼️ How to use Canvas to capture video frames
  • ⚙️ Key HTML attributes and APIs (<video>, <audio>, MediaRecorder, etc.)

🎬 VIDEO HANDLING

Let’s start with how to play and control video in your web app.

▶️ Playing Video

1. Play Static File 

<video src="/assets/sample.mp4" controls></video>
Enter fullscreen mode Exit fullscreen mode

2. Play by Link 

<video src="https://example.com/video.mp4" controls></video>
Enter fullscreen mode Exit fullscreen mode

⚠️ CORS Alert:
If your video is hosted on another domain, the server must allow cross-origin access using headers like:

Access-Control-Allow-Origin: https://yourwebsite.com

🎨 Control Customization

Show or Hide Controls 

<!-- Default browser controls -->
<video src="movie.mp4" controls></video>

<!-- Hide controls (for custom UI) -->
<video src="movie.mp4"></video>
Enter fullscreen mode Exit fullscreen mode

Enable Download 

<a href="/assets/video.mp4" download>Download Video</a>
Enter fullscreen mode Exit fullscreen mode

Custom Controls Example 

<video id="myVideo" src="video.mp4"></video>
<button onclick="document.getElementById('myVideo').play()">Play</button>
<button onclick="document.getElementById('myVideo').pause()">Pause</button>
Enter fullscreen mode Exit fullscreen mode

💡 Tip: Support keyboard shortcuts like Space (play/pause) or ↑/↓ (volume) for accessibility.

🌐 Streaming Video

Streaming allows smooth playback without downloading the entire file upfront.

How It Works

  1. The backend stores multiple versions of the same video at different bitrates.
  2. The player fetches small chunks (.ts or .mp4 segments).
  3. Based on network speed, it automatically adjusts quality.

Common Streaming Formats

Format Extension Used For Frontend Library
HLS .m3u8 Apple devices / most browsers hls.js
MPEG-DASH .mpd Android / Smart TVs dash.js

Example (Using HLS.js) 

const video = document.getElementById('video');
if (Hls.isSupported()) {
  const hls = new Hls();
  hls.loadSource('https://example.com/video.m3u8');
  hls.attachMedia(video);
}
Enter fullscreen mode Exit fullscreen mode

🎧 Multi-Track Support

Multiple Audio Tracks

  • Supported in MP4 and MKV formats.
  • Access via the audioTracks API.

Multiple Subtitles

Keep separate .vtt files for each language:

<video controls>
  <source src="movie.mp4" type="video/mp4">
  <track src="subtitles_en.vtt" kind="subtitles" srclang="en" label="English">
  <track src="subtitles_es.vtt" kind="subtitles" srclang="es" label="Spanish">
</video>
Enter fullscreen mode Exit fullscreen mode

🧩 Useful Attributes

Attribute Description
poster="thumbnail.jpg" Shows a preview image before playback
preload="none", preload="metadata", preload="auto" Controls how the browser preloads video data
controlsList="nodownload" Disables the default download button in the browser’s video controls
crossorigin="anonymous" Enables cross-origin requests for external video sources (required when using subtitles or CORS-enabled videos)

🎞️ Video Codecs & Formats

Format Codec Notes
MP4 H.264 Most common, supported everywhere
WebM VP8/VP9 Open-source, efficient
MKV H.264/HEVC For multi-track videos
AVI — Old, rarely used now

✅ Best Choice: MP4 (H.264) for broad compatibility and quality balance.

🎵 AUDIO HANDLING

▶️ Playing Audio

1. Static File 

<audio src="/assets/song.mp3" controls></audio>
Enter fullscreen mode Exit fullscreen mode

2. By Link 

<audio src="https://example.com/song.mp3" controls></audio>
Enter fullscreen mode Exit fullscreen mode

3. Enable Download 

<a href="/assets/song.mp3" download>Download Audio</a>
Enter fullscreen mode Exit fullscreen mode

🎨 Custom Controls 

<audio id="myAudio" src="song.mp3"></audio>
<button onclick="document.getElementById('myAudio').play()">Play</button>
<button onclick="document.getElementById('myAudio').pause()">Pause</button>
Enter fullscreen mode Exit fullscreen mode

📡 Streaming Audio

Audio streaming also uses HLS or DASH formats.

  • Backend stores multiple bitrates.
  • Player adapts based on bandwidth.
  • You can include lyrics/subtitles using <track>. 
<audio controls>
  <source src="song.mp3" type="audio/mp3">
  <track src="lyrics_en.vtt" kind="subtitles" srclang="en" label="English">
</audio>
Enter fullscreen mode Exit fullscreen mode

🎧 Common Audio Formats

Format Type Use Case
MP3 Lossy Most supported
AAC / M4A Lossy Better compression, ideal for streaming
OGG Open-source Not supported on Safari
FLAC Lossless High quality, large size

✅ Preferred: MP3 (for support) or AAC (for quality).

📡 STREAMING OVERVIEW

Here’s how adaptive streaming works behind the scenes 👇

  1. The backend stores video/audio in different bitrates.
  2. The player downloads small chunks as per current network speed.
  3. The manifest file (.m3u8 or .mpd) defines all available versions.
  4. Subtitles and audio tracks are synced via timestamps in the manifest.

🎥 RECORDING & CAPTURING MEDIA

Modern browsers let you capture or record directly — no plugins required.

📸 Capture Photo (Webcam + Canvas Frame)

You can capture a live photo (frame) from a video stream using the <canvas> element.

<video id="camera" autoplay playsinline width="400" height="300"></video>
<canvas id="snapshot" width="400" height="300"></canvas>
<button id="capture">📸 Capture Photo</button>

<script>
  // Step 1: Start webcam stream
  navigator.mediaDevices.getUserMedia({ video: true })
    .then(stream => {
      document.getElementById('camera').srcObject = stream;
    })
    .catch(err => console.error('Camera access denied:', err));

  // Step 2: Capture a frame on click
  document.getElementById('capture').addEventListener('click', () => {
    const video = document.getElementById('camera');
    const canvas = document.getElementById('snapshot');
    const ctx = canvas.getContext('2d');

    // Draw current frame on canvas
    ctx.drawImage(video, 0, 0, canvas.width, canvas.height);

    // Optional: Convert to image
    const imageData = canvas.toDataURL('image/png');
    console.log('Captured frame:', imageData);

    // Download captured frame
    const link = document.createElement('a');
    link.href = imageData;
    link.download = 'photo.png';
    link.click();
  });
</script>
Enter fullscreen mode Exit fullscreen mode

🧠 How it works:

  • The <video> displays live feed.
  • <canvas> takes a snapshot using drawImage().
  • You can save it, preview it, or send to your backend.

🎥 Capture Video + Audio 

navigator.mediaDevices.getUserMedia({ video: true, audio: true })
  .then(stream => {
    document.querySelector('video').srcObject = stream;
  });
Enter fullscreen mode Exit fullscreen mode

🖥️ Capture Screen or Tab 

navigator.mediaDevices.getDisplayMedia({ video: true, audio: true })
  .then(stream => {
    document.querySelector('video').srcObject = stream;
  });
Enter fullscreen mode Exit fullscreen mode

🔴 Record the Stream 

const recorder = new MediaRecorder(stream);
const chunks = [];

recorder.ondataavailable = e => chunks.push(e.data);
recorder.onstop = () => {
  const blob = new Blob(chunks, { type: 'video/webm' });
  const url = URL.createObjectURL(blob);
  document.querySelector('video').src = url;
};

recorder.start();
Enter fullscreen mode Exit fullscreen mode

🔗 Sending or Combining Streams

  • Send as Stream: Use WebRTC or WebSocket for live transfer.
  • Send as File: Upload the recorded Blob/File to backend.
  • Combine Screen + Video: Capture both and merge via Canvas or MediaStreamTrack.

🧠 Final Notes

✅ Play local and linked video/audio
✅ Build custom players and controls
✅ Stream using HLS/DASH
✅ Manage subtitles and multi-audio tracks
✅ Capture frames using Canvas
✅ Record from webcam, mic, or screen
✅ Choose the right formats and codecs

💡 Bonus Tips

  • Use preload="metadata" for faster page loads
  • Whitelist domains for CORS when using video links
  • Use CDNs for smooth video delivery
  • Test across browsers and devices
  • Use Canvas for filters or real-time frame capture

🏁 Quick Summary

Topic API/Tag Description
Video Playback <video> Play, control, preload, poster
Audio Playback <audio> Play with captions or custom UI
Streaming HLS/DASH Adaptive bitrate playback
Subtitles <track> For multiple languages
Capture getUserMedia() Access webcam/mic
Screen Share getDisplayMedia() Capture tab/window
Frame Capture <canvas> Snapshot from video stream
Recording MediaRecorder Record and export video/audio

🚀 Wrap-Up

Building rich media experiences in the browser is easier than ever.
With HTML5 media elements and powerful APIs like MediaRecorder, getUserMedia, and <canvas>, you can create experiences similar to YouTube or Zoom — right inside your web app.

Top comments (0)