Extracting Audio from Video in the Browser using React and FFmpeg WASM

In today's web development landscape, performing complex media operations directly in the browser has become increasingly possible. One such operation is extracting audio from video files without server-side processing. This blog post will guide you through implementing an audio extraction feature in a React application using FFmpeg WASM.

What We'll Build

We'll create a React application that allows users to:

1. Upload an MP4 video file
2. Extract the audio from the video
3. Download the extracted audio as an MP3 file

All of this will happen client-side, leveraging the power of WebAssembly through FFmpeg WASM.

Prerequisites

To follow along, you should have:

• Basic knowledge of React and TypeScript
• Node.js and npm installed on your machine
• A code editor of your choice

Setting Up the Project

First, let's set up a new React project using Vite:

npm create vite@latest audio-extractor -- --template react-ts
cd audio-extractor
npm install

Next, install the required dependencies:

npm install @ffmpeg/ffmpeg@0.12.10 @ffmpeg/util@0.12.1

Implementing the Audio Extractor

Let's break down the implementation into steps:

1. Importing Dependencies

import React, { useState, useRef } from 'react'
import { FFmpeg } from '@ffmpeg/ffmpeg'
import { toBlobURL, fetchFile } from '@ffmpeg/util'

We import the necessary React hooks and FFmpeg utilities.

2. Setting Up State and Refs

const [loaded, setLoaded] = useState(false)
const [videoFile, setVideoFile] = useState<File | null>(null)
const ffmpegRef = useRef(new FFmpeg())
const messageRef = useRef<HTMLParagraphElement | null>(null)

We use state to track whether FFmpeg is loaded and to store the selected video file. Refs are used for the FFmpeg instance and a message display element.

3. Loading FFmpeg

const load = async () => {
  const baseURL = 'https://unpkg.com/@ffmpeg/core-mt@0.12.6/dist/esm'
  const ffmpeg = ffmpegRef.current
  ffmpeg.on('log', ({ message }) => {
    if (messageRef.current) messageRef.current.innerHTML = message
  })
  await ffmpeg.load({
    coreURL: await toBlobURL(`${baseURL}/ffmpeg-core.js`, 'text/javascript'),
    wasmURL: await toBlobURL(`${baseURL}/ffmpeg-core.wasm`, 'application/wasm'),
    workerURL: await toBlobURL(`${baseURL}/ffmpeg-core.worker.js`, 'text/javascript'),
  })
  setLoaded(true)
}

This function loads the FFmpeg WASM core and sets up logging.

4. Extracting Audio

const extractAudio = async () => {
  if (!videoFile) {
    alert('Please select an MP4 file first')
    return
  }
  const ffmpeg = ffmpegRef.current
  await ffmpeg.writeFile('input.mp4', await fetchFile(videoFile))
  await ffmpeg.exec(['-i', 'input.mp4', '-vn', '-acodec', 'libmp3lame', '-q:a', '2', 'output.mp3'])
  const data = await ffmpeg.readFile('output.mp3')
  const audioBlob = new Blob([data], { type: 'audio/mp3' })
  const audioUrl = URL.createObjectURL(audioBlob)
  const link = document.createElement('a')
  link.href = audioUrl
  link.download = 'extracted_audio.mp3'
  document.body.appendChild(link)
  link.click()
  document.body.removeChild(link)
}

This function handles the audio extraction process using FFmpeg commands and creates a download link for the extracted audio.

5. Handling File Selection

const handleFileChange = (event: React.ChangeEvent<HTMLInputElement>) => {
  if (event.target.files && event.target.files[0]) {
    const file = event.target.files[0]
    if (file.type === 'video/mp4') {
      setVideoFile(file)
    } else {
      alert('Please select an MP4 file.')
      event.target.value = ''
    }
  }
}

This function ensures that only MP4 files are selected.

Configuring Vite

To ensure proper functioning of FFmpeg WASM, we need to configure Vite. Create a vite.config.ts file in your project root:

import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react-swc'

export default defineConfig({
  plugins: [react()],
  optimizeDeps: {
    exclude: ['@ffmpeg/ffmpeg', '@ffmpeg/util'],
  },
  server: {
    headers: {
      'Cross-Origin-Opener-Policy': 'same-origin',
      'Cross-Origin-Embedder-Policy': 'require-corp',
    },
  },
})

This configuration excludes FFmpeg from dependency optimization and sets necessary headers for WASM to work correctly.

Conclusion

We've successfully implemented a browser-based audio extraction feature using React and FFmpeg WASM. This approach allows for efficient client-side processing of media files, reducing server load and improving user experience.

By leveraging WebAssembly technology, we can bring powerful media manipulation capabilities directly to the browser, opening up new possibilities for web-based media applications.

Full Code Reference

Here's the complete code for the AudioExtractor component:

import React, { useState, useRef } from 'react'
import { FFmpeg } from '@ffmpeg/ffmpeg'
import { toBlobURL, fetchFile } from '@ffmpeg/util'

const AudioExtractor: React.FC = () => {
  const [loaded, setLoaded] = useState(false)
  const [videoFile, setVideoFile] = useState<File | null>(null)
  const [message, setMessage] = useState('')
  const ffmpegRef = useRef(new FFmpeg())

  const load = async () => {
    const baseURL = 'https://unpkg.com/@ffmpeg/core-mt@0.12.6/dist/esm'
    const ffmpeg = ffmpegRef.current
    ffmpeg.on('log', ({ message }) => {
      setMessage(message)
    })
    await ffmpeg.load({
      coreURL: await toBlobURL(`${baseURL}/ffmpeg-core.js`, 'text/javascript'),
      wasmURL: await toBlobURL(`${baseURL}/ffmpeg-core.wasm`, 'application/wasm'),
      workerURL: await toBlobURL(`${baseURL}/ffmpeg-core.worker.js`, 'text/javascript'),
    })
    setLoaded(true)
  }

  const extractAudio = async () => {
    if (!videoFile) {
      alert('Please select an MP4 file first')
      return
    }
    const ffmpeg = ffmpegRef.current
    await ffmpeg.writeFile('input.mp4', await fetchFile(videoFile))
    await ffmpeg.exec([
      '-i',
      'input.mp4',
      '-vn',
      '-acodec',
      'libmp3lame',
      '-q:a',
      '2',
      'output.mp3',
    ])
    const data = await ffmpeg.readFile('output.mp3')
    const audioBlob = new Blob([data], { type: 'audio/mp3' })
    const audioUrl = URL.createObjectURL(audioBlob)
    const link = document.createElement('a')
    link.href = audioUrl
    link.download = 'extracted_audio.mp3'
    document.body.appendChild(link)
    link.click()
    document.body.removeChild(link)
  }

  const handleFileChange = (event: React.ChangeEvent<HTMLInputElement>) => {
    if (event.target.files && event.target.files[0]) {
      const file = event.target.files[0]
      if (file.type === 'video/mp4') {
        setVideoFile(file)
      } else {
        alert('Please select an MP4 file.')
        event.target.value = ''
      }
    }
  }

  return (
    <div className="my-8 rounded-lg border bg-gray-50 p-6">
      <h2 className="mb-4 text-2xl font-semibold">Audio Extractor</h2>
      {!loaded ? (
        <button
          onClick={load}
          className="rounded bg-blue-500 px-4 py-2 font-bold text-white hover:bg-blue-700"
        >
          Load FFmpeg
        </button>
      ) : (
        <>
          <input type="file" accept="video/mp4" onChange={handleFileChange} className="mb-4" />
          <br />
          <button
            onClick={extractAudio}
            disabled={!videoFile}
            className="rounded bg-green-500 px-4 py-2 font-bold text-white hover:bg-green-700 disabled:opacity-50"
          >
            Extract Audio
          </button>
          <p className="mt-4 text-sm text-gray-600">{message}</p>
        </>
      )}
    </div>
  )
}

export default AudioExtractor