If someone wants a transcript of a 2-hour screen recording, the file on their disk might be 4 GB. But the part a speech-to-text model needs is the sound, and speech models work at 16 kHz mono anyway. At 32 kbps Opus, an hour of mono speech is about 14 MB. Ten hours is about 144 MB.
So for TranscribeLink I stopped asking people to upload the video. The browser pulls the audio track out first and uploads only that. Here's how it works and what I had to get right.
The tools: WebCodecs + Mediabunny
WebCodecs gives the browser access to native audio encoders and decoders (AudioDecoder, AudioEncoder). On its own it doesn't read containers like MP4, MOV, MKV or WebM. For that I use Mediabunny, a small library that demuxes and muxes those formats in the browser and drives WebCodecs for you.
The core of it is a "conversion" that drops the video track and re-encodes the audio:
const input = new Input({ source: new BlobSource(file), formats: ALL_FORMATS });
if (!(await input.getPrimaryAudioTrack())) throw new Error('No audio track');
const output = new Output({ format: new OggOutputFormat(), target: new BufferTarget() });
const conversion = await Conversion.init({
input, output,
video: { discard: true },
audio: { codec: 'opus', numberOfChannels: 1, sampleRate: 16000, bitrate: 32000, forceTranscode: true },
});
conversion.onProgress = (p) => showProgress(p);
await conversion.execute();
const audio = new File([output.target.buffer], 'audio.ogg', { type: 'audio/ogg' });
BlobSource reads the file in slices, so a 5 GB video never has to fit in memory.
Fallbacks matter more than the happy path
Not every browser can encode Opus. I try two targets in order and check each one first with canEncodeAudio:
- Opus, mono, 16 kHz, 32 kbps, in an Ogg file (about 14 MB an hour)
- AAC, mono, 44.1 kHz, 48 kbps, in an MP4 file (about 22 MB an hour)
If the browser has no AudioEncoder at all (older Safari), or neither codec works, nothing breaks: it just uploads the original file. Extraction is an optimisation, never a requirement.
Don't wait for extraction: race it
Extracting audio from a long file takes time, and on a fast connection the original might upload before extraction finishes. So both start at once:
- The original file starts uploading in chunks right away.
- Extraction runs in parallel.
- When extraction finishes, compare: if the bytes still left to upload are more than the size of the extracted audio, stop the upload, tell the server to replace it, and send the audio instead. Otherwise cancel extraction and let the original finish.
const first = await Promise.race([
sending.then((result) => ({ sent: result })),
extraction.promise.then((audio) => ({ audio }), (error) => ({ audio: null, error })),
]);
if (first.audio && file.size * (1 - uploadedFraction) > first.audio.size) {
stopUpload();
await replaceOnServer(first.audio.size);
await upload(first.audio);
}
On a slow connection the audio almost always wins, and that's exactly where it saves the most time.
Small details that caught me
- Resumed uploads keep the original. I resume interrupted uploads by byte offset. Extracted audio isn't byte-identical between runs, so a resumed upload continues with the original file instead of re-extracting.
- Skip tiny files. Below about 1 MB, extraction isn't worth the overhead, so those go up as they are.
- Load the library early. Mediabunny is about 690 KB minified. I start loading it when the user opens the file picker, so it's ready by the time they've chosen a file.
-
Use XMLHttpRequest for chunks.
fetchdoesn't report upload progress, so a big chunk shows 0% until it's done.xhr.upload.onprogresslets the progress bar move smoothly, which matters when people are watching a multi-GB upload.
Result
A long video becomes an audio file of tens of megabytes before it leaves the user's machine. The upload that used to be the slowest step is now usually the shortest, and the server never has to store or process the video.
I use this in TranscribeLink, where you can upload files up to 10 hours or 5 GB, or paste a video link, and get a transcript in 100 languages. If you've done something similar, I'd like to hear how you handled browsers without WebCodecs.
Top comments (2)
The race can upload part of the original video before audio extraction wins, so it is worth telling users that the video may already have left their device. On the server, discard the partial chunks after switching and when an upload is abandoned.
The race between extraction and upload is the smartest part here. Starting the original upload in chunks while Mediabunny extracts in parallel avoids the obvious trap of making a fast connection wait for local processing, and comparing remaining bytes against the extracted size is a clean decision rule. Your fallback order also makes sense: Opus at 16 kHz mono for speech, then AAC when Opus is unavailable, with the original as the last resort. One thing I would check is how the UI explains the switch mid-upload, since a user watching progress might be confused when the total suddenly shrinks from gigabytes to tens of megabytes.