When I started building a MOV and MKV to MP4 converter for SquishyFile, my first version did what most online converters do. It decoded every frame and encoded it again as H.264. A 1 GB iPhone clip took minutes, the fan spun up, and the output looked slightly worse than the input.
Then I looked at what was actually inside those files. An iPhone MOV holds H.264 or HEVC video with AAC audio. A typical MKV holds H.264 with AAC, AC-3 or Opus. MP4 can carry H.264, HEVC, AAC and MP3 directly. So for most files, the "conversion" only needs to move the existing packets into a new box. Nothing needs decoding.
That one realization turned the tool from a slow transcoder into a remuxer that only re-encodes when it has to. This post covers how it decides.
Copy, encode, or drop
Every stream gets one of three actions:
export type StreamAction = 'copy' | 'encode' | 'none';
The planner is a pure function. It knows nothing about browsers or encoders, so I can test it with plain Node. It reads the probe result and returns a plan:
const KEEP_VIDEO = new Set(['avc', 'hevc']); // copied as-is
const H264_VIDEO = new Set(['avc']); // copied in "plays everywhere" mode
const COPY_AUDIO = new Set(['aac', 'mp3']);
const copyable = mode === 'keep' ? KEEP_VIDEO : H264_VIDEO;
const video = source.videoCodec && copyable.has(source.videoCodec) ? 'copy' : 'encode';
let audio: StreamAction = 'none';
if (source.hasAudio) {
audio = source.audioCodec && COPY_AUDIO.has(source.audioCodec) ? 'copy' : 'encode';
}
VP9, AV1, ProRes and old MPEG-4 Part 2 video get re-encoded. So do AC-3, DTS, FLAC, Opus, Vorbis and PCM audio. Everything else passes straight through.
The one choice I left to the user: HEVC
An HEVC MP4 is a perfectly valid file. It plays on iPhones, Macs, recent Android phones and new PCs. But plenty of Windows machines without the HEVC extension, older smart TVs and some video editors refuse to open it.
I couldn't pick the right answer for everyone, so the tool shows exactly one setting. "Keep" copies HEVC untouched and finishes in seconds. "H.264" re-encodes it so the MP4 opens anywhere. Every other decision happens automatically.
AC-3 is legal in MP4, and it still plays silent
This one surprised me. AC-3 (Dolby Digital) is allowed inside MP4, and a TV will play it fine. But no desktop browser decodes it. If I copied AC-3 audio into the MP4, the user would open the result in Chrome or Firefox and hear nothing.
So AC-3 and E-AC-3 get re-encoded to AAC, even though copying them would technically produce a "correct" file. I care more about the file working where people actually open it.
When a re-encode does happen, I cap the bitrate
For video that needs encoding, I size the bitrate from bits per pixel per frame:
const H264_BPP = 0.12;
const SOURCE_BITRATE_HEADROOM = 1.6;
const bppBitrate = width * height * frameRate * H264_BPP;
const sourceBitrate = (totalBytes * 8) / durationSec;
const videoBitrate = Math.max(
300_000,
Math.min(bppBitrate, sourceBitrate * SOURCE_BITRATE_HEADROOM)
);
The cap matters more than the formula. HEVC, VP9 and AV1 squeeze the same picture into fewer bits than H.264, so an H.264 copy needs some extra bitrate. Without a ceiling, a lean 3 Mbps phone clip could balloon into a 7 Mbps MP4 that looks no better. I let it grow to 1.6x the source and no further.
Two engines, and the fast one goes first
The fast path uses Mediabunny, which reads QuickTime and Matroska and writes MP4. For a copy, I just leave the codec unset and it moves packets without touching a frame. When something does need encoding, it uses the browser's WebCodecs encoders.
Before starting, I check that the browser can actually encode what the plan needs:
const codec = await getFirstEncodableAudioCodec(['aac'], {
bitrate: plan.audioBitrate,
numberOfChannels: plan.audioChannels
});
if (codec === null) return false; // hand off to ffmpeg.wasm
I added this check after a bug. A browser without an AAC encoder made Mediabunny silently drop the audio track, and the user got a silent MP4 with no error. Now I also inspect the tracks the conversion discarded. If any track got dropped for a reason other than my own discard, I throw and fall back to ffmpeg.wasm, which can decode AC-3 and DTS.
The ffmpeg path is slower and loads the whole file into memory, so it tops out around what a tab can allocate (roughly 2 GB). That's why it only runs when the fast path gives up.
Two small details that cost me an afternoon each
Audio that starts before zero. QuickTime files usually start the AAC track a few milliseconds before t=0, with an edit list hiding the encoder's priming samples. By default Mediabunny trims everything before zero, and trimming audio means decoding and re-encoding it. I start the output at the earliest timestamp instead, which keeps the copy pure and keeps audio in sync.
The hvc1 tag. When ffmpeg copies HEVC out of an MKV, it tags the stream hev1 by default. Apple's players want hvc1 and won't open the file otherwise. One flag fixes it:
if (plan.sourceVideoCodec === 'hevc') args.push('-tag:v', 'hvc1');
I also write the moov atom at the front of the file (fastStart in Mediabunny, +faststart in ffmpeg), so the MP4 starts playing before it's fully read.
What I'd tell anyone building a converter
Probe first, and only then decide what to do. The "convert" button in most tools re-encodes everything because that's the one code path guaranteed to work. But for container changes, re-encoding is usually the expensive and lossy choice, and the cheap and lossless one sits right there.
If you want to see it in action, the tool runs in the browser at squishyfile.com/mov-to-mp4, and the same engine handles MKV to MP4. The result panel tells you which streams it copied and which it re-encoded, which is a decent way to find out what's actually inside your files.
Top comments (2)
Two things the codec table doesn't cover.
Sample entry tags: an HEVC stream copied into MP4 keeps its hev1 sample entry by default, and hev1 expects parameter sets in-band on every keyframe. Apple players and QuickTime generally want hvc1 instead, where the parameter sets live in the container. Same bitstream, valid file, silent failure on iOS. So "keep HEVC" is really "copy HEVC and retag it to hvc1", which means the planner's output isn't just a codec decision but a container-level one.
Multi-track audio: an iPhone MOV commonly carries two tracks (stereo mix plus a mono one), and a naive "take the first audio stream" picks whichever the encoder put first, not necessarily the one the user wants. The probe result has to expose track count and language metadata, otherwise the copy path silently drops the second track.
One question: in the bpp cap, does totalBytes include the audio and container overhead? On a clip where video is 90% of the bytes that is close, but for a phone recording with a stereo + mono pair it shifts the ceiling by a few percent, and since the ceiling is what stops the ballooning, it is the one number worth being careful about.
I like that the result tells people what was copied and what was changed. Nice work on SquishyFile!