Sending an HD MP4 while it's still being merged: named pipes, ffmpeg stream copy and fragmented MP4

YT Video Downloader — Fri, 13 Nov 2026 08:17:53 +0000

When you build a video download tool, the first HD file teaches you something annoying. Above 360p, YouTube doesn't serve one file with picture and sound. It serves a video-only stream and a separate audio-only stream, and your tool has to put them back together into one MP4.

The obvious way to do that is slow:

Download the whole video stream to disk.
Download the whole audio stream to disk.
Run ffmpeg to merge them.
Send the finished file to the user.

For a 10 minute 1080p video that means a few hundred MB written to disk, a progress bar the user has to stare at, and a server that needs scratch space for every download running at once.

For YT Video Downloader we wanted the browser's download to start right away, with no file ever touching the disk. Here's how that works, with the actual code.

The idea

ffmpeg doesn't need files. It needs two inputs it can read from and one output it can write to. So we give it:

two named pipes (FIFOs) as inputs, one for video and one for audio
stdout as the output
-c copy, so nothing is re-encoded

Our Python server fills the two pipes from the source streams and forwards whatever ffmpeg writes to stdout straight into the HTTP response. The user's browser receives MP4 bytes while the merge is still happening.

Step 1: two pipes and an ffmpeg process

d = tempfile.mkdtemp(prefix="yt", dir=FIFO_ROOT)
vp, ap = os.path.join(d, "v"), os.path.join(d, "a")
os.mkfifo(vp)
os.mkfifo(ap)

proc = await asyncio.create_subprocess_exec(
    "ffmpeg", "-hide_banner", "-loglevel", "error", "-nostdin",
    "-i", vp, "-i", ap,
    "-map", "0:v:0", "-map", "1:a:0", "-c", "copy",
    "-movflags", "frag_keyframe+empty_moov+default_base_moof",
    "-f", "mp4", "pipe:1",
    stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE)

The important part is -movflags frag_keyframe+empty_moov. A normal MP4 keeps its index (the moov box) at the end of the file, or moves it to the front in a second pass. Both need the whole file first. A fragmented MP4 writes an empty moov up front and then a series of small self-contained fragments, so ffmpeg can emit playable bytes as soon as it has the first keyframe.

-c copy means ffmpeg only rewraps the existing H.264 and AAC data. CPU use stays tiny, and the output is bit for bit the quality of the source.

Step 2: feeding a pipe without blocking the event loop

Opening a FIFO for writing blocks until someone opens it for reading. In asyncio you don't want that, so we open it non-blocking and retry until ffmpeg has opened its side:

async def open_fifo_writer(path, timeout=30):
    loop = asyncio.get_running_loop()
    deadline = loop.time() + timeout
    while True:
        try:
            fd = os.open(path, os.O_WRONLY | os.O_NONBLOCK)
            break
        except OSError as e:
            if e.errno != errno.ENXIO or loop.time() > deadline:
                raise
            await asyncio.sleep(0.05)
    f = os.fdopen(fd, "wb", buffering=0)
    transport, protocol = await loop.connect_write_pipe(asyncio.streams.FlowControlMixin, f)
    return asyncio.StreamWriter(transport, protocol, None, loop)

ENXIO is what you get when nobody is reading yet. Once it opens, connect_write_pipe gives us a normal StreamWriter, which means we get backpressure for free:

async def feed(path, url, size):
    writer = await open_fifo_writer(path)
    try:
        async for chunk in fetch_in_ranges(url, size):
            writer.write(chunk)
            await writer.drain()
    except (BrokenPipeError, ConnectionResetError):
        return
    finally:
        writer.close()

await writer.drain() pauses the download when ffmpeg isn't reading fast enough. That matters because ffmpeg reads the two inputs in step with each other. If the video feeder raced ahead without backpressure, it would fill memory with data ffmpeg doesn't want yet.

Step 3: fetching the source in ranges, with resume

The source streams are plain HTTPS. Instead of one long request, we fetch 10 MB ranges. If a range drops halfway, we retry from the last byte we actually received, not from the start:

CHUNK = 10 * 1024 * 1024

async def fetch_in_ranges(url, total):
    pos = 0
    while pos < total:
        end = min(pos + CHUNK, total) - 1
        tries = 0
        while True:
            start_pos = pos
            try:
                async with client.stream("GET", url, headers={"Range": f"bytes={pos}-{end}"}) as r:
                    async for chunk in r.aiter_bytes(256 * 1024):
                        pos += len(chunk)
                        yield chunk
                break
            except (httpx.TransportError, httpx.TimeoutException):
                tries = tries + 1 if pos == start_pos else 1
                if tries >= 3:
                    raise
                await asyncio.sleep(tries)

The retry counter resets whenever progress was made, so a long download on a flaky link survives several short drops, but a link that's really dead fails after three tries.

Step 4: wiring it to the response

The two feeders run as tasks, and the response body is just ffmpeg's stdout:

feeders = [asyncio.create_task(feed(vp, video_url, video_size)),
           asyncio.create_task(feed(ap, audio_url, audio_size))]
try:
    while True:
        chunk = await proc.stdout.read(256 * 1024)
        if not chunk:
            break
        yield chunk
    rc = await proc.wait()
finally:
    for t in feeders:
        t.cancel()
    shutil.rmtree(d, ignore_errors=True)

The finally block matters more than it looks. If the user cancels the download, the client disconnects, the generator is closed, the feeders get cancelled, ffmpeg loses its inputs and exits, and the temp directory holding the FIFOs is removed. No orphaned processes, no leftover files.

The trade-offs

This isn't free, and it's worth being honest about what you give up:

No exact Content-Length. We don't know the final size until ffmpeg finishes, so the browser shows the download without a total. We show an estimate on the page before the user clicks, and a note explaining why the browser can't.
No resuming a half-finished download. The file is being built on the fly, so a byte-range request for the second half has nothing to point at. If a download breaks, it starts again.
Fragmented MP4 is slightly unusual. Most players and editors open it without trouble, but the layout differs from a classic MP4, so it's worth testing the ones your users rely on.

For a download tool, those are easy trades. The user gets a file that starts immediately, the server never stores a copy of anyone's video, and the disk usage per download is zero.

Small things that helped

Keep the FIFOs in their own directory, so a hard-killed process leaves nothing scattered around the filesystem.
Set -loglevel error. ffmpeg's stderr is a pipe too, and if it fills with progress logs nobody reads, ffmpeg stalls.
Pick formats that fit in an MP4 container (H.264 video, AAC audio) before you start. -c copy can't put VP9 or Opus into an MP4 that plays everywhere.

We also open-sourced the small parser we use to turn whatever people paste into a clean video ID: youtube-link-parser. It handles youtu.be links with tracking parameters, Shorts, live links, timestamps and links buried in pasted text.

If you've solved the Content-Length problem for live-muxed files in a cleaner way, I'd like to hear how.

DEV Community: YT Video Downloader