DEV Community: Aleksander mako

The Surprisingly Expensive Problem Behind MP4 Fast Start

Aleksander mako — Thu, 17 Sep 2026 14:46:00 +0000

I have been working on a browser-based video editor for a while now, and as a result I have run into plenty of video-related problems. While documenting as many of them as I could remember, I came back to one of my favourites: a problem for which I really struggled to find a satisfying solution.

Simply stated, I discovered that my implementation could require roughly twice the size of a video file in RAM just to optimize an MP4 for fast start.

Optimizing a video for fast start matters when serving video on the web because it allows playback to begin before the entire file has been downloaded. This can improve the user experience and, when the video is important to the initial page render, can also affect loading performance such as Largest Contentful Paint.

Fast start itself is not a Google ranking factor, although Core Web Vitals are used by Google's ranking systems.

But what exactly does fast start mean?

In a normal MP4, the file contains a box called moov.

You can think of it as the index of the video. It contains metadata that tells a player where the encoded samples are located, how large they are, when they should be decoded and presented, and which samples are keyframes.

For fast start, we want that index near the beginning of the file rather than at the end.

FFmpeg's faststart option does exactly this by running a second pass and moving the moov box to the beginning.

Without this optimization, a player may need to fetch the end of the file before it has enough information to start playback. With moov near the front, the player can discover the structure of the video early and begin requesting and decoding the media it needs.

To understand why moving this box is more complicated than it sounds, we need a few MP4 fundamentals.

A few MP4 fundamentals

An MP4 is made up of boxes. A real file can contain many of them, but three are particularly useful for this explanation:

ftyp, which describes the file format and compatibility;
mdat, which contains the encoded media data;
moov, which contains the metadata describing that media.

There is another complication: encoded video is not necessarily processed in the same order in which we eventually watch it.

A video stream can have a decode order and a presentation order. Some frames depend on other frames, including frames that may appear later in playback. The decoder therefore sometimes needs to process frames in one order and display them in another.

The container metadata is what allows all of this to be reconstructed.

Among other things, the tables inside moov tell the demuxer where samples live in the file and how they relate to time. This is why we cannot simply jump to an arbitrary byte offset, assume it contains H.264, and start decoding meaningful video from there.

A straightforward MP4 muxer can therefore do something very convenient: write the media data first, learn where everything ended up, and write the moov metadata afterwards.

By then all of the offsets and timing information are known.

Unfortunately, that gives us exactly the layout we do not want for fast start.

So we move moov to the front.

Moving `moov` breaks the offsets

And this creates an amusing problem: moov itself contains offsets pointing to the media data.

Once we insert moov before that media data, those offsets are no longer correct.

Suppose some media previously started at:

header + X

After inserting a moov box of size M before it, that same data is now roughly at:

header + M + X

The relevant chunk offsets inside the metadata therefore have to be rewritten to point to their new positions.

This is an important detail of the format: chunk offsets are absolute file offsets, so moving metadata in front of the media changes them.

The algorithm itself is conceptually simple:

Read the existing structure.
Move moov.
Adjust the affected offsets.
Write the rearranged MP4.

Where the new file gets written

The interesting problem for me was where the new file gets written.

The usual fast-start implementation is a second pass that produces rearranged output. If the original file is kept while the new one is being written, peak storage approaches the size of the input plus the size of the output.

In other words, roughly twice the video size.

On a normal machine, that usually means temporary disk usage.

I was doing it with an FFmpeg WASM build whose filesystem lived in WASM-backed memory.

That changed the problem completely.

By the time the last byte of the optimised video had been written, I could have both the original file and almost an entire second copy sitting in memory at the same time.

In practice, the process needs additional working memory as well, so the true peak can be even higher.

For a 100 MB video, this is annoying.

For a multi-gigabyte video, it becomes an architectural problem.

And because this editor runs entirely inside the browser, the machine doing the work belongs to the user. Memory is limited, browser environments have their own constraints, and the situation becomes even more restrictive on mobile devices, which I also want to support.

That was the part I originally underestimated.

Moving one piece of metadata from the end of a file to the beginning sounds almost trivial.

Doing it when the file might itself be larger than the memory available to your application is a very different problem.

I am glad I ran into it, because otherwise I probably would never have dug this far into how MP4 files are actually laid out.

What I did instead

I eventually reworked the pipeline around WebCodecs together with browser file APIs, which allowed me to rely on file-backed storage instead of keeping the entire transformation inside FFmpeg WASM's in-memory filesystem.

WebCodecs itself handles encoding and decoding; browser file APIs are what provide the file-backed part.

That solution, though, deserves its own article.

How to Scrub Huge Video Files in the Browser Without Blowing Up RAM

Aleksander mako — Wed, 16 Sep 2026 14:45:00 +0000

Vidstudio is a multi track video editor that works entirely inside the browser without any back end services supporting or doing any kind of heavy lifting. One of the most resource intensive features of an editor is in fact the timeline, because users need to be able to load assets and scrub back and forth along it. This article gives an overview of how the timeline works and how the RAM consumption is kept under control.
Video editing is notoriously RAM intensive, and part of that difficulty is a question with no comfortable answer: how do you let a user preview a file that is larger than the total available RAM, and still let them ask for gigabyte 1 or gigabyte 100 of it, without crashing the system? Under these circumstances the only way to not crash the system is to not load the entire file into RAM but stream it instead. That solves the memory problem and immediately creates a different one. For a requested time instant, instead of just drawing the frame we now need to run the whole pipeline that yields the frame to be drawn. Everything below is me buying that latency back.
The pipeline which yields a frame starts by reading a chunk of the file at the requested position, which is encoded, so we need to decode it and then display it in a two dimensional canvas in the preview. The next time the user moves the ruler ahead or back in time we need to start the process all over again. This naive implementation works, but it is not going to be efficient.
The first source of inefficiency we can tackle is the I/O round trip to disk to fetch encoded chunks, in the case where the user happened to scrub nearby rather than really far ahead. The way to do this is to have a sample cache which responds to requests for chunks out of RAM. Caching encoded samples works fairly well here because the encoded samples are not full frames and therefore cost a fraction of the RAM to hold. The arithmetic is worth doing once: a 1080p frame sitting in a canvas is 1920 by 1080 by 4 bytes, so roughly 8MB, while the encoded packet it came out of is more likely to be a few tens of kilobytes. Two orders of magnitude for the same moment in the video, which is why you can afford to keep a lot of one and very little of the other.
The next optimization to go after is keeping some decoded frames in RAM as well, in the proximity of the sought position, in case the user seeks nearby, since decoding is not free either. Here the 8MB figure sets the budget, so the window stays deliberately narrow. If they seek beyond your cached bounds then you have no choice but to start the pipeline again.
Restarting the pipeline is not cheap, because once more you have to reach to disk to fetch the chunks, decode, possibly apply more effects and then draw, so we should only do it when it is really necessary. And it is in fact likely that users will seek to truly random positions along the timeline, in short succession of one another. Here we can do something smart: the last seek wins. Vidstudio does this by debouncing the seek operation for a few ms and tracking the id of the most recently started decode operation, then cancelling anything that started before it. That way only the latest seek operation ends up consuming resources on the end user's device.
For a client side application, being frugal with the end user's resources matters, but so does the experience, and the two are in direct competition. Every one of the decisions above is the same trade made in a different place. Cache the cheap thing generously, cache the expensive thing narrowly, and throw away work the user has already changed their mind about. What the timeline does not do yet is anything clever about where in the file it starts decoding from, which turns out to be the largest cost of all. That one needs its own article.
If you want to check out the project

Online Video Editor (No Upload, No Signup) | VidStudio

Free online video editor that runs in your browser. Multi-track timeline, frame-accurate seek, powered by WebCodecs and FFmpeg WASM. Your files never leave your device.

vidstudio.app

Free Text-Based Video Editor (No Signup, No Upload) | VidStudio

Edit video by editing the transcript, free and in your browser. Delete a word and the video cuts to match. Transcription runs on your device, nothing uploads, and there is no signup or watermark.

vidstudio.app

It is really hard to write good software in Javascript

Aleksander mako — Tue, 15 Sep 2026 13:46:00 +0000

Six years of building Golang services gave me fewer race conditions than one year of writing Javascript. I have spent that year on vidstudio.app, an in browser client only video editor, which means I have had to write a lot of Javascript and have come to the conclusion that it is in fact a challenging language to master. The worst part is how easily I got there. Most of those races I introduced without noticing, and the one I saw coming is the one that taught me the most, because catching it is what let me say concretely why Javascript is so hard to write well.
I was reworking the export pipeline of the editor and had to replace one of my dependencies which was part of the export path. For context, my export has to demux a source file, read a section of it, decode it, draw it, apply some effect, encode it and finally assemble it, and the order of assembly in a media file is important. The dependency I wanted to replace was mp4box.js, better known as a demuxer but perfectly capable of muxing, where you hand it an encoded chunk and it synchronously consumes the chunk.
That synchronous consume was doing more for me than I understood at the time. Because the call returned when the work was finished, the loop handing over chunks in order was also the loop that had finished writing them in order, and I never had to think about either property. The new muxer, mediabunny, hands you back a promise instead, which resolves when the pipeline is ready to receive more data. The call no longer tells me that the previous chunk has been dealt with, and the sequencing I had been getting for free turned into something I had to state out loud. I am really glad I caught this one myself, but I still had to deal with serialising the assembly of the file, and while working on it I thought that this would never have happened to me in Golang.
The reason this would never have happened to me in Golang, C++, C# or Java is that by default you get linear program execution. The order in which I invoke the lines of code is the order they execute, and the only time that changes is when I decide to go non linear. That is a convention rather than a guarantee, since a library in any of those languages can spawn threads without telling me, but it is a strong enough convention that a library doing it silently is considered badly behaved. In Golang I decide for myself to spawn goroutines and complicate the execution flow of my program in exchange for scalability. Javascript has no such baseline. Anything that touches I/O is asynchronous whether I want it to be or not, and in the browser there is no synchronous version to fall back on, so if I need operations linearised I have to arrange that myself.
I suspect this is why the steady pipeline of articles trying to explain the event loop is still very much active, and still eagerly consumed by the next generation of developers, just as I consumed them when I first crossed paths with Node.js. Nobody is publishing a fresh explainer every month about how a for loop executes.
Now that I am using Javascript more frequently I can see how challenging it is. It really does take effort to write good and correct software while staying conscious of every I/O operation in your application. I take linear execution for granted in my day job, and on my side project I have to actively classify which parts of the application have to stay linear and which ones can be left to run in whatever order the event loop decides. The event loop is a powerful concept and I am not arguing that it should not exist. I am saying that it hands me a job that other runtimes do for me, and that the job never finish.
If you are interested in the project check it out
Here: video-editor
and Here:text-based video editor

DEV Community: Aleksander mako

The Surprisingly Expensive Problem Behind MP4 Fast Start

But what exactly does fast start mean?

A few MP4 fundamentals

Moving moov breaks the offsets

Where the new file gets written

What I did instead

How to Scrub Huge Video Files in the Browser Without Blowing Up RAM

Online Video Editor (No Upload, No Signup) | VidStudio

Free Text-Based Video Editor (No Signup, No Upload) | VidStudio

It is really hard to write good software in Javascript

Moving `moov` breaks the offsets