An eight-minute video is long enough to expose work that is easy to miss in short browser-editor demos: audio preparation, repeated frame conversions, background thumbnail decoding, and idle time between decode, composition, and encoding.
We recently optimized these paths in Timeline Studio, an open-source browser video editor. In a controlled comparison against the code before the first optimization round, average export time fell from 192.0 seconds to 131.9 seconds—a 31.3% reduction, saving about 60 seconds.
Source code on GitHub · Try the editor
Benchmark: what we actually measured
Both versions ran on the same machine with the same approximately eight-minute synthetic stress-test video. The input was H.264 at 1440 × 1080 and approximately 29.88 fps, with stereo AAC audio at 44.1 kHz.
Output settings were identical:
- H.264, 1440 × 1080, 30 fps
- 10 Mbps target video bitrate
- AAC audio at 192 kbps
- No effects or overlays in this benchmark
Timing started immediately before embedded-audio preparation and ended when export completed. Import and timeline-thumbnail preparation were excluded.
We ran old → new → new → old, starting a fresh browser process for each run.
| Version | First run | Second run | Mean total |
|---|---|---|---|
| Before the first optimization | 203.7 s | 180.4 s | 192.0 s |
| Current implementation | 141.6 s | 122.3 s | 131.9 s |
| Stage | Before, mean | After, mean |
|---|---|---|
| Audio preparation | 9.3 s | 0.9 s |
| Video export stage | 182.7 s | 131.0 s |
These are two runs per version, with visible runtime variation. The input is synthetic, so the 31.3% result should not be generalized to every camera recording, device, or effect stack. We also checked decoding and compatibility with a native Sony ZV-1 4K camera sample, but that sample is not the source of the eight-minute benchmark result.
1. Avoid decoding the entire audio track just to prepare it
The old preparation path used FFmpeg to decode embedded audio into WAV before subsequent processing. For long inputs, this adds whole-track decoding, PCM allocation, and intermediate-data handling.
For eligible AAC tracks, the new path extracts encoded packets and remuxes them instead. Final output audio is still encoded according to export settings; this change removes unnecessary work in the preparation stage.
Timestamp handling matters. AAC can contain preroll and negative timestamps. Simply copying packets while discarding that information can shift audio relative to the picture. The implementation preserves and compensates for the source offset and distinguishes storage duration from logical segment duration.
Other formats retain the fallback path. In this benchmark, mean audio preparation dropped from 9.3 seconds to 0.9 seconds.
2. Remove an intermediate frame conversion only when it is safe
At 30 fps, an eight-minute output contains approximately 14,400 frames. Even a small repeated operation is worth examining at that scale.
The new eligible path sends decoded VideoFrame objects directly to the shared compositor, avoiding an additional intermediate Canvas rasterization step.
Eligibility is deliberately narrow: original-size, unrotated, square-pixel frames with no relevant transformations or effects that require the existing rasterization path. Scaling, grading, animation, masks, and other affected cases keep their compatible Canvas path.
This restriction came from validation: different rasterization routes produced small pixel differences when resizing. We therefore did not extend the direct-frame optimization to those cases.
3. Overlap work with bounded queues
A fully serial loop can leave resources waiting:
Decode frame → Compose frame → Submit/wait for encoding → Repeat
The updated pipeline overlaps a limited amount of work:
- Prepare one decoded frame ahead.
- Compose the next frame while a video submission is in flight, subject to backpressure.
- Process audio and video concurrently.
- Release frames promptly after use.
The queue bounds are essential. Unrestricted prefetching can turn a throughput problem into a memory problem on long videos.
The exporter also probes supported hardware-encoding configurations and falls back when necessary. Actual hardware acceleration depends on the browser, operating system, and device.
Cancellation needs the same discipline: stop submitting new work, let bounded in-flight operations settle, and then release output resources. Closing everything immediately can leave asynchronous waits stranded.
4. Give playback priority over background thumbnails
Export was only part of the original problem. Playback and seeking also competed with background timeline work.
We changed scheduling so that:
- Playback pauses background thumbnail refinement.
- Scrubbing cancels background thumbnail work for the pointer gesture.
- Releasing the pointer resumes viewport-prioritized refinement.
- Starting playback prioritizes the latest seek instead of stale scrub requests.
- The playback clock pauses while decoding is seeking or buffering, reducing cases where the timeline advances while the picture is frozen.
For live thumbnail updates, confirmed decoded frames are drawn directly rather than repeatedly going through JPEG encoding and decoding. Source timestamps remain aligned with timeline sampling.
For locally imported media processed in the browser, these are client-side resource and coordination issues worth measuring alongside network behavior.
Validation: speed with the same output settings
We kept output resolution, frame rate, and target bitrate unchanged. Validation covered:
- Output video/audio tracks, duration, and dimensions.
- AAC packet extraction and source-offset handling.
- Pixel comparisons for eligible direct-frame cases.
- Canvas fallback behavior for scaling, grading, and animation.
- Encoded-packet consistency for a fixed effects case across the pipeline scheduling change.
- Cancellation and resource cleanup.
This does not establish byte-identical output between the original version and the final version, or cover every possible project. It checks the specific paths changed by this optimization.
Explore the implementation
The main gains came from reducing intermediate work and overlapping bounded tasks while retaining the processing paths required by effects.
If you reproduce a performance issue with your own footage, include your device, browser version, source codec parameters, export settings, and effects in a GitHub issue. Separating preparation time from export time makes the bottleneck much easier to discuss.
Top comments (0)