Short-form video has a brutal production bottleneck: you shoot ten minutes of footage and need thirty usable seconds — captioned, framed vertically, ready to post — before the trend moves on. Editing that on a phone is fiddly; editing it in a desktop suite means an install, an import, and a render queue.
The browser turns out to be a legitimate middle ground. <video>, <canvas>, MediaRecorder, and WebCodecs give you nearly the whole pipeline without leaving the tab. After spending a long stretch building and refining a browser-based video editor, here are the three areas where the real lessons live: trimming, captions, and vertical export.
1. Trimming is a UX problem, not a decoding problem
Decoding video in the browser is a solved problem — drop a file into a <video> element and it plays. The hard part is helping someone find the exact moment they want.
The first lesson: thumbnails beat timestamps. Scrubbing blind is miserable. Pre-generating a thumbnail strip by seeking a muted <video> element to intervals and painting frames to an offscreen canvas makes trimming feel instant, and it only needs to happen once per imported clip. On slower devices, generate fewer, larger thumbnails instead of a dense strip — perceived responsiveness matters more than precision at this stage.
The second lesson: seeks are async and approximate. Setting video.currentTime doesn't land exactly where you ask, and it definitely doesn't happen synchronously. The trim handles in your UI should snap to the seeked frame (listen for the seeked event) rather than the requested time. If the UI promises frame accuracy, the plumbing underneath has to deliver it.
The third lesson: preview the cut, not the clip. When someone drags a trim handle, the most useful feedback isn't the whole clip replaying — it's a short loop of the moment just before the cut point and the moment just after it. That micro-preview answers the only question that matters: "did I cut the good part off?"
2. Captions: render them, don't overlay them
Most short-form content is watched on mute, which makes captions non-negotiable. But how you deliver them matters:
Burn captions into the video rather than relying on platform caption files. Platforms recompress, reframe, and re-upload your video constantly — burned-in captions survive every re-upload, duet, and stitch. Caption sidecar files get lost the moment someone screen-records the clip.
Style for the platform, not the spec. Classic white text with a dark stroke still wins for one reason: it stays readable over any background. But positioning matters even more — keep captions inside the vertical safe zone (roughly the middle band of the frame) because TikTok, Reels, and Shorts all paint their own UI over the top and bottom of your video.
Word-by-word highlighting is worth the effort. Viewers follow along more easily when the currently spoken word is emphasized, karaoke-style. The implementation is straightforward: render each word as its own text run on the canvas using per-word timing data, highlight the active word, and dim the rest. It's more work than a single static block, but it's the biggest single readability upgrade you can make.
Generate the words, let humans fix them. Automatic speech recognition gets you most of the way there; the remainder — names, slang, brand terms — needs a fast correction UI. A caption editor that lets you click a word and retype it, with timing untouched, will get used constantly. One that forces you to re-time anything will get abandoned.
3. Vertical export: design the canvas first
Everything about short-form is vertical, and this is where browser editors either shine or fall apart. The key insight: treat the export canvas as the source of truth, not the preview.
Draw your final 1080x1920 composition on an offscreen canvas with drawImage, applying the same crop-and-scale math used everywhere else. Then capture that canvas with MediaRecorder (or WebCodecs when you need finer control) and mix in audio from the source clips. When the export canvas is the render target:
- What the creator sees while editing and what gets exported can't drift apart.
- Reframing landscape footage becomes a layout problem (pan-and-scan, blurred-fill backgrounds, subject tracking) instead of an export bug.
- Effects — zooms, captions, overlays — live in one compositor instead of being bolted onto the file afterward.
Two practical gotchas: first, MediaRecorder output codecs vary by browser, so validate your container and codec choices on every target browser rather than assuming H.264-in-MP4 everywhere. Second, keep preview resolution and export resolution separate — preview at whatever the device handles smoothly, and only render full resolution during export. Your users' laptop fans will thank you.
The through-line
Trimming, captions, and vertical export look like three separate features, but they're really one design principle: reduce the distance between having footage and having a finished post. Every seek that lags, every caption that needs manual re-timing, and every export that looks different from the preview adds friction exactly where a creator's motivation is thinnest.
Keep the pipeline in the browser and that distance shrinks dramatically — no installs, no uploads parked on a server, no round-trips. Raw clips go in, and a vertical, captioned MP4 comes out, minutes later.
Disclosure: I build and maintain ReelWorkshop, a browser-based studio for turning raw clips into TikToks, Reels, and Shorts — this is where I ran into everything above. The techniques themselves are portable to any web video project.
Top comments (0)