The Best Feature I Shipped Was Removing Friction: Rebuilding the UX of a Browser Video Editor

When I first shared Timeline Studio, it had around 80 GitHub stars.

The technical premise was already there: a local-first video editor that runs in the browser, with a multi-track timeline, voiceovers, captions, effects, and offline export.

The user experience was not.

Some early feedback was blunt:

Interesting project, but the interface feels clunky.

There are a lot of features, but some interactions feel unintuitive.

That feedback was fair. I had spent too much time proving that browser video editing was possible and not enough time asking whether ordinary editing actions behaved the way people expected.

The project has since passed 500 GitHub stars. The number is nice, but the more meaningful milestone was going back and fixing the small frustrations that made the entire editor feel harder than it needed to be.

This is a post about those fixes.

A feature can work and still be wrong

The early version had split, zoom, scrolling, track locking, audio separation, mobile controls, and export settings.

On paper, those features existed.

In practice, they often exposed the implementation instead of matching the user's mental model.

For example, the split action originally required an explicitly selected clip. A user could place the playhead in the middle of the main video, click the scissors button, and still be told to select a clip first.

The code was consistent:

selectedClip -> split(selectedClip, playheadTime)

The interaction was not.

The user was looking at the main visual under the playhead. They had already communicated enough intent.

The current rule is:

if a clip is explicitly selected:
    split that clip
else:
    find the main visual under the playhead
    select and split it

Explicit selections are still respected, but the common action now has a useful default.

This change added no new capability. It simply removed a question the editor should never have asked.

Lock the meaning of a continuous gesture

Timeline editors usually need several wheel behaviors:

• vertical track scrolling
• horizontal timeline panning
• timeline zoom
• Shift-modified panning
• Ctrl/Cmd wheel zoom
• trackpad pinch zoom

My initial implementation kept resolving the wheel behavior from whatever happened to be under the pointer.

That sounds reasonable until the timeline moves.

Imagine starting a vertical scroll over empty timeline space. As the tracks move, a clip slides under the stationary pointer. If the editor resolves the target again, the gesture can suddenly change from scrolling tracks to zooming the timeline.

From the user's perspective, the editor has changed its mind halfway through an action.

The fix was to treat a continuous wheel sequence as one gesture:

gestureTarget = hitTest(pointerPositionAtStart)
gestureMode = resolveMode(gestureTarget, modifierKeys)

while the gesture continues:
    keep gestureMode stable

after a short pause:
    allow the next gesture to resolve again

The resulting behavior is predictable:

• start over empty timeline space to scroll track lanes
• start over a clip to zoom the timeline
• use Shift for horizontal panning
• use Ctrl/Cmd or pinch for explicit zoom

The important part is not the exact mapping. It is that the mapping does not mutate while the user is still performing the same physical gesture.

Users manipulate clips, not source asset IDs

A split video often produces several timeline clips that reference the same source file.

Internally, that might look like this:

sourceAssetId: video-001

clip A -> source range 0s–5s
clip B -> source range 5s–12s
clip C -> source range 12s–20s

An early version of "Separate audio" relied too heavily on the shared source asset ID. That made whole-asset matching convenient, but it violated what the user had actually asked for.

When the command comes from clip B's context menu, the expected result is the audio for 5s–12s—not new mappings for every sibling clip backed by video-001.

The current behavior operates on the exact timeline segment:

• preserve its source trim
• preserve its playback speed
• place the extracted audio on an audio lane
• mute the corresponding picture-in-picture clip to prevent doubled sound
• select and reveal the resulting audio
• leave untouched sibling clips alone

Whole-asset matching still makes sense when an unsplit source asset is intentionally dragged onto a source-audio track. It does not make sense for a clip-scoped menu action.

The lesson was simple: internal identity is not the same thing as interaction scope.

Automatic layout should not rearrange confirmed work

Audio lane placement caused a similar problem.

When a new voice clip overlapped existing audio, an automatic placement algorithm could move an existing clip to another lane. The layout became technically valid, but the user's previously arranged work changed without permission.

Now existing clips stay where they are.

New voice or audio clips find another available lane, or a new ordinary audio lane is materialized when necessary. AI music always routes to the dedicated music track. Split audio can still be moved vertically between normal audio lanes, but adding something new never displaces something already confirmed.

Automation should solve the new placement problem, not reopen old decisions.

Keep the primary visual in view

Once a project contains overlays, captions, source audio, voiceovers, and music, vertical scrolling becomes unavoidable.

Letting every track scroll together meant the main visual could disappear while the user worked on lower lanes. That removed the most important reference point in the composition.

The main visual track is now pinned directly below the ruler. Overlay and lower tracks scroll underneath it.

The persistent scrollbar chrome is hidden, but both horizontal and vertical scrolling remain functional. The pinned track's lower edge also acts as the upper boundary for vertical drag auto-scroll, so cross-lane moves still behave correctly.

This required more than position: sticky. Pointer hit testing, drag targets, auto-scroll boundaries, and main-to-overlay or overlay-to-main moves all had to use the same geometry.

The visible result is much less dramatic: the main picture simply stops getting lost.

Mobile is not desktop with smaller buttons

The first mobile layout was essentially the desktop editor compressed into a narrow viewport.

It technically contained the same features. It was also exhausting to use.

The redesigned mobile workspace follows a different hierarchy:

• preview first
• timeline immediately below it
• a centered playhead with the timeline moving around it
• clip-specific actions after selection
• properties inside a focused bottom drawer
• touch-sized controls
• no desktop keyboard-shortcut guide

Desktop and mobile still edit the same project model. They do not need to expose that model through the same interface.

Responsive design is not preserving every panel at every width. It is preserving the user's task.

Small state mismatches create large trust problems

Several fixes were almost embarrassingly small:

• Clicking the Export button a second time now closes the export popover.
• Icon-only timeline controls have localized tooltips on hover and keyboard focus.
• Locked tracks retain a subdued treatment, but their clips no longer gain a dashed border that looks like selection.
• "Analyze person" and "Analyze object" use the primary action style whenever they are actionable; the neutral style is reserved for "Cancel analysis."
• Generated music and converted voices go to the asset library instead of silently modifying the timeline.
• Repeated voice generations append after the current voiceover end instead of stacking at 0 seconds or an unchanged playhead.

None of these belongs in an impressive model benchmark.

Together, they answer a more important question: can the user predict what will happen after the next click?

The product changed when the question changed

At the beginning, I kept asking:

Can this run in a browser?

Can WebCodecs export the composition? Can WebGPU run the model? Can the timeline support another track type? Can an agent modify the project?

Those were useful questions, but they produced a technology demonstration.

The editor started becoming a product when the question changed to:

What does the user reasonably expect to happen here?

That question led to fewer interruptions, stable gestures, clip-scoped actions, conservative automation, clearer state, and a mobile interface built around touch rather than CSS breakpoints.

Timeline Studio is still evolving. It combines a real multi-track editor with local-first browser AI, portable .timeline projects, and deterministic offline export. There are still plenty of rough edges to find.

But if you tried an early version and found the UI clunky, I would genuinely like to know whether the same problems still stand—and what remains confusing today.

• GitHub: https://github.com/MartinDelophy/ai-video-editor
• Live editor: https://video-editor.ai-creator.top/

The most useful feedback is not "add another AI model."

It is: "I expected this action to do X, but it did Y."