Last week I open sourced FableCut,
a Premiere-style video editor that runs in the browser and that AI agents can
operate. It hit the front page of Hacker News
(thread), and the questions
there made me realize the interesting part isn't the editor. It's one design
decision: the project file is the interface.
The usual way, and why I flipped it
Most AI video tools hide the edit behind an API. You call addClip(),
applyFilter(), and the tool owns the state. If you want a human to touch the
result, you build a whole collaboration layer.
FableCut does the opposite. The entire timeline lives in one JSON document,
project.json: media, clips, tracks, keyframes, transitions, markers. The
editor UI reads it. The export renders it. And anything that can write JSON
can edit video: Claude Code through MCP, a Python script, jq, or you with a
text editor.
{
"id": "c_title", "kind": "text", "track": "V3",
"start": 0, "duration": 2.2,
"props": { "text": "HANDMADE", "font": "Bebas Neue",
"glow": 45, "textAnim": "letter-pop" }
}
That clip is a glowing kinetic caption. There is no API call that creates it.
Writing it into the file IS creating it.
SSE as a doorbell, not a data channel
The first question on HN was "what's the benefit of SSE here?" Fair question,
because the SSE channel does almost nothing, and that's the point.
The server watches the project file with fs.watch, debounces 150ms, and
pushes the literal string change to the browser. No payload. The browser
re-fetches the project and re-renders. The whole mechanism is about 15 lines
on a bare node:http server.
Why not WebSockets? Because the data only flows one way. Everything that
writes (the UI, an agent, a shell script) goes through REST or the
filesystem. The browser only ever needs to hear "something changed, go look."
An event with no payload can't arrive out of order, and a missed event costs
nothing because the next fetch has the latest state anyway.
The revision counter, or: how a human and an agent share a timeline
The file carries an integer revision. Every write must bump it. If a write
arrives with a revision that isn't newer than what's on disk, the server
rejects it with a 409.
This one integer is the entire concurrency model. If I drag a clip in the UI
while an agent is mid-edit, the agent's stale write bounces, it re-reads,
re-applies its change on top of mine, and writes again. No operational
transforms, no CRDTs, no lock files. It works because edits are coarse
(a whole document) and rare (human speed), so last-writer-wins with a
staleness check is enough.
The trick I'm proudest of: frame-accurate CSS animations
FableCut has animated SVG overlays (lower thirds, confetti, sparkles) that
are plain .svg files animated with CSS @keyframes. The problem: a video
compositor needs to render the animation state at an exact time, and export
isn't realtime. You can't just let the animation play.
The solution: pause every animation and drive time by hand. The compositor
sets animation-delay: calc(var(--d, 0s) - t) where t is the clip's local
time. A negative delay means "you started in the past," so a paused animation
with delay -1.3s displays exactly its 1.3 second frame. Deterministic,
scrubbable, identical in preview and export. The only rule for SVG authors is
to never hardcode animation-delay and use the --d custom property for
staggering instead.
"You can just give Claude access to ffmpeg"
Someone said this on HN and it deserves a straight answer. For trims,
concats, and batch transcodes: yes, absolutely, do that.
The difference is the creative loop. ffmpeg is write-only. The agent builds a
filter graph, renders for minutes, and cannot see what it made. You give
feedback, everything re-renders. In FableCut an edit is a JSON diff, the open
browser updates in 150ms, and the timeline stays editable instead of being
baked into a filter string. It's not a replacement for ffmpeg anyway: the
export pipeline renders frames in the browser and pipes them to ffmpeg for
encoding. FableCut is the state and preview layer between the agent and
ffmpeg.
Honest limitations
The compositor is the browser, so export needs a browser open (headless
export is not there yet). It's Chromium-first. And an AI can misjudge a cut
just fine, which is why the human-in-the-loop part matters more than the AI
part: the agent does the labor, you do the taste.
Full disclosure since HN asked: Claude helped write the README, and large
parts of the editor were built in collaboration with it. That felt fitting
for a tool whose primary user is an AI agent, but the architecture decisions
above are the ones I'd defend in person.
Repo: https://github.com/ronak-create/FableCut. It's MIT, zero dependencies,
one node server.js. If you build something weird with it, I want to see it.
Top comments (11)
The project-file-as-interface idea is very strong. It gives the agent something durable to operate on instead of asking it to pretend the UI is the API. The hard part is making that file format stable enough that edits are reviewable, reversible, and not just a hidden pile of state mutations.
Treating the project file as the interface is a clean idea, it gives the agent something concrete to read and write against instead of inferring intent from a prompt. Curious whether that file format has needed versioning yet, since an agent's assumptions about what a field means will drift the moment the schema changes under it.
The revision counter as the entire concurrency model is a nice minimal-mechanism story, last-writer-wins with a staleness check works precisely because you named the actual constraint that makes it safe, edits are coarse and rare, and it would fall apart the moment either assumption breaks. SSE as a doorbell rather than a data channel is the same instinct in a different layer, an event with no payload can't arrive out of order and a missed one costs nothing because the next fetch has the latest state anyway, that's a good default any time you're tempted to reach for something heavier like websockets or CRDTs. The honest-limitations section is doing real work for credibility, naming that export needs a browser open rather than pretending headless export exists yet. One question on the 409-reject-and-reapply flow: when the agent's stale write bounces and it re-applies on top of a human's concurrent edit, does it re-derive the diff from the new base state, or does it just retry the same write verbatim and risk clobbering the human's change a second time?
A verbatim retry can't clobber, it just keeps bouncing. The re-apply is semantic, done by the agent, not mechanical, done by the server. And the recommended path sidesteps the whole dance.
Semantic re-apply by the agent, not mechanical retry by the server, is the right default and it's the part I'd have gotten wrong by instinct. Worth spelling out for anyone skimming: which path is "recommended" here, does the agent re-fetch and re-derive the diff against the new base state before reapplying, or does the 409 just hand it back the fresh state and trust the agent's next turn to notice what changed on its own?
it's re-fetch and re-derive, the 409 does not hand back the fresh state.
Re-fetch and re-derive is the answer I was hoping for, that's the version where a 409 costs a round trip instead of a silent clobber. Good to have it confirmed rather than assumed from the architecture description alone.
The negative animation-delay trick got me. Pausing every animation and steering time by hand through one custom property is such a clean answer to "export is not realtime" that I'm mildly annoyed it isn't already the standard pattern for browser compositors. Thanks for sharing!
One nice side effect of this approach is that it works with pretty much any CSS
@keyframesanimation. We simply pause the animation and drive it by updatinganimation-delayrelative to the current timeline position, so animations become a pure function of the editor's timeline instead of real time. That's what lets the preview and the export use the exact same rendering path, which is why SVG animations stay frame-accurate in both.The only small tradeoff is that staggered animations need to use a
--dcustom property instead of a fixedanimation-delay, since the engine takes control ofanimation-delaywhile scrubbing the timeline. It wasn't an obvious design choice at first, but it ended up making the whole animation system much more predictable and deterministic.How does FableCut handle complex editing tasks like multi-track audio syncing, and I'd love to hear more about your approach to AI-driven editing decisions, can you share some insights?
So audio is split across three dedicated tracks, A1 to A3, but here's the thing, every video clip also carries its own audio baked in, with its own volume and volume keyframes. Which sounds like it'd be a sync nightmare honestly, tracks plus per-clip audio all fighting for the same timeline. But it actually holds up because export never tries to record anything live. It just builds an offline mix first, every audio clip gets placed at its exact timeline position, and only after that mix is built does ffmpeg come in and stitch it to the rendered frames. So you're never fighting playback timing, the placement is just... fixed, deterministic, no drift.
That same logic is what saves you when you throw speed ramps into the mix. Slow-mo, speed-up, whatever, the audio gets time remapped right along with the video (the math is basically in-point plus the integral of speed over time, if you want the formal version), so nothing drifts out of sync even in the weird stretchy sections. And since beat markers exist too, you can actually snap your cuts to hit the beat instead of eyeballing it.
Then there's the AI editing side, which is really a different conversation but ties back to the same "nothing hidden" philosophy. The whole idea is the project file IS the interface, the agent isn't quietly deciding things you can't see. It's handed a compact JSON snapshot of where things stand, plus an analysis blueprint, shot boundaries, beats, BPM, energy per shot, where the drop lands, all pulled out with ffmpeg ahead of time. That analysis part is deterministic, it's not guessing. What's not deterministic is the creative call, which shot goes where, which cut, which effect, that's on the agent. But even those decisions get written back as plain readable JSON, so nothing's locked away, you can just open the timeline and override whatever it picked.