DEV Community

Cover image for Instagram gave its video editor an MCP. Here's what an agent-editable timeline actually needs
shansjl
shansjl

Posted on

Instagram gave its video editor an MCP. Here's what an agent-editable timeline actually needs

Last week Instagram shipped Edits for Mac with an MCP server, so Claude and other agents can operate a real video editor instead of generating a clip from scratch. Whatever you think of Instagram, that's a signal: editors are turning into toolboxes that agents call.

I've been building the open-source version of that idea since September. It's called CutVoke, an MIT-licensed, local-first editor that runs on Windows, macOS and Linux, with no account and no cloud. Disclosure up front: I'm the author, and it's an early prerelease.

Building it taught me that "give the agent an MCP server" is the easy part. The hard part is everything that happens after the agent's first cut, when a human wants to change something. Here are five things I had to get right.

1. Leases, not hope

The first version let the agent and the UI write to the same project at the same time. It worked fine until I dragged a clip while the agent was halfway through a batch of edits. One of us lost.

Now the agent takes an editing lease before it makes grouped changes. While the lease is held, the editor knows an agent is working, and the agent's batch can't be split by a manual edit landing in the middle. When the agent is done, the lease is released and the human has the timeline back. The next feature I'm working on is letting the human safely take over a lease the agent forgot to release.

2. Dry-runs before commits

Agents are confident. A command like "delete the sentence where I say 'um, basically'" sounds simple until it removes picture, audio and three captions on four tracks.

So supported commands have a preview mode. The agent calls command_preview first, gets back what would change, and only then commits. For transcript cuts it can see the deletion's impact before confirming. That one habit removed most of the "why did it delete that" moments.

3. Revision numbers, so nobody edits a stale project

Every commit carries an expectedRevision and a stable commandId. If the project changed since the agent last looked (because I edited something by hand), the commit is rejected instead of silently overwriting my change. The agent re-reads and tries again. It's boring optimistic concurrency, and it matters more for agents than for people, because agents don't notice that the screen changed.

Grouped edits also go through edit.batch, which makes the whole batch one undo step. If the agent's idea was bad, one Ctrl+Z undoes it.

4. Let the agent actually look

The biggest quality jump came from giving the agent eyes. preview_frames returns real rendered PNG frames at the timestamps it asks for, captions and effects included. Before that, the agent "knew" a title was centered because the numbers said so. Now it looks at the frame, sees the caption is covering a face, and moves it before export.

If you build an agent tool for any visual medium, I'd put this near the top of the list.

5. The output has to be a timeline, not an MP4

Most AI video tools give you a finished file. That's fine until you want to change one word in one subtitle, and the answer is to regenerate everything.

In CutVoke the result of the agent's work is a normal project: multitrack, captions as individual objects, keyframes, masks, speed ramps. I open it in the same editor and keep going by hand. The agent can also clone a project and swap in new media to make a second version. The editor, the MCP server, the HTTP API and the CLI all read and write the same project database, so nothing has to be exported or re-imported between the agent and the human.

Where it falls short today

Honestly, plenty. It's a v0.3.5 prerelease. The Windows build is unsigned, so SmartScreen will complain, and the macOS build isn't notarized. The project format may still change. It doesn't generate footage, and there's no LLM inside: you bring Claude Code, Codex, Cursor or any MCP client. Compared with mature editors like Kdenlive or Resolve, it's young and much smaller.

Try it or tear it apart

Setup is three steps: download a build, copy the MCP command from profile/mcp-config.json, and run claude mcp add cutvoke -- <command>. Two demo projects ship in the repo with their source media.

I'd really like to hear from people building agent tools for other creative apps. What did you need beyond "expose the commands"?

https://github.com/frankShan001/CutVoke

Top comments (2)

Collapse
 
fredabila profile image
Fred Abila •

Leases and dry-runs before commit are the two that matter most once a human wants to change the agent's first cut, and the Edits-for-Mac news backs up your point that editors are becoming toolboxes agents call. One I'd add to the list: let the agent pull a preview frame at a timestamp and look at it before it says it's done, because a timeline that validates can still have a caption over a face or a black frame at a cut. Disclosure: I build Vidmoat, a hosted AI video editor with an MCP server for Claude and Cursor, and that's how we handle it: edits land on a real timeline you can still adjust, and the agent can check preview frames before the final MP4. CutVoke being local-first and MIT covers the case where footage can't leave the machine, which a hosted editor doesn't.

Collapse
 
frankshan001 profile image
shansjl •

Agreed, "it validates" and "it looks right" are different checks. CutVoke already has preview_frames, which renders real frames (captions included) at timestamps the agent picks, and the agent Skill tells it to look before it calls an edit done. Your caption-over-a-face example is exactly what that catches. The gap I found after the last round of comments was before/after: an agent could see the result, but not compare it with what was there before the edit. PR #38 adds preview_compare for that. It dry-runs the command and returns before/after frames at up to three timestamps without committing anything: github.com/frankShan001/CutVoke/pu...

Thanks for the disclosure. And yes, the case where footage can't leave the machine is the main reason CutVoke is local-first.