Video Editing Skill for Claude Code: An Agent That Plans Every Cut, Then Renders With ffmpeg

Video editing skill is a literal name: it's a free, MIT-licensed agent skill for Claude Code, Codex, Cursor and OpenClaw that does the first edit of a talking-head recording. It transcribes with word timestamps, writes a cut list where every cut has a reason, waits for your approval, then renders a frame-exact edit with punch-ins on the jump cuts, levelled audio, an SRT caption file and YouTube chapters. Three Python scripts do the deterministic work with ffmpeg and Pillow on your machine; the agent does the reading and judging.

Repo: github.com/social-media-skills/video-editing-skill

The repo's worked example: a 6:39 public-domain NASA Goddard interview edited to 5:57, question cards kept. Footage courtesy of NASA; not endorsed by NASA.

The split: scripts render, the agent decides

Finding silence is a solved problem in ffmpeg. What makes a first edit usable is everything around it: knowing that "um" is safe to cut and "like" often isn't, telling a retake from a sentence that just happens to start the same way, spotting a tangent, keeping an interviewer's question on screen, and not clipping the last consonant of a word. Those calls need something that reads the transcript. So the skill is a SKILL.md workflow plus scripts that never guess:

video.mp4
  -> check_setup.py     Python, ffmpeg, Pillow, transcription routes, install commands
  -> transcribe.py      work/transcript.json (every word, start, end) + transcript.txt
       --terms "OSIRIS-REx, Bennu"   names and jargon spelled right
  -> agent reads the whole transcript, writes edit.json
       style, extra fillers, cut ranges with reasons, keep ranges, replace map, chapters
  -> edit_video.py --plan   edited/edit-plan.md: summary, removals, possible retakes
  -> you approve
  -> edit_video.py      edited.mp4, captions.srt, chapters.txt, cuts.jpg,
                        edit-report.md, manifest.json (with warnings)
  -> agent checks cuts.jpg and the warnings; re-renders if needed
  -> WoopSocial MCP     upload, validate, confirm, post

transcript.txt is one timestamped line per sentence, compact enough for an agent to read an hour of speech in one go. transcript.json holds the word timings the editor actually cuts on.

Getting the fillers and the names into the transcript

Transcript-driven editing quietly breaks when the transcript is too clean. OpenAI's speech-to-text guide notes that the model may leave out filler words unless the prompt contains some, and an "um" that was never written down can't be cut. transcribe.py sends a disfluent prompt on both the API and the local route, and appends whatever you pass with --terms:

FILLER_PROMPT = "Umm, let me think, like, hmm... Uh, okay, so, um, here's what I, uh, mean."
prompt = FILLER_PROMPT + (f" Names and terms: {a.terms}." if a.terms else "")

Names that still come out wrong get fixed in edit.json before any caption is written. In the example, the transcriber heard the mission name as "Cyrus Rex":

"replace": {"Cyrus Rex": "OSIRIS-REx", "gimbled": "gimbaled"}

Three transcription routes, picked in this order by --engine auto:

Route	Flag or env	Trade-off
Existing transcript	--from captions.srt	Free, instant; word timing estimated, most fillers missing
Groq or OpenAI	GROQ_API_KEY / OPENAI_API_KEY	Fast; Groq is about $0.04 per audio hour; long audio is sent in 20-minute chunks
Local	pip install faster-whisper	Free, private; roughly 0.5 to 1x realtime with --model small

Detection heuristics, and what is deliberately not automatic

Everything runs over the word list after lowercasing and stripping punctuation. Each rule returns a reason per word, and that reason ends up in the plan.

Fillers. The default set is um, uh, uhm, umm, er, erm, ah, hmm, cut only when the word has a small gap on at least one side; one fused to both neighbours would take part of them with it. "Mm hmm" as agreement stays. Habit words ("like", "so", "you know", "basically") are opt-in through extra_fillers, and single words are cut only next to a comma.

Stutters. The same word twice with under 0.25 s between and no comma, or the same two words twice. A stop list (very, really, that, had, is, no, yeah, so, well and a few more) is never touched, because "had had" and "no, no" are usually meant.

Retake cues. "let me say that again", "let me start over", "let me try that again", "scratch that", "take two". The cue goes. The sentence before it goes too only when the next sentence repeats at least half of its words, so a cue on its own can't delete a good sentence.

Restarts. An unfinished fragment (no end punctuation, eight words or fewer) followed by a sentence that opens with the same words is cut; the last take stays.

Possible retakes. Two finished sentences that open the same way ("It was great. It was fun.") are never cut automatically. They're listed in the plan, and the agent decides each one with you, adding real retakes to cut.

Agent picks. Tangents, "can you hear me?", a long set-up before the point go in edit.json as "cut": [[61.2, 74.8, "tangent about the weather"]]. "keep" ranges override every automatic rule, which is how the example keeps about 3 seconds of each interview question card.

Pauses become beats, and small cuts are skipped

Deleting every gap makes a speaker sound breathless. Each pause over the style's limit shrinks to a beat that depends on context:

Style	Shorten gaps over	Mid-sentence	After a sentence	Long gap or card	Skip removals under
youtube	0.50 s	0.30 s	0.45 s	0.90 s	0.40 s
tight	0.30 s	0.22 s	0.25 s	0.50 s	0.25 s
podcast	0.80 s	0.40 s	0.60 s	1.00 s	0.60 s

The last column matters more than it looks. Every cut inside one continuous shot is a visible jump, so a removal that saves less than the minimum is left alone: a dozen tiny cuts save a few seconds and make the video look nervous. Fillers, stutters and retakes are always cut regardless. A kept piece under 0.4 s joins a touching neighbour so nothing flickers, and the kept beat is the real room tone, never digital silence.

Cut edges placed by ear and by eye

Transcribers often put a word's end 50 to 150 ms early, so cutting on raw timestamps clips word endings. The script measures the audio's level in 10 ms windows and estimates the room's noise floor, then moves each edge:

• Out-points move forward until the audio has stayed below the noise floor plus 6 dB for 30 ms, up to 250 ms.
• In-points move back to where the sound starts, up to 200 ms, and never closer than 100 ms before the word.
• Edges never move into a neighbouring word.

The picture gets one pass too. ffmpeg's blackdetect and a scene-change select on a 160-pixel-wide copy find title cards and shot changes. A card inside a removed gap goes entirely, and a shot change within half a second of a kept edge pushes the edge past it, so no stray frames of a card or a second camera angle flash by.

Frame-exact rendering in groups

Cutting hundreds of pieces as separate files and concatenating them lets audio and video drift by a few milliseconds per join, which adds up. Here every edge is rounded to the frame grid, and each group of up to 60 pieces is trimmed and joined inside one filter graph:

[0:v]trim=start=S:end=E,setpts=PTS-STARTPTS,<crop/scale>[v0]
[0:a]atrim=start=S:end=E,asetpts=PTS-STARTPTS,afade=t=in:d=0.004,afade=t=out:...[a0]
...
[v0][a0][v1][a1]...concat=n=N:v=1:a=1[cv][ca]

The 4 ms fades stop clicks without dipping the room tone. After the render the script compares the audio and video stream durations and adds a warning to manifest.json if they differ by more than a frame.

Punch-ins where a jump cut needs hiding

Alternating between the full frame and a 1.1x crop makes a jump cut read as an edit. The script switches framing at every cut that removed a second or more, and otherwise only once 4 seconds (2.5 in tight) have passed on the same framing. It never punches in at a cut onto a new camera shot, which already changes the picture, and never on a title card. The crop is anchored to an eye line at 38% of the frame height so the eyes stay level between framings, the zoom is capped at 1.2x to keep 1080p sharp, and "punch_in": 0 turns it off for screen recordings.

The audio chain

Every filter here ships in standard ffmpeg builds, so no special build is needed on any OS:

1. volume to bring the recording to about -20 LUFS, a working level.
2. highpass=f=80 for rumble.
3. afftdn with its noise floor set from the measured room noise, skipped entirely when the room is already under -60 dB, so quiet word endings aren't eaten.
4. acompressor, 3:1 from -24 dB, 5 ms attack, 120 ms release.
5. alimiter, set so peaks end at -2 dB after the final gain, leaving headroom for the AAC encoder.
6. One linear volume gain to -14 LUFS, measured on the processed audio, so there's no pumping.

-14 LUFS is the level YouTube and the short-form apps are widely measured to play at; the podcast style uses -16.

Captions and chapters that survive the cuts

Every word keeps its source timestamps, and to_out() maps a source time onto the edited timeline by walking the kept spans. Captions are built from kept words only, with replace applied: at most 2 lines of 42 characters, 1 to 7 seconds each, gaps under 0.3 s closed, and no line break after a small word ("the", "of", "my") or inside a name. They go to captions.srt for upload in YouTube Studio.

With "burn": true, Pillow renders each caption as a transparent PNG at 4.2% of the frame height and ffmpeg overlays the sequence, so ffmpeg's text filters aren't needed. Vertical versions burn by default, one line of up to 32 characters. The skill tells the agent not to burn captions into a YouTube video that also gets the SRT, since viewers with captions on would see both.

Chapters follow the same mapping. The agent writes them in original-video times, named for what the viewer gets; the script moves them through the cut list and writes chapters.txt with the first at 0:00, warning if any chapter drops under YouTube's 10-second minimum.

The plan and the check sheet

--plan writes edit-plan.md without rendering. The example's summary line:

6:39 -> 5:57 (11% shorter; 9 s from inside the talking, 34 s from long gaps,
cards, opening and ending). 12 cuts, pace 183 words per minute.

Splitting time removed inside the talking from time removed in gaps, cards and the ends keeps the headline honest: in the example most of the saving is a 24-second ending and long gaps around the question cards, while the talking itself lost about 9 seconds, because the interview was already well delivered. The plan then lists every removal over a second with its reason and the possible retakes. The report flags a pace over 190 words per minute and any edit that removes more than 45%.

A render also writes cuts.jpg: frames just before and after the biggest cuts, side by side. The agent opens it to look for a flash of a card, a jump into another shot or a frozen expression, instead of claiming it watched the whole video.

Posting

After the render, the skill can post through WoopSocial, a social media API and MCP server for AI agents. The agent creates an upload session with the byte size from manifest.json, uploads the parts with upload_parts.py, builds a YouTube post (title, privacy, a description that starts with what the viewer gets and then the chapters), runs posts_validate, and shows you the whole post before calling posts_create. Caption files aren't sent through the API; you upload captions.srt in YouTube Studio. Setup per agent is at woopsocial.com/agents, and the free plan covers 2 social accounts. Disclosure: the skill is maintained by Frank Heijdenrijk and the team behind WoopSocial. Posting is optional; the edited files are yours either way.

Try it

npx skills add social-media-skills/video-editing-skill -g -a claude-code -y

Other routes: openclaw skills install @social-media-skills/video-editing-skill, gh skill install social-media-skills/video-editing-skill video-editing-skill, or gemini extensions install https://github.com/social-media-skills/video-editing-skill. You need Python 3.8+, ffmpeg and Pillow, and an agent that can run ffmpeg locally; CI runs a transcribe, plan and render on Ubuntu, Windows and macOS.

Then point it at a recording: "edit lesson3.mp4 for YouTube, cut the silences and ums, add chapters." Issues and pull requests are welcome on GitHub. More free skills for social media work are at social-media-skills.com.