Forem

Cover image for TIL: Turn a video to frames and then back again with ffmpeg
Corey Alexander
Corey Alexander Subscriber

Posted on • Originally published at coreyja.com

1

TIL: Turn a video to frames and then back again with ffmpeg

Here are a collection of bash commands and then a final script that I used this week to convert a video to frames.
Edit those frames a bit, and then convert the edited frames back to video

This was for my rmbg tool, which I discussed in this weeks newsletter.
The script at the end actually uses rembg which is the tool that inspired my rust port!

Disclaimer: It turns out this is NOT the "state of the art" background removal for videos! Keep your eyes peeled for a future TIL covering RobustVideoMatting

But the ffmpeg scripts for converting to frames are still useful!

Make PNGS from video

ffmpeg -i input_video.mp4 'frames/%06d.png'
Enter fullscreen mode Exit fullscreen mode

Make video from PNGS

ffmpeg -framerate 24 -pattern_type glob -i 'frames/*.png' output_video_without_audio.mp4
Enter fullscreen mode Exit fullscreen mode

Combine Video and Audio

ffmpeg -i output_video_without_audio.mp4 -i input_video.mp4 -map 0:v -map 1:a -c copy -shortest output.mp4
Enter fullscreen mode Exit fullscreen mode

Add overlay

ffmpeg -i video.mp4 -i overlay.png -filter_complex "[0:v][1:v] overlay=0:0" -c:a copy output.mp4
Enter fullscreen mode Exit fullscreen mode

Background Removal Script

input_video=$0

# Make PNGS from video
ffmpeg -i "$input_video" 'frames/%06d.png'

# Remove background from all frames
rembg p frames frames_out

# Make video from pngs
ffmpeg -framerate 24 -pattern_type glob -i 'frames_out/*.png' bg_removed.mp4

# Add back in the audio track
ffmpeg -i bg_removed.mp4 -i "$input_video" -map 0:v -map 1:a output.mp4
Enter fullscreen mode Exit fullscreen mode

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more →

Top comments (1)

Collapse
 
Sloan, the sloth mascot
Comment deleted

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay