TIL: Turn a video to frames and then back again with ffmpeg

#todayilearned #ffmpeg

Here are a collection of bash commands and then a final script that I used this week to convert a video to frames.
Edit those frames a bit, and then convert the edited frames back to video

This was for my rmbg tool, which I discussed in this weeks newsletter.
The script at the end actually uses rembg which is the tool that inspired my rust port!

Disclaimer: It turns out this is NOT the "state of the art" background removal for videos! Keep your eyes peeled for a future TIL covering RobustVideoMatting

But the ffmpeg scripts for converting to frames are still useful!

Make PNGS from video

ffmpeg -i input_video.mp4 'frames/%06d.png'

Make video from PNGS

ffmpeg -framerate 24 -pattern_type glob -i 'frames/*.png' output_video_without_audio.mp4

Combine Video and Audio

ffmpeg -i output_video_without_audio.mp4 -i input_video.mp4 -map 0:v -map 1:a -c copy -shortest output.mp4

Add overlay

ffmpeg -i video.mp4 -i overlay.png -filter_complex "[0:v][1:v] overlay=0:0" -c:a copy output.mp4

Background Removal Script

input_video=$0

# Make PNGS from video
ffmpeg -i "$input_video" 'frames/%06d.png'

# Remove background from all frames
rembg p frames frames_out

# Make video from pngs
ffmpeg -framerate 24 -pattern_type glob -i 'frames_out/*.png' bg_removed.mp4

# Add back in the audio track
ffmpeg -i bg_removed.mp4 -i "$input_video" -map 0:v -map 1:a output.mp4

Create and maintain end-to-end frontend tests

Learn best practices on creating frontend tests, testing on-premise apps, integrating tests into your CI/CD pipeline, and using Datadog’s testing tunnel.

Download The Guide