DEV Community

Cover image for Adding sound wave overlays to videos and pictures using FFMPEG
Christian Heilmann
Christian Heilmann

Posted on • Originally published at christianheilmann.com

6 2

Adding sound wave overlays to videos and pictures using FFMPEG

In order to fix a broken video where the camera stopped recording, I wanted to replace the part with a video with a sound wave overlay. There are many services for that, and it is also a feature of video editing software. But I am stingy and a geek, so I wanted to use FFMPEG on the command line instead. This also allows for batch processing.

Note: I didn't embed the videos in this post but GIFs instead. The real videos play the video and audio and the sound wave moves accordingly.

Let's say this is our original video:

Video of me talking about public speaking

This is what it looked like in the end:

The same video with a sound wave on top of it

Here's the command I used to create the above version:

ffmpeg -i Understandable.mp4 \
 -filter_complex "[0:a]showwaves=colors=0xff1646@0.3\
 :scale=sqrt:mode=cline,format=yuva420p[v];\
 [v]scale=1280:400[bg];\
 [v][bg]overlay=(W-w)/2:H-h[outv]"\
  -map "[outv]" -map 0:a -c:v libx264 -c:a copy \
  waveform-sqrt-cline.mp4
Enter fullscreen mode Exit fullscreen mode

OK, let's unwrap this:

  • Understandable.mp4 is the input file
  • 0xff1646@0.3 is the hexadecimal colour of the waves I want to create. The @0.3 is the opacity, 30%.
  • The first scale is how the bars should be resized to stay in a certain range. I found squareroot to look the best. Other options are available in the documentation
  • mode is what soundwave you want. This version, cline is a centered, filled line. Other options are point, line and p2p.
  • scale is the size of the generated video of the wave, in this case 1280x400 pixels (as the video is 1280x1080)
  • The overlay defines where on the background video the wave should appear. In this case centred in the horizontal direction and on the bottom of the video.
  • waveform-sqrt-cline.mp4 is the name of the final video.

Change location of the sound wave

You can also center the sound wave in both directions using overlay=(W-w)/2:(H-h)/2:

ffmpeg -i Understandable.mp4 \
 -filter_complex "[0:a]showwaves=colors=0xff1646@0.3\
 :scale=sqrt:mode=cline,format=yuva420p[v];\
 [v]scale=1280:400[bg];\
 [v][bg]overlay=(W-w)/2:(H-h)/2[outv]"\
  -map "[outv]" -map 0:a -c:v libx264 -c:a copy \
  waveform-sqrt-cline-centered.mp4
Enter fullscreen mode Exit fullscreen mode

And the result looks like this:

Video showing the sound wave in the vertical center

Other sound wave styles

And you can change the output type to line and change the colour:

ffmpeg -i Understandable.mp4 \
 -filter_complex "[0:a]showwaves=colors=0xffffff@0.5\
 :scale=sqrt:mode=line,format=yuva420p[v];\
 [v]scale=1280:400[bg];\
 [v][bg]overlay=(W-w)/2:H-h[outv]"\
  -map "[outv]" -map 0:a -c:v libx264 -c:a copy \
  waveform-sqrt-line.mp4
Enter fullscreen mode Exit fullscreen mode

Video showing the sound wave as lines

This is what the point to point mode looks like:

ffmpeg -i Understandable.mp4 \
 -filter_complex "[0:a]showwaves=colors=0xffffff\    
 :scale=sqrt:mode=p2p,format=yuva420p[v];\ 
 [v]scale=1280:400[bg];\
 [v][bg]overlay=(W-w)/2:H-h[outv]"\
  -map "[outv]" -map 0:a -c:v libx264 -c:a copy \
 waveform-sqrt-p2p.mp4 
Enter fullscreen mode Exit fullscreen mode

Video with point to point line

Adding a sound wave to a static image

If you only want the audio from a video and use a background image instead, this is what to use. This creates a 400x400 pixel video with the image as the background and the sound wave on top:

ffmpeg -i Understandable.mp4  -i chris.jpg\
 -filter_complex "[0:a]showwaves=colors=0xff1646@0.3\
 :scale=sqrt:mode=cline,format=yuva420p[v];\
 [1:v]scale=400:400[bg];\
 [bg][v]overlay=(W-w)/2:(H-h)/2[outv]"\
  -map "[outv]" -map 0:a -c:v libx264 -c:a copy \
  static-image.mp4
Enter fullscreen mode Exit fullscreen mode

Soundwave on top of a static image

FFMPEG is amazing

Granted, the syntax of FFMPEG is pretty nuts, but it is also super powerful. Want to know for example how I created the GIFs of this post?

First, I resized all the MP4s in the folder to 1/3 of their size and saved them as small*:

for f in *.mp4 ; \
do ffmpeg -i "$f" -vf scale=w=iw/3:h=ih/3 "small-$f" ;\
done             
Enter fullscreen mode Exit fullscreen mode

Then I took those and created gifs

for f in small-*.mp4 ; \
do ffmpeg -i "$f" -sws_dither ed -ss 5 -t 1 "e-$f.gif" ; \
done      
Enter fullscreen mode Exit fullscreen mode

In English: Get all small-*.mp4s in the current folder, and create a GIF from second 5 of the video, one second long, using error diffusion.

Automate all the things!

Soundwave from transformers pressing a button

Heroku

Build apps, not infrastructure.

Dealing with servers, hardware, and infrastructure can take up your valuable time. Discover the benefits of Heroku, the PaaS of choice for developers since 2007.

Visit Site

Top comments (1)

Collapse
 
ingosteinke profile image
Ingo Steinke, web developer

Thanks for using and explaining ffmpeg! not everything visual / multimedia needs to be done in flowblade / iMovie / Premiere / whatever GUI tool. I have been using ffmpeg for a long time and rediscovered it every few years. Thanks a lot for your post! I have bookmarked it on my reading list.

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs