DEV Community

Cover image for Sync folders using diff + awk
Sérgio Araújo
Sérgio Araújo

Posted on

4 4

Sync folders using diff + awk

Real life diff use:

I want to copy only the missing 'jpg' files at the ~/tmp/wallpapers

diff -u ~/img/backgrounds ~/tmp/wallpapers

Only in /home/sergio/tmp/wallpapers: .git
Only in  /home/sergio/tmp/wallpapers: .gitignore
Only in  /home/sergio/tmp/wallpapers: README.md
Only in  /home/sergio/tmp/wallpapers: thumbs
Only in  /home/sergio/img/backgrounds: wallpaper-1791868.jpg
Only in  /home/sergio/img/backgrounds: wallpaper-2099153.jpg
Only in  em /home/sergio/img/backgrounds: wallpaper-2226037.jpg
Enter fullscreen mode Exit fullscreen mode

Using awk with two field separators [: ] I only need the third and fifth fields to compose a copy command:

diff -u ~/img/backgrounds ~/tmp/wallpapers | awk -F'[: ]' '/backgrounds/ {print $3"/"$5}'

/home/sergio/img/backgrounds/wallpaper-1791868.jpg
/home/sergio/img/backgrounds/wallpaper-2099153.jpg
/home/sergio/img/backgrounds/wallpaper-2226037.jpg
Enter fullscreen mode Exit fullscreen mode

Now I only have to add some strings to my awk command and pipe the result to the shell

"cp " ............... copy comand with space
$3  ................ the third fild - the basename
"/" ................ add the slash so we do not mess things up
$5 ................. the missing file name
" ~/tmp/wallpapers"  destination folder
| sh ................ pipe the preceding command to the shell

diff -u ~/img/backgrounds ~/tmp/wallpapers \
| awk -F'[: ]' '/backgrounds/ {print "cp " $3"/"$5" ~/tmp/wallpapers/"}' | sh
Enter fullscreen mode Exit fullscreen mode

An interesting thing is that the backgrounds folder has no .git files on it but the wallpapers do, that's why I am filtering '/backgrounds/' with awk and not using recursion '-r', because the wallpapers folder has a thumbs subfolder

Image of Timescale

Timescale – the developer's data platform for modern apps, built on PostgreSQL

Timescale Cloud is PostgreSQL optimized for speed, scale, and performance. Over 3 million IoT, AI, crypto, and dev tool apps are powered by Timescale. Try it free today! No credit card required.

Try free

Top comments (0)

AWS Security LIVE!

Tune in for AWS Security LIVE!

Join AWS Security LIVE! for expert insights and actionable tips to protect your organization and keep security teams prepared.

Learn More

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay