Sync one directory into another with compression:
rsync -az source/directory destination/directory
Sync over SSH:
rsync -az firstname.lastname@example.org:path/to/file destination
rsync -az source/directory email@example.com:destination/path
Sync does not delete existing files at destination. To make sure the source and destination folders look identical, use the
rsync -az --delete source/directory destination/directory
Rsync is this neat utility that commonly exists on Linux systems (and other Unix-likes). It's like a copy command but the destination (or source) can be remote.
It syncs files rather than copies them so if the destination file is already present, it doesn't get overwritten. This is great because you can sync large folders several times but only the missing files will get copied.
So while you might run something like
cp -r ~/projects/dotfiles/.bashrc ~ to copy a
.bashrc file from a folder to your home directory, rsync lets you do it in a remote fashion like so:
rsync firstname.lastname@example.org:path/to/file ~. Or the other way around
rsync path/to/file email@example.com:path/to/destination.
To me, what makes rsync complicated are the flags. It's always the flags that trip me up and push me to feel uncertain about the command.
You lose a lot by not providing flags. Without flags, you can also sync one file at a time.
rsync source/file destination
You can also do remote copy:
rsync serverUser@serverAddressOrIP.com:path/to/file destination
ssh for remote connections so make sure you have access via ssh!
Copying the other way is the same deal:
rsync path/to/file serverUser@serverAddressOrIP.com:destination
rsync -flags source destination
There are several helpful flags and if you follow any guides online, you'll most likely come across these same exact ones.
a- stands for
archive. It is necessary for recursive copies (ie. copying whole directories)
zflag enables compression for faster copying across networks
v- for verbose output
archive option has always been confusing to me. Luckily, there's a great server fault post detailing the archive flag. It's a compilation of other flags that allows recursive syncing - so directory copy/sync support. And if you're copying locally, it preserves permissions and a bunch of other stuff.
z flag enables compression via Zlib (at least that's what Wikipedia tells me!). In case you wanted to know, zlib uses the same compression technique as
gzip which you can read about in my Demystifying TAR article.
The verbose output should display files copied file-by-file. A typical rsync command might look like this:
rsync -azv ./files-to-upload firstname.lastname@example.org:~/uploads
Don't read this section of rsync is already giving you a headspin. But if you ever want to refer back to this article, you can check these flags out. I wouldn't use any of these by default you might come across them in tutorials/articles or you might want to use them yourself:
n- stands for
dry run. This will check that you actually can transfer those files.
delete- delete files from destination that don't exist in source
Pflag is a combination of the
n flag is handy because you can use it to make sure you're running your command correctly. Combined with a verbose output, you can see exactly what's going to happen without making it happen. I found this useful when preparing to upload a lot of files and not wanting to screw up where they upload or which files upload.
delete flag is also super useful in that it doesn't just copy files to the destination, it ensures that files in the source directory don't exist in the destination. Super useful when syncing from a source that you consider the "source of truth" and want to have identical copies elsewhere...without having to delete everything and copy everything from scratch.
I don't often use the
P flag but for many, it's the default. The
P flag will do 2 very important things:
- allow partial file upload + resume
- display a progress bar
If you check the screenshot above, I ran that copy with a
P flag. At the end, it'll let you know neat stuff about how much data you've transferred if you combine it with the
The wonderful part about partial file uploads is that if your connection fails, you can run the command again and it'll pick up where it left off with the partially uploaded files. Without the
P flag, rsync would not be able to finish uploading the interrupted file.