rsync is a command for move and synchronizing files. This command help us to manage files or directories effectively when backing up data on a regular basis. rsync compares any differences and only transfer those differences.
scp is used for copy files and directories in secure way. Basically, It is a plain linear copy. scp reads the source file and writes it to the destination.
Basic usage :
$ rsync OPTION SOURCE DESTINATION
We need to check if we have
rsync, because some machines might not have this command.
Not passing the
It will sync between two directories, but only files in the main directory, not include files in the subdirectories.
# Sync all files in original directory to duplicate directory $ rsync SOURCE/* DESTINATION/
Sync all files in all the main directory and subdirectories
# Sync all files in original directory and its subdirectories $ rsync -r SOURCE/ DESTINATION/
This option allow us to copy almost everything, including file permissions, user & group ownership and timestamps.
It gives us a summary about the data transferred.
We will get the result in human readable format.
Compress file data during the transfer
To specify the type of protocol we use. We can choose using
# Sync using ssh protocol $ rsync -avhze ssh SOURCE/ DESTINATION/
To show the progress while transferring the data
# Show the progress during transfer $ rsync -avhze ssh --progress SOURCE/ DESTINATION/
To updated more recently on the local filesystem. Files that don't exist are copied. Files that already exist, but have a newer timestamp are also copied.
# Update only if there is a newer version $ rsync -avhze ssh --progress --update SOURCE/ DESTINATION/
To remove files that successfully transferred.
# Sync using ssh protocol $ rsync -avhze ssh --remove-source-files SOURCE/ DESTINATION/
Delete files that have been deleted in the original directory
# Update duplicate directory if there are files deleted in original directory $ rsync -avhze ssh --delete SOURCE/ DESTINATION/
This option will include files that we specify in the parameter
# Include files that matched with include paramaeter $ rsync -avhze ssh --include 'KEYWORD' SOURCE/ DESTINATION/
This option will exclude files that we specify in the parameter
# Exclude files that matched with exclude paramaeter $ rsync -avhze ssh --exclude 'KEYWORD' SOURCE/ DESTINATION/
Change file permission in the destination directory
# Copied files permisson are changed form user to group $ rsync -avhe ssh --chown=USER:GROUP SOURCE/ DESTINATION/
This option perform a trial run and will not make any changes, but gives us the same result as a real run. If the results are as expected, then we can remove the
# Perfom dry run $ rsync --dry-run -avhze ssh --delete SOURCE/ DESTINATION/
Set the maximum files to transfer. This option will only transfer files that are equal or smaller than the max size.
# Set max size 200k $ rsync -avhze ssh --max-size='200k' SOURCE/ DESTINATION/
We can set the bandwidth limit when we sync data from a machine to another machine.
# Set bandwidth rate 5000KBytes per second $ rsync -avhze ssh --bwlimit=5000 SOURCE/ DESTINATION/
Basic usage :
$ scp OPTION SOURCE DESTINATION
- Copy a single file
# Copy a single file $ scp SOURCE_FILE.txt DESTINATION/
- Copy multiple files
# Copy a multiple file $ scp FILE1.txt FILE2.txt FILE3.txt DESTINATION/
- Copy .pem file
# Copy aws key to vm $ scp -i file.pem file.pem DESTINATION/
-rto copy a directory recursively
# Copy all files in a directory $ scp -r SOURCE/ DESTINATION/
-Pto specify the port
scpwill use port 22, but we can use another port for security reason
# Use port 2249 $ scp -P 2249 SOURCE/ DESTINATION/
To show detailed information of the scp process
# Use port 2249 $ scp -v SOURCE/ DESTINATION/
-llimit the bandwidth usage
# Set the bandwidth limit $ scp -l 400 SOURCE/ DESTINATION/
-qfor quite mode
Run the command in quite mode, and only display critical errors
-pfor preserves modification times, access times, and modes from the original file
-uto delete source files after the transfer is successful
Both of the commands are similar, lets choose what we use based on our case