DEV Community

Cover image for Automated Backups with cron and RClone
Marco Antonio Bet
Marco Antonio Bet

Posted on

Automated Backups with cron and RClone

#productivity #linux #bash #programming

This article will show you how to use RClone and cron to automated file backups in a Linux Operating System.

It is Friday past 6 PM, and I needed to back up some work and school files to Google Drive. Yes, I use git for coding projects, but this kind of work is better kept in Google Drive. So I wonder if there was a quicker, simple, and automated way of doing backups to Google Drive, and interesting, I found RClone.

In this article you will learn:

  • Whats is RClone.
  • What is cron.
  • Basic usage of RClone and cron.
  • How to automatically back up files every Friday at 6:30 PM.
  • Cron logging.
  • How to uninstall RClone.

Whats is RClone?

Basically, RClone is a command-line computer program that can manage, transfer and sync files from your computer to cloud storage, or even between two cloud storages.

Whats is cron?

Cron is a command-line utility software with the main purpose of scheduling jobs. Jobs are cron command with two parts basically, the first one is the schedule expression , the last is the script to be executed. Usually, all Linux OS comes with cron installed, if not:

CentOS/RHEL 7/8/5



yum install cronie
Enter fullscreen mode Exit fullscreen mode

Ubuntu



apt-get install cron
Enter fullscreen mode Exit fullscreen mode

With cron, you can schedule a command or a shell script to run at a specific time, and that is what we going to do. Schedule an RClone command to run every Friday at 6:30PM.

Setting up RClone

To start, install RClone.

For Linux, macOS, and BSD, just run:



curl https://rclone.org/install.sh | sudo bash
Enter fullscreen mode Exit fullscreen mode

After installing RClone, let’s run the RClone configuration command.



rclone config
Enter fullscreen mode Exit fullscreen mode

Here you will follow these steps:

  1. Choose n for a New remote.
  2. Type the name of the new RClone config, for example, mygdrive.
  3. Type 15 to use Google Drive as remote storage.
  4. For the next two steps, just type enter to continue.
  5. Now choose 1 for full access.
  6. For the next two choices, just type enter to continue.
  7. Sign-in into your google account in the RClone popup. Now you are ready to go.

Building a shell script to backup files to Google Drive

We are going to build a shell script that contains RClone command lines to sync folders from my computer to the Google Drive storage.

Open your terminal and type:



mkdir sh ; cd sh
Enter fullscreen mode Exit fullscreen mode

You don’t need to create a sh folder in your $HOME directory, you can create in another place or even do not create, but it is a good idea to organize your scripts in one place. If you create in another place, just remind yourself to change the path in the next steps.

Now if you want to use vim then:



vim backup.sh
Enter fullscreen mode Exit fullscreen mode

But if you want to use another text editor, just create a file named backup.sh in the $HOME/sh folder or any other place, but $HOME/sh will be my location for this article.

With the backup.sh file opened, let's write a line of code that syncs a folder from my computer to the Google Drive storage.



rclone sync -v --create-empty-src-dirs /$HOME/Desktop/Work mygdrive:/Work
Enter fullscreen mode Exit fullscreen mode

Now let’s dive into the explanations of the command.



rclone sync
Enter fullscreen mode Exit fullscreen mode

Sync will copy the files from the source and send them to the destination, but will not modify the source, only the destination.



--create-empty-src-dirs
Enter fullscreen mode Exit fullscreen mode

By default empty folders in the source are not synced, so to work around, this flag creates empty folders in the destination.

In the last part of the command, you have two paths separated by a space, the first one is the source path and the second one is the destination path. By now you should probably notice that the second path has a mygdrive: which is the same name as the rclone config name. So basically I am syncing data from my computer to Google Drive, using mygdrive configuration.

You can have multiple RClone configurations and this allows you to sync or copy files, not just from your computer to the cloud, but from one cloud to another. Just like this:



rclone sync -v --create-empty-src-dirs mygdrive:/Work anothergdrive:/Work
Enter fullscreen mode Exit fullscreen mode

To see if the backup.sh file works, just run the command below and check if the destination has changed.



source backup.sh
Enter fullscreen mode Exit fullscreen mode

After backup.sh working properly, let's understand how we can schedule the backup.sh to run every day.

Final script:



rclone sync -v --create-empty-src-dirs /$HOME/Desktop/Work mygdrive:/Work
rclone sync -v --create-empty-src-dirs /$HOME/Desktop/School mygdrive:/School
Enter fullscreen mode Exit fullscreen mode

Setting up cron to run Shell file

To schedule any job in cron we need to edit the crontab file, and to edit this file, open your terminal and type:



crontab -e
Enter fullscreen mode Exit fullscreen mode

If you never used crontab, then you will have to choose a default text editor, in my case it is vim.

It will open the crontab file with your default text editor. When the file is open you will see only comments.

To schedule in cron you will need a specific type of expression. Below you will see the structure of the expression:



.---------------- minute (0 - 59)
 |  .------------- hour (0 - 23)
 |  |  .---------- day of month (1 - 31)
 |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
 |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed
 |  |  |  |  |
 *  *  *  *  *  command to be executed
Enter fullscreen mode Exit fullscreen mode

Examples — Expression to schedule a job (my_shell.sh) to run every Monday at 8:49 AM is:



49 8 * * 1 my_shell.sh
Enter fullscreen mode Exit fullscreen mode

If you want to run a job (my_shell.sh) from Monday to Friday at 11:26 AM you can write:



26 11 * * 1-5 my_shell.sh
Enter fullscreen mode Exit fullscreen mode

More examples of expressions:



0 * * * *     => Every hour
*/2 * * * *   => Every two minutes
0 9-17 * * *  => At minute 0 past every hour from 9 through 17
Enter fullscreen mode Exit fullscreen mode

For more examples or to test if your expression will return the same logic as you expect, go to crontab for more information.

With the crontab file open, type the line below:



30 18 * * 5 /$HOME/sh/backup.sh
Enter fullscreen mode Exit fullscreen mode

What this command does is, every Friday at 6:30 PM backup.sh will be called, and it will back up my files to Google Drive.

Tips — Where is the cron log file?

By default the cron log file is located at:



/var/log/syslog
Enter fullscreen mode Exit fullscreen mode

And to see the cron log in this file, run the code below:



grep CRON /var/log/syslog
Enter fullscreen mode Exit fullscreen mode

or to continuously watch the file:



tail -f /var/log/syslog | grep CRON
Enter fullscreen mode Exit fullscreen mode

This command will return something like this:



Dec 15 07:35:01 rainbowone CRON[242416]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Dec 15 07:45:01 rainbowone CRON[245607]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Dec 15 07:55:01 rainbowone CRON[248793]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Dec 15 08:00:01 rainbowone CRON[249386]: (rainbow) CMD (/$HOME/Desktop/sh/daily.sh)
Enter fullscreen mode Exit fullscreen mode

These messages only show which command cron ran, if you want to see the backup.sh log you will need to personalize the backup.sh to create its own log file, and write the content of it.

But assuming you want to log the cron message to a specific file, luck of yours I will show you how.

First of all, open this file:



/etc/rsyslog.d/50-default.conf
Enter fullscreen mode Exit fullscreen mode

Uncomment the line that starts with #cron.* and then restart syslog with:



sudo service rsyslog restart
Enter fullscreen mode Exit fullscreen mode

After that you will see cron logs in /var/log/cron.log and /var/log/syslog.

How to uninstall RClone?

It is very simple to uninstall RClone from a Linux operating system, first of all, let’s find where the rclone configuration file is, to find the file run the command below.



rclone config file
Enter fullscreen mode Exit fullscreen mode

It will list one or more RClone configuration paths.



Configuration file is stored at:
/home/$USER/.config/rclone/rclone.conf
Enter fullscreen mode Exit fullscreen mode

To remove the configuration file:



sudo rm -rf /home/$USER/.config/rclone/rclone.conf
Enter fullscreen mode Exit fullscreen mode

And to finally remove RClone form the system:



sudo rm /usr/bin/rclone
sudo rm /usr/local/share/man/man1/rclone.1
Enter fullscreen mode Exit fullscreen mode

At this point, you are probably able to automate file backups with cron and RClone in a Linux operating system.

If you like this article you can give a ❤️, and 🔖 for later in the save button.

And if you like content about Git, Linux, Productivity tips, Typescript, and Python please follow me Marco Antonio Bet

GitHub logo itsbetma / CronAndRCloneTutorial

Repository for the How to use RClone and cron to automate backup in Linux.

CronAndRCloneTutorial




View on GitHub


Top comments (2)

Collapse
 
dlibian profile image
Ian Lessing •

rclone also has a --log-file parameter which could be utilized to log what rclone does.

Collapse
 
stochiometric profile image
Man •

There's an issue with this tutorial:

The bash script must be made executable by "chmod +x "thelocationtoyourscript" "
Also instead of "rclone" in the script, type "/usr/bin/rclone"

Read next

mikeyoung44 profile image

Microsoft's Phi-4: Smaller AI Model Achieves Big Results Through Clean Training Data

Mike Young -

pragativerma18 profile image

Understanding JavaScript Deobfuscation in Web Scraping

Pragati Verma -

msnmongare profile image

How to SSH into a Server Using IP Address, Username, and Password

Sospeter Mong'are -

respect17 profile image

My 2024 Journey: Learning from My Mistakes as a Junior Dev

Kudzai Murimi -