DEV Community

Cover image for Simple backup service with rclone, restic and systemd on WebDAV (or another storage)
Patrice Ferlet
Patrice Ferlet

Posted on

Simple backup service with rclone, restic and systemd on WebDAV (or another storage)

Let me show you how I back up my computers with restic and rclone using simple systemd configuration.

On my previous article, I presented how I was able to have a nice NAS (storage) with Nextcloud. It has got the good idea to propose a WebDAV endpoint that can be mounted on our Linux computers, or on Windows with the client.

By chance, Gnome provides an 'Online Account' option to automatically mount my remote hard drive on Nautilus, the file manager, and to synchronize contacts, calendars, etc.

But, I wanted to have a “real” backup service too.

Because a "real" backup system is not only to send files on a remote drive. It is a complete snapshot management, with incremental checks, compression, security...

This signifies that I intend to create backups for certain directories, excluding others, to retrieve files from a period of 7 days, 1 week, 1 month, or even 1 year ago.

This is what we call an "incremental" backup system.

☝️ This article proposes a backup configuration for a "user" on Linux but you can adapt the configuration to backup a server or your system as an administrator.

💡 What we will see in this article

  • What are Restic and Rclone commands
  • How to back up some directories and restore them
  • How to automate the backup with systemd for a Linux user (or for an admin account)

TL;DR

# Create your configuration to connect a storage
# if restic doesn't provide the connector (e.g. for WebDav).
# Name your connection, for example "nas"
rclone config

# initialize the backup storage in a directory,
# e.g. Backups
restic -r rclone:nas:Backups init

# Create a simple backup one shot
restic -r rclone:nas:Backups backup $HOME/Images --verbose

# Forget some backups:
restic -r rclone:nas:Backups \
    forget --prune \
    --keep-daily 7 \
    --keep-weekly 4 \
    --keep-monthly 6 \
    --keep-yearly 1

# This is to make management easiers:
# Create files to list directories to backup and what to exclude.
# It support "patterns"
echo "$HOME/Documents" >> ~/.config/backup.list
echo "*.tmp" >> ~/.config/backup.exclude


# example of backup command:
restic -r rclone:nas:Backups backup \
    --files-from ~/.config/backup.list \
    --exclude-file ~/.config/backup.exclude

# Scroll to the bottom of the article to find systemd
# service and timers

# Done !
Enter fullscreen mode Exit fullscreen mode

Please read the article to see how to restore a snapshot, to simplify the management, …

Incremental backup and GFS Backups?

An incremental backup is a method of backing up solely the modifications from the most recent backup to conserve space.

Moreover, we can do a GFS backup.

GFS? What is this?

GFS means " Grandfather-Father-Son Backup", this is a method to make daily backup every day but:

  • All backups older than a certain date, such as 7 days old, are removed.
  • Keep a backup each week for four weeks.
  • Keep a backup every month for six months.
  • Keep one backup per year, per year.

This facilitates the storage of substantial data in a timely manner while utilizing minimal space.

It's a very common method, I use it at work to backup dozen of servers..

🎉 ❤️ And yes, Restic can do all of this!

Restic

Restic is a backup command line. It can use many local or remote endpoints like SFTP, Samba, or S3 protocol.

Restic can make backups, and make snapshot restoration, cleanup, mounting snapshots to navigate into, secure data by encryption, and many more! 👏

Unfortunatly, Restic doesn't support WebDav, but there is a solution: using rclone adapter.

Restic will create a “repository”. It's a remote or local path where Restic will keep snapshots, data, and status.

The Restic repository on my NAS

We will initialize it in a while.

Rclone

Rclone is a command line to send / synchronize and manage local or remote directories. It is compatible with a huge number of providers like Google Drive, Amazon S3, S3 compatible, Samba, Proton Drive, Dropbox… and WebDAV!

Create a Rclone configuration

OK, in my case, I need to contact my local WebDAV (nas.home is a domain locally assigned by my router, an Orange Livebox)

On my Nextcloud account, browsing my files, I can see the WebDAV URL in the settings:

Where to find WebDAV URL

In a terminal, I only do:

rclone config
Enter fullscreen mode Exit fullscreen mode

I needed to find the WebDAV option, then Nextcloud, then give my URL, user and password.

Another way is to use one command line to set it up all:

# change the url, user and password of course
rclone config create nas \
webdav \
url https://nas.home/remote.php/dav/files/Patrice \
user Patrice \
vendor nextcloud \
pass $(rclone obscure 'TYPE YOUR NextCloud Password here')
Enter fullscreen mode Exit fullscreen mode

⚠️ The obscure command is not a encrption method. It "obscures" the password. It is only made to avoid "direct reading". Rclone can read the original string.

To check if it works:

# liste directories only
rclone lsd nas:
Enter fullscreen mode Exit fullscreen mode

If it works, you may continue

Restic, initialization

We have to initialize a Restic repository. This will create the data and configuration on the remote storage.

So, we created a nas: Rclone configuration to access our WebDAV endpoint; let's use it:

restic -r rclone:nas:Backups init
Enter fullscreen mode Exit fullscreen mode

The command will ask you to provide a password. This password is used to encrypt your backup! This is not your WebDAV password, this is a secret to avoid others to read your backups.

⚠️ If you lose this secret, you will never be able to restore snapshots later!

Now, Restic is ready.

Test Restic backup

First, create a simple directory where we will store a simple file.

mkdir /tmp/data
echo "hello" > /tmp/hello.txt
date > /tmp/data/file.txt
Enter fullscreen mode Exit fullscreen mode

Let backup this directory:

restic -r rclone:nas:Backups backup /tmp/data
# it will ask you the repository password

# output
repository 52592651 opened (version 2, compression level auto)
no parent snapshot found, will read all files
[0:01] 100.00%  5 / 5 index files loaded

Files:           2 new,     0 changed,     0 unmodified
Dirs:            2 new,     0 changed,     0 unmodified
Added to the repository: 1.902 KiB (1.361 KiB stored)

processed 2 files, 38 B in 0:03
snapshot cb53a1b9 saved
Enter fullscreen mode Exit fullscreen mode

The directory is now snapshot in the Backups repository.

Let's take a look:

restic -r rclone:nas:Backups snapshots
# output
repository 52592651 opened (version 2, compression level auto)
ID        Time                 Host            Tags        Paths                           Size
--------------------------------------------------------------------
cb53a1b9  2025-01-22 22:13:21  patrice-laptop              /tmp/data                       38 B
--------------------------------------------------------------------
1 snapshots
Enter fullscreen mode Exit fullscreen mode

(Actually there are many other snapshots in my output as I have already made several backups)

OK, so the snapshot is here. Let's break our local directory:

# oops, accidentally removed :)
rm /tmp/data/hello.txt

# remove the date, write "broken"
echo "broken" > /tmp/data/file.txt
Enter fullscreen mode Exit fullscreen mode

How to recover?

The simplest is to use the snapshot ID and specify where to restore:

restic -r rclone:nas:Backups restore cb53a1b9 --target /
# output
repository 52592651 opened (version 2, compression level auto)
[0:03] 100.00%  7 / 7 index files loaded
restoring snapshot cb53a1b9 of [/tmp/data] at 2025-01-22 22:13:21.433737756 +0100 CET by metal3d@patrice-laptop to /tmp/data
Summary: Restored 4 files/dirs (38 B) in 0:04
Enter fullscreen mode Exit fullscreen mode

Why "/" and not "/tmp/data"? Because the full path is stored in restic.

ℹ️ You can use latest instead of the cb53a1b9 ID (which is different for you). latest means "the latest snapshot". Using the ID is interesting to get an older snapshot, for example if the latest snapshot contains a broken file.

Let it restoring, and…

$ tree /tmp/data
/tmp/data/
├── file.txt
└── hello.txt

# back!
$ echo /tmp/hello.txt
hello

# not broken :)
$ echo /tmp/file.txt
mer. 22 janv. 2025 22:12:47 CET
Enter fullscreen mode Exit fullscreen mode

Then:

  • you can create other snapshots, it “incrementally” adds files.
  • if nothing changed in your local directory, so the snapshot will be “empty”, it's normal
  • only the newest files and changed content will be stored in the following snapshots — it saves space!
  • you can recover snapshots at a certain date

But... We need to cleanup very old backups

To clean backups, the forget command is simple to use.

I want to keep 7 days of backups, but also 1 backup per week for 4 weeks, and 1 backup per month for 6 months, and finally one backup per year for one year…

restic -r rclone:nas:Backups forget \
--keep-daily 7 \
--keep-weekly 4 \
--keep-monthly 6 \
--keep-yearly 1 \
--prune
Enter fullscreen mode Exit fullscreen mode

OK, there are a lot of information. And there are plenty of options and commands in restic. Read the doc 😉

⚠️ Crucial:

If you never heard about incremental backup, you may wonder if a large directory will consume all your CPU and bandwidth every day.

No

This is why a “real” backup system is very intriguing. The first backup is a bit long as there is no track of the files you're backing up.

The next backup will only take into account the unknown and changed files.

So then:

  • you can revert files as if they were versioned
  • and the next backups will be faster, really faster

Some nice features

Something very interesting when you want to restore a file, is that you can use --include to only get one file or directory. Moreover, there is a find subcommand to find a file.

But, another way is to “mount” the snapshots on your host.

Yes, you can navigate the snapshots

Mounting snapshots

It only works on Linux... OK for me 😉

If you use your WebDAV drive, or the web interface of Nextcloud, the snapshots are encrypted and not usable:

On WebDAV, snapshots are not browsable

But, using restic mount:

mkdir /tmp/restic
restic -r rclone:nas:Backup mount /tmp/restic
Enter fullscreen mode Exit fullscreen mode

You can now browse the snapshots:

Restic mount snapshots, it's now OK

This is very comfortable to be able to navigate snapshots, at a certain date, check the content of a file and to be able to copy and paste it.

Now, automate with Systemd

There are several new things that are interesting to know:

  • you can set the Restic password as RESTIC_PASSWORD environment variable
  • you can set the Restic repository name in the RESTIC_REPOSITORY environment variable
  • you can provide several directories to snapshot at once
  • the list can be set in the command line, or inside a text file

So, with this information, we can create a user service and associated timer — this to automate backup.

💡 If your want to use Restic for a server or to backup a root system, so you only need to change the paths to /etc/systemd/system to save files, and do not use --user option.

The service in “~/.config/systemd/user/backup.service”:

[Unit]
Description=Backup service to NextCloud

[Service]
Type=oneshot
Nice=19
# environment variables to simplify commands
Environment=RESTIC_PASSWORD=YOUR PASSWORD HERE
Environment=RESTIC_REPOSITORY=rclone:nas:Backups

# Call sequentially
ExecStart=/usr/bin/restic backup --files-from %h/.config/backup.list --exclude-file %h/.config/backup.exclude --verbose
ExecStart=/usr/bin/restic forget --prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --keep-yearly 1

[Install]
WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

The Type=oneshot is important:

  • you can start several commands sequentially
  • it will not restart as other services that are expected to be continuously “alive”

As you can see, I'm using files to list directories and exclusion.

Let's create them:

# for example, backup Document folder
echo $HOME/Documents >> ~/.config/backup.list
# and exclude all .tmp files
echo '*.tmp' >> ~/.config/backup.exclude
Enter fullscreen mode Exit fullscreen mode

You can edit the file with a text editor, of course.

Then, create the timer in ~/.config/systemd/user/backup.timer:

[Unit]
Description=Planify the Restic backup service

[Timer]
# you can set specific date / time
# OnCalendar=*-*-* 02:00:00
# or simply leave the system default which
# is 3:00 am
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target
Enter fullscreen mode Exit fullscreen mode

It's important to have the same basename for the timer and the service.

OK, let's go:

systemctl --user daemon-reload
systemctl --user enable --now backup.timer
Enter fullscreen mode Exit fullscreen mode

And voilà! Every day, a snapshot will be made for all directories you've listed in the list file. And it avoids all directory or patterns that you've listed in the exclude file.

To check if the backup works, you can, of course, launch the service right now:

# Warning, if the backup is huge, this command will
# stay alive for a long time. Test with a tiny directory to backup.
systemctl --user start backup
Enter fullscreen mode Exit fullscreen mode

Conclusion

Restic and Rclone are both powerful. Typing a few command lines to manage backup is much more comfortable on Unix/Like and Linux systems (it works on FreeBSD, OpenBSD, macOS…)

It can work on Windows computers. But I never tried (I don't have any Windows machine at home)

Anyway, one more time systemd is IMHO one of the best things that happened to Linux. Whatever others think, I love it.

And, again, using a terminal command that does one thing, but does it well, is a power in our hands.

Do your career a big favor. Join DEV. (The website you're on right now)

It takes one minute, it's free, and is worth it for your career.

Get started

Community matters

Top comments (0)

👋 Kindness is contagious

Explore a sea of insights with this enlightening post, highly esteemed within the nurturing DEV Community. Coders of all stripes are invited to participate and contribute to our shared knowledge.

Expressing gratitude with a simple "thank you" can make a big impact. Leave your thanks in the comments!

On DEV, exchanging ideas smooths our way and strengthens our community bonds. Found this useful? A quick note of thanks to the author can mean a lot.

Okay