In this article, we’ll see how you a real case example of a Redis backup process, end-to-end.
We'll cover how to configure a Redis backup, which binaries you should use and the most important settings you need to be aware of, as well as how to store your backup remotely and how to restore it.
Let's get started!
What is Redis?
Redis is an open source in-memory key-value store written in C.
Redis stands for Remote Dictionary Server and is used as a database, cache, queue system, and message broker.
Redis is fast because its data is stored in memory, meaning, unlike traditional databases Redis doesn't need to access the disk.
While writing this article I learned that Redis is often called a "Data Structure Server" because it provides data types that are similar to those in modern programming languages.
Some data structures that Redis provides are Strings, Lists, Sets, Hashes, and Sorted Sets.
What is Redis used for?
Even though Redis could be used as your primary Database it's usually not what it is used for.
Here are the most common use cases for Redis:
- Cache: Redis is used as a cache, which is a fast way to store data in memory.
- Session storage: Redis is used to store session data. Writing and reading data out of Redis is super fast which makes it an ideal candidate for session storage.
- Message queue
There are a lot of different use cases for Redis, but these are the most common ones.
Where is Redis database stored?
As stated above Redis is storing its data in memory. But depending on your use case Redis can copy the data on your disk.
This comes obviously handy when you have a large amount of data, and you need to be able to restore it and this is also why you might be needing to back it up.
Redis regularly dumps its data to an RDB file on the disk based on how snapshots are configured.
Redis configuration
The
file contains your Redis configuration.
The configuration file is located at
```/etc/redis/redis.config```
and straighforward: it's a list of instructions.
You'll find a section called
```#### SNAPSHOTTING ####```
on which you can define if Redis should snapshot its data to the dis and how often it should do it.
```redis.config
save 60 1000
This configuration will make Redis dump the dataset to the disk every 60 seconds if at least 1000 keys are changed.
Learn more about Redis Configuration →
Redis also works with AOF (Append-Only File) which is a way to store the data on the disk by logging all write operations received by the server.
AOS won't be covered in this article but it is worth knowing it exists, especially when you'll need to backup and restore your data.
How to back up Redis data
Making a Redis backup is pretty easy. You'll need to make a fresh copy of the RDB file, compress it and save it somewhere safe.
Redis offers 2 methods to "force" a snapshot:
: This will force a snapshot to be taken synchronously.
-
```BGSAVE```
: This will force a snapshot to be taken asynchronously.
The easiest way is to use the
```SAVE```
method but it will block all other operations until the snapshot is done.
Using
```BGSAVE```
will make the server continue to accept commands and will not block other operations but you'll have to figure out when the snapshot is this one is asynchronous.
**So, if you want to make a backup you'll need to do the following:**
1. Review / update your Redis configuration.
2. Create a snapshot for your Redis data (known as a "dump"/"rdb file").
3. Save the RDB file to a remote location
### 1. Review / update your Redis configuration.
You'll need to know where your snapshot file will be generated using the redis-cli command described below.
The default location of your Redis config file is
```/etc/redis/redis.config```
.
You can also use this command to find the location of your Redis config file:
```redis-cli config get dir```
.
You can find the configuration file here [https://redis.io/topics/config](https://redis.io/topics/config).
### 2. Create a snapshot for your Redis data.
#### <u>Using redis-cli
```SAVE```
command</u>
This method will work synchronously to make a snapshot of your Redis database.
Just ssh into your server and run the following command:
Log in to the database command line interface:
redis-cli
You might have to authenticate to your database:
```shell{promptHost: 127.0.0.1}
auth YOUR_PASSWORD_HERE
SAVE
The output will be something like this:
OK
(1.23s)
You can then exit the redis-cli by typing
.
At this stage, the
```RDB```
file will be saved in
```/var/lib/redis/```
and will be named
```dump.rdb```
.
#### <u>Using redis-cli
```BGSAVE```
command</u>
Using the asynchronous dump function, you'll need to make sure you are aware of the end of the process.
One way to do it is to use
```inotifywait```
which will notify you when a change to the dump file is made.
### 3. Automate your Redis backups and store them on AWS S3
As stated in [Redis documentation](https://redis.io/docs/manual/persistence/), it's safe to copy the RDB file even if used by your running server:
> Redis is very data backup friendly since you can copy RDB files while the database is running: the RDB is never modified once produced, and while it gets produced it uses a temporary name and is renamed into its final destination atomically using rename(2) only when the new snapshot is complete.
>
> This means that copying the RDB file is completely safe while the server is running. This is what we suggest:
>
> - Create a cron job in your server creating hourly snapshots of the RDB file in one directory, and daily snapshots in a different directory.
> - Every time the cron script runs, make sure to call the find command to make sure too old snapshots are deleted: for instance you can take hourly snapshots for the latest 48 hours, and daily snapshots for one or two months. Make sure to name the snapshots with date and time information.
> - At least one time every day make sure to transfer an RDB snapshot outside your data center or at least outside the physical machine running your Redis instance.
We'll create a script that will create our dump file, then upload to Amazon S3.
#### <u>Create a shell script that will dump the Redis database</u>
```shell
cd ~
mkdir scripts
cd scripts
nano redis_backup.sh
Copy and paste the script below to it:
#!/bin/bash
rdb_file="/FOLDER_TO_YOUR_REDIS_DUMP_FILE/redis/dump.rdb"
redis_cli="/usr/bin/redis-cli"
DIR=`date +%d-%m-%y`
DEST=~/redis_backups/$DIR
mkdir $DEST
echo save| $redis_cli
exit 1
Send the Redis DUMP to an AWS S3 bucket
Append to
the following:
```shell
BUCKET_NAME="YOUR_S3_BUCKET_NAME"
aws s3 cp $rdb_file s3://YOUR_S3_BUCKET/redis_backups/ && echo "Backup copied to S3"
Schedule the script to run every day at midnight
First, let's CHMOD the script to make it executable:
chmod +x ~/scripts/db_sync.sh
Then create a cron job to run the script every day at midnight:
crontab -e
0 0 * * * ~/scripts/redis_backup.sh # take a backup every midnight
How to restore a Redis backup
Now that you've made a backup, we'll see how to restore it from a
file.
**We recommend you first try this on a fresh Redis server**
### Make sure AOF is disabled
AOF stands for Append-Only File, which will instruct Redis to log all operations in a `.aof` file.
Since we're restoring a backup, we need to disable AOF before restoring the data as we don't want Redis to log all these operations.
Open your configuration file (
```redis.config```
) and make sure
```appendonly```
is set to
```no```
.
appendonly no
### Stopping the Redis server
Before being able to replace the
```dump.rdb```
file, you'll need to stop the Redis server.
```shell
sudo service redis-server stop
Restoring the Redis database
Prior to restoring the database, you can rename the existing dump.rdb file in order to have a restore point in case something goes wrong.
sudo cp /home/redis/dump.rdb /home/redis/dump.rdb.bak
You can then copy the backup rdb file as follows:
sudo cp /redis_backups/20220810/dump.rdb /home/redis/dump.rdb
And finally make sure to apply the right permissions to the dump.rdb file:
sudo chmod 660 /home/redis/dump.rdb
Re-starting Redis server
sudo service redis-server start
Additional Resources
- Official Redis documentation: https://redis.io/docs/manual/persistence/
- Service to automate your Redis backups: https://simplebackups.com/redis-backup/
Conclusion
We've seen how to back up your Redis database and restore it, and we've seen how to automate the process.
SimpleBackups will save you a lot of time setting up scripts, ensuring they run without problems, all without code or maintenance.
It will alert you when things go wrong, and allows you to store your backups on many cloud storage services like Google, DigitalOcean, Wasabi, Dropbox, and more…
Top comments (0)