In the past, I was in charge of backup a MongoDB instance that runs in Azure Kubernetes Services (AKS). This database has at least 2 TB of data that was being persisted in an Azure Managed Disk with Kubernetes Persistent Volume (PV).
I could not run the “mongodump” without issues. This was overloading the database and take a long time to complete the backup process. The same behavior happened when I tried to restore the database backup.
Mongodump is a utility tool that allows us to create MongoDB backups. You can check this tool here.
Then, instead of run the “mongodump” tool, I applied options below that I found in Microsft Documentation:
- Take a backup from the Azure Managed Disk with Azure Backup.
- Create a scheduler backup to MongoDb’s PVC with the Velero tool.
You can see the documentation about AKS backup in this link from Microsoft Documentation.
The first option basically, allows you to take a snapshot from the disk that is being used by Kubernetes PVC. Later, you can restore this snapshot if you need it.
For the second option, you can use a tool called Velero to backup the whole AKS cluster, or just some Kubernetes resources, like Deployments, Services, persistent volume claims (PVC), PV, etc.
The backup destination created by Velero could be set up with plugins, which have support for the principal cloud providers, like Azure, AWS, and Google Cloud. In the case of Azure, the backup will be stored in Azure Account Storage.
In the end, what I did was create a backup from PV that MongoDB is using to persist its data.
So, in this article, I will present how to install Velero and how to use it to create and restore backups.
Setup Velero
The setup of Velero in the environment is very simple. Beyond the Azure account, you will need Azure Account Storage and AKS cluster.
After the resources created in Azure, you can apply the steps below to setup Velero in your environment:
- Install Azure CLI and kubectl tool in the machine that will apply the next steps.
- Connect to the AKS cluster. You can use this link to connect to your cluster.
- Create environment variables to store Azure Account Storage configurations.
# environment variables | |
export AZURE_STORAGE_ACCOUNT_ACCESS_KEY=<<your storage account access key>> | |
export BLOB_CONTAINER=<<blob container>> | |
export AZURE_BACKUP_RESOURCE_GROUP=<<your resource group>> | |
export AZURE_STORAGE_ACCOUNT_NAME=<<storage account name>> | |
export AZURE_BACKUP_SUBSCRIPTION_ID=<<your subscription id>> |
- Create a credential file. This file will be used to configure the Azure Account Storage credentials in Velero.
# Azure credential file | |
cat << EOF > /tmp/credentials-velero | |
AZURE_STORAGE_ACCOUNT_ACCESS_KEY=${AZURE_STORAGE_ACCOUNT_ACCESS_KEY} | |
AZURE_CLOUD_NAME=<<any name>> | |
EOF |
- Install Velero client. You can find the Velero client on its home page.
- Finally, install Velero in the AKS cluster. After that, we can create backups and restore them.
velero install \ | |
--provider azure \ | |
--plugins velero/velero-plugin-for-microsoft-azure:v1.1.0 \ | |
--bucket $BLOB_CONTAINER \ | |
--secret-file /tmp/credentials-velero \ | |
--backup-location-config resourceGroup=$AZURE_BACKUP_RESOURCE_GROUP,storageAccount=$AZURE_STORAGE_ACCOUNT_NAME,storageAccountKeyEnvVar=AZURE_STORAGE_ACCOUNT_ACCESS_KEY,subscriptionId=$AZURE_BACKUP_SUBSCRIPTION_ID \ | |
--use-volume-snapshots=false |
The Velero installation will create a set of new Kubernetes resources in the AKS cluster. Velero will use the Kubernetes namespace “velero”. Then, to remove Velero from the AKS cluster, apply the command below:
kubectl delete namespace/velero clusterrolebinding/velero | |
kubectl delete crds -l component=velero |
Also, like any Kubernetes resources, we can update it to achieve your requirements. For example, we can update resource limits from its Deployment:
kubectl patch deployment velero -n velero --patch '{"spec":{"template":{"spec":{"containers":[{"name": "velero", "resources": {"limits":{"cpu": "3", "memory": "14Gi"}, "requests": {"cpu": "2", "memory": "7Gi"}}}]}}}}' |
Create and Restore Backups
Now that you have Velero in the AKS cluster, you can create and schedule backups, and restore them with the Velero client.
Below, there are some commands to help you create and restore backups:
- Create a new backup:
# | |
# syntax | |
# velero backup create <<backup name>> [options] | |
# | |
# backup the AKS cluster | |
velero backup create kubernetes-cluster | |
# backup only persistent volume (PV) | |
velero backup create pv-backup --include-resources pv |
- Create a new schedule:
# | |
# syntax | |
# velero schedule create <<backup name>> [options] | |
# | |
# backup the AKS cluster every week with TTL 30 days | |
velero schedule create kubernetes-weekly --schedule="@weekly" --ttl 720h0m0s | |
# backup only persistent volume (PV) every week | |
velero schedule create pv-backup-weekly --schedule="@weekly" --include-resources pv |
- Restore backup:
# | |
# syntax | |
# velero restore create <<restore name>> --from-backup <<backup name>> [options] | |
# | |
# restore the AKS cluster backup | |
velero restore create kubernetes-restore --from-backup kubernetes-cluster | |
# restore persistent volume (PV) | |
velero restore create pvc-restore --from-backup pv-backup |
You can find more commands and options in the Velero’s home page. Click here to check them.
When the commands above are executed by Velero, new folders are created in its Azure Blob Storage. Here is an example:
Beyond backup and restore the cluster, Velero can be useful in the migration process.
We can use that to migrate our AKS cluster to a new AKS cluster or a new Kubernetes cluster from another Cloud Provider.
For now, that’s all. I hope that can be helpful for you!
Top comments (1)
How to perform backups of files generated by pg_dump?