Containers have become a great facility to easily deploy applications, whether locally or on orchestrated clusters.
However, containers are ephemeral, meaning their data should be stored externally. When possible, they can be stored using databases or object storage. Most often though, you will need to resort to using data volumes, mounted inside your containers. How then can be perform a backup of this data?
Contrarily to the traditional situation in application deployment, the location of critical data in containers is known, since it uses named volumes. We can thus connect to the Docker socket or the API managing the volumes to list them and perform the backups.
Bivac is a tool created to do just that. It can be plugged to either a Docker socket, a Rancher API, or a Kubernetes server. It will then list the volumes on the platform and automatically back them up on a regular basis, using Restic to transfer the data to an object storage provider (e.g. AWS S3).
In addition, Bivac can provide metrics on the backup statuses as it exposes a Prometheus endpoint.
Using the REST client, backups can be listed, executed on demand, and it is also possible to restore volumes.
Bivac can easily be installed a binary or a container. Here are some examples, deploying it locally on Docker, or using Kubernetes.
docker-compose.yml file can be used to deploy the Bivac manager:
--- version: '3' services: bivac: image: camptocamp/bivac:2.2 command: "manager -v" ports: - "8182:8182" volumes: - "/var/run/docker.sock:/var/run/docker.sock:ro" environment: BIVAC_AGENT_IMAGE: camptocamp/bivac:2.1 BIVAC_SERVER_PSK: super-secret-psk RESTIC_PASSWORD: not-so-good-password BIVAC_TARGET_URL: s3:my-bucket AWS_ACCESS_KEY_ID: XXXXX AWS_SECRET_ACCESS_KEY: XXXXX
Additionally, you can also deploy a local Prometheus server to retrieve the metrics. See the full example.
The easiest way to deploy a Bivac manager on Kubernetes is to use Camptocamp's Helm chart:
$ helm repo add camptocamp http://charts.camptocamp.com $ helm install camptocamp/bivac --version 1.0.0
The CLI can be downloaded from the releases page. Once the binary is installed, you can use it to list backups, perform backups, or restore data.
The CLI needs to be connected to the Bivac manager, using its HTTP URL and PSK (defined in the deployment). This can be performed using either the
--server.psk options, or by setting the
bivac volumes command lets you list the volumes managed by Bivac:
While Bivac automatically performs backups at a regular interval, the CLI can also be used to trigger backups manually:
Bivac stores restic backups on object storage and lets you restore them using the
backup restore command:
More features are available, such as the ability to manage a remote Restic repository. See the documentation for more information.
Bivac allows to easily backup data, monitor their status and restore them, whether you are using raw Docker, Rancher volumes or Kubernetes.
This post was originally published on https://www.camptocamp.com/actualite/backup-your-container-data/
All coders welcome
Level up every day