VoluSnap — Cloud Volume Auto Snapshot Go Server

VoluSnap — Cloud Volume Auto Snapshot Go Server

Do you deploy your web apps to cloud virtual machines such as Digital Ocean Droplets or Scaleway Cloud Instances ?

If your app saves any kind of data to disk and you’re worried about data loss due to software bugs, human errors or malicious actors (amongst other risks) then you need a backup strategy.

Retro Data Backups :)

Cloud Volume Backups

Most cloud providers provide a Hot Backup feature which takes a snapshot of your volume data without the need to stop your instances during the backup process (but there is a data loss risk involved, please read up on this topic and make an informed decision).

However, automated recurring backups are not always available, and sometimes the configuration options are not granular enough. At the time of writing this article Digital Ocean Backups are taken once a week, and Scaleway only offers manual backups.

Fortunately, most cloud providers provide API access. I’ve needed an automated backup solution for my personal cloud deployments for a while, so I decided to build an open source solution:

VoluSnap: A self hosted solution built with Go

Volusnap is an API server built with Go. It allows triggering automated recurring snapshots of cloud provider volumes. Here is a simplified schema explaining how VoluSnap works:

API Server + Rules Checker Goroutine

The basic workflow is :

• User signs up
• User logs in, generating a JWT token
• User adds a Cloud Account and relevant Cloud API access token
• User lists Cloud Volumes
• User creates Snapshot Rule(s) for the chosen Volume(s)
• Rules Checker Service checks the snapshot rules periodically, triggers snapshots via Cloud Provider API and saves snapshot Metadata to the Database

VoluSnap uses:

• gorilla/mux for routing
• sqlboiler for data persistence to PostgreSQL
• sql-migrate for database migrations
• viper for configuration
• testify for unit tests and mocks

Components:

• REST API
• SnapRules Checker Service

The REST API is documented in the github repo. No client code is provided at the moment.

VoluSnap API Docs

At the moment both the REST API and the SnapRules Checker components run in the same Go process. The SnapRules Checker Service runs in a goroutine at the moment but it would be doable to split it into its own process in the future if needed:

Adding Cloud Providers

VoluSnap currently supports Digital Ocean and Scaleway. I’ve tried to make adding other Cloud providers easy (Hint: PRs welcome ❤️). As you can see in the code below, pRegistry is a package level singleton service you can register provider services with. Your provider just has to implement the interface ProviderSvcer :

type ProviderSvcer interface {
    ListVolumes() ([]Volume, error)
    TakeSnapshot(snapRule \*models.SnapRule) (string, error)
}

And don’t forget the corresponding unit tests

Conclusion

I hope that you’ll find VoluSnap useful for your Cloud Volume Backups. Definitely leave comments if you like/hate/have questions about the project or the code/testing patterns used !