Manage your secrets in Git with SOPS

We use Git for everything now, from code source to organization, history, and even for Kubernetes Cluster Management (aka GitOps).

But, there is still something not widely adopted... managing our secrets in Git. Some tools like HashiCorp Vault, Google Secret Management, or AWS Secret Manager provide us a solution to manage our secrets in a dedicated system, but they are still not in sync with our source code.

We will see here, thanks to Mozilla SOPS how to integrate our secrets management directly in Git.

mozilla / sops

Simple and flexible tool for managing secrets

SOPS: Secrets OPerationS

sops is an editor of encrypted files that supports YAML, JSON, ENV, INI and BINARY formats and encrypts with AWS KMS, GCP KMS, Azure Key Vault, age, and PGP (demo)

---

1   Download

1.1   Stable release

Binaries and packages of the latest stable release are available at https://github.com/mozilla/sops/releases.

1.2   Development branch

For the adventurous, unstable features are available in the develop branch, which you can install from source:

$ mkdir -p $GOPATH/src/go.mozilla.org/sops/
$ git clone https://github.com/mozilla/sops.git $GOPATH/src/go.mozilla.org/sops/
$ cd $GOPATH/src/go.mozilla.org/sops/
$ git checkout develop
$ make install

(requires Go >= 1.17)

If you don't have Go installed, set it up with:

$ {apt,yum,brew} install golang
$ echo 'export GOPATH=~/go' >> ~/.bashrc
$ source ~/.bashrc
$ mkdir $GOPATH

Or whatever variation of the above fits your system and shell.

To use sops as a library, take a look at…

View on GitHub

DISCLAIMER: I've previously written an article on the same subject about a project named kubesec specialized in Kubernetes Secret. The project seems to be stopped and Mozilla SOPS is a better alternative right now, because it can manage every kind of secrets, not only Kubernetes ones.

Sops Installation

Sops is very simple to install, like every golang application, you just have to download the binary for your specific Operating System (Linux, Mac, Windows) directly from the release page on GitHub.

By the way, you can install it thanks to brew on Mac & Linux (sops formuale).

Use case

What we will try to achieve is to store secrets in Git but with restrictions on "who can access what".

For example, we have 4 environments, dev_a, dev_b, int, and prod and 3 team members, Alice, Bobby, and Devon.

We want to restrict secrets access with the following requirements:

• dev_a secrets should be only accessible by Alice.
• dev_b secrets should be only accessible by Bobby.
• int secrets should be accessible by Alice, Bobby, and Devon.
• prod secrets should be accessible only by Devon.

Each of them already has configured their GPG key pairs. If you need to set them up, you can follow the official GitLab documentation 🦊 about this.

In this example, secrets are just plain old env files.

dev_a configuration

Alice will generate a file containing a secret:

Thanks to the command:

sops -pgp 5844C613B763F4374BAB2D2FC735658AB38BF93A -e dev_a.env > dev_a.encrypted.env

Alice has encrypted the file dev_a.env and stored the result in dev_a.encrypted.env. She is the only one able to decrypt it.

Let's see, if we run this as Bobby:

dev_b and prod configurations are similar to the one created by Alice. Then, Bobby and Devon will respectively create dev_b and prod.

int configuration

In this configuration, we would like every developers to be able to read this file. But, only developers from the project and not everyone with access to the git repository... so we still have to encrypt this file.

To do so, Devon will execute the following commands:

Devon has to create the secret with the command

sops --pgp 57E6DA39E907744429FB07871141FE9F63986243,\
           5844C613B763F4374BAB2D2FC735658AB38BF93A,\
           AE0D6FD0242FF896BE1E376B62E1E77388753B8E \
           -e int.env > int.encrypted.env`

This command contains every public key ids, comma sparated.

We can check that both Alice and Bobby can decrypt the int.encrypted.env file:

alice $ sops -d int.encrypted.env
secret=5Tz2QNxki789YFDa

bobby $ sops -d int.encrypted.env
secret=5Tz2QNxki789YFDa

All the *.encrypted.env files are now stored in Git and can be managed like any other resources, with history and diff in commits. Only those defined during encryption can read them edit them.

You can find the source code of this article, files, and scripts in this GitLab repository.

I hope this will help you to use Git & SOPS to manage your secrets.

Comments

[🦄N B🛡]

but they are still not in sync with our source code.

I make the case, here, that that is a feature.

Take it from someone who has lead the charge on this kind of thing before, yup, there's a lot of history down that road.

If you have someone crowing about how they don't need to worry about etcd backups, because they can restore their entire application from .yaml files, shouldn't that raise an eyebrow, or maybe even some questions? Like, "What about sensitive data or identification is needed by your applications?"

Not to mention that kind of thing sort of screws up the way source control and version control is supposed to work.

Am I going to git bisect and get stuck with old, hopefully expired versions of credentials, too?

shakes his cane at you

mutters incomprehensibly

[Dmitri Telinov]

ansible-vault is an alternative