Michael Bogan for Heroku

Posted on Oct 7, 2020

How to Manage Your Secrets with git-crypt

#security #webdev #devops #tutorial

Many software projects use secrets - usually, keys to external APIs or credentials to access an external resource such as a database. Your application needs these keys at runtime, so you need to be able to provide them when you deploy your application, or as a step in preparing your deployment environment.

In this article, I'm going to show you how to use git-crypt so that you can safely keep your application secrets in your source code repositories, even if they're public.

The Problem With Application Secrets

Most projects have some sort of secret keys or credentials. For example, if your application is hosted on Heroku, you might provide an API key to your Heroku application using a command like this:

$ heroku config:set API_KEY=my-sooper-sekrit-api-key

By running this command before you (re)deploy your application, you give it an environment variable at runtime called API_KEY with the value my-sooper-sekrit-api-key. However, keeping track of these secret values outside of Heroku (or wherever you deploy your application) is still a challenge.

I always try to set up my projects so that I can run a single command to deploy them from scratch without any separate, manual steps. For our example, this means I need to store the value my-sooper-sekrit-api-key somewhere so that my deployment code can use it (in this case, to run the heroku config:set... the command above).

My project source code is always stored in git and usually hosted on github.com or bitbucket.com or some other source code hosting service. I could store my API_KEY value in my source code repository, however, there are some downsides to this:

I can't share my repository with anyone else unless I'm comfortable with them accessing my secrets. This means all my application repositories with secrets in them need to be private.
Presumably many staff members at Github/bitbucket/wherever would also have access to my secrets, which I might not be okay with (depending on the secret).
It's easy to forget about the secrets in a private repository if I later choose to make it public. So I could accidentally disclose important secrets.

I could store my secrets somewhere separate from my application source code, but this has its own problems:

I need a way to get my secrets from wherever they're stored, at or before deployment time, and give my deployment code access to them.
My secrets may not be stored as robustly as my source code. For example, I could keep secrets in a .env file on my laptop, and make sure I never check that into the git repository. However, if I lose that file (such as if my laptop gets damaged/stolen), then I also lose that secret.

git-crypt

Git-crypt aims to solve this problem by encrypting your secrets whenever you push them to your git repository, and decrypting them whenever you pull them. This happens transparently, from your point of view. So the secrets are in cleartext as far as you and your deployment code are concerned, but nobody else can read them, even if your source code is in a public Github repository.

Let's look at an example.

1. Install git-crypt.

There are instructions for Linux, Mac, and Windows on the git-crypt install page

If like me, you're using a Mac with Homebrew installed, you can run:

$ brew install git-crypt

2. Create a new git repository.

$ mkdir myproject 
$ cd myproject 
$ git init 
$ echo "This is some text" > file.txt 
$ git add file.txt 
$ git commit -m "Initial commit"

Now we have a git repository containing a single text file.

3. Set up the repository to use git-crypt.

$ git-crypt init

You should see the output:

Generating key...

Before we do anything else, please run the following command:

$ git-crypt export-key ../git-crypt-key

This command creates a copy of the git-crypt symmetric key that was generated for this repository. We're putting it in the directory above this repository so that we can re-use the same key across multiple git repositories.

By default, git-crypt stores the generated key in the file .git/git-crypt/keys/default so you can achieve the same result by running cp .git/git-crypt/keys/default ../git-crypt-key

This git-crypt-key the file is important. It's the key that can unlock all the encrypted files in our repository. We'll see how to use this key later on.

4. Tell git-crypt which files to encrypt.

Imagine our application needs an API key, and we want to store it in a file called api.key.

Before we add that file to our repository, we will tell git-crypt that we want the api.key file to be encrypted whenever we commit it.

We do that using the .gitattributes file. This is a file we can use to add extra metadata to our git repository. It's not specific to git-crypt, so you might already have a .gitattributes file in your repository. If so, just add the relevant lines—don't replace the whole file.

In our case, we don't have a .gitattributes file, so we need to create one. The .gitattributes file contains lines of the form:

[file pattern] attr1=value1 attr2=value2

For git-crypt, the file pattern needs to match all the files we want git-crypt to encrypt, and the attributes are always the same: filter and diff, both of which we set to git-crypt.

So, our .gitattributes file should contain this:

api.key filter=git-crypt diff=git-crypt

Create that file, and add and commit it to your git repository:

$ echo "api.key filter=git-crypt diff=git-crypt" > .gitattributes 
$ git add .gitattributes 
$ git commit -m "Tell git-crypt to encrypt api.key"

I've used the literal filename api.key in my .gitattributes file, but it can be any file pattern that includes the file(s) you want to encrypt, so I could have used *.key, for instance. Alternatively, you can just add a line for each file you want to encrypt.

It can be easy to make a mistake in your .gitattributes file if you're trying to protect several files with a single pattern entry. So, I strongly recommend reading this section of the git-crypt README, which highlights some of the common gotchas.

5. Add a secret.

Now that we have told git-crypt we want to encrypt the api.key file, let's add that to our repository.

It's always a good idea to test your setup by adding a dummy value first, and confirming that it's successfully encrypted, before committing your real secret.

$ echo "dummy value" > api.key

We haven't added api.key to git yet, but we can check what git-crypt is _going _to do by running:

$ git-crypt status

You should see the following output:

    encrypted: api.key 
not encrypted: .gitattributes 
not encrypted: file.txt

So, even though the api.key file has not yet been committed to our git repository, this tells you that git-crypt is going to encrypt it for you.

Let's add and commit the file:

$ git add api.key 
$ git commit -m "Added the API key file"

6. Confirm our secret is encrypted.

We've told git-crypt to encrypt, and we've added api.key to our repository. However, if we look at, nothing seems different:

$ cat api.key 
dummy value

The reason for this is that git-crypt transparently encrypts and decrypts files as you push and pull them to your repository. So, the api.key file looks like a normal, cleartext file.

$ file api.key 
api.key: ASCII text

One way to confirm that your files really are being encrypted is to push your repository to GitHub. When you view the api.key file using the GitHub web interface, you'll see that it's an encrypted binary file rather than text.

An easier way to see how the repository would look to someone without the decryption key is to run:

$ git-crypt lock

Now if we look at our api.key file, things are different:

$ file api.key 
api.key: data 

$ cat api.key 
GITCRYPTROܮ7y\R*^

You will see some different garbage output to what I get, but it's clear the file is encrypted. This is what would be stored on GitHub.

To go back to having a cleartext api.key file, run:

$ git-crypt unlock ../git-crypt-key

The ../git-crypt-key the file is the one we saved earlier using git-crypt export-key...

Checkpoint

Let's do a quick review of where we are now.

Initialize git-crypt on a git repository using git-crypt init
Use file patterns in .gitattributes to tell git-crypt which files to encrypt
git-crypt lock will encrypt all the specified files in our repository
git-crypt unlock [path to keyfile] will decrypt the encrypted files

The git-crypt-key the file is very important. Without it, you won't be able to decrypt any of the encrypted files in your repository. Anyone who has a copy of that file has access to all of the encrypted secrets in your repository. So you need to keep that file safe and secure.

Re-using Your git-crypt Key File

We used git-crypt init and git-crypt export-key to create our git-crypt-key file. But, if we have to have a separate key file for each of our repositories, then we haven't improved our secret management very much.

Fortunately, it's very easy to use the same git-crypt key file for multiple git repositories.

To use an existing key file, just use git-crypt unlock instead of git-crypt init when you set up your git repository to use git-crypt, like this:

$ mkdir my-other-project   # At the same directory level as `myproject` 
$ cd my-other-project 
$ git init 
$ echo "Something" > file.txt 
$ git add file.txt 
$ git commit -m "initial commit" 
$ git-crypt unlock ../git-crypt-key

If you run the git-crypt unlock command before adding any files to your git repository, you will see a message like this:

fatal: You are on a branch yet to be born 
Error: 'git checkout' failed 
git-crypt has been set up but existing encrypted files have not been decrypted

This still works just fine, but it's a bit confusing, so I made sure to add and commit at least one file before running git-crypt unlock...

Re-using your git-crypt key file is convenient, but it does mean that if anyone else gets a copy of your key file, all of your encrypted secrets are exposed.

This is the same kind of security trade-off as using a password manager like LastPass or 1password. Rather than managing multiple secrets (passwords), each with its own risk of exposure, you keep them all in a secure store and use a single master password to unlock that.

The idea here is that it's easier to manage one important secret than many lesser secrets.

When NOT to Use git-crypt

Git-crypt is a great way to keep the secrets your applications need right in the git repository, alongside the application source code. However, like every other security measure, it's not always going to be appropriate or advisable.

Here are some things to consider to decide whether it's the right solution for your particular project:

git-crypt is designed for situations where the majority of the files in your git repository can remain in cleartext, and you just need to encrypt a few files that contain secrets. If you need to encrypt most or all of the files in your repository, then other solutions may be a better fit.
There is no easy way to revoke access to the secrets in a repository once someone has the key file, and no easy way to rotate (i.e. replace) a key file (although changing the git-crypt key file doesn't help much unless you also rotate all of the actual secrets in the repository).
git-crypt only encrypts the contents of files. Therefore, it's not suitable if the metadata of your repository is also sensitive (i.e. filenames, modification dates, commit messages, and so on).
Some GUI git applications may not work reliably with git-crypt. (Although the specific case of Atlassian SourceTree, mentioned in the README, has been fixed.)

There is more information in this section of the git-crypt README.

A Better Way to Use git-crypt

Rather than managing your git-crypt key file directly, there is a better way to manage encrypted repositories by integrating git-crypt with gpg, so that you can use your gpg private key to decrypt the git repository. This also allows you to add multiple collaborators to a git repository without transmitting any secrets between the parties. However, this requires a more complicated setup, so we'll save that for another article.