Google Cloud Functions makes it easy to add environment variables. This post will show you how you can use the Google Cloud Key Management Service to safely and securely put secrets in these variables, and then use those secrets in your function.
Using environment variables to store plain-text strings that should be "secret", like API keys, secret tokens for cookies, etc. generally isn't recommended. Any third-party dependency or library you use has access to these same environment variables. Furthermore, when these environment variables are stored, they are usually stored unencrypted.
However, it's very convenient to store your secrets this way along side your function. You can still do it, as long as you store these secrets safely and securely.
We'll do this by encrypting our secret ahead of time with our key, and decrypting secrets at the application layer. Doing so will limit access to the secret to just members of your team who have access to the KMS key (and, of course, the function when it's running on Google Cloud). Furthermore, using KMS lets you audit access to the key -- you can see every time it is used to decrypt the secret.
Note: If you're using a product other than Cloud Functions, or a language other than Python, you might want to check out Berglas, a newly launched tool for managing secrets on Google Cloud. Unfortunately, if you're using Python on Cloud Functions, this tool does not yet work for you, and you'll need to use the more manual steps outlined below.
I'll assume that you've already created a Google Cloud project with a name (mine is
my_cloud_project). If you haven't, go ahead and do that now.
First, we'll use the Cloud Key Management Service to create a cryptographic key which will be used to encrypt and decrypt our secrets.
Once you've enabled the service for your project, click Create Key Ring to create a key ring, which will group multiple keys together to keep them organized. You can pick a name and a region for your keyring, or you can make it
global, which is the default. I'll name mine
Next, click Create Key to create a new key. You can name your key whatever you'd like, I'll call mine
It's important that you leave the key set on its default purpose of "Symmetric encrypt/decrypt" -- this allows us to use the same key to encrypt and decrypt our secret.
Finally, while it's generally a good idea to rotate your keys, let's set the rotation period to "Never (manual rotation)". But don't forget to manually rotate your keys!
Now that you've created a key, the next step is to encrypt a secret with it. We can do this on the command line with Python! Also, we'll do it via Cloud Shell, so no credentials ever have to leave the cloud.
First, launch Cloud Shell for your project.
You should see a prompt that looks like this:
Welcome to Cloud Shell! Type "help" to get started. Your Cloud Platform project in this session is set to my_cloud_project. Use “gcloud config set project [PROJECT_ID]” to change to a different project. di@cloudshell:~ (my_cloud_project)$
Next, install the
di@cloudshell:~ (my_cloud_project)$ pip3 install google-cloud-kms --user Collecting google-cloud-kms ... Successfully installed google-cloud-kms-0.2.0
Now, we can start a Python interpreter and create a KMS client:
di@cloudshell:~ (my_cloud_project)$ python3 Python 3.5.3 (default, Jan 19 2017, 14:11:04) [GCC 6.3.0 20170118] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from google.cloud import kms >>> kms_client = kms.KeyManagementServiceClient()
We'll use the project name, keyring name and region, and the key name to create a "resource name" for the key, which is sort of like a path to a file. For my project, this would be:
>>> resource_name = ( ... "projects/my_cloud_project/" + ... "locations/global/" + ... "keyRings/my_key_ring/" + ... "cryptoKeys/my_key_name/" + ... "cryptoKeyVersions/1" ... )
Finally, we can use this along with our secret string to generate an encrypted secret (note that we're converting our string to bytes):
>>> secret = kms_client.encrypt( ... resource_name, ... bytes("The chickens are in the hayloft.", "ascii") )
We can retrieve the ciphertext from the secret, but it will be a binary string:
>>> secret.ciphertext b'\n$\x00W^\xf2\x1b\x89\xbf\xb6#\xfc\xf6D`\x19_\x05=X\x08\xd2i\x02\x12\xa5hSL\\\x98\xdd\xce\x90)\x95\xc1#\x12I\x00G\xf0<\x9e\x81\x04\xb0y1>\x1f\xb5\x195\xe7,x \x1e\xe6,\xf1\xd2#4\xad\xbe\xfd\xf2\xcd\x13P\xa4\xc6\xfav\xe6\xf3|/H\x1a\xf5\xfe\xce\xf3\x8eCu\xb2\x13\x0f\n\x9b\x98\x93j\x9cjv\xb0c0\xed|\xf8\x94\x9a+\xac\xf1\x85'
To make it safer for storage and transport (and a little easier to look at as a human), we can base64 encode it:
>>> import base64 >>> base64.b64encode(secret.ciphertext) b'CiQAV17yG4m/tiP89kRgGV8FPVgI0mkCEqVoU0xcmN3OkCmVwSMSSQBH8DyegQSweTE+H7UZNecseCAe5izx0iM0rb798s0TUKTG+nbm83wvSBr1/s7zjkN1shMPCpuYk2qcanawYzDtfPiUmius8YU='
Copy the base64-encoded result to your clipboard for the next step.
At this point, we'll switch to your local development environment where you'll be writing your function. I like to put environment variables and such in a
.env.yaml file. Now that we've encrypted and encoded our secret, we can include it there as well, along with the resource name:
SECRET_RESOURCE_NAME: projects/my_cloud_project/locations/global/keyRings/my_key_ring/cryptoKeys/my_key_name SECRET_STRING: CiQAV17yG4m/tiP89kRgGV8FPVgI0mkCEqVoU0xcmN3OkCmVwSMSSQBH8DyegQSweTE+H7UZNecseCAe5izx0iM0rb798s0TUKTG+nbm83wvSBr1/s7zjkN1shMPCpuYk2qcanawYzDtfPiUmius8YU=
Be sure to add this line to your
.gcloudignore files to make sure it doesn't accidentally get committed to your source repository, or uploaded with your function:
# Ignore environmental variables .env.yaml
Now we'll write our function. Here's our whole function that we'll deploy, that will live in a file called
import os import base64 from google.cloud import kms kms_client = kms.KeyManagementServiceClient() secret_string = kms_client.decrypt( os.environ["SECRET_RESOURCE_NAME"], base64.b64decode(os.environ["SECRET_STRING"]), ).plaintext def secret_hello(request): return secret_string
This function is essentially undoing everything we just did to encrypt our secret: it's getting the resource name and encrypted string out of the function's environment, decoding the string from Base64, and then decrypting it with our key, and finally returning the unencrypted string.
Also note how the function is doing this work outside of the Python function itself. This will speed up our function by only decrypting the secret once when the Cloud Function is instantiated, and not once for every single request.
We'll also need to tell our function to install the same
google-cloud-kms library we used to encrypt the function, so it can decrypt it. In a file called
requirements.txt, add the line:
At this point, your directory structure should look something like this:
. ├── .env.yaml ├── .gcloudignore ├── .gitignore ├── main.py └── requirements.txt
The last step is to deploy our function with the
gcloud tool along with it's environmental variables. The important part here is that you are using the
--env-vars-file flag to deploy your environmental variables along with the function:
$ gcloud beta functions deploy secret_hello --env-vars-file .env.yaml --runtime python37 --trigger-http
Now we can test our function once it's deployed:
$ curl https://us-central1-my-cloud-project.cloudfunctions.net/secret_hello The chickens are in the hayloft.
There's lots more you can do with Cloud Functions! Follow the links below to learn how to:
- Write "background" trigger functions that respond to events
- Monitor functions at a high level with Stackdriver
- See additional tips & tricks for writing Cloud Functions
All code © Google w/ Apache 2 license