Terraform: manage secrets with SOPS

Securing sensitive data, such as Database connection details, API keys, access tokens etc, is crucial in any infrastructure project. One way to handle them is by utilizing SOPS (Secrets OPerationS), a tool for securely encrypting and decrypting files. By leveraging AWS KMS for encryption and SOPS for management, you can seamlessly integrate secrets into your Terraform configurations.

Folder Structure

For example, secrets for each environment (dev, qa and prod) are stored in a secrets directory, with a folder per environment:

/secrets
  /dev
    dev-enc.yml
  /qa
    qa-enc.yml
  /prod
    prod-enc.yml

The yml files are encrypted using AWS KMS and decrypted dynamically during Terraform runs. Also note that you can use "-enc" suffix as a proper naming convention to indicate that file is encrypted.

Decrypting Secrets in Terraform

The sops_file data block reads and decrypts the encrypted files:

data "sops_file" "env_secrets" {
  source_file = "secrets/${local.env}/${local.env}-enc.yml"
}

This block dynamically points to the environment-specific encrypted secrets file. The data block of the sops_file resource exposes the decrypted secrets for use.

Using Decrypted Secrets in Resources

Now secrets can be injected into other Terraform resources. For example:

1. If you need to secure AWS MemoryDB password, you can configure it like this:

resource "aws_memorydb_user" "lambda_user" {
  user_name     = "lambda-user-${local.env}"
  access_string = "on ~* &* +@all"

  authentication_mode {
    type      = "password"
    passwords = [data.sops_file.env_secrets.data["MEMORY_DB_PASSWORD"]]
  }
}

1. If you have Lambda environment variables that you want to secure:

lambda_env_variables = {
  CLIENT_ID     = data.sops_file.env_secrets.data["CLIENT_ID"]
  CLIENT_SECRET = data.sops_file.env_secrets.data["CLIENT_SECRET"]
}

Encrypting and Decrypting Secrets

Encrypt a file:
Use sops with KMS key to encrypt secrets:

sops -kms arn:aws:kms:{region}:{account_id}:key/XXX dev-enc.yml

Decrypt a file:
Use sops to view plaintext:

sops -d dev-enc.yml

Of course, ensure that KMS key is deployed and accessible for encryption and decryption.

KMS key and Terraform pipeline user

When utilizing SOPS with a KMS key, it is essential to configure proper key policies that define who can control the key and who can use it for encryption and decryption. In this use case that is for sure AWS IAM user created for Terraform. Regarding key usage permissions, you would need actions like: Encrypt, Decrypt, GenerateDataKey, ReEncrypt, DescribeKey.

Important to configure

Before the encryption of the yml file you need to have SOPS tool installed locally as well as AWS creds configuration for the specific account so that you have permissions to open and edit the file.

Benefits of SOPS with Terraform

• Security: Secrets are encrypted at rest and dynamically decrypted only during Terraform runs.
• Simplicity: Direct integration into Terraform configurations.
• Flexibility: Supports multiple encryption backends.

Conclusion

Managing secrets with SOPS in Terraform provides a secure, efficient, and maintainable approach to handling sensitive data. By combining SOPS with AWS KMS and Terraform's dynamic configuration capabilities, you can safeguard your infrastructure without introducing unnecessary complexity.