In this tutorial, you'll use Open Policy Agent (OPA) to enforce fine-grained policy control across development and production environments for Terraform deployments. This tutorial assumes that you are already using Terraform.
The challenge
Rapu started at Crab Inc. as a Junior DevOps Engineer. He is shadowing a senior engineer on the team to learn how the team deploys PostgreSQL, Redis, Kafka, and other data-related services across development and production environments.
The development team is based in Montreal, Canada and they should only create cloud resources in the Google Cloud Montreal region. However, to ensure high availability for the company's North American customers, the production environment supports multiple AWS cloud regions in the US East location. Previously, there were no guardrails in place and Rapu deployed to cloud regions where he wasn't supposed to deploy.
These are the specific regions for Prod and Dev.
- Prod: "aws-us-east1"
- Dev: "google-northamerica-northeast1"
Your goal is to help Rapu enforce these policies so that resources don't get created in the wrong cloud or region.
Prerequisites
The concept of the tutorial is agnostic to the Terraform provider you choose. For the sake of a demo, I'll choose Aiven Terraform Provider. Aiven provides highly-available and scalable data infrastructure based on open-source technologies. For this tutorial, you'll create a free Aiven account and an Aiven authentication token.
Install jq (optional).
The Story
In this section, you'll help Rapu create Terraform files that describe an Aiven for Redis® resource. This can be any cloud resource for your use case and Aiven for Redis is used as an example. Besides the Aiven for Redis resource in the services.tf file, create a provider.tf file for the provider and version details. You'll also create a variables.tf to define the required variables for Aiven Terraform Provider.
Our protagonist Rapu will learn how to decouple and enforce policies using some common tools. When decoupling policies using Open Policy Agent, the structure is pretty consistent no matter the tool or service.
There is a tool/service, in this case we will be using Terraform. This tool will generate some data that will be used as Input for our decision. The input file will be sent to OPA to be compared against the Policy(written in Rego) and any additional Data.
As an added bonus Rapu will learn how to write Unit tests for his policies, which is part of clean code and best practicies.
Here's a high-level overview of the system:
Here's a detailed version of the same system:
In this tutorial, you run terraform and opa commands manually and from your local machine. The above diagram shows how these tools can be used in an automated way. For example, a CI/CD pipeline can deploy cloud resources using Terraform if the OPA policy allows. A deny from OPA might result in a Slack or email notification to the developer.
Set up Terraform files
In an empty directory, create these three files:
provider.tf file:
terraform {
required_providers {
aiven = {
source = "aiven/aiven"
version = "~> 4.1.0"
}
}
}
provider "aiven" {
api_token = var.aiven_api_token
}
variables.tf file:
variable "aiven_api_token" {
description = "Aiven console API token"
type = string
}
variable "project_name" {
description = "Aiven console project name"
type = string
}
service.tf file:
resource "aiven_redis" "redis-demo" {
project = var.project_name # Find your Aiven project name from top-left of Aiven console
plan = "hobbyist" # For this exercise, the hobbyist plan will do
service_name = "redis-demo" # Choose any service name
cloud_name = "google-northamerica-northeast1" # Choose any cloud region from https://docs.aiven.io/docs/platform/reference/list_of_clouds
}
To set the values for the environment variables like aiven_api_token or project_name, you can either use TF_VAR_name, use variables on the command line, or use a variable definition file.
Prepare the Terraform manifest for OPA
Before executing your Terraform manifest with OPA, it's important to add two environment variables so that Terraform client knows about two variables that Aiven Terraform Provider requires. Please replace the placeholder values with the actual values for your Aiven API token and Aiven project name.
export TF_VAR_aiven_api_token=YOUR_AIVEN_API_TOKEN
export TF_VAT_project_name=YOUR_AIVEN_PROJECT_NAME
Now initialize this directory with the terraform init command and ask Terraform to calculate what changes it will make and store the output in plan.binary.
terraform init
terraform plan --out tfplan.binary
Use the command terraform show to convert the Terraform plan into JSON so that OPA can read the plan.
terraform show -json tfplan.binary > tfplan.json
For improved readability you can pipe the output through jq before saving the file.
terraform show -json tfplan.binary | jq > tfplan.json
Here is a sample output of tfplan.json:
{
"format_version": "1.1",
"terraform_version": "1.2.8",
"variables": {
"aiven_api_token": {
"value": "3Xi1J+E0G3vo0fwr2vWMLl0XgHwRyA6pCX8C6rQQZVhoyFfz9WAMreaGZPAI+jRUGWgtslQKQtIZTICCDZlZkQn3sRYHGBcAxgXqoiT3l9cYVbVvyPNSVPGHrSvBhSCXIYgWX3AXkOG/kQiJ1r0CZn0Y0gK/pRyiti6dImIzyEsZWja9FZk+mV/M/6BAZMKpa/EokkKUj4puMpUX4B3//slU9yUdicr2wCe/uyx53K64rU/OWZYCbqfTI6QcsjZc1wd8/a+0aLsv651qZwxmgTAenmj0JC5tXWD+Dx89NaiZcUdGxhyg58ZYfYh6U3YDm5S/ovDcvq9m/ffMKbb2Sut2vVELPO1l6AA70U1besBR0dE="
},
"project_name": {
"value": "devrel-dewan"
}
},
"planned_values": {
"root_module": {
"resources": [
{
"address": "aiven_redis.redis-demo",
"mode": "managed",
"type": "aiven_redis",
"name": "redis-demo",
"provider_name": "registry.terraform.io/aiven/aiven",
"schema_version": 0,
"values": {
"additional_disk_space": null,
"cloud_name": "google-northamerica-northeast1",
"disk_space": null,
"maintenance_window_dow": null,
"maintenance_window_time": null,
"plan": "hobbyist",
"project": "devrel-dewan",
"project_vpc_id": null,
"redis_user_config": [],
"service_integrations": [],
"service_name": "redis-demo",
"service_type": "redis",
"static_ips": null,
"tag": [],
"termination_protection": null,
"timeouts": null
},
"sensitive_values": {
"components": [],
"redis": [],
"redis_user_config": [],
"service_integrations": [],
"tag": []
}
}
]
}
},
"resource_changes": [
{
"address": "aiven_redis.redis-demo",
"mode": "managed",
"type": "aiven_redis",
"name": "redis-demo",
"provider_name": "registry.terraform.io/aiven/aiven",
"change": {
"actions": [
"create"
],
"before": null,
"after": {
"additional_disk_space": null,
"cloud_name": "google-northamerica-northeast1",
"disk_space": null,
"maintenance_window_dow": null,
"maintenance_window_time": null,
"plan": "hobbyist",
"project": "devrel-dewan",
"project_vpc_id": null,
"redis_user_config": [],
"service_integrations": [],
"service_name": "redis-demo",
"service_type": "redis",
"static_ips": null,
"tag": [],
"termination_protection": null,
"timeouts": null
},
"after_unknown": {
"components": true,
"disk_space_cap": true,
"disk_space_default": true,
"disk_space_step": true,
"disk_space_used": true,
"id": true,
"redis": true,
"redis_user_config": [],
"service_host": true,
"service_integrations": [],
"service_password": true,
"service_port": true,
"service_uri": true,
"service_username": true,
"state": true,
"tag": []
},
"before_sensitive": false,
"after_sensitive": {
"components": [],
"redis": [],
"redis_user_config": [],
"service_integrations": [],
"service_password": true,
"service_uri": true,
"tag": []
}
}
}
],
"configuration": {
"provider_config": {
"aiven": {
"name": "aiven",
"full_name": "registry.terraform.io/aiven/aiven",
"version_constraint": "~> 4.1.0",
"expressions": {
"api_token": {
"references": [
"var.aiven_api_token"
]
}
}
}
},
"root_module": {
"resources": [
{
"address": "aiven_redis.redis-demo",
"mode": "managed",
"type": "aiven_redis",
"name": "redis-demo",
"provider_config_key": "aiven",
"expressions": {
"cloud_name": {
"constant_value": "google-northamerica-northeast1"
},
"plan": {
"constant_value": "hobbyist"
},
"project": {
"constant_value": "devrel-dewan"
},
"service_name": {
"constant_value": "redis-demo"
}
},
"schema_version": 0
}
],
"variables": {
"aiven_api_token": {
"description": "Aiven console API token"
},
"project_name": {
"description": "Aiven console project name"
}
}
}
}
}
Note The api token in the above example is invalid and is shown as an example only.
Write OPA policies in Rego
OPA policies are written in Rego. The following Rego checks if Rapu can deploy to a development environment or a production environment based on the type of Terraform resource and the cloud region they choose.
In the same folder, create a sub-folder called policy and create a file within called terraform.rego. Add the following code to that file:
terraform.rego file:
package terraform.analysis
import input as tfplan
import future.keywords
dev_env_cloud_prefix := "google-northamerica-northeast1"
prod_env_cloud_prefix := "aws-us-east"
resource_types := {"aiven_kafka", "aiven_pg", "aiven_opensearch", "aiven_redis"}
default allow_dev_deployment := false
default allow_prod_deployment := false
allow_dev_deployment if {
some resource in tfplan.planned_values.root_module.resources
resource.type in resource_types
startswith(resource.values.cloud_name, dev_env_cloud_prefix)
}
allow_prod_deployment if {
some resource in tfplan.planned_values.root_module.resources
resource.type in resource_types
startswith(resource.values.cloud_name, prod_env_cloud_prefix)
}
Let's analyze this file. Crab Inc. uses PostgreSQL for their relational database, Redis for caching, OpenSearch for search and analytics, and Apache Kafka as the central message bus. Therefore, the resource_types field limits the use to these four resources. Crab Inc. allows developers to deploy in the Montreal, Canada region only. The dev_env_cloud_prefix field takes care of that requirement. Similarly, production deployments are allowed at any one of Aiven's supported AWS cloud regions in the US East coast which the prod_env_cloud_prefix field takes care of.
Execute the following command from the main directory to find out if OPA allows the Terraform deployment to go through:
./opa exec --decision terraform/analysis/allow_prod_deployment --bundle policy/ tfplan.json
With the current Terraform service definition, the output of the above command is:
{
"result": [
{
"path": "tfplan.json",
"result": false
}
]
}
If you have jq installed on your machine, you can find the exact result with:
./opa exec --decision terraform/analysis/allow_prod_deployment --bundle policy/ tfplan.json | jq '.result[0].result'
The opa exec command is taking in the tfplan.json as an input and validating this against the policy we defined in the allow_prod_deployment section under policy/terraform.rego file. terraform/analysis is denoting the package name in that Rego.
Let's make a change in the services.tf file and change the cloud_name field to aws-us-east1. Now if you repeat the previous steps to create the tfplan.json and run opa exec command, the output should be true.
hint:
terraform plan --out tfplan.binary
terraform show -json tfplan.binary | jq > tfplan.json
opa exec --decision terraform/analysis/allow_prod_deployment --bundle ./policy tfplan.json | jq '.result[0].result'
Create data block in Rego
Rapu has done a great job implementing his first policy. However, typically data isn't hard coded in the policy. So let's rewrite the current policy and create some news ones.
Create the following data.json file in the policy folder this should be right next to our rego file:
Filename data.json:
{
"team": "devrel",
"app": "crab_cast",
"dev": {
"cloud": "google-northamerica-northeast1"
},
"prod": {
"cloud": "aws-us-east1"
}
}
Now that we have a data file, we can go back and update our Rego policy.
file name terraform.rego:
package terraform.analysis
import input as tfplan
import future.keywords
# notice we removed the hard coded variables
resource_types := {"aiven_kafka", "aiven_pg", "aiven_opensearch", "aiven_redis"}
default allow_dev_deployment := false
default allow_prod_deployment := false
allow_dev_deployment if {
some resource in tfplan.planned_values.root_module.resources
resource.type in resource_types
startswith(resource.values.cloud_name, data.dev.cloud) # referencing the new data block
}
allow_prod_deployment if {
some resource in tfplan.planned_values.root_module.resources
resource.type in resource_types
startswith(resource.values.cloud_name, data.prod.cloud) # referencing the new data block
}
Now we can rerun the policy check. Remember we added the data file to our policy folder so OPA should be aware of the new data.
terraform plan --out tfplan.binary
terraform show -json tfplan.binary | jq > tfplan.json
opa exec --decision terraform/analysis/allow_prod_deployment --bundle ./policy tfplan.json
Unit testing in Rego
Now that we have a few policies in place, we are going to add unit tests to ensure our policies are good before we enforce them in production.
Create a rego file for our tests.
file policy/test_terraform.rego:
package terraform.test_analysis
import data.terraform.analysis
test_allow_dev_deployment {
analysis.allow_dev_deployment with input as {"planned_values": {"root_module": {"resources": [{
"address": "aiven_redis.redis-demo",
"mode": "managed",
"type": "aiven_redis",
"name": "redis-demo",
"provider_name": "registry.terraform.io/aiven/aiven",
"schema_version": 1,
"values": {
"cloud_name": "google-northamerica-northeast1",
"plan": "hobbyist",
"project": "devrel-dewan",
"service_name": "redis-demo",
"service_type": "redis",
},
}]}}}
}
test_not_allow_prod_deployment {
not analysis.allow_prod_deployment with input as {"planned_values": {"root_module": {"resources": [{
"address": "aiven_redis.redis-demo",
"mode": "managed",
"type": "aiven_redis",
"name": "redis-demo",
"provider_name": "registry.terraform.io/aiven/aiven",
"schema_version": 1,
"values": {
"cloud_name": "google-northamerica-northeast1",
"plan": "hobbyist",
"project": "devrel-dewan",
"service_name": "redis-demo",
"service_type": "redis",
},
}]}}}
}
With our testing file in place let's run the tests. In this command we are calling the OPA binary, with the subcommand test on the target folder policy.
opa test policy
Some homework for you
Now that you have learned about writing and testing OPA policies for your data infrastructure in place, please write the following two policies and two unit tests:
Policies to add
- project must start with team name
- service_name must include app name
Unit tests to add
- test that Aiven project name must contain team name
- test that Aiven service name must contain app name
If you need a hint or two, take a look at the solutions.
Great job helping Rapu succeed
Let's look at all the things Rapu has accomplished on his first project at Crab, Inc.
- Created an Aiven Terraform file
- Converted the Terraform plan into binary
- Converted the binary output into JSON
- Created a Rego policy to verify the resource configuration
- Tested our Rego policy on our local CLI
- Cleaned up our Rego policy by moving hard coded data to a data file
- Wrote addition policies to very more specifics of the resources
- Added unit tests for each of our policies
Thanks for spending the time learning with us today. Here are some additional resources to help you learn about the tools in this tutorial.
| Link | Description |
|---|---|
| Aiven Docs | The Aiven docs will help you get unblocked using any Aiven resources |
| Terraform Docs | The Terraform docs are a great reference to get started with Terraform |
| Rego Playground | The Rego playground is available to interactively test your rego policies |
| OPA Docs | The OPA docs are extensively written to answer any questions you may have |
If you have any question on this tutorial, please check out the FAQ page, ask on Aiven community forum, or raise an issue.
Top comments (0)