Terraform Development & Testing Introduction

I'm starting to play around with kitchen-terraform for testing and compliance of Terraform modules. I'm currently using the inspec-gcp resource pack and the inspec-gcp-cis-benchmark profile. This post will focus on the latter. The drivers for doing this are testing code's self-evident benefits and shifting some reasonable base security standards left.

To keep things simple, I wanted to start from what you might do first when you begin building cloud infrastructure out in Google. The base of any Google Cloud Platform service is a project. The module I will be talking about is here for reference and covers building a project. It's a reasonably opinionated terraform module to fit some use cases I had; however, I think it shows the example of what I'm trying to do without being too complex and confusing. Because honestly, this stuff is hard for non-developer, ex-operational mindset people like me.

NOTE: I'm not going into the basics of setting up Terraform, Ruby, Kitchen-Terraform, Chef Inspec, etc. If that's interesting, let me know, and maybe I can do another post.

"Local" Development

Learning from the practices of software development, we need to be able to develop "locally." When I say locally here, we can consider it a sandboxed area in the Google Cloud Platform resource hierarchy where you can create and build infrastructure safely. When you start your Terraform module development, you will most likely end up with something like this in your main.tf:

# Project Resource
# https://www.terraform.io/docs/providers/google/r/google_project.html

resource "google_project" "this" {
  name                     = var.project_id
  project_id               = var.project_id
  billing_account          = var.billing_id
  folder_id                = "folders/${var.folder_id}"
}

and a variables.tf file like this:

variable "billing_id" {
  description = "Billing ID for the project to use"
  type        = string
}

variable "project_id" {
  description = "Project ID (This will be used for the project name as well)"
  type        = string
}

variable "folder_id" {
  description = "Folder ID for the project to be created in."
  type        = string
}

We can now run the terraform and create our Google Cloud project:

terraform init

terraform plan -out plan.out -var="billing_id=00000C-AZAZAZ-EFEFEF" \
-var="project_id=test-del-me-4876des" \
-var="folder_id=993877078800"

terraform apply plan.out

Now you've got a project up and running, but you may be surprised that you have already violated a bunch of CIS Benchmarks! Thanks to Google and the work folks have done in the inspec-gcp-cis-benchmark GitHub repository; we can test it!

NOTE: This is not an officially supported Google product, but hopefully, they maintain this repo with help from the community.

One thing I love about this GitHub project is that it runs directly against a specific Google Cloud project. For me, that's precisely the level at which I want to run the tests. I'll get into more details on that later, but for now, lets inspec the project we just built to see what we need to do:

inspec exec https://github.com/GoogleCloudPlatform/inspec-gcp-cis-benchmark.git \
-t gcp:// --input gcp_project_id=test-del-me-4876des

After running that command, you're going to see a list of violations. In the spirit of Test Driven Development (TDD), we have our failing test, and now we can code to fix it. We will integrate all this is the next post. For the sake of simplicity, let's focus on two of them:

×  cis-gcp-4.4-vms: [VMS] Ensure oslogin is enabled for a Project
×  cis-gcp-3.1-networking: [NETWORKING] Ensure the default network does not exist in a project

The idea here is that we address these issues as far left in the process as possible. I'm still developing "locally." If you deploy a bunch of stuff on default networks, and three years later, your security team says, we need to get everything CIS compliant; you will have an unruly amount of operational work! Let's fix these compliance issues now. To do that we can add a few lines to the Terraform module code. Then, everyone in your organization that consumes it will meet the standard.

# Project Resource
# https://www.terraform.io/docs/providers/google/r/google_project.html

resource "google_project" "this" {
  name                = var.project_id
  project_id          = var.project_id
  billing_account     = var.billing_id
  folder_id           = "folders/${var.folder_id}"
  auto_create_network = false
}


# Project Metadata Resource
# https://www.terraform.io/docs/providers/google/r/compute_project_metadata.html

resource "google_compute_project_metadata" "this" {
  project = google_project.project.project_id
  metadata = {
    enable-oslogin = true
  }
}

Next, you can destroy your previous project and recreate it using this code, and you will see you've passed the tests:

✔  cis-gcp-3.1-networking: [NETWORKING] Ensure the default network does not exist in a project
✔  cis-gcp-4.4-vms: [VMS] Ensure oslogin is enabled for a Project

Not only that but we went from:

Profile Summary: 4 successful controls, 16 control failures, 21 controls skipped
Test Summary: 12 successful, 40 failures, 79 skipped

to:

Profile Summary: 5 successful controls, 11 control failures, 22 controls skipped
Test Summary: 7 successful, 12 failures, 80 skipped

I feel like the results speak for themselves. In the next post, we will focus on kitchen-terraform.