Terraform Cookbook: Development Environment Recipe

Denis — Thu, 02 Jan 2025 22:32:40 +0000

TLDR: This tutorial provides a step-by-step guide to setting up a complete Terraform development environment, including version management (tenv), documentation generation (terraform-docs), linting (TFLint), security scanning (Trivy, Checkov), pre-commit hooks, and others. We use a cookbook-style approach for easy understanding, but if you just want the command script to set up the environment, you can get it here.

Terraform Cookbook: Development Environment Recipe

A quick and easy recipe to set up a killer development environment for writing Terraform modules. Tested on Linux Ubuntu 24.04.1 LTS for x86_64 architecture (should work on other debian-based systems. For other Linux distributions, some tweaks are required).

Building robust and secure infrastructure with Terraform requires more than just the terraform CLI. A well-equipped development environment is crucial for maximizing productivity, ensuring code quality, and preventing costly mistakes. This tutorial serves as your complete recipe for setting up a top-notch Terraform development environment. I'll guide you through installing and configuring essential tools. By the end of this tutorial, you will have a streamlined workflow, enabling you to focus on building infrastructure rather than managing complex configurations. We use a cookbook-style approach for easy understanding.

Ingredients:

Linux utilities (curl, jq, tar, unzip, git)
tenv: version manager for Terraform
terraform CLI: HashiCorp's Terraform CLI
terraform-docs: tool for generating module documentation
TFLint: terraform linter
Trivy: security scanner for IaC and dependencies
venv: virtual environments for Python
Pre-commit: git hook manager
Pre-commit-terraform: terraform git hooks
Checkov: static analysis for IaC
VSCode: code editor with Terraform extension

Preparation Steps:

1. Know Your Kitchen, VSCode.

First things first, every chef needs a well-equipped kitchen. In our case, Visual Studio Code is our kitchen, the place where we'll craft all our infrastructure recipes. But a bare kitchen isn't enough. We need the right tools and appliances to prepare delicious dishes. That's where the Terraform extension by Hashicorp comes in. The extension provides syntax highlighting, code completion, among other features.

# Follow the [VSCode installation guide](https://code.visualstudio.com/docs/setup/linux) for other instalation methods.
sudo apt-get install curl

curl -L -o vscode.deb  https://go.microsoft.com/fwlink/?LinkID=760868

sudo apt install ./vscode.deb

code 
# After VSCode launch, use Quick Open (Ctrl+P), paste the command "ext install HashiCorp.terraform" and press enter.
# Or install the extension @id:HashiCorp.terraform using the VSCode GUI.

2. Equip Your Kitchen with `tenv`.

Manage multiple Terraform versions like a pro. Simplifies the management of multiple Terraform versions, ensuring compatibility with different projects.

Grab your Terraform version manager, tenv. Every good chef needs a well-organized kitchen. When it comes to managing different versions of Terraform, tenv will keep your environment neatly organized and easily accessible. It also supports OpenTofu, Terragrunt, and Atmos. Let's get it installed:

TENV_LATEST=$(curl -s https://api.github.com/repos/tofuutils/tenv/releases/latest | jq -r '.assets[] | select(.name | endswith("Linux_x86_64.tar.gz")) | .browser_download_url')

curl -L -O $TENV_LATEST

mkdir ~/.tenv

tar xvzf $(echo $TENV_LATEST | grep -o -E "tenv_v.*") -C ~/.tenv

export PATH="$HOME/.tenv:$PATH"

tenv completion bash > ~/.tenv/tenv_completion.bash

echo -e '\n# tenv and terraform tools\nexport PATH="$HOME/.tenv:$PATH"\nsource $HOME/.tenv/tenv_completion.bash' >> ~/.bashrc

The lines added to .bashrc ensure that tenv is always available when we start cooking terraform modules.

3. Grab Your Trusty `terraform` CLI Using `tenv`.

It's time to grab our most essential infrastructure cooking utensil, our pan: the terraform CLI. This is the workhorse of our infrastructure kitchen – the pan we'll use to cook up all our infrastructure recipes. We'll use tenv to fetch and install the perfect pan for the job. If you wish, use Opentofu, which is an "more open" source altenative.

tenv terraform install

terraform -install-autocomplete

This is like reaching into our neatly organized tools rack and selecting the perfect terraform CLI pan.

4. Compile Your Infrastructure Recipe Notebook With `terraform-docs`.

Automatically generate documentation for your terraform modules (inputs, outputs, etc.).

A good chef keeps a detailed recipe notebook, documenting each dish's ingredients, instructions, and variations. In the world of infrastructure as code, terraform-docs is our tool to maintain recipe documents. It automatically generates documentation for our Terraform modules, neatly organizing inputs, outputs, and other important details. This ensures that anyone (including our future selves) can easily understand and use our infrastructure recipes. Let's add it to our kitchen:

TDOC_LATEST=$(curl -s https://api.github.com/repos/terraform-docs/terraform-docs/releases/latest | jq -r '.assets[] | select(.name | endswith("linux-amd64.tar.gz")) | .browser_download_url')

curl -L -O $TDOC_LATEST

tar xzf $(echo $TDOC_LATEST | grep -o -E "terraform-docs-.+") -C ~/.tenv terraform-docs

terraform-docs completion bash > ~/.tenv/terraform-docs_completion.bash

echo -e '\nsource $HOME/.tenv/terraform-docs_completion.bash' >> ~/.bashrc

5. Employ TFLint Quality Control.

A linter for Terraform code. Catch syntax errors, enforce standards, and keep your Terraform configs clean.

Maintaining quality control is crucial. In the Terraform kitchen, TFLint scans our Terraform code for syntax errors and potential issues, ensuring our code is sound. Let's add this to our kitchen:

TFLINT_LATEST=$(curl -s https://api.github.com/repos/terraform-linters/tflint/releases/latest | jq -r '.assets[] | select(.name | endswith("tflint_linux_amd64.zip")) | .browser_download_url')

curl -L -O $TFLINT_LATEST

unzip -o $(echo $TFLINT_LATEST | grep -o -E "tflint_linux.+") -d ~/.tenv

The .tflint.hcl file is the configuration file for TFLint. Its purpose is to define the rules, plugins, and configurations that TFLint will use when analyzing your Terraform code. This allows you to enforce best practices, identify issues, and customize the linter's behavior to align with your project requirements. The configuration I currently use for terraform AWS resources is like the bellow example.

Sample `.tflint.hcl` Configuration

tflint {
  required_version = "~> 0.54.0"
}

plugin "terraform" {
  enabled = true
  preset  = "all"
}

plugin "aws" {
  enabled = true
  version = "0.36.0"
  source  = "github.com/terraform-linters/tflint-ruleset-aws"
  deep_check = true
}

rule "aws_resource_missing_tags" {
  enabled = true
  tags = [
    "env"
  ]
}

6. Double Strain Your Infrastructure Code With `Trivy` And `Checkov`.

Trivy scan your IaC files, container images, and dependencies for vulnerabilities.

Checkov detects misconfigurations, security risks, and compliance violations in Terraform and other IaC templates.

Ensuring the safety of your code is paramount. You wouldn't want any unwanted bits sneaking into your dishes. Just as a chef uses strainers to remove impurities, we use Trivy and Checkov to filter out potential vulnerabilities and misconfigurations from our infrastructure code. Think of them as two different types of strainers, each catching different kinds of unwanted elements. Let's add them:

TRIVY_LATEST=$(curl -s https://api.github.com/repos/aquasecurity/trivy/releases/latest | jq -r '.assets[] | select(.name | endswith("Linux-64bit.tar.gz")) | .browser_download_url')

curl -L -O $TRIVY_LATEST

tar xzf $(echo $TRIVY_LATEST | grep -o -E "trivy_.+") -C ~/.tenv trivy

# We are going to install checkov using python pip and venv
sudo apt install python3-pip python3.12-venv

python3 -m venv ~/.venv

source ~/.venv/bin/activate

pip install checkov

Now, with both Trivy and Checkov in our kitchen, we can confidently cook a safe and secure code. This double-straining approach provides a more comprehensive security check, catching both specific vulnerabilities and broader misconfigurations.

7. Set Up Your `pre-commit` Kitchen Workflow Checklist

Ensures consistent code quality by running checks (e.g., linting, formatting, security scans) before committing code.

In a busy kitchen, chefs rely on checklists and workflows to ensure every step is followed and no crucial task is missed. pre-commit acts as our kitchen workflow checklist, ensuring we systematically employ all our quality control and tools (like TFLint, Trivy, and Checkov) before we "serve" (commit) our infrastructure code.

pre-commit uses git hooks, which are scripts that run automatically at specific points in the Git workflow. In this case, we're going to use pre-commit hooks, which run before each git commit. This ensures that our code is always checked before it's added to the project's history. Let's set up our kitchen workflow checklist:

source ~/.venv/bin/activate

pip install pre-commit

Now that we have pre-commit installed, we'll configure it to use our quality control tools. The .pre-commit-config.yaml file is the configuration file for the Pre-commit framework that define the pre-commit hook.

Sample `.pre-commit-config.yaml` Configuration.

repos:
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.96.2
    hooks:
      - id: terraform_validate
      - id: terraform_fmt
      - id: terraform_tflint
        args:
          - --args=--config=__GIT_WORKING_DIR__/.tflint.hcl
      - id: terraform_trivy
      - id: terraform_checkov
        name: "Terraform validate with Checkov"
        args:
          - --args=--quiet
      - id: terraform_docs

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v5.0.0
    hooks:
      - id: detect-aws-credentials
        name: "Pre-commit detect AWS credentials"
      - id: detect-private-key
        name: "Pre-commit detect private keys"
      - id: end-of-file-fixer
        name: "Pre-commit fix end of files"
      - id: trailing-whitespace
        name: "Pre-commit remove trailing whitespaces"

Beyond quality and safety, presentation matters too! Just as chefs ensure consistent dish presentation, we also use terraform fmt and others pre-commit hooks to format our code in a consistent way. This makes our code easier to read, understand, and maintain, both for ourselves and our colleagues.

Now that our kitchen is equipped, it's time to gather the ingredients for our infrastructure main dish: the Terraform root module.

8. Our Main Infrastructure Dish: The Terraform Root Module.

Just like any good recipe, it requires the right ingredients, organized in the right way. HashiCorp suggests a standard module structure, which we'll follow to ensure a well-structured and maintainable code. Here is an example:

root-module/
├── README.md
├── main.tf
├── variables.tf
├── outputs.tf
├── providers.tf
├── modules/
│   ├── child-module-A/
│   │   ├── README.md
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   ├── outputs.tf
│   ├── child-module-B/
│   ├── .../
├── .pre-commit-config.yaml
└── .tflint.hcl

With our main dish structure defined, let's dive into a sample recipe specifically for AWS dishes.

A Sample Recipe for AWS

A template that serves as a foundation for creating Terraform modules.

Here is a handy module template to bootstrap your AWS resources. It serves as a foundation starting point for various AWS infrastructure creations, and includes essential ingredients for a robust and secure AWS setup, including:

An S3 Bucket for State Storage. It includes server-side encryption, versioning, and lifecycle rules for noncurrent object transition and expiration, ensuring the safety and management of our stored data. It also blocks public access, preventing unwanted access.
A DynamoDB Table for State Locking. This acts as a coordination system in our kitchen, preventing conflicts when we have multiple chefs.

Template usage example

aws configure # You will need the AWS CLI installed. Set up AWS credentials with privileges to provision resources. Remember to use temporary credentials or store permanent credentials outside the git repository.

git clone https://github.com/zidenis/terraform-module-template-aws.git my-terraform-module

cd my-terraform-module

git remote remove origin # We remove the original remote to start our own module

# Do not run this script if your intention is to develop the module locally.
chmod +x backend_bootstrap.sh; ./backend_bootstrap.sh # optionally, use this bash script to set up the remote backend. 

source ~/.venv/bin/activate

pre-commit install --install-hooks # set up your git repository to run pre-commit hooks automatically

pre-commit run -a # optionally, run pre-commit hooks against all the files.

You can now start developing your specific module, using the provided structure and backend setup as a solid foundation. Using pre-commit, you can run all the quality and security checks we set up earlier, ensuring a perfect and secure final dish.

pre-commit usage example

source ~/.venv/bin/activate

echo "# cooking a new module" >>  main.tf

git add main.tf

git commit -m "Cooking a new module"
# [INFO] Initializing environment for https://github.com/antonbabenko/pre-commit-terraform.
# [INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
# [INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
# [INFO] Once installed this environment will be reused.
# [INFO] This may take a few minutes...
# Terraform validate.......................................................Passed
# Terraform fmt............................................................Passed
# Terraform validate with tflint...........................................Passed
# Terraform validate with trivy............................................Passed
# Terraform validate with Checkov..........................................Passed
# Terraform docs...........................................................Passed
# Pre-commit detect AWS credentials........................................Passed
# Pre-commit detect private keys...........................................Passed
# Pre-commit fix end of files..............................................Passed
# Pre-commit remove trailing whitespaces...................................Passed

Conclusion

By following this recipe, you've equipped your development environment with essential tools to write, validate, and secure Terraform modules effectively. This tutorial was designed to install the latest versions of the tools, but remember to regularly update your tools and configurations for optimal performance and compliance.

Are there any additional tools or configurations you'd like to explore to complement the setup? are you using Generative AI to develop your terraform modules? please, tell me how. While Generative AI has the potential to streamline Terraform development by automating certain tasks like code generation or identifying potential issues. However, at this stage, I think it's likely not a core component for most users setting up a development environment. This is something that I plan to investigate.

Let me know your thoughts and suggestions to improve this guide further. Happy coding!

Tools versions at the time of writing

tenv version
# tenv version v3.2.11

terraform version
# Terraform v1.10.3
# on linux_amd64

terraform-docs version
# terraform-docs version v0.19.0 af31cc6 linux/amd64

tflint --version
# TFLint version 0.54.0
# + ruleset.terraform (0.10.0-bundled)

trivy --version
# Version: 0.58.0

checkov --version
# 3.2.343

pre-commit --version
#pre-commit 4.0.1

Cleanup

Clean up downloaded archives.

rm $(echo $TENV_LATEST | grep -o -E "tenv_v.*")

rm $(echo $TDOC_LATEST | grep -o -E "terraform-docs-.+")

rm $(echo $TFLINT_LATEST | grep -o -E "tflint_linux.+")

rm $(echo $TRIVY_LATEST | grep -o -E "trivy_.+")

rm vscode.deb

DEV Community: Denis

Terraform Cookbook: Development Environment Recipe