Terraform: Using Dynamic Blocks and for_each

In Terraform proper usage of dynamic blocks, for_each and conditionals allow you to build adaptable and reusable infrastructure configurations. In this blog, you will see practical examples of how to use these features to simplify complex setups and manage multi-environment deployments efficiently and flexibly.

Dynamic Blocks for Complex Resources
Dynamic blocks are ideal when you need repeatable sub-resources within a resource block. For example, you need to deploy Cloud Watch Metric Stream to continually stream Metrics to specific destinations of your choice with near real-time delivery. Without going into details of how that works, let's say that you must pass specific metric namespaces to the aws_cloudwatch_metric_stream. We refer to those as filters so that we can stream only specified metrics and have many. An example of the Metric Stream would be:

resource "aws_cloudwatch_metric_stream" "main" {
  name          = "my-metric-stream"
  role_arn      = aws_iam_role.metric_stream_to_firehose.arn
  firehose_arn  = aws_kinesis_firehose_delivery_stream.s3_stream.arn
  output_format = "json"

  include_filter {
    namespace    = "AWS/EC2"
    metric_names = ["CPUUtilization", "NetworkOut"]
  }

  include_filter {
    namespace    = "AWS/EBS"
    metric_names = []
  }

  include_filter {
    namespace    = "AWS/Timestream""
    metric_names = []
  }
}

Without using dynamic block you will have to repeat include_filter for each AWS Service you want to stream. However, if you introduce a dynamic block you can avoid duplicating configuration:

resource "aws_cloudwatch_metric_stream" "main" {
  name          = "my-metric-stream"
  role_arn      = aws_iam_role.metric_stream_to_firehose.arn
  firehose_arn  = aws_kinesis_firehose_delivery_stream.s3_stream.arn
  output_format = "json"

  dynamic "include_filter" {
    for_each = var.metric_namespaces
    content {
        namespace    = include_filter.key
        metric_names = include_filter.value
    }
 }
}

And metric_namespaces would look like this:

variable "metric_namespaces" {
  type = map(list(string))
  default = {
    "AWS/EC2"        = ["CPUUtilization", "NetworkOut"]
    "AWS/EBS"        = []
    "AWS/Timestream" = []
  }
}

Since we have a Metric namespace and then for each namespace we can have multiple metrics we should use a variable of type map(list(string)). The map's key will be a namespace value such as AWS/EC2 and the map's value will be a list of metrics such as CPUUtilization and NetworkOut. With this approach, we won't have a need to change the resource block but instead just add new services to variable metric_namespaces, and our dynamic block will do its job.

For_each for Efficient Resource Creation
Previous examples also showcase how we use for_each to effectively iterate over maps or lists. This is particularly useful for managing collections. However for_each is not limited to dynamic blocks, meaning you can use to create multiple resources based on a single resource block and variable or local. If for example, you want to deploy multiple EC2 instances you can use the map to specify multiple amis:

resource "aws_instance" "example" {
  for_each = var.ami_ids

  ami           = each.value
  instance_type = var.instance_type
}

variable "ami_ids" {
  type    = set(string)
  default = ["ami-12345", "ami-abcde"]
}

variable "instance_type" {
  type    = string
  default = "t2.micro"
}

ami_ids is a type set of strings where is string is an AMI ID and instance_type is a simple variable of type string that holds the same instance type for all instances that we are deploying. If in the future we need additional EC2 instances we could just add a new AMI ID. Additionally more advanced approach would be to use a variable that is type list(map(string)):

variable "instances" {
  type = list(map(string))

  default = [
    { ami = "ami-12345", instance_type = "t2.micro" },
    { ami = "ami-abcde", instance_type = "t3.micro" }
  ]
}

In this scenario aws_instance block would look like:

resource "aws_instance" "example" {
  for_each = { for idx, instance in var.instances : idx => instance }

  ami           = each.value.ami
  instance_type = each.value.instance_type
}

For_each argument utilizes a for expression to iterate over the var.instances list of maps. Each instance is paired with an index (idx), creating a map where the index serves as a unique key and the instance object is the corresponding value. This mapping allows Terraform to manage each EC2 instance separately, enabling efficient resource creation.
Another approach would be to use locals instead of variables if instance configurations are only needed within the current module as it keeps the module self-contained:

locals {
  instances = [
    { ami = "ami-12345", instance_type = "t2.micro" },
    { ami = "ami-abcde", instance_type = "t3.micro" }
  ]
}

resource "aws_instance" "example" {
  for_each = { for idx, instance in local.instances : idx => instance }

  ami           = each.value.ami
  instance_type = each.value.instance_type
}

In this example we define a local value to make complex structures more readable. For_each iterates over the local.instances, allowing you to create multiple instances based on specific configuration.

Conclusion
Leveraging dynamic blocks and for_each in Terraform enhances flexibility and scalability by enabling conditional resource creation and iteration over collections. By understanding and utilizing these constructs, you can create more modular and maintainable Terraform configurations that respond effectively to changing infrastructure needs.