DEV Community

Cover image for Terraform Dynamic Blocks 101
Panchanan Panigrahi
Panchanan Panigrahi

Posted on

Terraform Dynamic Blocks 101

#aws #cloud #terraform #devops

Imagine scrolling through a lengthy Terraform configuration file, filled with repetitive code that feels never-ending. It’s not only tedious but also makes the whole setup hard to understand and maintain, especially when changes are needed. This is where dynamic blocks come to the rescue!

Dynamic blocks in Terraform are designed to simplify and clean up your code by reducing repetition, making it more efficient and easier to manage. They help you follow the "Don't Repeat Yourself" (DRY) principle, turning complex and cluttered configurations into concise, reusable, and elegant blocks.

In this guide, we’ll break down how dynamic blocks work, why they’re game-changers, and how you can start using them to take your Terraform scripts to the next level!

What Are Dynamic Blocks?

In Terraform, dynamic blocks provide a way to dynamically generate repeated nested blocks within resource, data, provider, and provisioner blocks. They're commonly used in resource blocks to make your configurations more flexible and follow the "Don't Repeat Yourself" (DRY) principle.

Here's the basic structure of a dynamic block in Terraform:

resource "resource_type" "resource_name" {
  # Resource block configuration

  dynamic "label" {
    for_each = collection_to_iterate
    iterator = item

    content {
      # Content of the dynamically generated block
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Breaking Down the Dynamic Block

Let’s walk through the key elements that make up a dynamic block:

  • label: This represents the type of block you want to generate dynamically (e.g., subnet, security_group_rule). It essentially defines what nested block you’re constructing within the parent block.

  • for_each: This argument takes a list or map, representing the collection of values you want to iterate over to generate multiple blocks. It allows you to loop over a set of data and create multiple instances of the nested block.

  • iterator (optional): By default, this takes the name of the label block, but you can customize it to any name you prefer. It acts as a placeholder for each element in your collection as you iterate over it. This helps in accessing individual elements within the content block.

  • content: This is the heart of your dynamic block. It defines the actual content that gets generated for each item in the iteration. Here, you use the iterator to reference properties or values of the current item in the loop.

Why Use Dynamic Blocks?

  • Eliminate Repetition: Write once, use multiple times. Dynamic blocks reduce code duplication, making your Terraform scripts more concise.
  • Increase Maintainability: Changes are easier to manage when code is less cluttered and more focused.
  • Enhance Readability: With dynamic blocks, you can structure your configuration in a way that is both efficient and easier to understand.

Example 1: Simplifying AWS Security Group Rules with Terraform Dynamic Blocks

Without Using Dynamic Blocks

Let's start by looking at a typical Terraform configuration without using dynamic blocks. This configuration sets up an AWS VPC, a public subnet, and an AWS Security Group.

resource "aws_vpc" "main_vpc" {
  cidr_block = "10.0.0.0/16"
  tags = {
    Name = "main-vpc"
  }
}

resource "aws_subnet" "public_subnet" {
  vpc_id                  = aws_vpc.main_vpc.id
  cidr_block              = "10.0.1.0/24"
  availability_zone       = "us-east-1a"
  map_public_ip_on_launch = true
  tags = {
    Name = "public-subnet"
  }
}

resource "aws_security_group" "main_sg" {
  vpc_id = aws_vpc.main_vpc.id

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = [aws_vpc.main_vpc.cidr_block]
  }

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = [aws_vpc.main_vpc.cidr_block]
  }

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = [aws_vpc.main_vpc.cidr_block]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = [aws_vpc.main_vpc.cidr_block]
  }

  tags = {
    Name = "main-sg"
  }
}
Enter fullscreen mode Exit fullscreen mode

Overview

  • VPC: Defines a Virtual Private Cloud with a CIDR block 10.0.0.0/16.
  • Subnet: Creates a public subnet within the VPC.
  • Security Group: The security group (aws_security_group.main_sg) allows inbound traffic on ports 22 (SSH), 80 (HTTP), and 443 (HTTPS) and all outbound traffic.

However, the ingress and egress rules are repetitive, which makes the configuration lengthy and difficult to maintain, especially as the rules grow. This is a prime candidate for refactoring using Terraform dynamic blocks.

Refactoring with Terraform Dynamic Blocks

Let's make our code more efficient and adhere to the DRY principle using dynamic blocks.

Using Terraform Dynamic Blocks

Variables for Ingress and Egress Settings

First, we create variables to store our ingress and egress rules, making them reusable and easy to maintain:

variable "ingress_settings" {
  type = list(object({
    description = string
    port        = number
    protocol    = string
  }))
  default = [
    {
      description = "Allows SSH access"
      port        = 22
      protocol    = "tcp"
    },
    {
      description = "Allows HTTP traffic"
      port        = 80
      protocol    = "tcp"
    },
    {
      description = "Allows HTTPS traffic"
      port        = 443
      protocol    = "tcp"
    }
  ]
}

variable "egress_settings" {
  type = list(object({
    description = string
    port        = number
    protocol    = string
  }))
  default = [
    {
      description = "Allows any protocol with any port access"
      port        = 0
      protocol    = "-1"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode
  • ingress_settings: Contains a list of ingress rules with description, port, and protocol.
  • egress_settings: Contains egress rules with similar attributes.

By storing these rules in variables, you make them flexible and easier to modify or expand.

Implementing Dynamic Blocks in Resource Block 

resource "aws_vpc" "main_vpc" {
  cidr_block = "10.0.0.0/16"
  tags = {
    Name = "main-vpc"
  }
}

resource "aws_subnet" "public_subnet" {
  vpc_id                  = aws_vpc.main_vpc.id
  cidr_block              = "10.0.1.0/24"
  availability_zone       = "us-east-1a"
  map_public_ip_on_launch = true
  tags = {
    Name = "public-subnet"
  }
}

resource "aws_security_group" "main_sg" {
  vpc_id = aws_vpc.main_vpc.id

  dynamic "ingress" {
    for_each = var.ingress_settings

    content {
      description = ingress.value["description"]
      from_port   = ingress.value["port"]
      to_port     = ingress.value["port"]
      protocol    = ingress.value["protocol"]
      cidr_blocks = [aws_vpc.main_vpc.cidr_block]
    }
  }

  dynamic "egress" {
    for_each = var.egress_settings

    iterator = main_sg_egress

    content {
      description = main_sg_egress.value["description"]
      from_port   = main_sg_egress.value["port"]
      to_port     = main_sg_egress.value["port"]
      protocol    = main_sg_egress.value["protocol"]
      cidr_blocks = [aws_vpc.main_vpc.cidr_block]
    }
  }

  tags = {
    Name = "main-sg"
  }
}
Enter fullscreen mode Exit fullscreen mode

Let us Break Down the Dynamic Block Implementation

Explanation of the ingress Dynamic Block

Here's the ingress block from our Terraform configuration:

dynamic "ingress" {
  for_each = var.ingress_settings

  content {
    description = ingress.value["description"]
    from_port   = ingress.value["port"]
    to_port     = ingress.value["port"]
    protocol    = ingress.value["protocol"]
    cidr_blocks = [aws_vpc.main_vpc.cidr_block]
  }
}
Enter fullscreen mode Exit fullscreen mode

Let’s break down each part to understand what's happening:

1. dynamic ingress Block

  • Just like with the egress block, the dynamic keyword allows us to create multiple ingress blocks based on the values in var.ingress_settings.
  • This means we can generate as many ingress rules as needed without having to hard-code each one.

2. for_each = var.ingress_settings

  • for_each tells Terraform to iterate over the list defined in the ingress_settings variable.
  • Each item in this list represents an individual ingress rule with attributes like description, port, and protocol.

3. The Absence of an Explicit iterator Definition

  • Unlike the egress block, we have not defined an iterator for the ingress block.
  • When you omit the iterator attribute, Terraform defaults to using the block label (in this case, ingress) as the iteration variable.
  • This means that within the content block, ingress.value refers to the current item in the for_each loop.

Why Is This Important?

  • Using ingress.value is the default behavior, making the code simpler and cleaner when you don't need a custom iterator name.
  • This approach is ideal when you don’t have multiple nested dynamic blocks or when there is no risk of confusion.

4. content Block

  • The content block defines the structure of each dynamically generated ingress rule:
    • description = ingress.value["description"]: Accesses the description field of the current rule.
    • from_port and to_port: Both set to ingress.value["port"], indicating the port range for this rule.
    • protocol = ingress.value["protocol"]: Sets the communication protocol (e.g., tcp, udp).
    • cidr_blocks = [aws_vpc.main_vpc.cidr_block]: Restricts access to the CIDR block defined by our main VPC.

Explanation of the egress Dynamic Block

Here's the relevant part of the code that uses a dynamic block for the egress rules:

dynamic "egress" {
  for_each = var.egress_settings

  iterator = main_sg_egress

  content {
    description = main_sg_egress.value["description"]
    from_port   = main_sg_egress.value["port"]
    to_port     = main_sg_egress.value["port"]
    protocol    = main_sg_egress.value["protocol"]
    cidr_blocks = [aws_vpc.main_vpc.cidr_block]
  }
}
Enter fullscreen mode Exit fullscreen mode

Let's break this down thoroughly:

1. dynamic egress Block

  • The dynamic keyword allows us to generate multiple instances of the egress block based on an iterable set of data (var.egress_settings in this case).
  • This approach provides a way to avoid manually duplicating the egress block for each rule, making our Terraform code much more maintainable and scalable.

2. for_each = var.egress_settings

  • This line tells Terraform to loop through each item in the var.egress_settings list and create an egress block for each element.
  • var.egress_settings is a variable of type list(object({ ... })), which contains all the rules we want to define for our egress traffic.

3. iterator = main_sg_egress

  • By default, Terraform uses the keyword egress as the iteration variable to refer to the current item in the for_each loop. However, using the iterator keyword, we override this default behavior and provide our own custom name, main_sg_egress.
  • This means that instead of accessing egress.value to get the current item's properties, we now access main_sg_egress.value.

Why Use iterator?

  • The iterator attribute is particularly useful when you want to enhance the readability of your code or avoid conflicts when nesting multiple dynamic blocks that might otherwise share the same default name. In this case, ideally we do not need any iterator but to understand this concept we are using it.

4. content Block

  • The content block defines what the egress rules should look like for each item.
  • We access each value of the current item using main_sg_egress.value. For example:
    • main_sg_egress.value["description"] provides the description of the current egress rule.
    • main_sg_egress.value["port"] specifies the port number for from_port and to_port.
    • main_sg_egress.value["protocol"] gives the protocol type (e.g., tcp, udp, or -1 for all).

Example 2: Using map(object) with Dynamic Blocks for Simplified Ingress and Egress Rules

In this example, we'll take a more efficient approach by combining ingress and egress rules into a single variable using a map(object) type. This design allows us to handle security group rules with greater flexibility and simplicity, reducing redundancy and making it easier to manage changes.

Step 1: Define a map(object) Variable for Security Group Rules

We start by defining a variable named sg_settings, which holds all our security group rules (both ingress and egress) in a single structure. This variable is a map containing object entries that define each rule's details.

variable "sg_settings" {
  type = map(object({
    type        = string   
    description = string   
    port        = number   
    protocol    = string   
  }))
  default = {
    ssh_ingress = {
      type        = "ingress"        
      description = "Allows SSH access"
      port        = 22               
      protocol    = "tcp"
    },
    http_ingress = {
      type        = "ingress"        
      description = "Allows HTTP traffic"
      port        = 80               
      protocol    = "tcp"
    },
    https_ingress = {
      type        = "ingress"        
      description = "Allows HTTPS traffic"
      port        = 443              
      protocol    = "tcp"
    },
    all_egress = {
      type        = "egress"         
      description = "Allows all outbound traffic"
      port        = 0                
      protocol    = "-1"            
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Explanation:

  • The sg_settings variable is defined as a map(object), meaning it's a collection of objects that you can look up by a key (like ssh_ingress or all_egress).
  • Each entry (ssh_ingress, http_ingress, https_ingress, all_egress) contains:
    • type: Determines if the rule is ingress (incoming) or egress (outgoing).
    • description: Provides a human-readable explanation of the rule.
    • port: Specifies which port to allow traffic through.
    • protocol: Indicates the protocol type (e.g., tcp, udp, or -1 for all protocols).

By combining both ingress and egress rules into this single variable, we make the structure cleaner and avoid repetitive definitions.

Step 2: Use Dynamic Blocks to Define Ingress and Egress Rules in the Security Group Resource

Next, we create our aws_security_group resource, where we'll use Terraform's dynamic blocks to handle both ingress and egress rules based on the type specified in sg_settings.

resource "aws_security_group" "main_sg" {
  vpc_id = aws_vpc.main_vpc.id  


  dynamic "ingress" {
    for_each = { for key, rule in var.sg_settings : key => rule if rule.type == "ingress" }

    content {
      description = ingress.value.description  
      from_port   = ingress.value.port         
      to_port     = ingress.value.port         
      protocol    = ingress.value.protocol    
      cidr_blocks = [aws_vpc.main_vpc.cidr_block]  
    }
  }


  dynamic "egress" {
    for_each = { for key, rule in var.sg_settings : key => rule if rule.type == "egress" }

    content {
      description = egress.value.description   
      from_port   = egress.value.port          
      to_port     = egress.value.port          
      protocol    = egress.value.protocol      
      cidr_blocks = [aws_vpc.main_vpc.cidr_block]  
    }
  }

  tags = {
    Name = "main-sg"  
  }
}
Enter fullscreen mode Exit fullscreen mode

Explanation:

  • resource "aws_security_group" "main_sg" creates an AWS security group within the specified VPC (aws_vpc.main_vpc.id).
  • The dynamic block generates multiple ingress or egress blocks based on the contents of our sg_settings variable.

Detailed Breakdown of the dynamic Blocks:

  1. Dynamic ingress Block:

    • for_each iterates over each entry in sg_settings where type is "ingress".
    • Inside content, the actual security group rule is created using values from sg_settings.
    • This dynamic block allows us to generate multiple ingress rules based on the entries in our variable.

  2. Dynamic egress Block:

    • Similarly, for_each iterates over rules in sg_settings where type is "egress".
    • The content block sets the values for each egress rule based on the data in the variable.
    • This ensures all outbound traffic rules are handled efficiently.

Why This Approach Is Better

  • Unified Configuration: All rules are defined in a single, easy-to-manage variable (sg_settings), reducing complexity and improving readability.
  • Reduced Redundancy: The use of dynamic blocks means we avoid manually writing multiple ingress and egress blocks, making the code cleaner.
  • Flexibility: Adding or removing rules is as simple as modifying the sg_settings variable, without needing to change the resource definition.

Example 3: Dynamic Blocks in Terraform for AMI Filtering

In this example, we leverage Terraform's dynamic blocks to filter Amazon Machine Images (AMIs) based on specific tags, enabling streamlined resource creation. This approach enhances configuration ability and allows for efficient management of EC2 instances to specific environments.

Variables and AMI Filtering 

variable "ami_tag_filters" {
  description = "A list of tag filters to locate specific Amazon Machine Images (AMIs)"
  default = [
    {
      name   = "tag:Purpose"
      values = ["WebServer"]
    },
    {
      name   = "tag:Environment"
      values = ["Production"]
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

The ami_tag_filters variable is defined as a list of filters that contain key-value pairs of tags. The variable is used to dynamically search for AMIs based on tags like Purpose and Environment. Here, the AMI is tagged with Purpose = WebServer and Environment = Production.

Data Source for AMI Selection 

data "aws_ami" "filtered_ami" {
  most_recent = true
  owners      = ["self"]

  dynamic "filter" {
    for_each = var.ami_tag_filters
    content {
      name   = filter.value.name
      values = filter.value.values
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

The data "aws_ami" "filtered_ami" block is used to dynamically filter AMIs based on the specified tags. The most_recent = true parameter ensures that the most recent AMI that matches the filters is selected. The owners attribute is set to "self", meaning only AMIs owned by the current AWS account will be considered.

Dynamic blocks are used in this section to iterate over the ami_tag_filters list. The for_each parameter takes each filter in the list and applies it to the AMI search. Inside the content block, the tag name and values are applied to each filter. This flexibility allows you to add or remove filters without modifying the data block itself.

EC2 Instance Creation 

resource "aws_instance" "web_server_instance" {
  count         = 2
  ami           = data.aws_ami.filtered_ami.id
  instance_type = "t2.micro"

  tags = {
    Name = "WebServer-${count.index}"
  }
}
Enter fullscreen mode Exit fullscreen mode

The aws_instance resource provisions two EC2 instances using the t2.micro instance type. The AMI is dynamically retrieved from the filtered AMI data source. The count parameter creates two instances, and the instance's name tag is dynamically generated using the ${count.index} syntax, which appends an index to the name (WebServer-0, WebServer-1).

Example 4: Helm Release Using Dynamic Blocks

Before Dynamic Block 

resource "helm_release" "metric_server" {
  name       = "metrics-server"
  chart      = "metrics-server"
  repository = "https://kubernetes-sigs.github.io/metrics-server/"
  version    = "3.12.0"

  set {
    name  = "replicas"
    value = "2"
  }

  set {
    name  = "metrics.enabled"
    value = "true"
  }

  set {
    name  = "serviceMonitor.enabled"
    value = "true"
  }
}
Enter fullscreen mode Exit fullscreen mode

This block of code works perfectly well, but it becomes difficult to scale as the number of settings increases. If you need to add more options or tweak existing ones, you'd have to repeat the same set block multiple times, which leads to redundancy and reduced maintainability. Now, let's optimize it using dynamic blocks to make it more modular, reusable, and maintainable.

After Dynamic Block 

# Define variables
variable "metrics_server_settings" {
  type = map(string)
  default = {
    "replicas"            = "2"
    "metrics.enabled"     = "true"
    "serviceMonitor.enabled" = "true"
  }
}

resource "helm_release" "metric_server" {
  name       = "metrics-server_release"
  chart      = "metrics-server"
  repository = "https://kubernetes-sigs.github.io/metrics-server/"
  version    = "3.12.0"

  dynamic "set" {
    for_each = var.metrics_server_settings
    content {
      name  = set.key
      value = set.value
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Breakdown of the Dynamic Configuration

  1. Variable Definition:

    • The metrics_server_settings variable is defined as a map(string). This type allows us to store multiple key-value pairs for our Helm release configuration.
    • The default values in this example include replicas, metrics.enabled, and serviceMonitor.enabled. These correspond to the settings we need for the metrics-server Helm chart.

  2. Dynamic Block:

    • The dynamic block leverages the for_each argument to loop over the key-value pairs in var.metrics_server_settings.
    • For every key in the map, Terraform generates a set block, where the key becomes the Helm chart setting (e.g., replicas or metrics.enabled), and the value is passed to the set.value.

Why the Dynamic Block is Better

The use of dynamic blocks transforms a static, repetitive setup into a flexible and concise solution. Instead of writing multiple set blocks, you can define all your settings in a map variable and then iterate over them, automatically generating the necessary set blocks. This approach not only reduces code duplication but also allows for easy updates and future scalability.

For example, if you want to add another setting, such as configuring resource limits, you would simply modify the variable:

variable "metrics_server_settings" {
  type = map(string)
  default = {
    "replicas"                = "2"
    "metrics.enabled"         = "true"
    "serviceMonitor.enabled"  = "true"
    "resources.limits.cpu"    = "100m"
  }
}
Enter fullscreen mode Exit fullscreen mode

With this, the dynamic block automatically picks up the additional setting without needing any changes to the core Helm release resource code.

Example 5: Nested Dynamic Blocks (EC2 Instance with Dynamic EBS Volumes and Tags)

When dealing with cloud infrastructure, flexibility is key—especially when managing resources like EBS volumes attached to EC2 instances. In this example, we'll use nested dynamic blocks to dynamically attach multiple EBS volumes to an EC2 instance, each with its own configuration and tags.

This scenario reflects a more realistic infrastructure setup where you may need to attach additional storage to your EC2 instance based on different workload needs, such as adding extra volumes for data storage, backup, or application-specific requirements. We'll also apply unique tags to these volumes, helping with organization, cost tracking, or environment differentiation.

Let’s dive in to see how dynamic blocks can help make your Terraform configuration flexible and scalable.

Variables Definition Using map(object)

We’ll define the EBS volumes, their sizes, types, and specific tags using a map(object) variable.

variable "ebs_volumes" {
  type = map(object({
    size = number
    type = string
    tags = map(string)
  }))
  default = {
    "data_volume" = {
      size = 50
      type = "gp3"
      tags = {
        Name  = "Data-Volume"
        Env   = "Prod"
        Usage = "Database"
      }
    },
    "backup_volume" = {
      size = 100
      type = "gp3"
      tags = {
        Name  = "Backup-Volume"
        Env   = "Prod"
        Usage = "Backup"
      }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

In this example:

  • The ebs_volumes variable is a map where:
    • Keys (data_volume, backup_volume) represent unique identifiers for the volumes.
    • Each volume is defined as an object containing:
    • size: The volume size in GB.
    • type: The type of the EBS volume (e.g., gp3).
    • tags: A map of key-value pairs to tag each volume, such as its purpose, environment, and other metadata.

Terraform Configuration with Nested Dynamic Blocks

Using the dynamic block setup, we’ll create an EC2 instance that dynamically attaches the specified volumes and applies unique tags to each volume.

resource "aws_instance" "example" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t3.medium"

  dynamic "ebs_block_device" {
    for_each = var.ebs_volumes

    iterator = ebs_volume

    content {
      device_name = "/dev/sd${ebs_volume.key}"
      volume_size = ebs_volume.value.size
      volume_type = ebs_volume.value.type

      dynamic "tags" {
        for_each = ebs_volume.value.tags

        iterator = volume_tag

        content {
          key   = volume_tag.key
          value = volume_tag.value
        }
      }
    }
  }

  tags = {
    Name = "App-EC2-Instance"
    Env  = "Prod"
  }
}
Enter fullscreen mode Exit fullscreen mode

Explanation:

  1. Dynamic Block for ebs_block_device:

    • The dynamic block uses the ebs_volume iterator to loop over the var.ebs_volumes map. Each key in this map represents an individual volume, and the block dynamically configures the device_name, volume_size, and volume_type based on the attributes of each ebs_volume.
    • The device_name is generated dynamically using the ebs_volume.key to ensure each attached volume has a unique identifier. The volume_size and volume_type are set according to the values provided in the ebs_volumes variable.

  2. Nested Dynamic Block for tags:

    • Inside each ebs_volume block, the nested dynamic block iterates over the volume's tags using the volume_tag iterator. This enables Terraform to apply a set of key-value tags to each specific volume.
    • The volume_tag.key and volume_tag.value ensure that each volume receives the exact tags as defined in the ebs_volumes map.

By using meaningful iterators like ebs_volume and volume_tag, the structure of the Terraform configuration becomes clearer, enhancing readability and maintainability. The use of nested dynamic blocks ensures flexibility and future-proofing, allowing for efficient management of multiple EBS volumes and consistent application of tags. This approach keeps the configuration clean, scalable, and easy to adapt as infrastructure needs grow.

Conclusion

Get hooked on Terraform dynamic blocks! These powerful features streamline your infrastructure as code configurations, reducing redundancy, enhancing readability, and improving maintainability. By leveraging dynamic blocks, you can create more flexible and scalable solutions, simplifying complex setups and managing resources with ease. So, dive into the world of Terraform dynamic blocks and experience the transformation power they bring to your infrastructure management.

Top comments (0)