DEV Community

Cover image for Terraform for_each: Examples, Tips and Best Practices
env0 Team for env0

Posted on • Originally published at env0.com

Terraform for_each: Examples, Tips and Best Practices

Why do we need Looping in Terraform?

When managing Infrastructure-as-code (IaC) with the Terraform CLI, one often encounters scenarios where multiple resources that are similar but not identical need to be separately created. 

This could range from deploying several instances across different availability zones, setting up multiple DNS records, or managing numerous user accounts. Writing out configurations manually for each resource becomes tedious and introduces a higher chance of errors and inconsistencies.

This is where looping in Terraform comes into play. Looping constructs, like the for expression, for_each, and count meta-arguments, provide a way to generate similar resources dynamically based on a collection or count.

Meta-arguments and Expressions for Terraform Looping

Meta-arguments, in a nutshell, are unique arguments that can be defined for all Terraform resources, altering specific behaviors of resources, such as their lifecycle, how they are provisioned, and their relationship with other resources. 

Expressions in Terraform are used to reference or compute values within your infrastructure configuration (like dynamic calculations, data access, and resource referencing).

These are mainly used in a Terraform resource block or a module block.

There are five different types of meta-arguments for Terraform resources, but we are going to focus on expressions and meta-arguments that help in looping for Terraform resources:

1.for_each - a meta-argument used to create multiple instances of a resource or module.  As the name implies, the for_each argument takes a map or a set of strings, creating an instance for each item. It provides more flexibility than count by allowing you to use complex data structures and access each instance with a unique identifier.

2.count - a meta-argument allows you to create multiple instances of a resource based on the given count. This is useful for creating similar resources without having to duplicate configuration blocks. For example, if count=5 for an EC2 instance resource configuration, Terraform creates five of those instances in your cloud environment.

3.for - a versatile expression for iterating over and manipulating collections such as lists, sets, and maps. The for expression can be used to iterate over elements in a collection and apply a transformation to each element, optionally filtering elements based on a condition.

For example:

locals {
  original_set = {1, 2, 3, 4, 5}
  even_set = {for i in local.original_set : i if i % 2 == 0}
}
Enter fullscreen mode Exit fullscreen mode

This creates an even_set containing only the even numbers from original_set.

for vs. for_each vs. count

Here are some key points that differentiate the for expression from the for_each and count meta arguments.

for vs. for_each vs. count

How does for_each work?

Let us take a real-world example to better understand the for_each meta-argument.

Say you have a map of instance configurations where the string value for each key is an identifier for the EC2 instance, and the value is another map containing the instance type and AMI ID.

#main.tf
variable "instances" {
  description = "Map of instance configurations"
  type = map(object({
    ami           = string
    instance_type = string
  }))
  default = {
    "amzlinux" = {
      ami           = "ami-02d3fd86e6a2f5122"
      instance_type = "t2.micro"
    },
    "ubuntu" = {
      ami           = "ami-0ce2cb35386fc22e9"
      instance_type = "t2.small"
    }
  }
}
resource "aws_instance" "servers" {
  for_each      = var.instances
  ami           = each.value.ami
  instance_type = each.value.instance_type
  tags = {
    Name = "env0-Server-${each.key}"
  }
}
Enter fullscreen mode Exit fullscreen mode

Let’s break everything down:

  • variable "instances" defines a map where each element represents an EC2 instance configuration. For instance, "amzlinux" and "ubuntu" are identifiers for these configurations, each specifying an AMI ID and an instance type.
  • resource "aws_instance" "servers" uses for_each to iterate over each element in the var.instances map. For each element, it creates an EC2 instance with the specified AMI and instance type.
  • each.key in this context refers to the key in the map (e.g., "amzlinux", "ubuntu"), which we use to uniquely name each instance with the Name tag.
  • each.value.ami and each.value.instance_type access the nested values for each instance's configuration.

After successfully running the Terraform workflow (init->plan->apply), we have provisioned these two instances using for_each.

Collections for for_each

Maps

Maps are collections of key-value pairs. When using for_each with maps, each iteration gives you access to both the map key and the value of the current item. Maps are ideal when you need to associate specific attributes or configurations with unique identifiers.

#Example config
variable "instance_tags" {
  type = map(string)
  default = {
    "Role" = "Web-server"
    "Environment" = "Production"
  }
}
resource "aws_instance" "server_tags" {
  for_each = var.instance_tags
  # Other Configuration...
  tags = {
    "${each.key}" = "${each.value}"
  }
}
Enter fullscreen mode Exit fullscreen mode

Sets

Sets are collections of unique values. When iterating over a set with for_each, the value for each.key and each.value will be the same since sets do not have key-value pairs but just a list of unique values.

Sets are useful when you need to ensure uniqueness and don't require associated values.

#Example config
variable "availability_zones" {
  type = set(string)
  default = ["us-west-2a", "us-west-2b"]
}
resource "aws_subnet" "list_subnets" {
  for_each = var.availability_zones
  availability_zone = each.key
  # Other Configuration...
}
Enter fullscreen mode Exit fullscreen mode

Lists

Directly, for_each cannot iterate over lists because lists do not provide a unique key for each item.

However, you can use the tomap() or toset() function to convert a list into a set or a map, allowing for_each to iterate over it.

#Example config
variable "availability_zones" {
  type = list(string)
  default = ["us-east-1a", "us-east-1b"]
}
resource "aws_subnet" "example" {
  for_each = toset(var.availability_zones)    #or tomap(var.availability_zones)
  availability_zone = each.key
  # Other configurations...
}
Enter fullscreen mode Exit fullscreen mode

Practical Use Cases for for_each

1. Resource Chaining

Resource chaining involves creating dependencies between resources where the configuration of one resource depends on the output of another. This is common in infrastructure setups where certain resources must be provisioned sequentially.

Let us take a common scenario of setting up a VPC and deploying subnets within the VPC, to better understand resource chaining.

#variable "networks" {
  description = "VPC Network Map"
  type = map(object({
    cidr = string
  }))
  default = {
    "network1" = {
      cidr = "10.0.1.0/24"
    },
    "network2" = {
      cidr = "10.0.2.0/24"
    }
  }
}
resource "aws_vpc" "main" {
  for_each = var.networks
  cidr_block = each.value.cidr
  tags = {
    Name = "VPC-${each.key}"
  }
}
resource "aws_subnet" "subnets" {
  for_each          = var.networks
  vpc_id            = aws_vpc.main[each.key].id
  cidr_block        = each.value.cidr
  availability_zone = "us-west-2a"
  tags = {
    Name = "Subnet-${each.key}"
  }
}
Enter fullscreen mode Exit fullscreen mode

In this example, each VPC is created based on the networks map, and then a subnet is created within each VPC. The aws_subnet resource uses the ID of the aws_vpc created in the same for_each loop, demonstrating resource chaining.

2. Tagging Resources Dynamically

Dynamic tagging allows you to assign metadata to resources based on their configuration or other dynamic inputs, improving resource management, billing, and automation.

We can take an example of dynamically tagging s3 buckets using for_each:

variable "buckets" {
  description = "Map of S3 bucket configurations"
  type = map(object({
    name = string
    tags = map(string)
  }))
  default = {
    "bucket1" = {
      name = "env0-app-logs",
      tags = {
        "Environment" = "Production",
        "Application" = "Logging",
      }
    },
    "bucket2" = {
      name = "env0-app-data",
      tags = {
        "Environment" = "Staging",
        "Application" = "DataStore",
      }
    }
  }
}
resource "aws_s3_bucket" "env0_app_bucket" {
  for_each = var.buckets
  bucket = each.value.name
  tags = each.value.tags
}
Enter fullscreen mode Exit fullscreen mode

This setup dynamically applies tags to each S3 bucket based on the tags defined in the buckets map.

3. Deploying Resources to Multiple Regions

Deploying resources across multiple regions can enhance disaster recovery and reduce latency. for_each can be used to manage such deployments efficiently.

We can deploy s3 buckets to multiple regions using for_each like this example below.

variable "regions" {
  description = "Regions to deploy S3 buckets"
  type        = map(string)
  default     = {
    "us-east-1" = "env0-app-us-east-1",
    "eu-central-1" = "env0-app-eu-central-1"
  }
}
provider "aws" {
  alias  = "useast1"
  region = "us-east-1"
}
provider "aws" {
  alias  = "eucentral1"
  region = "eu-central-1"
}
resource "aws_s3_bucket" "multi_region_buckets" {
  for_each = var.regions
  bucket   = each.value
  provider = each.key == "us-east-1" ? aws.useast1 : aws.eucentral1
}
Enter fullscreen mode Exit fullscreen mode

In this example, S3 buckets are created in both the us-east-1 and eu-central-1 regions, using separate provider instances for each region. The provider attribute dynamically selects the correct provider based on the region key from the regions map.

Advantages of using for_each

1. Dynamic Resource Management

for_each enables the dynamic creation, management, and destruction of resources based on collections (maps or sets). This dynamic approach allows infrastructure to adjust automatically to changes in the input data without requiring manual updates to the Terraform configuration.

For example, you can write an IaC for infrastructure provisioning to provision a dynamic number of S3 buckets based on a list of project names like the example below:

variable "project_names" {
type = set(string)
default = ["DisasterRecovery", "VPCNetworkSetup"]
}
resource "aws_s3_bucket" "project_bucket" {
for_each = var.project_names
bucket = each.value
}
Enter fullscreen mode Exit fullscreen mode




2. Conditional Resource Creation

Combined with Terraform's conditional expressions, for_each can be used to conditionally create resources based on specific criteria within the data it iterates over. This allows for more granular control over which resources are created, updated, or destroyed.

For instance, you can tailor your IaC in such a way that creates an S3 bucket only if create=true:

variable "bucket_configs" {
type = map(object({
name = string
create = bool
}))
default = {
"bucket1" = { name = "env0-1", create = true },
"bucket2" = { name = "env0-2", create = false }
}
}
resource "aws_s3_bucket" "conditional_bucket" {
for_each = { for k, v in var.bucket_configs : k => v if v.create }
bucket = each.value.name
}
Enter fullscreen mode Exit fullscreen mode




3. Improved Code Reusability

Instead of duplicating resource blocks for each instance of a resource, for_each allows you to define a single resource (or a module) block that can be applied to each item in a collection (like a map or set).

This approach abstracts the common configuration elements into a single, parameterized block, where the specific details for each resource instance are dynamically derived from the collection it iterates over.

For example, you can incorporate for_each with the use of modules to keep your code DRY:

variable "environments" {
type = map(string)
default = {
"dev" = "ami-02d3fd86e6a2f5122",
"prod" = "ami-0ce2cb35386fc22e9",
}
}
module "dry_module_ec2" {
for_each = var.environments
source = "terraform-aws-modules/ec2-instance/aws"
ami = each.value
instance_type = "t2.micro"
}
Enter fullscreen mode Exit fullscreen mode




Commonly Asked Questions/FAQs

Q. Can I use Terraform for_each and count for the same resource?

No, for_each and count cannot be used together within the same resource or module block. You must choose one based on your use case. 

count is used to create a specified number of identical resources. for_each iterates over items in a map or set to create multiple resources with different configurations.

Q. What is the use of count.index in Terraform?

The count.index in Terraform is a built-in variable that starts with a current zero-based index of the resource created in a block where the count meta-argument is used. 

It's primarily used when you're creating multiple instances of a resource with count and need to differentiate between these individual instances, with a numeric identifier.

Q. Can for_each be used with modules?

Yes, for_each can be used with Terraform modules, enabling you to create multiple instances of a module based on the items in a map or a set. When using for_each with a module, you can define a collection (map or set) containing the values you want to iterate over.

Q. Can count be used to conditionally create a resource?

Yes, count can be used to create a resource in Terraform conditionally. You can specify whether to create a resource based on a condition by leveraging the count meta-argument. For example, if the condition evaluates to true, create a single instance of the resource, and prevent the resource creation if it evaluates to false.

Q. Is it possible to migrate from count to for_each?

Yes, it is possible to migrate from count to for_each in Terraform, but the process requires careful planning and execution. You can check here for detailed information on migrating count to for_each.

Top comments (2)

Collapse
 
ravidsinghbiz profile image
Ravi D. Singh

Wondering if the for_each could be used to bulk import in Terraform.

Collapse
 
chpl profile image
chpl