Why do we need Looping in Terraform?
When managing Infrastructure-as-code (IaC) with the Terraform CLI, one often encounters scenarios where multiple resources that are similar but not identical need to be separately created.
This could range from deploying several instances across different availability zones, setting up multiple DNS records, or managing numerous user accounts. Writing out configurations manually for each resource becomes tedious and introduces a higher chance of errors and inconsistencies.
This is where looping in Terraform comes into play. Looping constructs, like the for expression, for_each, and count meta-arguments, provide a way to generate similar resources dynamically based on a collection or count.
Meta-arguments and Expressions for Terraform Looping
Meta-arguments, in a nutshell, are unique arguments that can be defined for all Terraform resources, altering specific behaviors of resources, such as their lifecycle, how they are provisioned, and their relationship with other resources.
Expressions in Terraform are used to reference or compute values within your infrastructure configuration (like dynamic calculations, data access, and resource referencing).
These are mainly used in a Terraform resource block or a module block.
There are five different types of meta-arguments for Terraform resources, but we are going to focus on expressions and meta-arguments that help in looping for Terraform resources:
1.for_each - a meta-argument used to create multiple instances of a resource or module. As the name implies, the for_each argument takes a map or a set of strings, creating an instance for each item. It provides more flexibility than count by allowing you to use complex data structures and access each instance with a unique identifier.
2.count - a meta-argument allows you to create multiple instances of a resource based on the given count. This is useful for creating similar resources without having to duplicate configuration blocks. For example, if count=5 for an EC2 instance resource configuration, Terraform creates five of those instances in your cloud environment.
3.for - a versatile expression for iterating over and manipulating collections such as lists, sets, and maps. The for expression can be used to iterate over elements in a collection and apply a transformation to each element, optionally filtering elements based on a condition.
For example:
locals {
original_set = {1, 2, 3, 4, 5}
even_set = {for i in local.original_set : i if i % 2 == 0}
}
This creates an even_set containing only the even numbers from original_set.
for vs. for_each vs. count
Here are some key points that differentiate the for expression from the for_each and count meta arguments.
How does for_each work?
Let us take a real-world example to better understand the for_each meta-argument.
Say you have a map of instance configurations where the string value for each key is an identifier for the EC2 instance, and the value is another map containing the instance type and AMI ID.
#main.tf
variable "instances" {
description = "Map of instance configurations"
type = map(object({
ami = string
instance_type = string
}))
default = {
"amzlinux" = {
ami = "ami-02d3fd86e6a2f5122"
instance_type = "t2.micro"
},
"ubuntu" = {
ami = "ami-0ce2cb35386fc22e9"
instance_type = "t2.small"
}
}
}
resource "aws_instance" "servers" {
for_each = var.instances
ami = each.value.ami
instance_type = each.value.instance_type
tags = {
Name = "env0-Server-${each.key}"
}
}
Let’s break everything down:
-
variable "instances"defines a map where each element represents an EC2 instance configuration. For instance, "amzlinux" and "ubuntu" are identifiers for these configurations, each specifying an AMI ID and an instance type. -
resource "aws_instance" "servers"usesfor_eachto iterate over each element in thevar.instancesmap. For each element, it creates an EC2 instance with the specified AMI and instance type. -
each.keyin this context refers to the key in the map (e.g., "amzlinux", "ubuntu"), which we use to uniquely name each instance with theNametag. -
each.value.amiandeach.value.instance_typeaccess the nested values for each instance's configuration.
After successfully running the Terraform workflow (init->plan->apply), we have provisioned these two instances using for_each.
Collections for for_each
Maps
Maps are collections of key-value pairs. When using for_each with maps, each iteration gives you access to both the map key and the value of the current item. Maps are ideal when you need to associate specific attributes or configurations with unique identifiers.
#Example config
variable "instance_tags" {
type = map(string)
default = {
"Role" = "Web-server"
"Environment" = "Production"
}
}
resource "aws_instance" "server_tags" {
for_each = var.instance_tags
# Other Configuration...
tags = {
"${each.key}" = "${each.value}"
}
}
Sets
Sets are collections of unique values. When iterating over a set with for_each, the value for each.key and each.value will be the same since sets do not have key-value pairs but just a list of unique values.
Sets are useful when you need to ensure uniqueness and don't require associated values.
#Example config
variable "availability_zones" {
type = set(string)
default = ["us-west-2a", "us-west-2b"]
}
resource "aws_subnet" "list_subnets" {
for_each = var.availability_zones
availability_zone = each.key
# Other Configuration...
}
Lists
Directly, for_each cannot iterate over lists because lists do not provide a unique key for each item.
However, you can use the tomap() or toset() function to convert a list into a set or a map, allowing for_each to iterate over it.
#Example config
variable "availability_zones" {
type = list(string)
default = ["us-east-1a", "us-east-1b"]
}
resource "aws_subnet" "example" {
for_each = toset(var.availability_zones) #or tomap(var.availability_zones)
availability_zone = each.key
# Other configurations...
}
Practical Use Cases for for_each
1. Resource Chaining
Resource chaining involves creating dependencies between resources where the configuration of one resource depends on the output of another. This is common in infrastructure setups where certain resources must be provisioned sequentially.
Let us take a common scenario of setting up a VPC and deploying subnets within the VPC, to better understand resource chaining.
#variable "networks" {
description = "VPC Network Map"
type = map(object({
cidr = string
}))
default = {
"network1" = {
cidr = "10.0.1.0/24"
},
"network2" = {
cidr = "10.0.2.0/24"
}
}
}
resource "aws_vpc" "main" {
for_each = var.networks
cidr_block = each.value.cidr
tags = {
Name = "VPC-${each.key}"
}
}
resource "aws_subnet" "subnets" {
for_each = var.networks
vpc_id = aws_vpc.main[each.key].id
cidr_block = each.value.cidr
availability_zone = "us-west-2a"
tags = {
Name = "Subnet-${each.key}"
}
}
In this example, each VPC is created based on the networks map, and then a subnet is created within each VPC. The aws_subnet resource uses the ID of the aws_vpc created in the same for_each loop, demonstrating resource chaining.
2. Tagging Resources Dynamically
Dynamic tagging allows you to assign metadata to resources based on their configuration or other dynamic inputs, improving resource management, billing, and automation.
We can take an example of dynamically tagging s3 buckets using for_each:
variable "buckets" {
description = "Map of S3 bucket configurations"
type = map(object({
name = string
tags = map(string)
}))
default = {
"bucket1" = {
name = "env0-app-logs",
tags = {
"Environment" = "Production",
"Application" = "Logging",
}
},
"bucket2" = {
name = "env0-app-data",
tags = {
"Environment" = "Staging",
"Application" = "DataStore",
}
}
}
}
resource "aws_s3_bucket" "env0_app_bucket" {
for_each = var.buckets
bucket = each.value.name
tags = each.value.tags
}
This setup dynamically applies tags to each S3 bucket based on the tags defined in the buckets map.
3. Deploying Resources to Multiple Regions
Deploying resources across multiple regions can enhance disaster recovery and reduce latency. for_each can be used to manage such deployments efficiently.
We can deploy s3 buckets to multiple regions using for_each like this example below.
variable "regions" {
description = "Regions to deploy S3 buckets"
type = map(string)
default = {
"us-east-1" = "env0-app-us-east-1",
"eu-central-1" = "env0-app-eu-central-1"
}
}
provider "aws" {
alias = "useast1"
region = "us-east-1"
}
provider "aws" {
alias = "eucentral1"
region = "eu-central-1"
}
resource "aws_s3_bucket" "multi_region_buckets" {
for_each = var.regions
bucket = each.value
provider = each.key == "us-east-1" ? aws.useast1 : aws.eucentral1
}
In this example, S3 buckets are created in both the us-east-1 and eu-central-1 regions, using separate provider instances for each region. The provider attribute dynamically selects the correct provider based on the region key from the regions map.
Advantages of using for_each
1. Dynamic Resource Management
for_each enables the dynamic creation, management, and destruction of resources based on collections (maps or sets). This dynamic approach allows infrastructure to adjust automatically to changes in the input data without requiring manual updates to the Terraform configuration.
For example, you can write an IaC for infrastructure provisioning to provision a dynamic number of S3 buckets based on a list of project names like the example below:
variable "project_names" {
type = set(string)
default = ["DisasterRecovery", "VPCNetworkSetup"]
}
resource "aws_s3_bucket" "project_bucket" {
for_each = var.project_names
bucket = each.value
}
2. Conditional Resource Creation
Combined with Terraform's conditional expressions, for_each can be used to conditionally create resources based on specific criteria within the data it iterates over. This allows for more granular control over which resources are created, updated, or destroyed.
For instance, you can tailor your IaC in such a way that creates an S3 bucket only if create=true:
variable "bucket_configs" {
type = map(object({
name = string
create = bool
}))
default = {
"bucket1" = { name = "env0-1", create = true },
"bucket2" = { name = "env0-2", create = false }
}
}
resource "aws_s3_bucket" "conditional_bucket" {
for_each = { for k, v in var.bucket_configs : k => v if v.create }
bucket = each.value.name
}
3. Improved Code Reusability
Instead of duplicating resource blocks for each instance of a resource, for_each allows you to define a single resource (or a module) block that can be applied to each item in a collection (like a map or set).
This approach abstracts the common configuration elements into a single, parameterized block, where the specific details for each resource instance are dynamically derived from the collection it iterates over.
For example, you can incorporate for_each with the use of modules to keep your code DRY:
variable "environments" {
type = map(string)
default = {
"dev" = "ami-02d3fd86e6a2f5122",
"prod" = "ami-0ce2cb35386fc22e9",
}
}
module "dry_module_ec2" {
for_each = var.environments
source = "terraform-aws-modules/ec2-instance/aws"
ami = each.value
instance_type = "t2.micro"
}
Commonly Asked Questions/FAQs
Q. Can I use Terraform for_each and count for the same resource?
No, for_each and count cannot be used together within the same resource or module block. You must choose one based on your use case.
count is used to create a specified number of identical resources. for_each iterates over items in a map or set to create multiple resources with different configurations.
Q. What is the use of count.index in Terraform?
The count.index in Terraform is a built-in variable that starts with a current zero-based index of the resource created in a block where the count meta-argument is used.
It's primarily used when you're creating multiple instances of a resource with count and need to differentiate between these individual instances, with a numeric identifier.
Q. Can for_each be used with modules?
Yes, for_each can be used with Terraform modules, enabling you to create multiple instances of a module based on the items in a map or a set. When using for_each with a module, you can define a collection (map or set) containing the values you want to iterate over.
Q. Can count be used to conditionally create a resource?
Yes, count can be used to create a resource in Terraform conditionally. You can specify whether to create a resource based on a condition by leveraging the count meta-argument. For example, if the condition evaluates to true, create a single instance of the resource, and prevent the resource creation if it evaluates to false.
Q. Is it possible to migrate from count to for_each?
Yes, it is possible to migrate from count to for_each in Terraform, but the process requires careful planning and execution. You can check here for detailed information on migrating count to for_each.
Top comments (2)
Wondering if the for_each could be used to bulk import in Terraform.
Sure