DEV Community: Adarsh Gupta

AWS Terraform IAM User Management

Adarsh Gupta — Mon, 15 Dec 2025 06:43:06 +0000

Managing IAM users manually in AWS can quickly become complex, error-prone, and difficult to scale. As teams grow, you need a repeatable, auditable, and secure way to manage users, groups, permissions, and security controls like MFA.

In this blog, we’ll explore how to implement AWS IAM user management using Terraform, driven by a CSV file as the single source of truth.

Why Use Terraform for IAM User Management?

Using Terraform for IAM offers several advantages:

Centralized and version-controlled user management
Easy onboarding and offboarding of users
Consistent security policies across teams
Reduced manual errors
Idempotent and auditable changes

Instead of clicking through the AWS Console, we define everything declaratively.

Architecture Overview

This setup includes:

IAM users created dynamically from a CSV file
IAM groups for Education, Managers, and Engineers
Automatic group membership based on user attributes
Console access with forced password reset
MFA enforcement across all groups
Group-based permission management

The CSV file becomes the single source of truth for identity data.

Reading User Data from CSV

We begin by loading user data from a CSV file.

locals {
  users = csvdecode(file("users.csv")) // It gives list of maps for the data
}

Terraform converts each row into a map, allowing us to loop over users dynamically. This makes adding or removing users as simple as editing the CSV.

Creating IAM Users Dynamically

IAM users are created using for_each, ensuring scalability and consistency.

resource "aws_iam_user" "users" {
  for_each = { for user in local.users : user.first_name => user }

  name = lower("${substr(each.value.first_name, 0, 1)}${each.value.last_name}")
  path = "/users/"

  tags = {
    DisplayName = "${each.value.first_name} ${each.value.last_name}"
    Department  = each.value.department
    JobTitle    = each.value.job_title
    Email       = each.value.email
    Phone       = each.value.phone
  }
}

Key points:

Usernames are generated automatically (first initial + last name)
Tags store rich metadata for filtering, auditing, and policies
No hardcoding of users

Enabling Console Access Securely

To allow AWS Console access, we create login profiles.

resource "aws_iam_user_login_profile" "users" {
  for_each                = aws_iam_user.users
  user                    = each.value.name
  password_reset_required = true

  lifecycle {
    ignore_changes = [
      password_reset_required,
      password_length
    ]
  }
}

This enforces a password reset on first login, aligning with security best practices.

Creating IAM Groups

Groups are used to manage permissions collectively.

resource "aws_iam_group" "education" {
  name = "Education"
  path = "/groups/"
}

resource "aws_iam_group" "managers" {
  name = "Managers"
  path = "/groups/"
}

resource "aws_iam_group" "engineers" {
  name = "Engineers"
  path = "/groups/"
}

This allows permissions to be managed at the group level instead of per user.

Dynamic Group Membership Assignment

Users are automatically assigned to groups based on their attributes.

Education Group

resource "aws_iam_group_membership" "education_members" {
  name  = "education-group-membership"
  group = aws_iam_group.education.name

  users = [
    for user in aws_iam_user.users : user.name
    if user.tags.Department == "Education"
  ]
}

Managers Group (based on job title)

resource "aws_iam_group_membership" "managers_members" {
  name  = "managers-group-membership"
  group = aws_iam_group.managers.name

  users = [
    for user in aws_iam_user.users : user.name
    if contains(keys(user.tags), "JobTitle") &&
       can(regex("Manager|CEO", user.tags.JobTitle))
  ]
}

Engineers Group

resource "aws_iam_group_membership" "engineers_members" {
  name  = "engineers-group-membership"
  group = aws_iam_group.engineers.name

  users = [
    for user in aws_iam_user.users : user.name
    if user.tags.Department == "Engineering"
  ]
}

This approach eliminates manual group assignment and ensures accuracy.

Enforcing MFA for All Users

Terraform cannot create MFA devices, but it can enforce MFA usage through IAM policies.

resource "aws_iam_policy" "require_mfa" {
  name        = "Require-MFA"
  description = "Deny access unless MFA is enabled"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid      = "DenyAllExceptMFA"
        Effect   = "Deny"
        Action   = "*"
        Resource = "*"
        Condition = {
          BoolIfExists = {
            "aws:MultiFactorAuthPresent" = "false"
          }
        }
      }
    ]
  })
}

This policy denies all AWS actions unless MFA is enabled.

Attaching MFA Policy to Groups

resource "aws_iam_group_policy_attachment" "mfa_enforcement" {
  for_each = {
    education = aws_iam_group.education.name
    managers  = aws_iam_group.managers.name
    engineers = aws_iam_group.engineers.name
  }

  group      = each.value
  policy_arn = aws_iam_policy.require_mfa.arn
}

Once attached, all users in these groups must enable MFA to access AWS.

Attaching Permissions to Groups

Permissions are granted using AWS managed policies.

resource "aws_iam_group_policy_attachment" "education_readonly" {
  group      = aws_iam_group.education.name
  policy_arn = "arn:aws:iam::aws:policy/ReadOnlyAccess"
}

resource "aws_iam_group_policy_attachment" "managers_admin" {
  group      = aws_iam_group.managers.name
  policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess"
}

resource "aws_iam_group_policy_attachment" "engineers_poweruser" {
  group      = aws_iam_group.engineers.name
  policy_arn = "arn:aws:iam::aws:policy/PowerUserAccess"
}

This ensures:

Education users have read-only access
Managers have full administrative access
Engineers can manage resources but not IAM

AWS Account Verification

data "aws_caller_identity" "current" {}

This is useful for outputs, debugging, and ensuring Terraform is operating in the correct AWS account.

Security and Best Practices

MFA enforced across all users
Permissions applied via groups, not users
Metadata stored as tags for auditing
CSV-driven user lifecycle management
Infrastructure fully reproducible and auditable

For production environments, AWS IAM Identity Center (SSO) is recommended instead of IAM users.

Conclusion

This Terraform-based IAM user management solution demonstrates how identity can be treated as code, not configuration. By combining CSV-driven data, dynamic group membership, MFA enforcement, and least-privilege access, you get a scalable and secure IAM architecture suitable for real-world environments.

This approach not only simplifies administration but also aligns closely with modern DevOps and security best practices.

Embedded Video Tutorial

@piyushsachdeva

AWS VPC Peering Using Terraform

Adarsh Gupta — Sun, 14 Dec 2025 15:49:10 +0000

Introduction

As cloud architectures grow, it becomes common to split workloads across multiple Virtual Private Clouds (VPCs). These VPCs may exist in different AWS regions for reasons such as fault tolerance, regulatory compliance, or performance optimization. To enable secure communication between such isolated networks, AWS provides VPC Peering.

This blog walks through a complete Terraform-based implementation of cross-region VPC peering, explaining each resource in detail and showing how two VPCs can communicate using private IP addresses without relying on the public internet.

What Is VPC Peering?

VPC peering is a networking connection between two VPCs that allows traffic to be routed privately using AWS’s internal network. Once peered, resources in both VPCs can communicate as if they were part of the same network, provided routing and security rules are configured correctly.

Key characteristics:

Traffic stays on the AWS backbone
No internet gateway, VPN, or NAT is required for inter-VPC communication
CIDR blocks must not overlap
Peering is non-transitive

Architecture Overview

This implementation creates:

Two VPCs in different AWS regions (Primary and Secondary)
One public subnet in each VPC
Internet gateways for outbound connectivity
Custom route tables with peering routes
A cross-region VPC peering connection
Security groups allowing controlled inter-VPC traffic
One EC2 instance in each VPC for testing connectivity

Provider and Region Strategy

Terraform does not support dynamic regions inside a single provider block. To solve this, provider aliases are used.

aws.primary manages resources in the primary region
aws.secondary manages resources in the secondary region

This approach enables clean multi-region deployments within a single Terraform configuration.

Creating the VPCs

Two VPCs are created with DNS support and hostnames enabled.

resource "aws_vpc" "primary" {
  cidr_block           = var.primary_vpc_cidr
  provider             = aws.primary
  enable_dns_support   = true
  enable_dns_hostnames = true

  tags = {
    Name = "Primary-VPC-${var.primary}"
  }
}

DNS support is critical for resolving private hostnames across peered VPCs, especially when services rely on DNS instead of hardcoded IPs.

The secondary VPC follows the same structure but uses the secondary provider and CIDR block.

Subnet Configuration

Each VPC contains one public subnet:

resource "aws_subnet" "primary_subnet" {
  provider                = aws.primary
  vpc_id                  = aws_vpc.primary.id
  cidr_block              = var.primary_vpc_cidr
  availability_zone       = data.aws_availability_zones.primary.names[0]
  map_public_ip_on_launch = true
}

The subnet is configured to automatically assign public IPs, enabling direct SSH access to EC2 instances for testing purposes.

Internet Gateways

An Internet Gateway is attached to each VPC to allow outbound internet access:

resource "aws_internet_gateway" "primary_igw" {
  provider = aws.primary
  vpc_id   = aws_vpc.primary.id
}

This is necessary for:

Software installation
SSH access
Package updates on EC2 instances

Route Tables and Associations

Custom route tables are created to control traffic flow.

resource "aws_route_table" "primary_rt" {
  provider = aws.primary
  vpc_id   = aws_vpc.primary.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.primary_igw.id
  }
}

Each subnet is explicitly associated with its route table:

resource "aws_route_table_association" "asso_primary" {
  provider       = aws.primary
  subnet_id      = aws_subnet.primary_subnet.id
  route_table_id = aws_route_table.primary_rt.id
}

Associating route tables at the subnet level ensures consistent routing behavior for all instances within that subnet.

VPC Peering Connection

The peering connection is initiated from the primary VPC:

resource "aws_vpc_peering_connection" "primary_to_secondary" {
  provider    = aws.primary
  vpc_id      = aws_vpc.primary.id
  peer_vpc_id = aws_vpc.secondary.id
  peer_region = var.secondary
  auto_accept = false
}

Since this is a cross-region peering, the connection must be explicitly accepted by the secondary VPC.

resource "aws_vpc_peering_connection_accepter" "secondary_accepter" {
  provider                  = aws.secondary
  vpc_peering_connection_id = aws_vpc_peering_connection.primary_to_secondary.id
  auto_accept               = true
}

This two-step process reflects AWS’s security model for cross-region peering.

Routing for VPC Peering

Peering alone does not enable traffic flow. Routes must be added in both VPCs.

Primary route table:

resource "aws_route" "primary_to_secondary" {
  provider                  = aws.primary
  route_table_id            = aws_route_table.primary_rt.id
  destination_cidr_block    = var.secondary_vpc_cidr
  vpc_peering_connection_id = aws_vpc_peering_connection.primary_to_secondary.id
}

Secondary route table:

resource "aws_route" "secondary_to_primary" {
  provider                  = aws.secondary
  route_table_id            = aws_route_table.secondary_rt.id
  destination_cidr_block    = var.primary_vpc_cidr
  vpc_peering_connection_id = aws_vpc_peering_connection.primary_to_secondary.id
}

These routes instruct AWS to forward inter-VPC traffic through the peering connection.

Security Groups

Security groups enforce network-level access control.

Primary VPC security group allows:

SSH from anywhere
ICMP from the secondary VPC
All TCP traffic from the secondary VPC

ingress {
  from_port   = 0
  to_port     = 65535
  protocol    = "tcp"
  cidr_blocks = [var.secondary_vpc_cidr]
}

The secondary security group mirrors this configuration, allowing symmetric communication.

EC2 Instances for Validation

One EC2 instance is launched in each VPC:

resource "aws_instance" "primary_instance" {
  provider      = aws.primary
  subnet_id     = aws_subnet.primary_subnet.id
  instance_type = var.instance_type
}

User data scripts install a web server and expose simple HTTP endpoints, making it easy to validate connectivity using curl or ping.

Traffic Flow Summary

EC2 instance sends traffic to the destination private IP
Subnet-associated route table matches the peered CIDR
Traffic is routed through the VPC peering connection
Security groups allow the request
Response returns via the same private path

No traffic traverses the public internet during inter-VPC communication.

Key Takeaways

VPC peering provides secure, low-latency private connectivity
Route tables are mandatory for peering traffic
Security groups must explicitly allow cross-VPC CIDRs
Provider aliases enable clean multi-region Terraform deployments
DNS support is essential for scalable architectures

Conclusion

This Terraform implementation demonstrates a production-grade approach to AWS VPC Peering across regions. By combining proper routing, security controls, and provider aliasing, you can build scalable, secure multi-region architectures without introducing unnecessary complexity.

This setup forms a strong foundation for extending into private subnets, NAT gateways, monitoring, and more advanced networking patterns such as Transit Gateway.

Embedded Video Tutorial

@piyushsachdeva

Host A Static Website in AWS S3 and CloudFront (Using Terraform)

Adarsh Gupta — Wed, 10 Dec 2025 15:21:59 +0000

Hosting a static website is simple in theory, but hosting it in a way that is fast, secure, and globally accessible requires the right architecture. AWS provides two services that work extremely well together for this purpose: Amazon S3 for storage and Amazon CloudFront for global delivery.

In this blog, I want to walk you through why S3 + CloudFront is such a powerful combination and how I implemented the entire solution in Terraform.

Why Use S3 and CloudFront Together?

1. S3 as the Website Storage

Amazon S3 is a great place to store static content like HTML, CSS, JavaScript, and images. It's reliable, scalable, and inexpensive. However, S3 alone is not optimized for global content delivery. If your bucket is in Mumbai and a user in Europe tries to access your website, the latency will be noticeable.

2. CloudFront for Global Performance

CloudFront solves this problem by delivering your content through a worldwide network of edge locations. When a user accesses your site, CloudFront serves the content from the nearest location. This reduces delays, speeds up page load time, and improves the overall user experience, no matter where the user is located.

3. Security Through OAC

Instead of making your S3 bucket public, CloudFront can access the bucket on behalf of the users through an Origin Access Control (OAC). This ensures that your S3 bucket remains private while still allowing your website to be accessible globally. It’s a secure and modern approach.

4. HTTPS Support

CloudFront also enables HTTPS by default using an AWS-managed certificate. Even if you don’t have a custom domain, CloudFront provides a secure .cloudfront.net URL for your website.

Terraform Implementation

Below is the exact Terraform code I used to deploy the entire setup. It covers S3 bucket creation, file upload, OAC configuration, bucket policies, and CloudFront distribution.

// Create an S3 bucket with a dynamic name using prefix and name variables
resource "aws_s3_bucket" "website" {
  bucket = "${var.bucket_prefix}${var.bucket_name}"
}

// Block public access to the S3 bucket
resource "aws_s3_bucket_public_access_block" "website_public_access_block" {
  bucket = aws_s3_bucket.website.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

// Create a CloudFront Origin Access Control for the S3 bucket
resource "aws_cloudfront_origin_access_control" "oac" {
  name                              = "${var.bucket_prefix}-oac"
  description                       = "OAC for ${var.bucket_name}"
  origin_access_control_origin_type = "s3"
  signing_behavior                  = "always"
  signing_protocol                  = "sigv4"
}

// S3 Bucket Policy to allow access from cloudfront origin access control
resource "aws_s3_bucket_policy" "website_bucket_policy" {
  bucket = aws_s3_bucket.website.id
  depends_on = [aws_s3_bucket_public_access_block.website_public_access_block]
  policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Sid" : "AllowCloudFrontAccess",
        "Effect" : "Allow",
        "Principal" : {
          "Service" : "cloudfront.amazonaws.com"
        },
        "Action" : [
          "s3:GetObject"
        ],
        "Resource" : "${aws_s3_bucket.website.arn}/*"
        Condition = {
          StringEquals = {
            "AWS:SourceArn" = aws_cloudfront_distribution.s3_distribution.arn
          }
        }
      }
    ]
  })
}

// upload files to the S3 bucket
resource "aws_s3_object" "website_files" {
  for_each = fileset("${path.module}/www", "**/*")
  bucket   = aws_s3_bucket.website.id
  key      = each.value
  source   = "${path.module}/www/${each.value}"
  etag     = filemd5("${path.module}/www/${each.value}")
  content_type = lookup({
    "html" = "text/html",
    "css"  = "text/css",
    "js"   = "application/javascript",
    "json" = "application/json",
    "png"  = "image/png",
    "jpg"  = "image/jpeg",
    "jpeg" = "image/jpeg",
    "gif"  = "image/gif",
    "svg"  = "image/svg+xml",
    "ico"  = "image/x-icon",
    "txt"  = "text/plain"
  }, split(".", each.value)[length(split(".", each.value)) - 1], "application/octet-stream")
}

// Create a CloudFront Distribution to serve content from the S3 bucket
resource "aws_cloudfront_distribution" "s3_distribution" {
  origin {
    domain_name              = aws_s3_bucket.website.bucket_regional_domain_name
    origin_id                = local.origin_id
    origin_access_control_id = aws_cloudfront_origin_access_control.oac.id
  }

  enabled             = true
  is_ipv6_enabled     = true
  default_root_object = "index.html"

  default_cache_behavior {
    allowed_methods  = ["GET", "HEAD"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = local.origin_id

    forwarded_values {
      query_string = false
      cookies {
        forward = "none"
      }
    }

    viewer_protocol_policy = "redirect-to-https"
    min_ttl                = 0
    default_ttl            = 3600
    max_ttl                = 86400
  }

  price_class = "PriceClass_100"
  restrictions {
    geo_restriction {
      restriction_type = "none"
    }
  }
  viewer_certificate {
    cloudfront_default_certificate = true
  }
}

Final Output

After running the usual Terraform commands—init, validate, plan, and apply—you get a CloudFront URL that securely serves your static website. The S3 bucket stays completely private while the content is delivered quickly and securely across the world.
This setup gives you a production-ready hosting environment that is fast, secure, and automated.

Reference Video

@piyushsachdeva

Terraform Data Source (AWS)

Adarsh Gupta — Tue, 09 Dec 2025 13:51:14 +0000

When working with Terraform, one of the most powerful concepts you’ll use is data sources. These allow you to fetch and reference existing resources rather than creating new ones. In real-world cloud environments—especially in large organizations—you often work with infrastructure that already exists, such as shared VPCs, pre-defined subnets, approved AMIs, or centrally managed security groups.

Instead of hardcoding these values or manually copying IDs, Terraform’s data sources give you a clean, dynamic, and error-free way to retrieve them.

This blog explains what Terraform data sources are, why they matter, and how to use them with practical AWS examples, including VPC, Subnet, and AMI lookups.

What Are Terraform Data Sources?

A data source in Terraform is a read-only lookup to an existing resource. Instead of creating something new, Terraform queries the cloud provider (AWS in this case) and returns information that can be used inside your configuration.

You use data sources when:

A resource is already created (shared VPCs, existing AMIs).
Another team manages the resource (network or security team).
Your Terraform module should not own or recreate the resource.
You need the latest or filtered version of something (latest AMI).
You want to avoid hardcoding identifiers such as IDs or ARNs.

This leads to cleaner, more dynamic infrastructure code.

Example 1: Fetching VPC ID Using a Data Source

In many organizations, networking is centralized. The VPC already exists, and your Terraform code will only deploy application resources inside it.

With the following data source, we fetch a VPC by matching its Name tag:

data "aws_vpc" "vpc_name" {
  filter {
    name   = "tag:Name"
    values = ["default-vpc"]
  }
}

Here’s what this does:

Searches for a VPC where the tag Name = default-vpc
Returns the VPC’s ID
Allows you to use the ID later using data.aws_vpc.vpc_name.id

This avoids the need to manually capture or maintain the VPC ID.

Example 2: Fetching Subnet ID from a Specific VPC

Once the VPC is retrieved, you often need a subnet inside it.
This subnet might also be managed by another team, or it may vary by environment.

data "aws_subnet" "shared_subnet" {
  filter {
    name   = "tag:Name"
    values = ["subnet-a"]
  }
  vpc_id = data.aws_vpc.vpc_name.id
}

Important details:

It fetches a subnet with Name = subnet-a
It ensures the subnet belongs to the VPC we fetched earlier
It returns a single subnet ID

This helps Terraform deploy EC2 or Lambda resources into the correct shared subnet without hardcoding anything.

Example 3: Fetching the Latest Amazon Linux 2 AMI

AMI IDs change frequently across regions, and using outdated or hardcoded AMIs leads to deployment failures.
With a data source, Terraform automatically selects the most recent approved AMI:

data "aws_ami" "linux2" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  filter {
    name   = "architecture"
    values = ["x86_64"]
  }
}

This configuration ensures:

You get the latest Amazon Linux 2 AMI
Only official images from the Amazon account are selected
The AMI matches the required architecture and virtualization type

This is a perfect example of where data sources solve a real problem—keeping images up to date.

Using the Data Sources to Launch an EC2 Instance

Once the VPC, Subnet, and AMI are fetched, we can provision an EC2 instance using those dynamic values:

resource "aws_instance" "ec2-one" {
  ami           = data.aws_ami.linux2.id
  instance_type = var.instance_type
  subnet_id     = data.aws_subnet.shared_subnet.id
  tags          = var.tags
}

This resource:

Uses the AMI from the data source
Places the EC2 inside the shared subnet
Applies the user-provided instance type and tags

The result is a reusable, environment-independent, and future-proof Terraform configuration.

Why Data Sources Matter

1. Avoids Hardcoding

No need to store IDs, ARNs, AMIs manually.

2. Enables Multi-Team, Multi-Account Use

Teams can reference central resources without needing permissions to modify them.

3. Improves Reusability

Modules become generic and work across dev, test, and prod seamlessly.

4. Supports Dynamic and Automated Infrastructure

Fetching latest AMIs ensures security and consistency.

5. Reduces Human Error

Manual copy-paste of IDs is error-prone; data sources eliminate this.

Conclusion

Terraform data sources are essential for building dynamic, secure, and production-ready infrastructure. They allow your code to interact with existing resources in AWS—like VPCs, subnets, AMIs, and more—without recreating them. The examples above represent real-world scenarios where infrastructure teams rely heavily on these patterns, especially in shared network environments.

By using data sources effectively, your Terraform setup becomes more scalable, maintainable, and aligned with best DevOps practices.

Reference Video

@piyushsachdeva

AWS Terraform Functions – Part 2

Adarsh Gupta — Mon, 08 Dec 2025 16:24:06 +0000

In this Part-2 guide on AWS Terraform Functions, we explore a set of practical and commonly used Terraform functions applied in real automation scenarios. These examples are based on a working Terraform configuration that demonstrates validation, file handling, list operations, cost calculation, and timestamp formatting.

The goal of this blog is to help you understand not just the functions, but also how they fit naturally inside real Terraform workflows.

1. Instance Validation Using Terraform Functions

Validating inputs at the variable level helps prevent mistakes before infrastructure is created. Terraform provides several helpful validation functions such as length(), can(), and regex().

Below is a real example that validates an EC2 instance type:

Validate instance type length

variable "instance_type" {
  default = "t2.micro"

  validation {
    condition     = length(var.instance_type) >= 2 && length(var.instance_type) <= 20
    error_message = "Instance type must be 2 and 20 characters."
  }

Validate instance type pattern using regex

  validation {
    condition     = can(regex("^t[2-3]\\.", var.instance_type))
    error_message = "Instance type must start with t2. or t3."
  }
}

This ensures that the instance type is not only of valid length but also begins with either t2. or t3..

2. Backup Configuration and Sensitive Handling

Backup naming conventions and secure values are important for safe automation. Terraform offers functions like endswith() and the sensitive attribute.

Ensuring backup name format

variable "backup_name" {
  default = "daily_backup"

  validation {
    condition     = endswith(var.backup_name, "_backup")
    error_message = "Backup name must end with '_backup'"
  }
}

Marking credentials as sensitive

variable "credentials" {
  default   = "xyzabc123"
  sensitive = true
}

Terraform will hide the value anywhere it appears in logs or CLI output, improving security.

3. Location Management with Lists and Sets

Managing user locations, default locations, and ensuring uniqueness is a common scenario in multi-region AWS deployments. Terraform simplifies this using concat() and toset().

Merging and deduplicating locations

variable "user_locations" {
  default = ["us-west-1", "us-east-1", "eu-central-1", "ap-south-1"]
}

variable "default_location" {
  default = ["ap-south-1"]
}

locals {
  all_locations     = concat(var.user_locations, var.default_location)
  unique_locations  = toset(local.all_locations)
}

Here, concat() merges the lists, and toset() removes duplicates automatically.

4. Cost Calculation and Value Processing

Terraform’s numeric functions make it easy to compute totals, calculate averages, or normalize values.

Example cost dataset

variable "monthly_costs" {
  default = [-50, 100, 150, 200, 250] # negative value indicates refunds
}

Processing the costs

locals {
  positive_cost = [for cost in var.monthly_costs : abs(cost)]
  max_cost      = max(local.positive_cost...)
  min_cost      = min(local.positive_cost...)
  total_cost    = sum(local.positive_cost)
  avg_cost      = local.total_cost / length(local.positive_cost)
}

This configuration uses:

abs() to convert negative values into positive
max() and min() to identify the cost range
sum() to compute total expenditure
A manual calculation for average cost

5. Timestamp and Date Formatting

Terraform provides built-in timestamp utilities that are extremely helpful for naming, versioning, or logging.

Generating formatted timestamps

locals {
  current_timestamp = timestamp()
  format1           = formatdate("DD-MM-YYYY", local.current_timestamp)
  format2           = formatdate("YYYY/MM/DD HH:MM:SS", local.current_timestamp)
  timestamp_name    = "backup-${local.format1}"
}

This enables time-based resource naming such as:

backup-05-12-2025

6. File Path Processing and JSON Handling

Terraform can work directly with files on disk. This is particularly useful for loading configuration files or template data.

Extracting directory name and checking if a file exists

locals {
  dir_name            = dirname("./config.json")
  config_file_exists  = fileexists("${local.dir_name}/config.json")
}

Conditionally loading JSON file content

locals {
  config_file_data = local.config_file_exists ? jsondecode(file("./config.json")) : null
}

Here, the workflow is:

Extract the directory name using dirname()
Check if the configuration file exists using fileexists()
If it exists, load the content using file()
Convert JSON to an object using jsondecode()

This approach is extremely useful in modular Terraform setups where external JSON files drive configuration.

Conclusion

The examples in this blog highlight how Terraform's built-in functions make configurations smarter, safer, and more flexible. Whether you're validating user input, processing files, managing regions, computing costs, or formatting timestamps, these functions help you build reliable and automated AWS infrastructure.

Understanding these helper functions is essential for writing reusable and production-grade Terraform modules.

Video Walkthrough – Terraform Functions Part 2

@piyushsachdeva

Terraform Functions Part-1

Adarsh Gupta — Sat, 06 Dec 2025 14:51:45 +0000

Terraform is more than just an infrastructure provisioning tool—it is a powerful configuration language that helps you automate cloud deployments efficiently. One of the most valuable components of Terraform is its rich set of built-in functions, which allow you to manipulate data, transform inputs, and create clean, dynamic configurations.

While Terraform does not support custom functions, its inbuilt functions are more than enough to handle real-world infrastructure requirements.

You can open the console using:

terraform console

1. String Functions

String functions are extremely useful for naming resources, creating tags, formatting paths, or transforming any text values.

Examples

upper("string")
# Output: "STRING"

lower("STRING")
# Output: "string"

trim("  adarsh  ", " ")
# Output: "adarsh"

replace("hello adarsh", " ", "-")
# Output: "hello-adarsh"

substr("adarsh", 0, 3)
# Output: "ada"

2. Numeric Functions

Numeric functions are useful when calculating dynamic sizes, thresholds, limits, cost-based decisions, or autoscaling configurations.

Examples

max(34, 2, 24)
# Output: 34

min(34, 2, 24)
# Output: 2

abs(-45)
# Output: 45

3. Collection Functions

Collections (lists, sets, maps) are foundational in Terraform. These functions allow you to transform, combine, and organize them efficiently.

length

length([1,3,4])
# Output: 3

concat

concat([1,2], [56,45])
# Output: [1,2,56,45]

merge

merge({a=1}, {b=24})
# Output: { "a" = 1, "b" = 24 }

split — convert string to list

split(",", "a,b,c")
# Output: ["a", "b", "c"]

join — convert list back to string

join("-", ["hello", "adarsh"])
# Output: "hello-adarsh"

split + join is especially useful when parsing CSV values, dynamic tags, or environment names.

4. Lookup Function

The lookup function provides a safe way to fetch values from a map. It prevents errors in case the key doesn’t exist and allows you to specify a default.

Example

lookup({ env = "prod", region = "ap-south-1" }, "region", "default-region")
# Output: "ap-south-1"

If the key is not found, Terraform returns the fallback value you provide.

5. Type Conversion Functions

Terraform often requires converting values between data types—especially when reading variables, external inputs, or combining different modules.

Examples

toset(["a","b","a"])
# Output: toset(["a","b"])

tonumber("23")
# Output: 23

tostring(34)
# Output: "34"

6. Date & Time Functions

These functions help you generate timestamps and format date strings, often used in tagging, versioning, and resource naming.

Examples

timestamp()
# Output: "2025-12-06T13:25:23Z"

formatdate("DD-MM-YY", timestamp())
# Output: "06-12-25"

Useful CLI Commands When Working With Functions

A few Terraform commands help you experiment and keep your workspace up to date:

terraform refresh
Updates the state file to reflect actual resource details.
terraform init -upgrade
Downloads the latest compatible provider versions.

Both commands are helpful when testing changes or adding new functions to your configuration.

Conclusion

Terraform functions are essential tools that help you write clean, efficient, and highly dynamic infrastructure code. Whether it’s manipulating strings, performing calculations, splitting lists, joining text, or safely looking up values, these functions make your Terraform configurations far more flexible and reusable.

Mastering these foundational functions will significantly improve your ability to work with complex Terraform projects and create production-ready infrastructure automation.

Reference Video — Terraform Functions Explained

@piyushsachdeva

Terraform Expressions — Conditional, Dynamic & Splat Expressions

Adarsh Gupta — Wed, 03 Dec 2025 18:09:18 +0000

Introduction

Terraform provides a powerful configuration language that allows you to build reusable, scalable, and environment-aware infrastructure. Three of the most useful features within Terraform’s expression system are conditional expressions, dynamic blocks, and splat expressions.

Understanding these three concepts helps you write cleaner, more flexible code while avoiding duplication and complexity. This blog explains each expression in detail, including when to use them, when not to use them, and how they work together.

Conditional Expressions

What They Are

Conditional expressions return one of two values depending on whether a given condition is true or false.

Syntax

condition ? true_value : false_value

How They Work

If the condition is true, the first value is returned.
If the condition is false, the second value is returned.
Similar to ternary operators in traditional programming languages.

Use Cases

Switching instance types based on environment
Enabling or disabling optional features
Selecting AMIs per region
Setting resource counts dynamically
Applying environment-specific tags

Benefits

Reduces code duplication
Makes environment differences explicit
Allows a single configuration to serve multiple use cases

When to Use

Feature toggles
Environment-based settings
Conditional resource attributes

When Not to Use

When logic becomes too complex
When separate configuration files would be clearer
When the same behavior is required in all environments

Dynamic Blocks

What They Are

Dynamic blocks allow you to programmatically generate nested blocks inside Terraform resources. They are useful when a resource needs multiple similar nested configurations.

Syntax

dynamic "block_name" {
  for_each = var.collection
  content {
    # content for each generated block
  }
}

How They Work

for_each loops over a list or map
Terraform creates one nested block for each item
content holds the fields of the nested block

Use Cases

Multiple ingress/egress rules in security groups
Multiple EBS volumes
Multiple IAM policy statements
Listener rules in load balancers
Route table routes

Benefits

Eliminates repetitive blocks
Driven by variables for better flexibility
Produces cleaner, maintainable configurations

When to Use

When nested blocks repeat
When the number of blocks should be dynamic
When variables drive configuration

When Not to Use

When only one or two static blocks are needed
When readability decreases
For top-level resources

Best Practices

Use descriptive iterator names
Avoid deep nesting
Document expected data structures

Splat Expressions

What They Are

Splat expressions extract a specific attribute from every element in a list using the [*] operator.

Syntax

resource_list[*].attribute_name

How They Work

Works on lists of objects or resources
Returns a list containing only the selected attribute

Use Cases

Collecting instance IDs
Gathering private or public IPs
Extracting security group IDs
Producing lists of subnets or ARNs
Simplifying outputs from multiple instances

Benefits

Very concise
Easy extraction of repeated attributes
Ideal for resources created using count

Limitations

Works only with lists or sets
For maps, a different expression is needed
Empty lists produce empty outputs—must be handled upstream

Example Using All Three Expressions

variable "environment" {
  type    = string
  default = "dev"
}

variable "ingress_rules" {
  type = list(object({
    from_port   = number
    to_port     = number
    protocol    = string
    cidr_blocks = list(string)
  }))

  default = [
    { from_port = 80,  to_port = 80,  protocol = "tcp", cidr_blocks = ["0.0.0.0/0"] },
    { from_port = 443, to_port = 443, protocol = "tcp", cidr_blocks = ["0.0.0.0/0"] }
  ]
}

resource "aws_security_group" "web_sg" {
  name = "web-sg-${var.environment}"

  dynamic "ingress" {
    for_each = var.ingress_rules
    content {
      from_port   = ingress.value.from_port
      to_port     = ingress.value.to_port
      protocol    = ingress.value.protocol
      cidr_blocks = ingress.value.cidr_blocks
    }
  }

  description = var.environment == "prod" ? "Production SG" : "Development SG"
}

resource "aws_instance" "web" {
  count         = var.environment == "prod" ? 3 : 1
  ami           = "ami-xxxxxxxx"
  instance_type = var.environment == "prod" ? "t3.medium" : "t3.micro"

  vpc_security_group_ids = [aws_security_group.web_sg.id]
}

output "instance_ids" {
  value = aws_instance.web[*].id
}

Conclusion

Conditional expressions, dynamic blocks, and splat expressions are foundational tools for writing flexible, modular, and reusable Terraform code. Correct use of these features helps you:

Reduce redundancy
Manage infrastructure across multiple environments
Maintain cleaner, more scalable configurations

Mastering these expressions greatly enhances your Terraform workflow and prepares you for larger, more complex infrastructure projects.

Embedded Video

@piyushsachdeva

AWS Terraform Lifecycle Rules

Adarsh Gupta — Tue, 02 Dec 2025 15:33:50 +0000

Infrastructure as Code (IaC) is most powerful when you have full control over how resources behave during updates, replacements, and deletions. Terraform provides a set of lifecycle meta-arguments that help you fine-tune this behavior, avoid unexpected outages, and enforce organizational policies. These lifecycle rules are essential when working with production-grade AWS environments where stability, predictability, and compliance matter.

This article explains each Terraform lifecycle rule in detail, how they work, practical use cases, and why they are critical in real-world AWS deployments.

1. create_before_destroy

This rule ensures that a replacement resource is created before the existing one is destroyed. This is especially important when you want zero downtime.

Why it matters

By default, Terraform first destroys the old resource and then creates the new one. This can cause a service interruption if the resource is critical, such as an EC2 instance serving live traffic. With create_before_destroy, Terraform safely provisions the new instance first, shifts dependencies, and then removes the old one.

Use Cases

EC2 instances behind a load balancer
RDS resources requiring uninterrupted availability
Blue-green deployment patterns
Infrastructure where gaps are unacceptable

Example

lifecycle {
  create_before_destroy = true
}

2. prevent_destroy

This rule prevents Terraform from destroying a resource entirely. If any operation attempts a destroy, Terraform throws an error.

Why it matters

Accidental deletions can cause irreversible damage. Databases, S3 buckets with sensitive logs, and stateful systems should never be deleted unintentionally. This rule adds a safety net, forcing manual intervention before critical resources can be removed.

Use Cases

Production databases
Critical S3 buckets
IAM roles used in multiple systems
Any resource containing sensitive or compliance-required data

Example

lifecycle {
  prevent_destroy = true
}

3. ignore_changes

Terraform normally tries to bring resources back to the desired state written in code. Sometimes external systems intentionally modify attributes (for example, AWS Auto Scaling Groups). ignore_changes tells Terraform not to track or revert specific attributes.

Why it matters

This prevents unnecessary churn and avoids Terraform fighting with AWS-managed services or other teams modifying resources outside Terraform.

Use Cases

Desired capacity of Auto Scaling Groups
Tags added automatically by monitoring tools
Security groups managed across multiple teams
Values controlled by AWS services

Example

lifecycle {
  ignore_changes = [desired_capacity]
}

4. replace_triggered_by

This rule forces a resource to be recreated whenever another resource changes. It is useful in cases where a dependent change affects the functionality of another component.

Why it matters

Even if an EC2 instance configuration hasn’t changed, a change to its security group might require a replacement for security or compliance reasons. This rule ensures freshness and consistency.

Use Cases

Replace EC2 instances when security groups change
Recreate resources based on configuration updates
Enforce immutable infrastructure practices

Example

lifecycle {
  replace_triggered_by = [aws_security_group.app_sg.id]
}

5. precondition

This rule validates certain conditions before resource creation or modification. If the condition fails, Terraform stops and displays a custom error message.

Why it matters

It prevents misconfigurations early in the workflow. You can ensure only approved regions, required tags, or valid parameters are used before deployment begins.

Use Cases

Allowing deployments only in specific AWS regions
Validating tag presence
Ensuring required environment variables exist
Checking resource limits

Example

precondition {
  condition     = contains(var.allowed_regions, data.aws_region.current.name)
  error_message = "Region not allowed for deployment."
}

6. postcondition

This rule verifies conditions after the resource is created or updated. If the condition fails, Terraform marks the resource as invalid.

Why it matters

It ensures the final state complies with policies or expectations. For example, you can enforce that a bucket must have specific tags even after creation.

Use Cases

Ensuring important tags exist
Validating resource attributes
Enforcing compliance with organization standards
Checking resource state after creation

Example

postcondition {
  condition     = contains(keys(self.tags), "Environment")
  error_message = "Environment tag is required."
}

Why Lifecycle Rules Matter in Real Deployments

Terraform lifecycle rules are essential for building resilient AWS infrastructure. They help prevent outages, protect critical data, ensure compliance, and maintain predictable behavior during updates. With these tools, you control not only what resources get created, but how they behave throughout their lifecycle.

Whether you're deploying production workloads, implementing automation pipelines, or managing infrastructure across teams, lifecycle rules bring safety, consistency, and reliability to your Terraform workflows.

Video Reference

@piyushsachdeva

AWS Terraform Meta Arguments | Count, depends_on, for_each

Adarsh Gupta — Mon, 01 Dec 2025 12:32:20 +0000

Terraform becomes truly powerful when you start using meta-arguments — special parameters that change how resources are created, managed, and related to each other. While Terraform syntax is easy to learn, mastering meta-arguments like count, for_each, and depends_on is what enables you to build dynamic, scalable, and production-ready AWS infrastructure.

This guide explains each meta-argument in detail with clear examples, best practices, and common real-world scenarios where these features become essential.

Understanding Meta Arguments in Terraform

Meta-arguments are not specific to any resource type. Instead, they can be applied to any Terraform resource, module, or data source to control creation patterns and dependency behavior.

The three foundational meta-arguments are:

count → create multiple instances of a resource using index numbers
for_each → create multiple resources using map or set keys (more predictable)
depends_on → explicitly define resource ordering and dependencies

Let’s explore each one in detail.

1. count — Create Resources Dynamically Using Indexing

count is the simplest way to create multiple instances of the same resource. It works by creating resources based on a number, and you use count.index to reference each instance.

Example — Creating Multiple S3 Buckets Using count

variable "bucket_names" {
  type    = list(string)
  default = ["app-logs", "media-storage", "archive-backups"]
}

resource "aws_s3_bucket" "buckets" {
  count  = length(var.bucket_names)
  bucket = var.bucket_names[count.index]
}

Terraform will create three buckets and name them based on list values. This is the simplest way to scale resource creation.

When to Use count

When resources are identical
When order doesn’t matter
When your inputs are lists, not maps
When index-based naming is acceptable

Limitations

The main drawback of count is index shifting.
If the order of your list changes, Terraform may destroy and recreate resources unnecessarily. For example, adding a new item in the middle of the list changes all subsequent indexes.

2. for_each — Create Resources with Stable Keys

for_each solves the indexing problem by using keys instead of positions. This ensures resources remain stable even when items are added or removed.

Example — Creating Buckets Using for_each

variable "buckets" {
  type = map(string)
  default = {
    logs   = "app-logs"
    media  = "media-storage"
    backup = "archive-backups"
  }
}

resource "aws_s3_bucket" "bucket" {
  for_each = var.buckets
  bucket   = each.value
}

Here, resources are created using keys (logs, media, backup) instead of index numbers.

When to Use for_each

When you want stable instances
When input is a set or map
When you need meaningful Terraform addresses like:

aws_s3_bucket.bucket["logs"]

Why for_each Is Better Than count

Unlike count, adding a new key does not recreate other resources. Keys remain stable indefinitely, making this the preferred method for production infrastructure.

3. depends_on — Explicitly Define Resource Dependencies

Terraform generally detects dependencies automatically. For example, if an EC2 instance references a subnet ID, Terraform understands the subnet must be created first.

However, sometimes you must explicitly declare dependencies — especially when:

Provisioners are involved
Resources don’t reference each other directly
Modules need ordering
Upstream actions must complete first

Example — Wait for S3 Bucket Before Running Local Script

resource "aws_s3_bucket" "example" {
  bucket = "demo-meta-argument-bucket"
}

resource "null_resource" "notify" {
  depends_on = [aws_s3_bucket.example]

  provisioner "local-exec" {
    command = "echo S3 bucket created!"
  }
}

When to Use depends_on

When implicit dependency cannot be detected
When actions must occur in strict sequence
When you must enforce ordering between modules

Best Practice

Use depends_on only when needed.
Terraform is good at understanding dependencies automatically, so avoid overusing it to prevent unnecessary serialization of your plan.

Putting It All Together — Real-World Example

Here’s a scenario combining all three meta-arguments:

variable "apps" {
  type = map(string)
  default = {
    app1 = "app1-logs"
    app2 = "app2-logs"
  }
}

resource "aws_s3_bucket" "buckets" {
  for_each = var.apps
  bucket   = each.value
}

resource "null_resource" "verify" {
  count      = length(var.apps)
  depends_on = [aws_s3_bucket.buckets]

  provisioner "local-exec" {
    command = "echo Created bucket ${count.index}"
  }
}

In this setup:

for_each ensures stable bucket creation
count runs verification for each bucket
depends_on ensures verification runs after bucket creation

This pattern appears frequently in CI/CD pipelines, multi-environment deployments, and state validation workflows.

Conclusion

Meta-arguments like count, for_each, and depends_on are the backbone of dynamic Terraform configurations. They help you scale your AWS infrastructure safely, maintain predictable behavior, and enforce dependency flow.

If you want to grow as a Terraform engineer, understanding these concepts is essential. Use count for simple repetition, for_each for stable and meaningful keys, and depends_on when Terraform needs guidance on ordering.

Reference Video

@piyushsachdeva

AWS Terraform Type Constraints Explained

Adarsh Gupta — Sun, 30 Nov 2025 15:03:44 +0000

Type constraints in Terraform play a crucial role in writing predictable, reusable, and maintainable infrastructure-as-code. When working with AWS resources, ensuring that your variables are properly typed helps prevent configuration errors, enforces standards, and makes your modules easier to understand and more robust. This post explains Terraform’s type system in depth and demonstrates how these type constraints apply to AWS infrastructure deployments.

Understanding Type Constraints in Terraform

Terraform uses type constraints to ensure that variables receive values in the correct format. Without proper typing, it becomes easy to pass the wrong kind of data into your AWS resources, leading to failed deployments or hidden configuration issues. Types also help teams maintain consistency, validate expectations, and build self-documenting code.

Terraform supports primitive types, collection types, and complex types. Each category serves different use cases, especially in AWS environments where configurations often involve nested structures like VPC settings, security groups, subnet definitions, tags, and scaling rules.

Primitive Types

string

Represents text values and is commonly used for names, environment stages, and ARNs.

variable "environment" {
  type = string
  default = "dev"
}

number

Used for numerical values such as port numbers, instance counts, or CIDR index extraction.

variable "instance_count" {
  type = number
  default = 2
}

bool

Controls conditional logic, for example enabling CloudWatch monitoring or versioning.

variable "enable_versioning" {
  type = bool
  default = true
}

Collection Types

Collection types are essential when defining multiple AWS configurations.

list(type)

An ordered list of values, often used for availability zones or port lists.

variable "availability_zones" {
  type = list(string)
  default = ["us-east-1a", "us-east-1b"]
}

set(type)

An unordered collection of unique values. Commonly used for security group ports.

variable "allowed_ports" {
  type = set(number)
  default = [22, 80, 443]
}

map(type)

A key-value structure frequently used for tagging AWS resources.

variable "tags" {
  type = map(string)
  default = {
    Environment = "dev"
    Owner       = "CloudOps"
  }
}

Tuple and Object Types

These types become powerful when modeling AWS infrastructure because they allow strict structural definitions.

tuple([type1, type2, ...])

Useful when the order of distinct typed elements matters.

variable "network_ports" {
  type = tuple([number, number])
  default = [80, 443]
}

object({ key = type, ... })

Object types help enforce detailed structural definitions such as instance sizing, VPC settings, or application configuration.

variable "server_config" {
  type = object({
    name    = string
    cpu     = number
    memory  = number
    is_prod = bool
  })

  default = {
    name    = "web-server"
    cpu     = 2
    memory  = 4
    is_prod = false
  }
}

Validation and Business Rules

Type constraints ensure the correct data format, but validation blocks enforce business rules on top of that.

Example: enforcing valid AWS instance count

variable "instance_count" {
  type = number

  validation {
    condition     = var.instance_count >= 1 && var.instance_count <= 10
    error_message = "Instance count must be between 1 and 10."
  }
}

Example: validating CIDR blocks for VPC creation

variable "vpc_cidr" {
  type = string

  validation {
    condition     = can(cidrhost(var.vpc_cidr, 0))
    error_message = "Invalid CIDR block format."
  }
}

Best Practices for Using Type Constraints

Always Specify Types

Defining explicit types leads to predictable behavior and better self-documentation.

Validate Complex Inputs

Nested objects, maps, and lists should always have constraints to prevent logical errors.

Use the Right Collection Type

Choose list when order matters, set for unique values, and map for structured key-value metadata.

Keep Error Messages Clear

Meaningful validation errors help teams quickly fix configuration issues.

Document Type Requirements

Descriptions in variable blocks make modules user-friendly and easier to maintain.

Final Thoughts

Terraform’s type constraints are more than just syntax—they represent a fundamental design practice that improves clarity, reliability, and collaboration. When working with AWS infrastructure, using proper type definitions significantly reduces the chances of misconfigurations and ensures your deployments behave exactly as intended. As AWS environments grow in complexity, mastering Terraform types becomes essential for maintaining high-quality infrastructure-as-code.

Reference Video

@piyushsachdeva

AWS Terraform Project Structure Best Practices

Adarsh Gupta — Sat, 29 Nov 2025 14:31:19 +0000

Organizing Terraform configurations properly is one of the most important aspects of building scalable, readable, and maintainable Infrastructure as Code. A well-structured project helps teams collaborate better, simplifies debugging, and makes large infrastructures easier to understand and extend.

This guide explains how Terraform loads files, recommended directory structures, common patterns used in production environments, and best practices followed by DevOps teams.

How Terraform Loads Files

Terraform automatically loads all files ending with .tf in the current directory. These files are merged into a single configuration before execution. File names do not affect functionality but play a major role in how readable and maintainable the project becomes.

Key points about file loading:

Terraform loads files in lexicographical (alphabetical) order
All .tf files are combined into one configuration
File names are for organization only
A clean structure improves clarity without changing behavior

Recommended Terraform File Layout

A commonly used and scalable layout looks like this:

project-root/
├── backend.tf
├── provider.tf
├── variables.tf
├── locals.tf
├── main.tf
├── vpc.tf
├── security.tf
├── compute.tf
├── storage.tf
├── database.tf
├── outputs.tf
├── terraform.tfvars
└── README.md

Purpose of Each File

backend.tf
Stores remote backend configuration such as S3 state storage and DynamoDB locking.

provider.tf
Defines provider configuration and required versions.

variables.tf
Contains input variables used across the project.

locals.tf
Includes computed local values such as tags, naming conventions, and derived expressions.

main.tf
General resources or the entry point for additional modules.

vpc.tf, storage.tf, compute.tf, security.tf, database.tf
Separate files for grouping resources logically by service.

outputs.tf
Exports useful attributes such as VPC IDs, subnet IDs, or bucket names.

terraform.tfvars
Stores the actual values for variables.

README.md
Explains how to use the project.

Principles for Organizing Terraform Files

Separation of Concerns

Group resources based on function. For example, keep VPC, subnets, route tables, and gateways together in networking-related files.

Logical Grouping

Organize the project by AWS service type—compute, security, networking, databases, and storage.

Consistency

Use predictable, descriptive file names. Lowercase + hyphens or lowercase + underscores are widely adopted.

Modularity

Keep files focused on a single responsibility. Larger projects typically use modules for VPC, EC2, security, etc.

Documentation

Document variable usage, architecture decisions, and module purposes. A well-written README accelerates onboarding and collaboration.

Advanced File Organization Patterns

Environment-Based Structure

When managing multiple environments such as dev, staging, and production:

environments/
├── dev/
│   ├── backend.tf
│   ├── terraform.tfvars
│   └── main.tf
├── staging/
│   ├── backend.tf
│   ├── terraform.tfvars
│   └── main.tf
└── production/
    ├── backend.tf
    ├── terraform.tfvars
    └── main.tf

Modules are stored separately:

modules/
├── vpc/
├── security/
└── compute/

This approach reduces duplication and enforces structure across environments.

Service-Based Structure

Larger teams often organize files by domain:

infrastructure/
├── networking/
├── security/
├── compute/
├── storage/
└── data/

This mirrors how AWS services are grouped and keeps complexity manageable.

Best Practices for Terraform Project Structure

Clear Naming Conventions

Use descriptive names like vpc.tf, security-groups.tf, or s3.tf.

Group Resources Logically

Place related resources in the same file or folder.

Manage File Size

Split files when they grow too large. Anything above 500 lines should be reviewed.

Maintain Order

Place backend and provider configuration first, then variables and locals, and outputs at the end.

Document Everything

Include a README and add comments for complex logic. Good documentation saves countless hours.

Commands for Validating the Structure

terraform validate
terraform fmt -recursive
terraform plan
terraform apply

Regular validation ensures the structure is clean and consistent.

Common Mistakes to Avoid

Putting everything into main.tf
Using inconsistent or unclear file names
Mixing random resources in one file
Keeping no documentation
Over-structuring a small project

A simple, predictable layout is always better than an overly sophisticated one.

Reference Video

@piyushsachdeva

AWS Terraform Project Structure | Best Practices

Adarsh Gupta — Sat, 29 Nov 2025 14:30:02 +0000