š Introduction
Welcome, Devs, to the exciting world of Infrastructure and Cloud computing!
If you have been navigating the cloud landscape recently, you have undoubtedly encountered the industry buzzword: Infrastructure as Code (IaC). Simply put, IaC is the practice of specifying your infrastructure requirements in a code format rather than manually configuring servers and networks. This approach is the bedrock of modern DevOps, ensuring reliability and consistency across all your environments.
While there is a robust ecosystem of tools offering IaC capabilitiesāincluding Pulumi, Chef, Puppet, and AWS CloudFormationātoday we are zeroing in on the undisputed heavyweight of this segment: Terraform.
If you are just starting out or want to dive deeper into the core concepts, I have covered a lot of ground regarding Terraform in my previous posts. I highly recommend checking out the full series here to get up to speed:
š Terraform Series
In today's guide, we are shifting our focus to Optimisation. As your infrastructure grows, so does the complexity and the time required for deployments. We will dive into the best practices to improve the performance of your Terraform configurations, ensuring your infrastructure remains efficient, fast, and secure.
So, without further ado, letās get started!
š½ļø Youtube Demonstration
š Pre-Requisites
Before we jump into the optimisation techniques, let's ensure you have the necessary tools ready to roll.
As usual, for this guide, you will need the following set up on your machine:
AWS CLI (Configured with an appropriate IAM user)
Terraform CLI
Docker and Docker-compose
If you don't have these installed yet or aren't sure how to configure them, donāt worry! I have walked through the entire process step-by-step in my previous guide. Just follow the instructions there, and you will be good to go:
š How to Install AWS CLI and Terraform
Once you are all set up, proceed to the next section where we start optimizing!
š How to Optimise Terraform
Now that we are set up, let's get into the meat of the matter. Here are 5 Best Practices to turbocharge your Terraform performance.
1. Use Remote Backends for State Management
By default, Terraform stores its "state" (the file that maps your code to real-world resources) locally on your machine (terraform.tfstate). While this works for solo side projects, it kills performance and collaboration in real teams.
The Fix: Store your Terraform State file in a remote location, preferably AWS S3.
Why? It prevents conflicts when multiple team members (Dev, Stage, Prod) try to apply changes at the same time.
The Performance Boost: Moving to a remote backend can improve I/O performance by 10-30% for large state files.
State Locking: When configured correctly with State Lock enabled, you ensure that no two people can write to the state simultaneously. (Note: While traditionally done with DynamoDB, ensuring your backend supports locking is key to preventing corruption).
How to do it:
You simply need to create an S3 bucket and reference it in your configuration:
terraform {
backend "s3" {
bucket = "my-terraform-state-bucket"
key = "prod/terraform.tfstate"
region = "us-east-1"
use_lockfile = true
}
}
Itās that simple!
2. Modularise Your Code
Writing one giant main.tf file is a common rookie mistake. If you are defining a secure EC2 instance, you likely also need a VPC, Security Groups, Subnets, etc. Lumping this all together makes Terraform work harder to calculate dependencies.
The Fix: Break your code down into Modules.
Instead of rewriting the same sub-components for every service, use the DRY (Don't Repeat Yourself) principle. Create a module for your network stack, a module for your compute stack, etc.
Benefit: This reduces complex dependency graphs.
Speed: It enables better parallel execution because Terraform can process distinct modules concurrently.
3. Unlock True Parallelism
Terraform is capable of walking through the dependency graph and creating non-dependent resources at the same time. However, the default setting is often too conservative.
The Fix: Adjust the -parallelism flag.
By default, Terraform limits concurrent operations to 10. For modern systems, this is quite low.
Guideline: You can safely increase this to 30-100 for normal development changes.
Constraint: Don't go too high, or you might hit AWS API rate limits (throttling). Aim for roughly 512MB of system memory per 1,000 resources.
The Result:
Using this flag can cut down plan and apply times significantlyāpotentially dropping a 3-5 minute operation down to just 30-60 seconds.
terraform apply -parallelism=30
4. Optimise Provider Configuration
Sometimes, Terraform performs checks that aren't strictly necessary for every single run, especially in non-production environments.
The Fix: Tune your AWS Provider.
In your Testing, Staging, or Dev environments, you can tell the provider to skip certain validations to save time on API calls. You can also configure retry behaviors to handle network blips gracefully without failing the whole run.
Example Configuration:
provider "aws" {
region = "us-east-1"
skip_credentials_validation = true
skip_metadata_api_check = true
max_retries = 5
}
Note: While this is a great time-saver for Dev/Test environments, check your organization's policy before using strict validation skipping in Production.
5. Use Resource Targeting
When managing large infrastructure, Terraform's default behavior is to analyze the entire configuration graph. For routine, independent updates (e.g., adding a new, isolated logging bucket), you might want to limit the scope of the operation to save time.
The Use Case: Use the -target flag when dealing with literally independent resources that are not connected to other resources that are changing. This allows you to apply changes only to specific resources, acting like a surgical strike rather than a full deployment.
# Only apply changes to the specific bucket, ignoring the rest
terraform apply -target=aws_s3_bucket.my_website_bucket
š Practical Demonstration
Enough with the theory! Let's get our hands dirty with some real Terraform code and the AWS Console.
To demonstrate these optimizations, we are going to deploy aĀ Request Counter ApplicationĀ onĀ AWS Elastic Beanstalk. This isn't just a "Hello World"; it is a multi-container setup running on an ECS Cluster involving:
Nginx:Ā As a reverse proxy and load balancer.
Node.js:Ā The application server (running two instances).
Redis:Ā To store the request count data persistently.
Step 1: Create Your ECR Repositories
First, we need a place to store our Docker images. We will create two Public Repositories in AWS Elastic Container Registry (ECR)āone for our web app and one for our custom Nginx image.
Run the following commands in your terminal:
aws ecr-public create-repository --repository-name nginx-node-redis-web
aws ecr-public create-repository --repository-name nginx-node-redis-nginx
Step 2: Get the Code
The complete source code for this project is available on my GitHub.
ForkĀ the repo.
CloneĀ it to your local system.
Open it in your code editor (like VS Code).
šĀ GitHub Repo: Nginx-Node-Redis Project
Step 3: Test Locally (Get the Vibe)
Before we deploy to the cloud, letās make sure it works on your machine. Inside the project directory, run:
docker-compose up --build
This will spin up the Nginx, Redis, and two Web containers. Open your browser and go toĀ http://localhost:8080.
You should see a nice Request Counter app. Every time you refresh, the counter increments, and you will see the server ID toggle betweenĀ web1Ā andĀ web2Ā as Nginx load-balances the traffic.
Once you are satisfied, pressĀ CTRL+CĀ in your terminal to stop the application.
Step 4: Build and Push to Cloud
Now, let's move these images to AWS.
Navigate to theĀ
webĀ directory in your terminal.Go toĀ AWS Console ā ECR ā Public Repositories.
SelectĀ
nginx-node-redis-webĀ and click theĀ "View Push Commands"Ā button.Execute those commands one by one to build and push your web image.
Repeat for Nginx:
Navigate to theĀ
nginxĀ directory.In the AWS Console, selectĀ
nginx-node-redis-nginx.ClickĀ "View Push Commands"Ā and execute them to push your Nginx image.
Step 5: The Terraform Configuration
Navigate to theĀ terra-performance-configĀ directory. This is where we have applied all the optimizations we discussed earlier:
provider.tf: We are skipping unnecessary validation checks (skip_metadata_api_check, etc.) to speed up the provider.backend.tf: We are using anĀ S3 bucketĀ to host our state file remotely and enablingĀlock_fileĀ to prevent write conflicts.main.tf: We are using a Terraform Module for Elastic Beanstalk. It utilizes aĀDockerrun.aws.jsonĀ file (similar toĀdocker-composeĀ but for ECS) which is uploaded to S3 and referenced in the module.dockerrun.aws.json: Skelton of your Docker containers, make sure to replace the image URI of web1, web2 and Nginx, you can find it in your ECR public Repo.variables.tf: Contains our standard configuration using the default VPC and subnets inĀus-east-1.
ā ļø CRITICAL: Architecture Check (ARM vs AMD)
I built the default Docker images on aĀ MacBook Air M3 (ARM64 architecture).
If you are usingĀ Windows or Linux (Intel/AMD), your architecture is likelyĀ AMD64. YouĀ mustĀ make these two small changes in the code before proceeding, or the deployment will fail:
- InĀ
variables.tf: Change the instance type fromĀt4g.microĀ (ARM-based) toĀt3.microĀ orĀt3.small.
- InĀ
Dockerrun.aws.json: Locate the Redis container definition. Change the image from the specific ARM tag to generic:Ā"image": "redis:alpine".
Step 6: Testing Parallelism
Now for the moment of truth. Let's see how fast we can provision this stack using theĀ ParallelismĀ optimization.
Run the following commands:
cd terra-performance-config
# Create the S3 bucket for our artifacts
aws s3 mb s3://pravesh-ebs-terra-performance-bucket
# Initialize Terraform
terraform init
# Apply with high parallelism
time terraform apply --auto-approve -parallelism=30
The Result:Ā Usually, a multi-container Elastic Beanstalk environment takesĀ 10-30 minutesĀ to provision. WithĀ -parallelism=30, you will likely see this drop toĀ 3-8 minutes.
Once finished, Terraform will output the Elastic Beanstalk URL. Click it to see your live app!
Step 7: Testing Resource Targeting
Finally, let's look at the power ofĀ Resource Targeting.
We have made a small change to theĀ Dockerrun.aws.jsonĀ file (e.g., adding an Environment Variable).
If we ran a normal apply, Terraform might check every single resource. Instead, we will targetĀ onlyĀ the specific components we are updating:
time terraform apply -parallelism=30 \
-target=aws_s3_object.dockerrun \
-target=aws_elastic_beanstalk_application_version.v1 \
-target=module.elastic-beanstalk-environment \
-auto-approve
The Result:Ā Without targeting, Terraform might attempt to refresh the state of the entire VPC and Security Group structure. With targeting, this update happens inĀ under a minute.
Cleanup
Don't forget to tear down your infrastructure to avoid unexpected AWS bills!
# Destroy Terraform resources
terraform destroy --auto-approve
# Clean up S3
aws s3 rm s3://pravesh-ebs-terra-performance-bucket --recursive
aws s3 rb s3://pravesh-ebs-terra-performance-bucket
# Delete ECR Repositories
aws ecr-public delete-repository --repository-name nginx-node-redis-web --force
aws ecr-public delete-repository --repository-name nginx-node-redis-nginx --force
š Conclusion
And thatās a wrap, folks!
We have successfully journeyed through the landscape ofĀ Terraform Optimization. We didn't just learn how to write Infrastructure as Code; we learned how to make itĀ efficient,Ā scalable, andĀ blazing fast.
From moving our state to aĀ Remote BackendĀ to unlocking the speed ofĀ Parallelism, and fromĀ ModularisingĀ our logic to performing surgical strikes withĀ Resource Targetingāyou now possess the toolkit to take your DevOps game to the next level.
Remember, optimization isn't just about saving a few minutes here and there. It's about creating a developer experience where feedback loops are short, deployments are reliable, and your infrastructure can scale without becoming a bottleneck.
I hope you enjoyed this deep dive and the hands-on project. Go ahead and apply these techniques to your own infrastructure, and let me know how much time you saved on your next deployment!
If you found this guide helpful or have any questions, feel free to connect with me. I talk about Cloud, DevOps, and everything in between.
š Let's Connect:
š¼Ā LinkedIn:Ā linkedin.com/in/pravesh-sudhaĀ
š¦Ā Twitter/X:Ā x.com/praveshstwtĀ
š¹Ā YouTube:Ā youtube.com/@pravesh-sudhaĀ
šĀ Website:Ā blog.praveshsudha.com
Happy Coding and Happy Clouding! āļøš













Top comments (4)
Very good post š¤©, I would add another tip to complete: caching providers (or baking them directly into the container image if possible)
Thanks, gonna incorporate it in the Video Demonstration.
Nice!
Thanks!