Mastering Terraform State Management with Proven Best Practices

Have you ever felt that your cloud infrastructure is getting out of control? or your resources mysteriously are disappearing or duplicating? If you're a developer or manage infrastructure, you know how tough it can be to keep everything organized and under control. This is where Terraform state comes to the rescue…. it's a crucial tool that helps keeping your projects on track.

Think of Terraform state as a detailed, up-to-date map of your digital setup. It not only shows you what resources are being used but also helps to prevent problems before they happen. Without it, your infrastructure could end up in chaos, with risks of errors and data loss.

In this article, we’ll take a look at how Terraform state helps you stay on top of your resources, boosts performance, catches unexpected issues, and makes teamwork easier. We’ll also cover what are the best practices for managing your state effectively and show you how to set it up.

Are you ready to change how you handle your infrastructure with Terraform state?

What is Terraform state?

Terraform state tracks and provides an overview of your infrastructure’s resources and configuration. It’s a reference point for Terraform to know what resources have been created, updated, or destroyed.

Why is Terraform State important?

• Tracks all resources of your infrastructure allowing Terraform to manage them without re-creating them.
• Improves performance allowing Terraform to skip repeatedly checking with the infrastructure provider to retrieve the current status of resources.
• Helps detecting drifts, which occurs when resources are changed outside of Terraform comparing the actual infrastructure with the desired configuration and updates.
• Storages a remote state to allow collaboration in teams, ensuring consistent updates and avoiding conflicts.

How to Manage Terraform State?

1. Local State Management: Terraform stated is stored by default in a JSON-formatted file called terraform.tfstate, is located in the root directory of your terraform project and contains the information abouts the resources managed like ID’s, attributes, dependencies, metadata, among others. This file is created after running terraform apply.
   
   It’s recommended to use it for small projects or individual use.

2. Remote State Management: Allows multiple users to share and manage the state file in a secure way supporting collaboration and avoiding conflicts. It’s recommended for team environments and large projects. This are some of popular backends that you can use:

   1. Amazon S3
   2. Azure blob storage
   3. Google cloud storage

Terraform state best practices

• Use a remote state: If you are working in a team environment, always use a remote backend so you won’t have to deal with conflicts and loss of information.
• Enable locking: Ensure that state locking is enabled, especially when you’re using a remote state, to prevent multiple users from making changes simultaneously and corrupt the state.
• Encrypt the state file: Always encrypt your state, either locally or remotely, due to it contains sensitive information.
• Use workspaces for multiple environments: Leverage workspaces or separate backends for managing state across different environments.
• Regular backups: Make backups regularly, especially when managing state locally.
• Avoid manual modifications: Never edit the state file manually, always use terraform commands like ‘terraform state’ to make adjustments.

Example

Requirements

• AWS account or administrator access
• S3 bucket created
• AWS CLI installed and configured
• Terraform installed

Terraform state code

The code bellow shows how to store your terraform state file inside an s3 bucket. Inside your root folder run the following cli comand and paste the example code.

touch backend.tf

terraform {
  backend 's3' {
    # The S3 bucket where Terraform will store the state file
    # Bucket must exists
    bucket = 'your-bucket-name' 

    # The path within the S3 bucket where the state file will be stored
    key    = 'states/your-app/example/terraform.tfstate'

    # The AWS region where the S3 bucket is located
    region = 'your-region'

    # The name of the DynamoDB table used for state locking
    dynamodb_table = 'terraform-state-lock-dynamo-dev' 

    # DynamoDB is used to prevent concurrent operations on the state file
    # by locking it during Terraform runs. If you change the table name,
    # make sure to update this value to reflect the new table name.
  }
}

Conclusion

Managing Terraform state remotely is crucial for effective collaboration, especially when working within a team of multiple developers. By sharing state files, you prevent resource duplication, reduce costs, and avoid conflicts. Even if you're working solo, remote state management offers significant advantages, such as the flexibility to switch between workspaces and devices without the hassle of transferring local state files. Using a remote storage solution like an S3 bucket ensures that your development process remains flexible, agile, and secure, making it the optimal choice for both team and individual projects.