How to manage and manipulate resources in Terraform state file - 2

This is in continuation with the 1st article - How to manage and manipulate resources in Terraform state file - 1, where you created an AWS Linux instance and security group, examined a state file, and replaced a resource using CLI to show how you manipulated the resources.

In this 2nd article, we will manipulate resources using these commands - move, remove and refresh to observe how important state is to your Terraform operations.

Please visit my GitHub Repository for Terraform articles on various topics being updated on constant basis.

Let’s get started!

Objectives:

4. Move a resource to a different state file

5. Remove a resource from state

6. Refresh modified infrastructure

7. Delete (Destroy) your infrastructure

Pre-requisites:

• AWS user account with admin access, not a root account.

• Cloud9 IDE with AWS CLI, and Terraform installed.

Resources Used:

For building this EC2 module, I have used a data source for pulling in an AMI ID instead of a hard-coded value. I have spent so much time to make this work, and after many, many attempts, I succeeded! I have also used Terraform documentation for this purpose.

Terraform documentation for AMI.

data source for pulling in an AMI ID.

Steps for implementation to this project:

4. Move a resource to a different state file

• The terraform state mv command moves resources from one state file to another.

• You can also rename resources with mv.

• The move command will update the resource in state, but not in your configuration file.

• Moving resources is useful when you want to combine resources from other states, but do not want to destroy and recreate the infrastructure.

• Let’s create the following organizational structure as shown below.

• The new_state subdirectory contains a new Terraform configuration.

• Create a main.tf file.

• This configuration creates a new EC2 instance named "aws_instance.ec2_new" and uses a data resource to use the same security group from your root configuration file.

# PROVIDER BLOCK

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.23"
    }
  }
  required_version = ">= 1.2.0"
}

provider "aws" {
  region  = "us-east-1"
}


# EC2 BLOCK

data "aws_ami" "linux" {
   most_recent = true
   owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}


data "terraform_remote_state" "root" {
  backend = "local"

  config = {
    path = "../terraform.tfstate"
  }
}

resource "aws_instance" "ec2_new" {
  ami                    = data.aws_ami.linux.id
  instance_type          = "t2.micro"
  vpc_security_group_ids = [data.terraform_remote_state.root.outputs.security_group]

  tags = {
    Name = "ec2_new"
  }
}


# OUTPUTS BLOCK

output "instance_id" {
  value = aws_instance.ec2_new.id
}

output "public_ip" {
  value       = aws_instance.ec2_new.public_ip
  description = "The public IP of the ec2 new server"
}

output "security_group" {
  value = data.terraform_remote_state.root.outputs.security_group 
}

• Change into the subdirectory.

cd new_state

• Run terraform init

• Run terraform apply to apply the configuration and type yes when prompted.

• Wait for the EC2 instance ec2_new to be created.

• Only 1 resource ec2_new added, not the security_group

• Now, you have a second state file with a managed resource and a data source.

• Move the new EC2 instance resource you just created, "aws_instance.ec2_new", to the old configuration's file in the directory above your current location, as specified with the -state-out flag.

• Set the destination name to the same name, since in this case there is no resource with the same name in the target state file.

terraform state mv -state-out=../terraform.tfstate aws_instance.ec2_new aws_instance.ec2_new

• Change into your root directory.

cd ..

• Run terraform state list to confirm that the new EC2 instance, "aws_instance.ec2_new", is present in the original configuration's state file.

• Run terraform plan.

• Because the new EC2 instance is present in state but not in the configuration, Terraform plans to destroy the moved instance, and remove the resource from the state file.

• Open the main.tf file in your root directory. Copy and paste the resource definition below.

resource "aws_instance" "ec2_new" {
  ami                = data.aws_ami.linux.id
  instance_type      = "t2.micro"
  vpc_security_group_ids = [aws_security_group.ec2_sg.id]

  tags = {
    Name = "ec2_new"
  }
}

• Run terraform apply

• Change into your new_state directory.

cd new_state

• Run terraform destroy and you should have no resources to destroy.

• Your security_group resource is a data source and you moved the aws_instance resource to another state file.

• Accept the changes by typing yes when prompted.

5. Remove a resource from state

• The terraform state rm subcommand removes specific resources from your state file.

• This does not remove the resource from your configuration or destroy the infrastructure itself.

• Change into your root directory.

cd ../

• Remove your security_group resource from state.

terraform state rm aws_security_group.ec2_sg

• Confirm the change by reviewing the state with terraform state list.

terraform state list

• The removed security_group resource does not exist in the state, but the resource still exists in your AWS account.

• Run terraform import to bring this security group back into your state file.

• Removing the security group from state did not remove the output value with its ID, so you can use it for the import.

terraform import aws_security_group.ec2_sg $(terraform output -raw security_group)

6. Refresh modified infrastructure

• The terraform refresh command updates the state file when physical resources change outside of the Terraform workflow.

• Delete the original EC2 instance from your AWS account using the AWS CLI or the AWS Console.

• Wait for 4-5 min for AWS to destroy your instance.

aws ec2 terminate-instances --instance-ids $(terraform output -raw instance_id)

• By deleting this piece of infrastructure, you have created a difference between your state and the real-world resources mapped to it.

• The state file no longer reflects the reality of your environment.

• Wait for 4-5 minutes for AWS to destroy your instance.

• Run the terraform refresh command to update your state file.

• Run terraform state list to confirm Terraform deleted the original aws_instance.2_orig resource from state.

Your state file now reflects the reality. You deleted the "aws_instance.ec2_orig" and the terraform refresh command removed it from state.

The terraform refresh command does not update your configuration file. Run terraform plan to review the proposed infrastructure updates.

• Remove the original "aws_instance.ec2_orig" resource, and public_ip, instance_id and security_group outputs from your main.tf file.

• This will prevent Terraform from recreating the resource in future Terraform operations.

# PROVIDER BLOCK

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.23"
    }
  }
  required_version = ">= 1.2.0"
}

provider "aws" {
  region  = "us-east-1"
}


# EC2 BLOCK

data "aws_ami" "linux" {
   most_recent = true
   owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}


######### REMOVE RESOURCE ---> ec2_orig ##########

resource "aws_instance" "ec2_orig" {
  ami                = data.aws_ami.linux.id
  instance_type      = "t2.micro"
  vpc_security_group_ids = [aws_security_group.ec2_sg.id]

  tags = {
    Name = "ec2_orig"
  }
}



# security group 
resource "aws_security_group" "ec2_sg" {
   name        = "web_sg"
   description = "allow inbound HTTP traffic"

   # HTTP from vpc
   ingress {
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]     
   }


  # outbound rules
  # internet access to anywhere
  egress {
     from_port   = 0
     to_port     = 0
     protocol    = "-1"
     cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
     name = "ec2_sg"
  }
}


######### REMOVE OUTPUTS ---> instance_id, public_ip, security_group ##########

# OUTPUTS BLOCK

output "instance_id" {
  value = aws_instance.ec2_orig.id
}

output "public_ip" {
  value       = aws_instance.ec2_orig.public_ip
  description = "The public IP of the ec2 server"
}

output "security_group" {
  value = aws_security_group.ec2_sg.id
}

• Run terraform apply, which will confirm that your configuration matches your state file, and remove their outputs from state. Accept the changes by typing yes when prompted.

• Run terraform state list

7. Delete (Destroy) your infrastructure

• Run terraform destroy to destroy your infrastructure. Accept the changes by typing yes when prompted.

• Wait for 4-5 minutes to destroy your resources.

What we have done so far

• In the 1st article on Terraform state, you created an EC2 Linux instance and corresponding security group. Then, you examined your local state file and replaced a resource using CLI to show how you manipulated the resources.

• In the 2nd article, you used Terraform state to move, remove, and modify your resources across multiple configurations.