DEV Community: frankfolabi

Managing Terraform State: Best Practices for DevOps

frankfolabi — Sun, 01 Sep 2024 05:01:52 +0000

How does Terraform knows which resources it manages? It uses a JSON format that records a mapping of the resources in the configuration files and the representation in those resources in the real world. This is known as Terraform state file. This is usually named terraform.tfstate. For personal project, the state file can live on the local machine called the local backend. However, when working with a team, having the state files on numerous local machines can be a disaster, neither would it be great to have it in a version control system due to some sensitive secrets that the state file contains. What are some best practices for managing state in Terraform?

Storage of State Files

The best practice for storing state files is to use remote backends. Amazon S3, Azure Storage, Google Cloud Storage, Terraform Cloud and Terraform Enterprise all support remote backends. These storage options help to avoid manual error, support locking so that no two persons would run terraform apply at the same time and protect secrets that may be contained in state files.

How to create a remote backend on S3

You can create an S3 bucket to store your state files. S3 buckets support versioning, encryption and security from public access. This makes the state file safe. You can also use DynamoDB to achieve state locking. Here is a Terraform configuration that can be deployed from another directory.

provider "aws" {
    region = "us-east-2"
}
// Create a unique bucket
resource "aws_s3_bucket" "tf_state" {
  bucket ="<your-bucket-name>"

  # Prevent accidental deletion of the S3 bucket
  lifecycle {
    prevent_destroy = true
  }
}

// Enable bucket versioning
resource "aws_s3_bucket_versioning" "enables" {
  bucket = aws_s3_bucket.tf_state.id
  versioning_configuration {
    status = "Enabled"
  }
}

// Enable S3 server side encryption 
resource "aws_s3_bucket_server_side_encryption_configuration" "default" {
  bucket = aws_s3_bucket.tf_state.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

// Block public access
resource "aws_s3_bucket_public_access_block" "public_access" {
  bucket = aws_s3_bucket.tf_state.id
  block_public_acls = true
  block_public_policy = true
  ignore_public_acls = true
  restrict_public_buckets = true
}

// Create DynamoDB for locking
resource "aws_dynamodb_table" "terraform_locks" {
  name = "<your-table-name>"
  billing_mode = "PAY_PER_REQUEST"
  hash_key = "LockID"

  attribute {
    name = "LockID"
    type = "S"

  }
}

You can now run the following Terraform commands in order

terraform init
terraform plan
terraform apply

An S3 bucket along with a DynamoDB table would be created. Next, configure the backend to store the state files remotely in the S3 bucket just created. This is the Terraform block to set the backend.

terraform {
  backend "s3" {
    bucket = "<your-bucket-name>"
    key = "global/s3/terraform.tfstate"
    region = "us-east-2"

    dynamodb_table = "<your-table-name>"
    encrypt = true
  }
}

Now you will need to run terraform init again which will allow Terraform to configure the remote backend and copy the state file to the specified S3 bucket.

After a confirmation of yes, the state file is no longer on a local machine but stored remotely. This will allow teams to collaborate efficiently without wreaking havoc on existing infrastructure. This is the best practice. However, how can environments be isolated? A future blog would provide a solution.

Managing High Traffic Applications with AWS Elastic Load Balancer and Terraform

frankfolabi — Sat, 31 Aug 2024 10:58:54 +0000

You just configured your infrastructure to auto scale when the traffic increases. How would you manage the traffic and ensure some servers are not overwhelmed while others are not underserved? This is the essence of load balancing - to ensure traffic is distributed to your servers in a manner not

AWS offers elastic load balancing resources for layers 3, 4, and 7 of the OSI model. These are the Gateway load balancer, Network load balancer, and the Application load balancer types. The application load balancer (ALB) would be used for this webapp.

Creating the ALB

resource "aws_lb" "webserver" {
  name = "tf-example"
  load_balancer_type = "application"
  subnets = data.aws_subnets.default.ids
  security_groups = [aws_security_group.alb.id]
}

Notice that the code block references the available subnet data, and the security group created for the ALB. The ALB security group would be covered soon. Next, create the ALB listener.

Creating the ALB listener

resource "aws_lb_listener" "webserver" {
  load_balancer_arn = aws_lb.webserver.arn
  port = 80
  protocol ="HTTP"

  default_action {
    type = "fixed-response"

    fixed_response {
      content_type = "text/plain"
      message_body ="404: page not found"
      status_code = 404
    }
  }
}

This resource used the port 80 and the http protocol. A fixed response was also set in case there is an error on the page and a status code of 404 was used.

Create security group for load balancer

resource "aws_security_group" "alb" {
  name = "tf-example"

  # Allow inbound HTTP request
  ingress {
    from_port = 80
    to_port = 80
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

   #Allow outbound requests
  egress {
    from_port = 0
    to_port = 0
    protocol = "-1"
    cidr_blocks = ["0.0.0.0/0"]   
  }
}

The security group rule for the ALB is different from the one set for the webapp. Here the ALB accepts traffic from port 80 from any traffic on the internet while it sends out request to all ports. Let's set the target group that would be used of the autoscaling.

Create target group for auto scaling group

resource "aws_lb_target_group" "webserver" {
  name = "tf-example"
  port =  var.server_port
  protocol = "HTTP"
  vpc_id = data.aws_vpc.default.id

  health_check {
    path = "/"
    protocol = "HTTP"
    matcher = "200"
    interval = 15
    timeout = 3
    healthy_threshold = 2
    unhealthy_threshold = 2
  }
}

There is a health check set up to confirm if the webservers are healthy or unhealthy. The target group will health check the webservers by periodically sending an HTTP request to each webserver. If the response matches the matcher then it determines it is healthy. Otherwise, it would be marked as unhealthy. The webserver will be using the port already defined in the variable and it will be placed in the default VPC. Finally, the ALB listener rules need to be created.

Create the ALB listener rules

resource "aws_lb_listener_rule" "asg" {
  listener_arn = aws_lb_listener.webserver.arn
  priority = 100

  condition {
    path_pattern {
      values = ["*"]
    }
  }
  action {
    type = "forward"
    target_group_arn = aws_lb_target_group.webserver.arn
  }
}

This listener rule sends request that matches any path to the terget group that contains the autoscaling group. The output block will filter out the application load balancer DNS name. This would be used to check the webapp.

output "alb_dns_name" {
    value = aws_lb.webserver.dns_name
    description = "The domain name of the load balancer"
}

You are now good to go. Run terraform validate to check if the configuration file syntax is valid. The run terraform plan to check the changes that will take place. After seeing the new resources that would be created, run terraform apply. Here is the output.

On the terminal, you can verify the webapp is running by using the curl comand. Run curl http://<alb-dns-name>

The webserver is now highly available and receiving its traffic from a load balancer which is more secure.

We can check the AWS console to see some of the resources created by Terraform. Here are the instances running

and the security groups created

as well as the load balancer

along with the autoscaling group.

To clean up all resources, simply run terraform destroy.

In the future, other components would be added to the architecture. Feel free to ask your questions.

Deploying a Highly Available Web App on AWS Using Terraform

frankfolabi — Wed, 28 Aug 2024 18:01:13 +0000

It is so exciting to launch a single webserver, and you are now online. The media has made your product popular, and the internet traffic to your website is increasing greatly. How would we handle such a large number of requests? How can we ensure that out server does not go down due to overwhelming request? It would make sense to have additional servers to help hand this situation. Here come scaling which is the ability to increase the number or instance size of your server which we call horizontal or vertical scaling. Would it not be nice to automatically do this? Let us improve our architecture from the previous post.

Configuring the web server with variables

You may have notice that the port number 8080 was hardcoded and duplicated. It may be challenging to make changes without an error if there are numerous occurrences. A best practice is to follow the Don't Repeat Yourself (DRY) principle. To male our code configurable we can define input variables.

Let us define the port number as a variable.

variable "server_port" {
  description = "The port the server will use for HTTP requests"
  type = number
  default = 8080
}

The type constraint will ensure that only a number value is acceptable. the code block also specific that the default port value as 8080. We can now refer to the variable in other places using var.server_port.
Let us now update the security group and the user data.

resource "aws_security_group" "instance" {
    name = "tf-sg"

    ingress {
        from_port = var.server_port
        to_port = var.server_port
        protocol = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
    }   
}

Creating a highly available webserver

To create a highly available, a launch configuration is needed and an auto scaling resource that will use this configuration.

resource "aws_launch_configuration" "webserver" {
    image_id = "ami-0fb653ca2d3203ac1"
    instance_type = "t2.micro"
    security_groups = [aws_security_group.instance.id]

    user_data = <<-EOF
    #!/bin/bash
    echo "Hello, World. Welcome to the use of Terraform in deploying infrastructure" > index.html
    nohup busybox httpd -f -p ${var.server_port} &
    EOF

    lifecycle {
      create_before_destroy = true
    }
}

Notice that the port in the user data script has been updated. The create_before_destroy would allow the autoscaling to be created first before destroying our single instance earlier launched.

resource "aws_autoscaling_group" "example" {
  launch_configuration = aws_launch_configuration.webserver.name
  vpc_zone_identifier = data.aws_subnets.default.ids

  target_group_arns = [aws_lb_target_group.webserver.arn]
  health_check_type = "ELB"

  min_size = 2
  max_size = 10

  tag {
    key = "Name"
    value = "tf-example"
    propagate_at_launch = true
  }
}

This block shows that a minimum number of servers that would run as 2 and the maximum as 10. This will ensure that a single point of failure is avoided.

In a future post, we will attach a load balancer to the autoscaling group and test traffic sent to it. Feel free to ask any questions and give feedback on how to improve.

Deploying Your First Server with Terraform: A Beginner's Guide

frankfolabi — Mon, 26 Aug 2024 03:54:31 +0000

Now you have installed Terraform, configured your AWS account to use the CLI, and installed VS Code. What next? Let us get deploy our first webserver.

On VS Code, create a directory to store your files with mkdir tf-files in the terminal
Next create a main.tf file with touch main.tf. You will write your configuration files here in Harshicorp Configuration Language.

Here is the block of code that you can add in your main.tf file

Provider Block

This shows that the provider is AWS and the region to be used is us-east-2

provider "aws" {
    region = "us-east-2"
}

Resource block

The resource block is used to define the resource Terraform would create. One interesting part is that Terraform would try to figure out which order to create these resources. In this resource block, we define the following:

Amazon Machine Image (AMI)
Instance type
Security group id
User data script
Tag

resource "aws_instance" "webserver" {
    ami = "ami-0fb653ca2d3203ac1"
    instance_type = "t2.micro"
    vpc_security_group_ids = [aws_security_group.instance.id]
    user_data = <<-EOF
    #!/bin/bash
    echo "Hello, World. Welcome to the use of Terraform in deploying infrastructure" > index.html
    nohup busybox httpd -f -p 8080 &
    EOF
    user_data_replace_on_change = true
    tags = {
        Name = "tf-webserver"
    }
}

This other resource block declares the security group to allow traffic from port 8080 from any IPv4 address

resource "aws_security_group" "instance" {
    name = "tf-sg"

    ingress {
        from_port = 8080
        to_port = 8080
        protocol = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
    } 
}

We can now save our main.tf file. Next, let us now deploy the webserver. First, we need to download the required provider plugins. Run terraform init

You may wonder, what are the provider plugin? These are executable binaries that the Terraform Core install on our laptop uses to communicate with the APIs of the cloud provider that has the resources we need. Notice that the screenshot uses the expression "Reusing previous version of . . ." This happens when you are running the terraform init command after the first time.

As a quick check to see if our file follows the right syntax, run terraform validate

What would our infrastructure look like? You can now run terraform plan

This information contains what resources would be created on the AWS. Now, let us proceed to deploy these resources. Run terraform apply. A confirmation will pop-up to ensure you really want to deploy these resources. The only acceptable answer to proceed is yes

So quick! You now have a webserver online. Did you notice how long it took to get the EC2 instance ready? 14 seconds. That's amazing!

How do I know that my webserver is actually working? Let us check some information about the webserver. Run terraform show

Did you notice the last entry displayed? That is the public IP address of the instance. Now, let us check the web content. Run curl http://<public_ip>:8080

Isn't that beautiful? Why not let us check out our EC2 instance on the AWS Console. Here you go!

What about our webpage? Type in your browser http://<public_ip>:8080

To clean up the resources and delete everything, run terraform destroy and enter yes as a confirmation to destroy the infrastructure.

Here is the architecture of what we built

Congratulations! We have deployed our first single web server, tested it and destroyed it. Can we improve on this? A future post will show us how. Feel free to ask questions regarding this post.

Step-by-Step Guide to Setting Up Terraform, AWS CLI, and VS Code

frankfolabi — Sat, 24 Aug 2024 03:42:51 +0000

You have heard of the benefit of Infrastructure as Code (IaC). You have seen the power, speed, and other beautiful features as well as the benefits of automation in provisioning your resources. You may now wonder, how can I setup my local machine to use this newfound tool and start deploying servers, databases, networking components, storage and the likes. This step-by-step guide would assist you in setting up Terraform, AWS CLI - which is the Command Line Interface to access your Amazon Web Services resource and VS Code - Visual Studio Code, a popular code editor. These tools can be installed on various operating system such as Linux, Mac, and Windows. Let us start with Terraform installation on Windows.

Terraform

Go to terraform.io

Click on Download Terraform

Go to the section for Windows

Click on the Download. Check if your computer is AMD64 or 386
A terraform.exe file would be downloaded in a zip file format that you have to unzip.

Go to your downloaded unzipped file

Copy the .exe file to the C drive and place it in a new folder named terraform.
Search for the Advanced System Settings.

Click on the Environment Variables

In the bottom section named System variables, scroll down to click on Path

Click on Edit
The click on New
Type in the path ex., C:\terraform
Click on OK

To check your installation, you can open your command line and type terraform version. You should see a version installed on the machine.

AWS CLI

To efficiently use the AWS CLI to communicate with AWS resources, you will need to sign up for an AWS account. You can sign up for a free account with your credit card on aws.amazon.com
After registering the account, you will have access to all resources on the AWS console. Best practice requires that you create an IAM user with administrative permissions that can be used subsequently to create other users.
Here are the steps to create an IAM admin user on the AWS Console.

Seach for IAM in the search bar of the console
Click on IAM
In the left pane under Access management, click on Users

Click on Create user

Type the user name

Note: Since we will be using the CLI, we left the box unchecked.

Click on next to set permissions. Best practice is to create a group and attach policies to that group. However, since we are dealing with a user, we will click Attach policies directly. Then click on AdministratorAccess

Review the details and click on Create user

To generate the Access Keys needed for CLI access, click on the newly created terraform-user

Click on the Security credentials tab

Scroll down to the Access keys section and click on create access key

Select Command Line Interface for your use case

Click on the confirmation then click Next

You may wish to add tags, thereafter, click on Create access key

Keep your access key safe. You may choose to download the .csv file then click Done

Next let us install the AWS CLI version 2 on our machine.

Go to the AWS documentation page at https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
Select the installation guide for Windows and follow the installer wizard
After installation, in your command prompt, you can run the aws --version

You can configure the CLI with the Access key credentials we generated. Run the command aws configure and enter the access key, secret access key, region and the default output format

VS Code

Now that we have configured our CLI, let us install VS Code.

Download Visual Studio Code at https://code.visualstudio.com/download
Follow the installer wizard instructions.
After installation and launching VS Code, click on Extensions in the left side pane and search for AWS Toolkit

Click on the Install button under the AWS Toolkit

You are now ready to use all these tools in your learning of Terraform. In the next post, we will start deploying infrastructure with Terraform.

What is Infrastructure as Code (IaC) and Why It's Transforming DevOps

frankfolabi — Wed, 21 Aug 2024 18:18:04 +0000

Infrastructure as Code (IaC) might look like a buzz word in the not so distant past but in today's DevOps world it is gradually becoming a standard. You may likely wonder, what is Infrastructure as Code and why is it transformational in DevOps? Let us take a quick look at DevOps.

DevOps is a curled from the Development team (Dev) and Operations team (Ops) in software creation. However, DevOps is said to make software delivery more efficient because it is about the people, process, tools and culture. Many companies that have adopted DevOps have reported significant improvement in their software delivery. Four core values have been identified in DevOps: Culture, Automation, Measurement, and Sharing. Infrastructure as Code is one of the numerous automation aspects of DevOps and the idea behind IaC is that you write and execute code to define, deploy, update, and destroy your infrastructure. You can manage everything in code be it servers, databases, networks, documentation and all other essentials.

Five broad categories of IaC tools are in the use mainly:

Ad Hoc Scripts: You can use your favorite scripting language such as Bash, Ruby, Python to define each step in a code and execute the script. However, the challenge is that you can write the code anyhow you love it, and it may be challenging to maintain a large repository of scripts.
Configuration Management Tools: Chef, Puppet and Ansible at tools that are designed to install and manage software on existing servers, and they are very efficient with such tasks.
Server Templating Tools: Docker, Packer and Vagrant can be found in this category. They are useful in creating images of a fully self-container that has the operating system, software, files, dependencies and other relevant details needed. These images can be virtual machine (VM) or containers which are light weight and fast.
Orchestration Tools: Kubernetes, Docker Swarm, Amazon Elastic Container Services (ECS) can be found in this category. These tools are useful in managing virtual machines and containers
Provisioning Tools: Terraform, Pulumi, CloudFormation are typical examples. These tools are used to create servers, networking components, databases and all other aspects of your infrastructures.

Why is Infrastructure as Code Beneficial?

Speed and safety: It is faster than a human clicking all around and safer since the process is automated, consistent and repeatable.
Version control: The code can be committed to Git which is easier to revert and debug.
Reusable: Modules can be created that can be a basis for other deployments.
Ease: Clicking around the same thing over and over can be boring but having these as code is just easy.

It is evident that infrastructure as code is here to stay. In future posts, we will dive deep into Terraform. Feel free you ask your questions, and I will be ready to give a response as soon as possible.