DEV Community: Jon Batista

How to Build a Simple AWS Test Environment with Terraform

Jon Batista — Wed, 08 Jan 2025 23:22:00 +0000

As a developer, sometimes you need to create a real environment to test your stuff. It can take a quite amount of your precious time. So if you want to create a cost-effective, manageable test environment, Terraform is the best choice. This simple article walks you through creating this kind of test environment on AWS that you can quickly launch or destroy, ensuring you avoid unexpected costs.

Think of it as a starting point and add the things you need to customize your environment. Use an LLM to help you create your own Terraform template. I’m sure it will save a lot of time.

Prerequisites

This is a straight-to-the-point tutorial, but before diving in, ensure you have:

An AWS account.
Terraform installed on your machine.
Basic understanding of Terraform and AWS concepts like VPC, subnets, and security groups.

If you don't know about these things, no worries. It's a simple configuration that can help you in your studies.

1. Provider Configuration

The first step is to create a Terraform file called main.tf. Then, configure the AWS provider to specify the region where your resources will be created:

provider "aws" {
  region = "us-east-1"
}

Here, I'm using the us-east-1 region, but feel free to choose your preferred AWS region.

2. Variables for Flexibility

To make the setup flexible and reusable, we define variables for key configurations:

variable "vpc_cidr" {
  default = "10.0.0.0/16"
}

variable "public_subnet_cidr" {
  default = "10.0.1.0/24"
}

variable "private_subnet_cidr" {
  default = "10.0.2.0/24"
}

variable "instance_type" {
  default = "t2.micro"
}

variable "ami" {
  description = "Amazon Linux 2 AMI"
  default     = "ami-0c02fb55956c7d316"
}

Using variables allows you to tweak the CIDR blocks, instance types, and AMI IDs without modifying the main configuration.

3. VPC Setup

The VPC (Virtual Private Cloud) is the backbone of your infrastructure:

resource "aws_vpc" "test_env_vpc" {
  cidr_block           = var.vpc_cidr
  enable_dns_support   = true
  enable_dns_hostnames = true
  tags = {
    Name = "test_env_vpc"
  }
}

This creates a VPC with DNS support and hostnames enabled, which are essential for networking.

4. Subnets: Public and Private

We create both public and private subnets within the VPC:

resource "aws_subnet" "public" {
  vpc_id                  = aws_vpc.test_env_vpc.id
  cidr_block              = var.public_subnet_cidr
  map_public_ip_on_launch = true
  availability_zone       = "us-east-1a"
  tags = {
    Name = "test_env_public_subnet"
  }
}

resource "aws_subnet" "private" {
  vpc_id            = aws_vpc.test_env_vpc.id
  cidr_block        = var.private_subnet_cidr
  availability_zone = "us-east-1a"
  tags = {
    Name = "test_env_private_subnet"
  }
}

The public subnet assigns public IPs for internet-facing instances, while the private subnet is reserved for internal resources.

5. Internet Gateway and Routing

To enable internet connectivity for resources in the public subnet:

resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.test_env_vpc.id
  tags = {
    Name = "test_env_igw"
  }
}

resource "aws_route_table" "public" {
  vpc_id = aws_vpc.test_env_vpc.id
  tags = {
    Name = "test_env_public_rt"
  }
}

resource "aws_route" "public_route" {
  route_table_id         = aws_route_table.public.id
  destination_cidr_block = "0.0.0.0/0"
  gateway_id             = aws_internet_gateway.igw.id
}

resource "aws_route_table_association" "public" {
  subnet_id      = aws_subnet.public.id
  route_table_id = aws_route_table.public.id
}

This setup routes public traffic through the internet gateway.

7. Instance Deployment

Launch a public EC2 instance with Terraform:

resource "aws_instance" "public_instance" {
  ami           = var.ami
  instance_type = var.instance_type
  subnet_id     = aws_subnet.public.id
  associate_public_ip_address = true
  vpc_security_group_ids = [aws_security_group.public_sg.id]

  tags = {
    Name = "test_env_public_instance"
  }
}

This creates a t2.micro instance running Amazon Linux 2 with public internet access.

8. Outputs

To view key information, we define outputs:

output "vpc_id" {
  value = aws_vpc.test_env_vpc.id
}

output "public_instance_public_ip" {
  value = aws_instance.public_instance.public_ip
}

These outputs make it easy to reference your resources post-deployment.

Wrapping Up

With this configuration, you have a simple, flexible AWS test environment that can be launched or destroyed in minutes. To launch it, run:

terraform init
terraform apply

And to tear it down:

terraform destroy

By managing your infrastructure with Terraform, you can experiment confidently without worrying about unexpected costs.

I hope this guide can help you!

How to integrate AWS CodeBuild with Jira

Jon Batista — Fri, 21 Jul 2023 15:01:50 +0000

Introduction

This article will guide you to configure an automation flow that will update your Jira board to the right status. To implement this I assume you already have an AWS account and a pipeline deploy using CodeBuild in place. So you will be able to plug it into your code as the last part of it.

This piece of code will be added to the buildspec.yml, inside the post-build block. It means when the artifact is in fact deployed to the environment, then the automation will update the stories correctly.

For reference, check the buildspec.yml file documentation.

Creating Lambda Function

To create a new Lambda function you new to go to Lambda's service:

Then, when creating the function, choose the python3.9 runtime:

Once the new Lambda's Python3.9 runtime does not support some dependencies we need to make it work, we will need to upload our package in a zip file. You can do this here:

So you can use docker or venv to zip all the dependencies with the python file.

I'm using docker so the dependencies are in this path /usr/local/lib/python3.12/site-packages

Make sure the name of the lambda code file is lambda_function.py and it is at the same level as the dependencies so that AWS can validate it.

The Flow

For instance, I'm using an SCM based on Gitflow, so I have the develop branch -- in which we will apply this -- and the master branch. This way allows me to compare those issues the automation should update the status.

By the way, another important thing is all of the commits should begin with the issue's key. Like this:

git commit -m "AE-28: <your-commit-message>"

CodeBuild

Every time the deploy is triggered manually, this snippet will be started as a post-build action.

Then it will verify on the remote repository if there were tickets deployed in the given job:

- TICKETS_LIST=$(git log origin/develop --no-merges --pretty=format:"%B" --not "origin/master" | tr a-z A-Z | tr ' ,' '\n' | grep -o '[A-Z]\+-[0-9]\+' | sort --version-sort | uniq | tr '\n' ',')

The result of this command is:

AE-28,AE-27,AE-30,AE-34,AE-35,

After that, if the command's output is not null, CodeBuild will send it to the Environment Variables on Lambda, and then, send the tickets list to the Environment Variables on Lambda, and finally triggers the jira_automation_tickets function passing the status 3 which is the transition ID to move issues to the QA lane:

- |
   if [ "$TICKETS_LIST" = "" ]; then 
       echo "No tickets deployed";
   else 
       aws lambda update-function-configuration --function-name jira_automation_tickets --environment "Variables={TICKETS_LIST='${TICKETS_LIST}'}"; 
       aws lambda invoke   \
       --function-name jira_automation_tickets  \
           --cli-binary-format raw-in-base64-out  \
               --payload '{"status": "3"}' output.txt; 
   fi

- cat output.txt

Tip: You can see that Environment Variable, on the Lambda Service Dashboard:

Configuration > Environment Variable

Tip: To know what the transition id, use this command:

curl -u <jira-user>:<token-api> -X GET -H "Content-Type: application/json" "https://<your-domain-jira-board>.atlassian.net/rest/api/3/issue/<issue-number>/transitions" | jq '.[]'

You will receive an array containing objects with all transitions' id as well as all status' id, like this one for QA which the transition id is 3 and the status id is 10004 -- I will use it here:

{
    "id": "3",
    "name": "QA",
    "to": {
      "self": "https://<your-domain-jira-board>/rest/api/3/status/10004",
      "description": "",
      "iconUrl": "https://<your-domain-jira-board>/",
      "name": "QA",
      "id": "10004",
      "statusCategory": {
        "self": "https://<your-domain-jira-board>/rest/api/3/statuscategory/4",
        "id": 4,
        "key": "indeterminate",
        "colorName": "yellow",
        "name": "In Progress"
      }
    },
    "hasScreen": false,
    "isGlobal": true,
    "isInitial": false,
    "isAvailable": true,
    "isConditional": false,
    "isLooped": false
  }

Lambda

We are using Python 3.9 as the runtime for this Lambda function. To use Jira SDK we needed to import it and its dependencies with the Lambda code as a zip file.

The code is simple is a simple one and it's commented line-by-line. This code will be committed to our repository to maintain a history of it. And the zip file with the necessary dependencies is attached as well.

First of all, create the file lambda_function.py -- it should have this name to avoid AWS errors -- then, import all dependencies we will need:

import boto3
import json
import os
import re
from jira import JIRA
from datetime import date
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText

The main method must have to be named lambda_handler. So create this method and initialize the credentials variables, then use the SDK to authenticate:

def lambda_handler(event, context):
    # set credentials using basic authentication
    jira_url = 'https://<your-domain-jira-board>.atlassian.net'
    jira_username = '<user-name>'
    jira_api_token = '<api-token>'

    # Connect to Jira
    jira = JIRA(server=jira_url, basic_auth=(jira_username, jira_api_token))

Now use the os to get the tickets list from the environment variables:

    # Get ticket list sent by CodeBuild and transform it in an Array/list
    ticket_list = re.split(r"\r\,|\,|\r", os.environ['TICKETS_LIST'])

Then, you can start to create the email template and initialize an empty list to control whether you should send an email or not.

    # Starts the email template
    email_body = """<p>List of tickets to be tested:</p>"""
    # Initiate an empty list/array to check its size later when it will send the email notification
    tickets_to_send_email = []

Now it's time to iterate that list and validate them. My criteria here was only to update the tickets that are not in the QA lane yet. In my case, the QA status id is 10004. It will increment a line in the email body.

    for ticket in ticket_list:
        if ticket != '':
            # instantiate Issue's object 
            issue = jira.issue(ticket)
            # get the current status
            current_status = issue.fields.status
            # 10004 is the id of the QA lane 
            if current_status.id != '10004':
                # increments the array to send notifications
                tickets_to_send_email.append(ticket)
                # include a issue link into the email template 
                email_body += f"""
                    <div>
                        <a href="{jira_url+'/browse/'+issue.key}">{issue.key} - {issue.fields.summary}</a>
                    </div>
                """
                # Change the status to status 3 -- QA
                jira.transition_issue(issue, event["status"])

Then, close the email body:


    # closing the email template
    email_body += """<p>If you find an issue with the ticket reach out to the reporter.</p>
    """

After validating and updating the status of the issues on Jira, set a variable called response_email to receive the funtion's response. Also, you will need to format this email with the correct MIME Type, to send it through the SES service:

    # setting default response to Lambda function
    response_email = "Email not sent"
    # checking if the array is populate to send email notification
    if len(tickets_to_send_email) > 0:
        # Compose the email subject and body
        subject = f'Tickets Available To Test In QA - {date.today()}'
        body = email_body
        # Create the email message
        email_message = MIMEMultipart()
        email_message['Subject'] = subject
        # Attach the HTML part
        html_part = MIMEText(body, 'html')
        email_message.attach(html_part)
        # Calling AWS SDK to send email
        ses_client = boto3.client('ses')
        response_email = ses_client.send_raw_email(
                Source='<authenticates-ses-email>',
                Destinations=['<email-1>, <email-2>'],
                RawMessage={'Data': email_message.as_string()}
            )

Finally, close your code returning a JSON with the send_raw_email method:


    return {
        'statusCode': 200,
        'body': json.dumps(response_email)
    }

Jira Board

Your board will be updated and those tickets that were merged in your development environment and are not in the QA lane already will be moved to this lane. This way the automation will assure the correct visibility.

Simple Email Service

Once the Lambda code was finished and there are changes to notify, it will trigger the SES (Simple Email Service) to send an email notification to the emails you passed as the destination.

The email will contain instructions to QAs to access the ticket by the links that will be listed on the email body, like this:

List of tickets to be tested:

AE-28: 'issue-title'
AE-27: 'issue-title'
AE-30: 'issue-title'
AE-34: 'issue-title'
AE-35: 'issue-title'

If you find an issue with the ticket reach out to the reporter.

Conclusion

Make sure your CodeBuild job has permission to write in Lambda:

As well as the Lambda has permission to trigger the SES service:

I'd be very happy if this article helps you anyhow. Feel free to collaborate with this solution. I'd like to see it getting as better as possible!