Deploying container to list EC2 instances in a region + Infra creation on AWS using terraform + Using free domain for testing

In the post I will explain how to create a docker container which lists all EC2 instance details in a region and how to deploy it by creating a basic application-load-balancer and auto-scaling-group setup along with registering your own free domain and using it in Route53 for testing purpose.

I have split the section into three different steps :

1. Building a basic application using python and boto3 to get desired response and exposing it on port 8000 and creating a docker container.

2. Getting a free domain and registering it on AWS account as hosted zone.

3. Infra creation using Terraform ( Route53 + ALB + ASG + Launch Configuration).

The source code for the application - https://github.com/MohammedSadiq11/python-listEC2-instances

First, create a app.py file and add flask and jsonify and map /ec2 path to an function.

from flask import Flask, jsonify

app = Flask(__name__)


@app.route('/ec2')
# ‘/’ URL is bound with hello_world() function.
def hello_world():

Then import boto3 and use it to connect to aws account by calling the client method and passing aws creds, region and the type of aws resource(which is ec2 in our case) as paramater.

import boto3

ec2client = boto3.client(
        'ec2',
        aws_access_key_id='<access-key>',
        aws_secret_access_key='<secret-key>',
        region_name='us-west-2'
    )

You can later edit the response by iteration through it however you want to display.

In my case I have used python list and dictionaries to map instance ID as key and other attributes as value in the dictionary.I have exposed the application on port 8000.

ec2detailsDict=dict()

    response = ec2client.describe_instances()

    for reservation in response["Reservations"]:
        for instance in reservation["Instances"]:
            print(instance)
            ec2detailsList=list()
            ec2detailsList.append(instance["InstanceId"])
            ec2detailsList.append(instance["PrivateIpAddress"])
            ec2detailsList.append(instance["Tags"])
            ec2detailsDict[instance["InstanceId"]]=[]
            ec2detailsDict[instance["InstanceId"]].extend(ec2detailsList)
    return jsonify(ec2detailsDict)


if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8000)

Next step is to dockerize the application and push it to an container registry. I have used docker hub for simplicity.

Write a basic docker file which uses a low size trimmed down version of ubuntu with python and install the required dependencies using requirements.txt file(these dependencies have been added using trail and error after building and running docker images and checking for logs for termination)

FROM python:3.8-slim-buster

WORKDIR /python-docker

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

COPY . .

CMD [ "python3", "-m" , "flask", "run", "--host=0.0.0.0", "--port=8000"]

Run the following commands to build and image and run it

sudo docker build -t image-name:tag .

sudo docker run -it -p 8000:8000 image-name:tag

You can use -d parameter to run the container in background and do sudo docker ps to check it's status.

After running the container you can hit the endpoint(GET call) which you get from the logs to check if the API is working

After verification login to your docker repo and push the image to the repo
Make sure the repo is public because we will be using it later in a script and we should be able to pull it without doing docker login

Now that we have an application which has an API which returns ec2 instance details, lets move on to infra creation.
But first let's buy a free domain and register it on the AWS account

We are going to use freenom.com to buy a free domain

1. Login to freenom.com using google creds(or signup)
2. Check for availability of your desired domain
3. If it available then add few personal details and buy it

This article explains the process in a very simple way - https://www.thoughtco.com/get-free-domain-name-4693744

Make sure that you do not enter any records on the website for the domain you buy,as we are going to do that in AWS Route 53

After buying the domain we have to register it as a hosted zone in AWS account where we are going to create infra as we will be using the domain in our terraform code for our application

1. Sign in to the AWS Management Console and open the Route 53 console

2. Select hosted zones from the left panel

3. Click on Create hosted zones

4. Add the domain name which you have created in the previous step and make sure that it is a public hosted zone.

Now, lets move to terraform and check our IAC code. Complete source code for Route53 + ALB + ASG + EC2 Launch configuration is present in this repo - https://github.com/MohammedSadiq11/ALB-ASG-EC2

The pem file which I have used for my configuration is kops-master.pem which is removed from the private-key folder in the repo for security reasons.
You can add your own pem file by downloading it from the AWS Key Pair.
https://docs.aws.amazon.com/cli/latest/userguide/cli-services-ec2-keypairs.html

Remember to replace the key references in the terraform files

There is an shell script called app1-install.sh on the repo which installs the docker application and exposes it on port 8000 on the EC2 machine during it's launch by Auto Scaling Group

sudo yum update -y 
sudo yum install docker -y 
sudo systemctl enable docker.service
sudo systemctl start docker.service
sudo docker pull mohammedsadiq/smallcase:v8
sudo docker run -d -p 8000:8000 mohammedsadiq/smallcase:v8

Replace the image name and tag with your public docker image of the application which you have pushed in the first step
Also, replace your domain name in the file c12-route53-dnsregistration.tf and c6-02-datasource-route53-zone.tf files.
I have used terraform version 0.14 and the module versions compatible with it.

Modules used with versions :-

• VPC - terraform-aws-modules/vpc/aws - 3.0.0
• Security Group - terraform-aws-modules/security-group/aws - 4.0.0
• EC2 - terraform-aws-modules/ec2-instance/aws - 2.17.0
• ALB - terraform-aws-modules/alb/aws - 6.0.0
• ASG - terraform-aws-modules/autoscaling/aws - 4.1.0

The configuration on repo will create a VPC with two public subnets and a private subnet in two availability zones*(us-west-2a, us-west-2b).*

It will create a bastion host and private ec2 instance which will have serve the application. The EC2 instance can be accessed through the bastion host. Private key to access the file will already be present in /tmp directory inside bastion host.

It will create a record in the Route53 which will be directed to the DNS of Application Load Balancer.

The load balancer is present in present in public subnet, but the ASG will spin up EC2 instance in private subnet only.

The application load balancer has target group healthcheck enabled on port 8000 and path /ec2(same path which returns list of EC2 instances).

The auto scaling group is associated to application load balancer through target group arns.

Run these commands after modifying the code according to your aws profile and region

terraform init - This will download all required modules and providers.
terraform validate - This will check the syntax of the files.
terraform plan - It will give an preview of all the resources which will be created. Type yes in prompt only after veriifying.
terraform apply - Wait for some time after running this command as it will take few minutes to create entire setup.

Verify the setup manually on AWS account after the last command is successful.Go to browser and hit the http://prefix.domainname.tk/ec2 path and check if it is giving proper response.

After verification terraform destroy should be used to delete all the resources created.

This constitutes our guide on containerising an application and deploying it on AWS with a domain name with autoscaling and load balancing.

Godspeed
Mohammed Sadiq.