DEV Community: Cory M

Cloud Resume Challenge Summary

Cory M — Tue, 24 May 2022 22:11:14 +0000

About the challenge

This project was completed in accordance with the Cloud Resume Challenge, which was created by @forrestbrazeal.

Certification

The first requirement of this challenge is to be AWS Cloud Practitioner certified, which I achieved in April 2022. After completing this certification, I began studying for the AWS Certified Solutions Architect which led to me discovering this challenge.

AWS Cloud Practitioner

Links to GitHub Repositories

Backend
Frontend

1 - Website Architecture

The website architecture is designed as follows:

A user requests the webpage via their browser.
The browser requests the IP from Route 53, which then resolves the DNS to the nearest CloudFront edge location.
CloudFront then forwards the request to the S3 bucket containing the website static files, and retrieves its contents.
The S3 bucket containing the frontend code is protected with Origin Access Identity (OAI), preventing access not routed through CloudFront.
The website runs JavaScript code which sends a GET request to API Gateway, to retrieve the current number of visitors stored in the DynamoDB database.
API Gateway forwards this request to lambda as JSON.
Lambda then identifies the type of request, performs a get/put operation on DynamoDB to retrieve and incrementally increase the number of visitors stored.
The returned visitor count is then processed and displayed on the website.
GitHub repositories of both frontend and backend data provide code version control, and CI/CD through GitHub Actions.
Terraform deploys the AWS infrastructure as detailed in the backend repository.

2 - Why Use Serverless?

Before I can explain why serverless was used, it is important to explain what serverless is. Serverless is not a new concept in computing, essentially how bygone mainframes and terminals worked. Its like that jacket you still have from high school, its so old that it has come back into fashion. Serverless computing is a development model in which code is executed on servers that have been abstracted from the user, meaning that the management and operation of the servers is external to the developers and users. An analogy would be a copy center: you don't own the printers, copiers, or fax machines, you simply pay for what you use in order to finish your task.

The reasons to use serverless are a combination of cost, scalability, availability, and performance. As the resources to execute code is not provisioned prior to use, the cost is only for the compute time that is actually used. Amazon manages the provision, operation, and scaling to execute workload and meet demand. In my case, the custom domain is an annual cost of $12, Route 53 is $0.50 monthly, KMS is $0.29 monthly, Lambda and DynamoDB are very far below usage levels that would incur fees.

3 - Infrastructure as Code

Infrastructure as Code is the use of definition files to provision and manage AWS infrastructure, instead of manually using the console or CLI to build and configure the needed infrastructure. I chose to use Terraform rather than AWS SAM tool, as Terraform is cross-platform and allows me to integrate additional technologies such as Kubernetes and virtual machines.

Before getting started it is important to take care of the following aspects:

Security: Terraform needs to have permission to deploy infrastructure on AWS. A user was created with access only to STS (Secure Token Service). A role was created with the IAM permissions needed to perform actions on the required AWS services.
Terraform State: A remote backend was configured using a S3 bucket to store the Terraform state file and DynamoDB was used to store the state lock. This configuration ensures that the infrastructure declared in .tf files, and the infrastructure deployed match.

The following AWS infrastructure was deployed via Terraform:

A private S3 bucket hosting the website frontend code.
A new table was created in DynamoDB with on-demand capacity and a primary key ID. A default item was created to store a value for the number of website visitors.
A Lambda function using Python runtime, the code of which was uploaded from GitHub.
An API Gateway configured as Lambda integration, which sends GET requests from CORS-compliant source to Lambda as JSON, and Lambda responds with JSON.

4 - Source Control

I chose to use GitHub to maintain control of the frontend and backend repositories. GitHub allows all additions, revisions, and deployments to be tracked and reviewed.

5 - Continuous Integration/Continuous Delivery

Continuous Integration/Continuous Delivery (CI/CD) is the method of automating and continually monitoring throughout the lifecycle of application, from integration and testing to delivery. To achieve this, I used GitHub Actions, which defines the process with YAML files. Any changes to the GitHub repositories triggers a new commit, which will deploy or modify the existing design.

6 - Lambda Function in Python

The Lambda function was written in Python code. This code uses the Boto3 SDK to interact with DynamoDB, specifically to read the visitor count table, and to incrementally increase the count value. In order for Lambda to have access to DynamoDB, IAM role permissions needed to be assigned. To secure access to the Lambda function, Cross-Origin Resource Sharing (CORS) requires that the source of GET must be the website. While I had some initial difficulties getting CORS Access-Control-Allow-Origin headers to function correctly, using * was not an option as it creates a large security risk.

7 - Building the Frontend

The frontend code was built using HTML, CSS, and JavaScript. The files are contained in a S3 bucket, which were deployed from the GitHub repository.

8 - Creating a Test

To test the functionality of the API, I wrote a Cypress test. This test sends the API a GET request, and evaluates the response type and to have a 200 status code.

9 - Conclusion

I found this challenge to be a great experience. There were many opportunities to learn the interactions between different AWS services, how to describe my design to Terraform, and how to implement CORS. I am deeply thankful to @forrestbrazeal for creating this wonderful challenge. I believe that this challenge has definitely improved my understanding of a myriad of AWS services, and given me direct experience in using Terraform and CI/CD.

Cloud Resume Challenge - Part 3

Cory M — Tue, 24 May 2022 05:23:12 +0000

I completed writing the backend as IaC in Terraform. As this was the first complicated project I have done in Terraform, I used an incremental approach to build each component and testing before expanding scope. While most of the essential parts, such as S3 buckets, IAM permissions, Route 53, and CloudFront were quite straightforward; there were a few issues that required a lot of research to fix, primarily due to cryptic error messages. For example, in the API Gateway Integration, I would receive an "BadRequestException: Enumeration value for HttpMethod must be non-empty" error. After digging through documentation and many searches on Stack Overflow, I discovered that this was due to not having used the integration_http_method argument, which the documentation identified as being optional, in addition to the required http_method argument.

After the Terraform code was competed and tested, the next step was to learn how to migrate the activation and distribution to GitHub Actions. This went fairly smoothly, as I had already learned the basics for the frontend code. I had a slight misstep in initializing tfvars, and in forgetting to remove reference to AWS credentials that had been stored locally during testing.

Once the backend had been proven as functional from GitHub Actions, the last step was for me to write a test. I had seen multiple references to a testing framework called Cypress, which interested me as an end-to-end tool for testing web applications. I used Cypress to request a GET on my API, then validate the response as 200 status and a number. This test was then integrated into my GitHub Action workflow to be run from a Docker image after Terraform apply has completed.

Cloud Resume Challenge - Part 2

Cory M — Thu, 19 May 2022 23:19:47 +0000

I solved the issue with the visitor counter not updating, it was an error with the CORS configuration. I had initially believed that my configuration of website calling to the CloudFront distribution would pass the correct headers; however I discovered that the API Gateway was not configured to correctly pass the header mappings through the Integration Response. Once that was corrected, the data was passed from the DynamoDB table without issue. Unfortunately, while I was receiving the data correctly, my JavaScript function would not result in any response other than null. After pouring through documentation and searching Stack Overflow without a solution that I could get working, I decided to abandon the use of fetch/response and instead implement XMLHttpRequest. I will need to learn more about JavaScript before I reiterate the function in the future.

I implemented the frontend code into GitHub, and began learning how to configure Actions to automate the CI/CD of any future changes. This led to learning how to set up secrets, and automate CloudFront invalidations through the GitHub Actions YAML configuration.

More work continues to write the scope of backend infrastructure into Terraform code.

Cloud Resume Challenge

Cory M — Tue, 17 May 2022 23:40:59 +0000

I have been working in Operational Technology (industrial control systems in power plants and oil refineries) for about 15 years, doing everything from initial construction and commissioning, to maintenance and modernization design reviews. I recently decided to transition to information security, and through studying and implementing projects, I fell in love with cloud services. This discovery pushed me to begin studying AWS services, which led me to find Cloud Resume Challenge.

Background

The project is a challenge that was created by Forrest Brazeal, who works in DevOps and cloud engineering. The challenge is to build a build a simple, static website of their resume using a multitude of cloud services. As the challenge allows you to choose your preferred cloud provider, I decided to used AWS because I already had some experience using it already, and had recently completed the AWS Certified Cloud Practitioner exam.

As I had chosen to use AWS, I began to lay out the infrastructure that would be needed to complete the project. S3 would host the static website, Route 53 and CloudFront would manage DNS requirements and secure website distribution. Creating a visitor counter would involve JavaScript calling API Gateway, triggering a Python Lambda function, which would read and append a DynamoDB table, and the necessary Roles would be managed with IAM. Eventually, the infrastructure would need to be written as "Infrastructure as Code" or IaC.

Implementation

In my experience doing AWS labs, I had used some individual parts of AWS, but not to the level that this project would demand. Because of this, I decided to start by manually setting up a S3 static website, Route 53, and CloudFront distribution. While everything went fine, I learned that any change of code necessitated an invalidation of the CloudFront cache to propagate the new code.

I am currently working on an error displaying the visitor counter updating correctly. I also have progress to make in the IaC front, I have been working in Terraform and have many bugs to work out. More to come soon.