DEV Community: andreapeterson

From Aspiring to Achieving: My Journey Conquering the AWS Resume Challenge

andreapeterson — Fri, 22 Dec 2023 04:02:14 +0000

Hello there! I'm absolutely thrilled to share my proudest project to date – my take on the Cloud Resume Challenge by Forrest Brazeal. Before we dive into my journey, let me introduce myself. I'm Andrea, residing in the greater Palm Bay-Melbourne area of Florida. I graduated from the University of Florida in May 2023 (Go Gators!) with a Bachelor of Science in Animal Sciences and a minor in Economics.

Now, you may be wondering how someone with a background in animal science has since found their way into the tech world. To meet the requirements for my minor in Economics, I enrolled in the only class that fit into my last semester schedule- 'Economic Analysis of Data'. This course, centered around R programming, introduced me to the world of coding. Since then, my curiosity has led me down a rabbit hole, guiding me on a path to transition into the IT industry.

Surprisingly, this year has seen me not only breeding my first horse for a class, but also constructing CI/CD pipelines for my website! The former is a story for another time... let's dig into the latter!

The Objective

The goal of this project is to create a dynamic resume website with a sleek front end, accompanied by a visitor counter on the back end. The visitor counter should be displayed on the website. It incorporates Infrastructure as Code (IaC) for the backend services, as well as CI/CD pipelines for both backend and frontend source code, automatically pushing changes to AWS. This project mimics real-world scenarios, providing an opportunity to delve deep into AWS services. More information can be found here.

My Take

One of my favorite aspects of this project is its versatility and the numerous avenues for customization. I decided to go the extra mile by putting not only the backend services into Infrastructure as Code but also the frontend resources, all within Terraform. Initially, I completed the project using clickops, then went back and migrated everything into Terraform. I completed this project over multiple months, aligning it with my studies. Below, I will briefly go over how I tackled each portion.

The HTML/CSS part

Before ever learning about the Cloud Resume Challenge, I initially embarked on the 100 Days of Code: The Complete Python Pro Bootcamp for 2023 by Dr. Angela Yu. I accumulated several Python projects that I wanted to showcase, so I started an HTML+CSS portfolio website.

Admittedly, at the time, HTML and CSS were alien concepts to me as I was just beginning Python. So, in my eagerness, I used a free template by ThemeFisher, tweaking it to fit my vision. At this point, before even beginning, I have my portfolio website complete.

The S3, Route53, CloudFront, & HTTPS part

After earning my AWS Certified Cloud Practitioner certification, I delved into Adrian Cantrill's AWS Certified Solutions Architect - Associate (SAA-C03) course. During it, I also started looking for more AWS-focused projects to further hone my skills, and that's when the Cloud Resume Challenge caught my eye.

This project seamlessly complemented my ongoing studies. Thanks to Adrian's course, I already had my portfolio website hooked up to a domain name via Route53, distributed through CloudFront, secure with HTTPS through a certificate in ACM, and my files stored safely in S3. Onto the backend!

The Lambda, DynamoDB, and JS part

Next, the challenge led me to the DynamoDB and Lambda portion. With limited experience in Lambda beyond the simple 'hello world', I decided to delve into the Lambda console. I even wrote a tutorial for sending automated emails through Lambda.

Also wanting to dive deeper into Boto3, I subsequently took AWS - Mastering Boto3 & Lambda Functions Using Python by Hari Kammana on Udemy. Then, I felt extremely confident and conquered this portion of the challenge. The Lambda code took shape, enriched with atomic counters, seamlessly connecting with the DynamoDB table.

Finally, I updated my website code to include Javascript to display the view count. Choosing Lambda's function URL over the conventional API gateway, I linked it to my homepage. Now, each load on my home page triggers Lambda, instructing the DynamoDB table to increment by 1, and the freshly minted view count is shown on the home page.

The Terraform and Github Actions

Finally, onto the last portion- the IaC and CI/CD chunk. I chose Terraform over CloudFormation due to its versatility across cloud platforms. Like most of this project, I started slow with Terraform's basics just to learn the lay of the land, initially creating an EC2 instance and gradually incorporating more features, rather than jumping right into incorporating Terraform into my project.

After I felt comfortable, I transitioned my backend services—Lambda and DynamoDB—into Terraform. Then, added my frontend source code to a Github repository with Github Actions, ensuring each commit to AWS. Yet, as I reached what I believed to be the final step(adding my Terraform code to a Github repository+Actions), a realization struck. Only half of my project is automated, which is a vulnerability in disaster recovery scenarios. Moreover, the dependency of my frontend JavaScript file on the backend Lambda function URL presented a manual intervention hurdle.

Choosing to face the challenge head-on, I made the decision to encapsulate everything within Terraform, resolving the dynamic function URL dilemma. The manual touch persisted only in crucial areas—my domain name within Route53, the certificate in ACM, and the reserved S3 bucket name mirroring my domain name. I fetched these items dynamically in my Terraform file using data blocks.

Finally, I added these files into a Github repository and set up Github Actions, triggering Terraform apply with each push to the main branch. The state file is stored remotely in S3.

Behold the culmination of this grand endeavor in all its glory: https://andrea-peterson.com

Conclusion

In conclusion, conquering the Cloud Resume Challenge proved to be an immensely rewarding experience defined by persistent problem-solving. The tech industry's vast support network, encompassing abundant resources like StackOverflow, YouTube tutorials, and insightful blog posts, played a pivotal role in navigating and resolving the challenges encountered. While I'd love to go into depth over every little problem encountered, the truth is I'd be writing this well into the greater part of 2024 with the amount of problems that occurred. Nonetheless, I did keep a Google Doc serving as a repository for the challenges that kept me awake for some nights—a testament to the learning process.

Overall, Terraform was my favorite aspect of this project and the area where I witnessed the most personal growth. Coming into this challenge, my knowledge of Terraform was limited to its benefits; however, by the project's end, I had completed the entire challenge within it. This underscores the transformative power of hands-on learning.

Future Plans

Looking ahead, I plan to expand on this project. Currently, I am gearing up to take the AWS Solutions Architect Associate exam and am also exploring Docker, Kubernetes, and more Linux. The road ahead promises new challenges and discoveries, and I am eager to embrace them with the same enthusiasm that fueled this Cloud Resume Challenge conquest.

Send Automated Emails (smtplib & Python) Through Lambda

andreapeterson — Fri, 18 Aug 2023 20:14:51 +0000

Hello there! I am so happy you're here and found my blog. I am new to the tech world and this is my very first project in AWS. It is a lovely beginner project especially if you have some Python skills up your sleeve, if not that's okay I will be sharing my code! This project took me through a whirlwind from CRON expressions to environmental variables- which are all new to me- and I am here to show you how I used them! I chose to set up an 8 a.m. email every Monday with some Monday Motivational Quotes. I've heard Lambda is an awesome, highly-used tool so what better way to learn it than to dive right in with a project? Lambda has a very generous free tier (1 million requests per month) and is relatively inexpensive so it is a good platform to play around with and discover the wonders of AWS.

I will admit this project took me longer than I would like to admit and so I would like to get this disclaimer out of the way:

This project, while run through Lambda, WILL NOT send emails to popular mail providers

I learned this the hard way. I learned that because of previous spam abuse sent from EC2 instances, popular mail providers block emails coming from EC2 instances. However, AWS offers Simple Email Service to fix this issue, which is a fully managed service. Before learning this, I spent hours (and hours) of fixing my code, researching, watching youtube videos, reading the entire lambda documentation (ouch), reading through StackOverflows, and starting all over a few times, but I could not figure out why my code wasn't sending the emails even though it was running the tests perfectly. It was my first time on Lambda, so I didn't know if it was something in my code or a setting messed up or what: but I finally came across this blog @ https://ivhani.medium.com/sending-an-email-with-aws-lamda-function-python-c4533aabf4af
and Ivhani Maselesele explained this problem perfectly. So, I switched the receiving email in my code to my aol account and ran the test again and heard the ping of joy, the email was finally in my inbox. Thank you Ivhani!!
If you are confused, don't worry I will explain what I am talking about when we get to that part. So, with that out of the way.. let's get started!

1. Setting Up the Main Code

I have been following along with Udemy's course "100 Days of Code: The Complete Python Pro Bootcamp for 2023" taught by Dr. Angela Yu. I originally got this course to supplement the "Python For Everybody" taught by Charles R. Severance (a really great, free course taught by an amazing instructor). I was only going to do a few days of the 100 Days of Code course, however I have been going consistently and am on day 32 and LOVE IT. It makes coding so fun and it is truly rewarding to see what you can create. Dr. Angela is amazing!! (DISCLAIMER: if you are interested in this course, they have REALLY great sales on Udemy and you shouldn't have to pay full price for anything)

With that being said, I got the Monday Motivation project idea from Dr. Angela. I challenge myself everyday by writing the code by myself and struggle through it(even if it takes more than 1 day) and only watch the videos for help, but the "quotes.txt" file is from Dr.Angela.

Below is my code before doing anything in AWS Lambda. This will send you an email with a quote from a file called "quotes.txt" if the day is equal to 1 (it is Tuesday, see 1e for notes about this).

Noteworthy mentions:

a. I put "XXXXX" where it was my sensitive information and you should add your own sending email and receiving email should you wish to do this project.

b. It is very important you recognize the password on line 6 is not your password to get into your email(most likely won't work), it is an APP password. This is because by default Gmail and other popular providers and picky about who can access your account, so if you want Python to send emails through your email account you have to set up an app password. I used a Gmail account on line 5 that I made just for this project, and I will show you how to set up an app password through Gmail. You can do this with other emails, it just might look a little different.

At the top right, click your picture/ icon then click "Manage Your Google Account"

Then, click on Security then 2-Step Verification. Once you set up your 2-step Verification, click on 2-Step Verification again and scroll down until you see "App Passwords"

Click on App Passwords.

For Select App, choose other

Then name it anything you want, I chose MondayMotivation. Then click generate, and your app password should pop up. Copy this and insert this into line 6 of code.

c. On line 16, where I have "smtp.gmail.com", if you are using another email provider for your sender email (line 5), just google the smtp for that email provider and insert it there. (Just a funny mention- I was confused at first and thought the smtp was for the RECEIVING email so I set up all these for and in statements for another project with different email recipients and it took me hours figuring out why only Gmail was working and if @aol and @me etc didn't work for this project, I switched the port and added smtp_ssl too and realized... the 'smtp.gmail.com' is for the SENDING email aka I don't need to change it for each receiving email... facepalm)

d. This code as well as the final ending code is on my Github (link at bottom of page)

e. This is also important- I wrote the code assuming the code can run every day at 8 a.m. but will only send an email if it is Monday. That is why you see : if day_of_week == 1: ...
For reference:
0= Monday
6= Sunday
I tested this on different days, so you will see different numbers throughout this blog. Be sure whatever day of the week it is when you're doing this, you put it so you can test it and have that statement be true so it will send an email. I'll remind you at the end to change it back to Monday when we are all done.

f. The disclaimer mentioned at the beginning has to do with receiving emails once we get to Lambda. Through VSCode or Pycharm etc, you should be able to send to any email.

h. Although not necessary, I recommend running this code in your favorite IDE just to make sure your passwords and emails are working.

2.Setting Up Lambda

a. Go to the AWS Lambda page and click create function.

I chose "Author From Scratch", a function name of your choosing, then be sure to select Python(I chose the latest right now which is 3.10), and I left the architecture as x86_64 and clicked create function.

b. Now, your screen should look like below if you click on the lamdba_function.py.

This file is important as we will be incorporating it into our MondayMotivation.py file, as seen below.

c. Update your MondayMotivation.py file with accordance to the lamda_function.py file:

d. Now, we will zip our file.

I have the MondayMotivation.py and quotes.txt in a folder named "Mondays" and I compressed that folder into a zip by right clicking on the folder and selecting compress.

You are then going to upload this zip file into AWS Lambda by clicking 'Upload from' then selecting .zip and uploading your zip file.

I then edited around the folders within lambda so I have below.

e. This part is important. Scroll down to RunTime settings and click edit. We need to edit the Handler name.

My .py file name is MondayMotivation2 and the def function is called lambda_handler within my .py file, so that is why my Handler info is like that above. Edit depending on your function name and .py file name.

3.Setting Up CloudWatch Events

Finally, let's set up a trigger so that Lambda will run this code. I would like this code to only run Monday's at 8 A.M.

a.

Click the add trigger button in the function overview.

b.

This is the information that I put in for my CloudWatch Event rule. This was my first time dealing with cron expressions so I was very confused and appalled at this notation with the question marks and asterisks, but this website explains cron expressions very well.https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html
Edit the cron expression to when you want it to run. (NOTE- if you want to do every day etc, make sure you edit "if day_of_week == .. code within your code to correspond) Click add and that's it!

4.Extra- Environment variables
This was my first instance of dealing with environment variables and I am very pleasantly surprised with how awesome they are, so I highly recommend you follow along and do this, it is simple and easy and makes your code more adaptable and flexible so you can quickly make changes and adjust your functions behavior without messing up the code. I used environment variables for: my email, app password, and receiving email. So for example, if I wanted to change my receiving email, I would change it through the environment variables tab and not have to touch my function code. It is a good habit to get into, so let's get started.

a. Click on the configuration tab, then click on environment variables and hit edit.

b. Click add environment variables and add 3. I added it for my sender email, receiving email, and app password that was on my MondayMotivation2.py file.

c. Click save.

d. Now, go back to code, and we are going to edit our code within here to reflect our environment variables. Make sure to click deploy changes when done!

I edited line line 7, 8, and 22 as well as added in line 4. REMEMBER- put a receiving email thats not popular(i.e. gmail won't work as the receiving email, but works great for the sending email. Aol works, check bottom of page for temporary email link that works)

Perfect, now you don't have sensitive information on the front of your Lambda.

5.Time To Run
We will now set up a test event and run our code through Lambda.

a. Set up a test by clicking the blue test box

I kept everything the same and just added a name. Click save.

b. Before clicking test, remember to have the day_of_week == to the day it is today. (see 1e)

c. Click test, and you should get a succeeded status in the execution results tab like below with a "Hello From Lambda" and a nice motivational quote in your email. Once you're done, switch the day_of_week back equal to 0 (Monday) and deploy your changes and you're good to go!

REMEMBER: It only works for receiving emails that still allow this. It did not work for my gmail but it worked for my aol account, but this could change. If you are getting a successful execution but no email, your email provider probably blocked it. You can use a website such as https://internxt.com/temporary-email , where you can get temporary access to an email and inbox and I have found this works as of now.

And that is it! I added the before code(without the Lambda function incorporation and environment variables- as it is seen in step 1), the final code, and the quotes.txt file into a GitHub repository @ https://github.com/andreapeterson/AWSLambda-SMTPLIB-Python

Good luck! Let me know if you try it.

For an extra challenge: Have Lambda run code that will run once a day and send an email if it is someones birthday. I uploaded my code to the same Github repository. Hint: this might be useful if you plan to use Pandas:https://stackoverflow.com/questions/63421063/unable-to-import-pandas-in-aws-lambda-layer