DEV Community: Harshad D

Embarking on Cloud Computing: My Journey Through the AWS 101 Workshop

Harshad D — Mon, 31 Mar 2025 06:55:28 +0000

Embarking on a journey into cloud computing can be both exciting and daunting. To bridge this gap, I recently completed the AWS 101 Workshop, an entry-level, hands-on experience designed to introduce newcomers to Amazon Web Services (AWS). In this blog, I'll share my insights and experiences from the workshop, aiming to guide others who are considering this learning path.

👉 Workshop Link: AWS 101 Workshop

Introduction to the AWS 101 Workshop

The AWS 101 Workshop is tailored for individuals new to AWS, offering practical experience in setting up a private EC2 web server within a custom Amazon Virtual Private Cloud (VPC). Participants learn to enable communication with public resources such as external HTTP APIs and Amazon Simple Storage Service (S3), all while ensuring the instance remains inaccessible directly from the public internet.
AWS Workshops

Workshop Content Overview

Throughout the workshop, several foundational AWS services are covered:

-Amazon Elastic Compute Cloud (EC2): Participants learn to launch and manage virtual servers, gaining insights into instance types, configurations, and the nuances of operating virtual machines in the cloud.

-Amazon Virtual Private Cloud (VPC): The workshop delves into creating isolated network environments, teaching how to set up subnets, route tables, and security groups to control traffic flow effectively.

-Amazon Simple Storage Service (S3): Attendees explore storing and retrieving data using S3, understanding bucket policies, object storage, and the integration of S3 with other AWS services.

-Identity and Access Management (IAM): The importance of securely managing access to AWS services is emphasized, with practical exercises on creating users, groups, roles, and policies to enforce least privilege access.

These sessions are designed to equip participants with the skills needed to set up and manage web servers, configure security groups, and grasp the fundamentals of AWS networking.

Personal Experience

My motivation for enrolling in the AWS 101 Workshop stemmed from a desire to build a solid foundation in cloud computing. As someone with limited prior exposure to AWS, I found the learning curve to be well-calibrated; the workshop's structured approach eased the assimilation of complex concepts.

One challenge I encountered was configuring the VPC to allow the EC2 instance to access public resources without exposing it to the internet. Through guided instructions and troubleshooting, I overcame this by setting up appropriate route tables and security group rules, reinforcing my understanding of AWS networking.

The hands-on labs were particularly beneficial. For instance, launching an EC2 instance and configuring it within a private subnet provided practical insights into managing cloud infrastructure securely.

Key Takeaways

The workshop significantly enhanced my understanding of AWS services and their interplay. I acquired practical skills in configuring EC2 instances, setting up VPCs, managing S3 buckets, and implementing IAM policies. A key insight was the emphasis on security best practices, such as the principle of least privilege and the importance of network segmentation.
Amazon Web Services, Inc.

Application of Knowledge

Armed with this foundational knowledge, I plan to embark on personal projects that involve deploying web applications on AWS, utilizing the services explored during the workshop. Professionally, this experience positions me to contribute more effectively to cloud-based initiatives and opens avenues for further specialization within the AWS ecosystem.

Recommendations

For those considering the AWS 101 Workshop, I recommend familiarizing yourself with basic networking concepts and having a general understanding of virtualization. Engaging actively with the hands-on exercises and not hesitating to seek clarification when challenges arise will enhance the learning experience.

Conclusion

Completing the AWS 101 Workshop has been a rewarding endeavor, laying a robust foundation for my cloud computing journey. I encourage others to explore this and other AWS learning opportunities to build their skills in this ever-evolving domain. AWS offers a variety of workshops catering to different skill levels and interests, which can be found on the AWS Workshops page.

Final result of the deployed service:

Happy learning!

AWS Three-Tier Architecture

Harshad D — Sun, 30 Mar 2025 03:56:36 +0000

Introduction

A 3-Tier Architecture is a widely used architectural pattern that separates an application into three layers:

Web Tier (Presentation Layer) – Handles user requests
App Tier (Business Logic Layer) – Processes data and logic
Database Tier (Data Layer) – Stores and retrieves data

This architecture enhances scalability, security, and maintainability. In this guide, we will set up a highly available 3-tier architecture on AWS using Amazon EC2, Elastic Load Balancer (ELB), and Amazon RDS (Aurora).

Part - 1

Architecture

Download Code from Github Repository

   - git clone https://github.com/Naveen3251/AWS_3Tier.git

S3 Bucket Creation

Navigate to the S3 service in the AWS console and create a new S3 bucket.
Give it a unique name, and then leave all the defaults as in. Make sure to select the region that you intend to run this whole lab in. This bucket is where we will upload our code later.

IAM EC2 Instance Role Creation

Navigate to the IAM dashboard in the AWS console and create an EC2 role.
Select EC2 as the trusted entity.
When adding permissions, include the following AWS managed policies. You can search for them and select them. These policies will allow our instances to download our code from S3 and use Systems Manager Session Manager to securely connect to our instances without SSH keys through the AWS console.
```
 - AmazonSSMManagedInstanceCore
 - AmazonS3ReadOnlyAccess
```

Give your role a name, and then click Create Role.

Part - 2

Networking and Security

VPC Creation

Navigate to the VPC dashboard in the AWS console and navigate to Your VPCs on the left hand side.

Make sure VPC only is selected, and fill out the VPC Settings with a Name tag and a CIDR range of your choice.

NOTE: Make sure you pay attention to the region you’re deploying all your resources in. You’ll want to stay consistent for this entier project.

NOTE: Choose a CIDR range that will allow you to create at least 6 subnets.

Subnet Creation

Next, create your subnets by navigating to Subnets on the left side of the dashboard and clicking Create subnet.
We will need six subnets across two availability zones. That means that three subnets will be in one availability zone, and three subnets will be in another zone. Each subnet in one availability zone will correspond to one layer of our three tier architecture. Create each of the 6 subnets by specifying the VPC we created in part 1 and then choose a name, availability zone, and appropriate CIDR range for each of the subnets.

NOTE: It may be helpful to have a naming convention that will help you remember what each subnet is for. For example in one AZ you might have the following:Public-Web-Subnet-AZ-1, Private-App-Subnet-AZ-1, Private-DB-Subnet-AZ-1.

NOTE: Remember, your CIDR range for the subnets will be subsets of your VPC CIDR range.

Your final subnet setup should be similar to this. Verify that you have 3 subnets across 2 different availability zones.

Internet Connectivity

Internet Gateway

In order to give the public subnets in our VPC internet access we will have to create and attach an Internet Gateway. On the left hand side of the VPC dashboard, select Internet Gateway.
Create your internet gateway by simply giving it a name and clicking Create internet gateway.

After creating the internet gateway, attach it to your VPC that you create in the VPC and Subnet Creation step of the workshop. You have a couple options on how to do this, either with the creation success message or the Actions drop down.

Then, select the correct VPC and click Attach internet gateway.

NAT Gateway

In order for our instances in the app layer private subnet to be able to access the internet they will need to go through a NAT Gateway. For high availability, you’ll deploy one NAT gateway in each of your public subnets. Navigate to NAT Gateways on the left side of the current dashboard and click Create NAT Gateway.
Fill in the Name, choose one of the public subnets you created in part 2, and then allocate an Elastic IP. Click Create NAT gateway.

Repeat step 1 and 2 for the other subnet.

Routing Configuration

Navigate to Route Tables on the left side of the VPC dashboard and click Create route table First, let’s create one route table for the web layer public subnets and name it accordingly.

After creating the route table, you'll automatically be taken to the details page. Scroll down and click on the Routes tab and Edit routes.

Add a route that directs traffic from the VPC to the internet gateway. In other words, for all traffic destined for IPs outside the VPC CDIR range, add an entry that directs it to the internet gateway as a target. Save the changes.
Edit the Explicit Subnet Associations of the route table by navigating to the route table details again. Select Subnet Associations and click Edit subnet associations.

Select the two web layer public subnets you created eariler and click Save associations.
Now create 2 more route tables, one for each app layer private subnet in each availability zone. These route tables will route app layer traffic destined for outside the VPC to the NAT gateway in the respective availability zone, so add the appropriate routes for that.
Once the route tables are created and routes added, add the appropriate subnet associations for each of the app layer private subnets.

Security Groups

Security groups will tighten the rules around which traffic will be allowed to our Elastic Load Balancers and EC2 instances. Navigate to Security Groups on the left side of the VPC dashboard, under Security.
The first security group you’ll create is for the public, internet facing load balancer. After typing a name and description, add an inbound rule to allow HTTP type traffic for your IP.

The second security group you’ll create is for the public instances in the web tier. After typing a name and description, add an inbound rule that allows HTTP type traffic from your internet facing load balancer security group you created in the previous step. This will allow traffic from your public facing load balancer to hit your instances. Then, add an additional rule that will allow HTTP type traffic for your IP. This will allow you to access your instance when we test.

The third security group will be for our internal load balancer. Create this new security group and add an inbound rule that allows HTTP type traffic from your public instance security group. This will allow traffic from your web tier instances to hit your internal load balancer.

The fourth security group we’ll configure is for our private instances. After typing a name and description, add an inbound rule that will allow TCP type traffic on port 4000 from the internal load balancer security group you created in the previous step. This is the port our app tier application is running on and allows our internal load balancer to forward traffic on this port to our private instances. You should also add another route for port 4000 that allows your IP for testing.

The fifth security group we’ll configure protects our private database instances. For this security group, add an inbound rule that will allow traffic from the private instance security group to the MYSQL/Aurora port (3306).

Part - 3

Database Deployement

Subnet Groups

Navigate to the RDS dashboard in the AWS console and click on Subnet groups on the left hand side. Click Create DB subnet group.

Give your subnet group a name, description, and choose the VPC we created.

When adding subnets, make sure to add the subnets we created in each availability zone specificaly for our database layer. You may have to navigate back to the VPC dashboard and check to make sure you're selecting the correct subnet IDs.

Database Deployement

Navigate to Databases on the left hand side of the RDS dashboard and click Create database.
We'll now go through several configuration steps. Start with a Standard create for this MySQL-Compatible Amazon Aurora database. Leave the rest of the defaults in the Engine options as default.

Under the Templates section choose Dev/Test since this isn't being used for production at the moment. Under Settings set a username and password of your choice and note them down since we'll be using password authentication to access our database.

Next, under Availability and durability change the option to create an Aurora Replica or reader node in a different availability zone. Under Connectivity, set the VPC, choose the subnet group we created earlier, and select no for public access.

Set the security group we created for the database layer, make sure password authentication is selected as our authentication choice, and create the database.

When your database is provisioned, you should see a reader and writer instance in the database subnets of each availability zone. Note down the writer endpoint for your database for later use.

Part - 4

App Instance Deployment

Navigate to the EC2 service dashboard and click on Instances on the left hand side. Then, click Launch Instances.

Select the first Amazon Linux 2 AMI.
We'll be using the free tier eligible T.2 micro instance type. Select that and click Next: Configure Instance Details.
Proceed without key-pair because we going to use ec2-instance login.

When configuring the instance details, make sure to select to correct Network, subnet, and IAM role we created. Note that this is the app layer, so use one of the private subnets we created for this layer.

Below we using already created VPC and Private Subnet AZ-1
Below we using already created private instance security group

Below we using already created IAM role

Finally click "Launch Instance"

Connect to Instance

Navigate to your list of running EC2 Instances by clicking on Instances on the left hand side of the EC2 dashboard. When the instance state is running, connect to your instance by clicking the checkmark box to the left of the instance, and click the connect button on the top right corner of the dashboard. Select the Session Manager tab, and click connect. This will open a new browser tab for you.

NOTE: If you get a message saying that you cannot connect via session manager, then check that your instances can route to your NAT gateways and verify that you gave the necessary permissions on the IAM role for the Ec2 instance.

When you first connect to your instance like this, you will be logged in as ssm-user which is the default user. Switch to ec2-user by executing the following command in the browser terminal:

        `sudo -su ec2-user`

Let’s take this moment to make sure that we are able to reach the internet via our NAT gateways. If your network is configured correctly up till this point, you should be able to ping the google DNS servers:

          `ping 8.8.8.8`

You should see a transmission of packets. Stop it by pressing cntrl c.

NOTE: If you can’t reach the internet then you need to double check your route tables and subnet associations to verify if traffic is being routed to your NAT gateway!

Configure Database

Start by downloading the MySQL CLI:

sudo yum install mysql -y

if above one causing issue like,

try the below commands to install without any errors

 sudo wget https://dev.mysql.com/get/mysql57-community-release-e17-11.noarch.rpm

 sudo rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2022

 sudo yum install https://dev.mysql.com/get/mysql57-community-release-e17-11.noarch.rpm

Initiate your DB connection with your Aurora RDS writer endpoint. In the following command, replace the RDS writer endpoint and the username, and then execute it in the browser terminal:

 mysql -h CHANGE-TO-YOUR-RDS-ENDPOINT -u CHANGE-TO-USER-NAME -p

You will then be prompted to type in your password. Once you input the password and hit enter, you should now be connected to your database.

Create a database called webappdb with the following command using the MySQL CLI:

 CREATE DATABASE webappdb;

You can verify that it was created correctly with the following command:

 SHOW DATABASES;

Create a data table by first navigating to the database we just created:

 USE webappdb;

Then, create the following transactions table by executing this create table command:

 CREATE TABLE IF NOT EXISTS transactions(id INT NOT NULL
 AUTO_INCREMENT, amount DECIMAL(10,2), description
 VARCHAR(100), PRIMARY KEY(id));

Verify the table was created:

 SHOW TABLES;

Insert data into table for use/testing later:

 INSERT INTO transactions (amount,description) VALUES ('400','groceries');

Verify that your data was added by executing the following command:

 SELECT * FROM transactions;

When finished, just type exit and hit enter to exit the MySQL client.

Configure App Instance

The first thing we will do is update our database credentials for the app tier. To do this, open the application-code/app-tier/DbConfig.js file from the github repo in your favorite text editor on your computer. You’ll see empty strings for the hostname, user, password and database. Fill this in with the credentials you configured for your database, the writer endpoint of your database as the hostname, and webappdb for the database. Save the file.

Upload the app-tier folder to the S3 bucket that you created in part 1.

Go back to your SSM session. Now we need to install all of the necessary components to run our backend application. Start by installing NVM (node version manager).

 curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
 source ~/.bashrc

Next, install a compatible version of Node.js and make sure it's being used.

 nvm install 16
 nvm use 16

PM2 is a daemon process manager that will keep our node.js app running when we exit the instance or if it is rebooted. Install that as well.

 npm install -g pm2

Now we need to download our code from our s3 buckets onto our instance. In the command below, replace BUCKET_NAME with the name of the bucket you uploaded the app-tier folder to:

 cd ~/
 aws s3 cp s3://BUCKET_NAME/app-tier/ app-tier --recursive

Navigate to the app directory, install dependencies, and start the app with pm2.

 cd ~/app-tier
 npm install
 pm2 start index.js

To make sure the app is running correctly run the following:

 pm2 list

If you see a status of online, the app is running. If you see errored, then you need to do some troubleshooting. To look at the latest errors, use this command:

 pm2 logs

NOTE: If you’re having issues, check your configuration file for any typos, and double check that you have followed all installation commands till now.

Right now, pm2 is just making sure our app stays running when we leave the SSM session. However, if the server is interrupted for some reason, we still want the app to start and keep running. This is also important for the AMI we will create:

 pm2 startup

After running this you will see a message similar to this.

 [PM2] To setup the Startup Script, copy/paste the following command: sudo env PATH=$PATH:/home/ec2-user/.nvm/versions/node/v16.0.0/bin /home/ec2-user/.nvm/versions/node/v16.0.0/lib/node_modules/pm2/bin/pm2 startup systemd -u ec2-user —hp /home/ec2-user

DO NOT run the above command, rather you should copy and past the command in the output you see in your own terminal. After you run it, save the current list of node processes with the following command:

 pm2 save

Test App Tier

Now let's run a couple tests to see if our app is configured correctly and can retrieve data from the database.

To hit out health check endpoint, copy this command into your SSM terminal. This is our simple health check endpoint that tells us if the app is simply running.

 curl http://localhost:4000/health

The response should looks like the following:

 "This is the health check"

Next, test your database connection. You can do that by hitting the following endpoint locally:

 curl http://localhost:4000/transaction

You should see a response containing the test data we added earlier:

 {"result":[{"id":1,"amount":400,"description":"groceries"},{"id":2,"amount":100,"description":"class"},{"id":3,"amount":200,"description":"other groceries"},{"id":4,"amount":10,"description":"brownies"}]}

If you see both of these responses, then your networking, security, database and app configurations are correct.

Your app layer is fully configured and ready to go.

Part - 5

Internal Load Balancing and Auto Scaling

App Tier AMI

Navigate to Instances on the left hand side of the EC2 dashboard. Select the app tier instance we created and under Actions select Image and templates. Click Create Image.
Give the image a name and description and then click Create image. This will take a few minutes, but if you want to monitor the status of image creation you can see it by clicking AMIs under Images on the left hand navigation panel of the EC2 dashboard.

Target Group

While the AMI is being created, we can go ahead and create our target group to use with the load balancer. On the EC2 dashboard navigate to Target Groups under Load Balancing on the left hand side. Click on Create Target Group.
The purpose of forming this target group is to use with our load blancer so it may balance traffic across our private app tier instances. Select Instances as the target type and give it a name.

Then, set the protocol to HTTP and the port to 4000. Remember that this is the port our Node.ja app is running on. Select the VPC we've been using thus far, and then change the health check path to be /health. This is the health check endpoint of our app. Click Next.

We are NOT going to register any targets for now, so just skip that step and create the target group.

Internal Load Balancer

On the left hand side of the EC2 dashboard select Load Balancers under Load Balancing and click Create Load Balancer.
We'll be using an Application Load Balancer for our HTTP traffic so click the create button for that option.

After giving the load balancer a name, be sure to select internal since this one will not be public facing, but rather it will route traffic from our web tier to the app tier.

Select the correct network configuration for VPC and private subnets.

Select the security group we created for this internal ALB. Now, this ALB will be listening for HTTP traffic on port 80. It will be forwarding the traffic to our target group that we just created, so select it from the dropdown, and create the load balancer.

Launch Template

Before we configure Auto Scaling, we need to create a Launch template with the AMI we created earlier. On the left side of the EC2 dashboard navigate to Launch Template under Instances and click Create Launch Template.
Name the Launch Template, and then under Application and OS Images include the app tier AMI you created.

Under Instance Type select t2.micro. For Key pair and Network Settings don't include it in the template. We don't need a key pair to access our instances and we'll be setting the network information in the autoscaling group.

Set the correct security group for our app tier, and then under Advanced details use the same IAM instance profile we have been using for our EC2 instances.

Auto Scaling

We will now create the Auto Scaling Group for our app instances. On the left side of the EC2 dashboard navigate to Auto Scaling Groups under Auto Scaling and click Create Auto Scaling group.
Give your Auto Scaling group a name, and then select the Launch Template we just created and click next.

On the Choose instance launch options page set your VPC, and the private instance subnets for the app tier.

For this next step, attach this Auto Scaling Group to the Load Balancer we just created by selecting the existing load balancer's target group from the dropdown. Then, click next.

For Configure group size and scaling policies, set desired, minimum and maximum capacity to 2. Click skip to review and then Create Auto Scaling Group.

After the above step Click next, next finally click "create autoscaling group"

Part - 6

Update Config File

Before we create and configure the web instances, open up the application-code/nginx.conf file from the repo we downloaded. Scroll down to line 58 and replace [INTERNAL-LOADBALANCER-DNS] with your internal load balancer’s DNS entry. You can find this by navigating to your internal load balancer's details page.

DNS Name is,

Copy the above DNS name and replace in your nginx.conf file

Then, upload this file and the application-code/web-tier folder to the s3 bucket you created for this lab.

Web Instance Deployment

Navigate to the EC2 service dashboard and click on Instances on the left hand side. Then, click Launch Instances.
Select the first Amazon Linux 2 AMI.

We'll be using the free tier eligible T.2 micro instance type. Select that and click Next: Configure Instance Details.

Proceed without key-pair because we going to use ec2-instance login.

When configuring the instance details, make sure to select to correct Network, subnet, and IAM role we created. Note that this is the app layer, so use one of the private subnets we created for this layer.

Below we using already created VPC and Public Subnet AZ-1 and Enable auto assign Public IP

Below we using already created web tier security group

Below we using already created IAM role

Finally click "Launch Instance"

Connect to Instance

Navigate to your list of running EC2 Instances by clicking on Instances on the left hand side of the EC2 dashboard. When the instance state is running, connect to your instance by clicking the checkmark box to the left of the instance, and click the connect button on the top right corner of the dashboard. Select the Session Manager tab, and click connect. This will open a new browser tab for you.

Test connectivity here via ping as well since this instance should have internet connectivity:

 sudo -su ec2-user 
 ping 8.8.8.8

Configure Web Instance

We now need to install all of the necessary components needed to run our front-end application. Again, start by installing NVM and node :

 curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
 source ~/.bashrc
 nvm install 16
 nvm use 16

Now we need to download our web tier code from our s3 bucket:

 cd ~/
 aws s3 cp s3://BUCKET_NAME/web-tier/ web-tier --recursive

Navigate to the web-layer folder and create the build folder for the react app so we can serve our code:

 cd ~/web-tier
 npm install 
 npm run build

NGINX can be used for different use cases like load balancing, content caching etc, but we will be using it as a web server that we will configure to serve our application on port 80, as well as help direct our API calls to the internal load balancer.

 sudo amazon-linux-extras install nginx1 -y

We will now have to configure NGINX. Navigate to the Nginx configuration file with the following commands and list the files in the directory:

 cd /etc/nginx
 ls

You should see an nginx.conf file. We’re going to delete this file and use the one we uploaded to s3. Replace the bucket name in the command below with the one you created for this workshop:

 sudo rm nginx.conf
 sudo aws s3 cp s3://BUCKET_NAME/nginx.conf .

Then, restart Nginx with the following command:

 sudo service nginx restart

To make sure Nginx has permission to access our files execute this command:

 chmod -R 755 /home/ec2-user

And then to make sure the service starts on boot, run this command:

 sudo chkconfig nginx on

Part - 7

External Load Balancer and Auto Scaling

Web Tier AMI

Navigate to Instances on the left hand side of the EC2 dashboard. Select the web tier instance we created and under Actions select Image and templates. Click Create Image.
Give the image a name and description and then click Create image. This will take a few minutes, but if you want to monitor the status of image creation you can see it by clicking AMIs under Images on the left hand navigation panel of the EC2 dashboard.

Target Group

While the AMI is being created, we can go ahead and create our target group to use with the load balancer. On the EC2 dashboard navigate to Target Groups under Load Balancing on the left hand side. Click on Create Target Group.

The purpose of forming this target group is to use with our load blancer so it may balance traffic across our public web tier instances. Select Instances as the target type and give it a name.

Then, set the protocol to HTTP and the port to 80. Remember this is the port NGINX is listening on. Select the VPC we've been using thus far, and then change the health check path to be /health. Click Next.

We are NOT going to register any targets for now, so just skip that step and create the target group.

Internet Facing Load Balancer

On the left hand side of the EC2 dashboard select Load Balancers under Load Balancing and click Create Load Balancer.

We'll be using an Application Load Balancer for our HTTP traffic so click the create button for that option.

After giving the load balancer a name, be sure to select internet facing since this one will not be public facing, but rather it will route traffic from our web tier to the app tier.

Select the correct network configuration for VPC and public subnets.

*Launch Template
*

Before we configure Auto Scaling, we need to create a Launch template with the AMI we created earlier. On the left side of the EC2 dashboard navigate to Launch Template under Instances and click Create Launch Template.

Name the Launch Template, and then under Application and OS Images include the app tier AMI you created.

Set the correct security group for our web tier, and then under Advanced details use the same IAM instance profile we have been using for our EC2 instances.

Auto Scaling

We will now create the Auto Scaling Group for our web instances. On the left side of the EC2 dashboard navigate to Auto Scaling Groups under Auto Scaling and click Create Auto Scaling group.

Give your Auto Scaling group a name, and then select the Launch Template we just created and click next.

On the Choose instance launch options page set your VPC, and the public subnets for the web tier.

For this next step, attach this Auto Scaling Group to the Load Balancer we just created by selecting the existing web tier load balancer's target group from the dropdown. Then, click next.

For Configure group size and scaling policies, set desired, minimum and maximum capacity to 2. Click skip to review and then Create Auto Scaling Group.

Finally we get our "WebApp"

Conclusion
Well done on successfully completing your three-tier architecture project on AWS! It's a significant accomplishment to have followed through from Part 1 to Part 7 of the blog series. Before wrapping up, it's essential to ensure that all the services you created for learning purposes are properly cleaned up to avoid any unnecessary costs or lingering resources.

_Thankyou !!!

Hope You have Learned !!!_

Step-by-Step Guide to Launch and Manage an AWS EC2 Instance with S3 Integration

Harshad D — Thu, 23 Jan 2025 17:02:42 +0000

1. Introduction to AWS EC2 and S3 Integration

AWS (Amazon Web Services) offers a wide range of cloud solutions, with EC2 being a popular service for scalable computing capacity. This guide walks you through launching an EC2 instance, connecting it via SSH using PuTTY, integrating with an S3 bucket, and cleaning up resources.

2. Creating an EC2 Instance

Step 2.1: Choose Amazon Linux as the AMI

Log in to your AWS Management Console.
Navigate to the EC2 Dashboard and click Launch Instance.
Select Amazon Linux as your Amazon Machine Image (AMI).

Step 2.2: Choose Instance Type

Select t2.micro (Free Tier eligible) as the instance type.

Step 2.3: Key Pair Creation

Create a new key pair or use an existing one.
Download the .pem file and store it securely (you’ll need it to connect via SSH).

Step 2.4: Configure Security Group

Set up inbound rules to allow SSH (port 22) access.
Review and launch your instance.

3. Connecting to the Instance with PuTTY

Step 3.1: Install PuTTY

Download PuTTY and PuTTYgen from the official website.
Install and extract PuTTY on your system.

Step 3.2: Edit Security Group Inbound Rules

Ensure that the SSH port is open to your local machine’s IP.

Step 3.3: Convert Key Pair Using PuTTYgen

Use PuTTYgen to convert the .pem file to a .ppk file for use with PuTTY.

Step 3.4: Connect Using PuTTY

Open PuTTY and enter the Public IPv4 address of your EC2 instance in the Host Name field.
Navigate to Connection > SSH > Auth and upload the .ppk file.
Click Open to connect to your EC2 instance as the ec2-user.

4. Assigning an IAM Role for S3 Bucket Access

Step 4.1: Create an IAM Role

Go to the IAM Dashboard and create a new role.
Assign the AmazonS3FullAccess policy to the role.

Step 4.2: Attach the Role to the EC2 Instance

Navigate to the EC2 Dashboard, select your instance, and choose Actions > Security > Modify IAM Role.
Attach the newly created role to the instance.

5. Uploading Files to S3 via EC2 Terminal

Step 5.1: Create an S3 Bucket

Navigate to the S3 Dashboard and create a new bucket.

Step 5.2: Upload a File

SSH into your EC2 instance.

Use the AWS CLI to upload a file to the S3 bucket:

      - aws s3 cp <local-file-path> s3://<your-bucket-name>/

6. Cleaning Up Resources

Step 6.1: Terminate the EC2 Instance

Go to the EC2 Dashboard and terminate the instance.

Step 6.2: Delete the IAM Role

Navigate to the IAM Dashboard and delete the role.

Step 6.3: Remove S3 Bucket (Optional)

Delete the S3 bucket if you no longer need it.

Amazon EMR

Harshad D — Thu, 23 Jan 2025 16:24:16 +0000

1. Service Overview

Service Name: Amazon EMR (Amazon Elastic MapReduce)

Tagline: "Amazon EMR: Simplifying Big Data Processing with Apache Hadoop at Scale."

Amazon EMR (Elastic MapReduce) is a cloud-based big data platform that allows you to process and analyze vast amounts of data using frameworks like Apache Spark, Hadoop, and Presto. It is optimized for large-scale data processing and supports various use cases, including data transformation, machine learning, log analysis, and real-time analytics.

2. Key Features

Top Features:

Scalable Big Data Processing: Amazon EMR provides a managed Hadoop framework, allowing you to process vast amounts of data seamlessly. It automatically scales clusters to meet your workload requirements.
Fully Managed Service: Amazon EMR handles provisioning, configuration, and maintenance of Hadoop clusters, saving time and operational effort.
Wide Tool Support: EMR supports Apache Hadoop, Apache Spark, Apache HBase, Apache Hive, and other open-source big data tools.
Integration with AWS Services: Seamlessly integrates with Amazon S3 for data storage, AWS Glue for ETL jobs, and Amazon CloudWatch for monitoring.
Cost Optimization: You can choose Spot Instances and Auto Scaling to optimize costs, and EMR pricing is based on a pay-as-you-go model.
Security and Compliance: Supports data encryption at rest and in transit, along with integration with AWS Identity and Access Management (IAM) for access control.

Technical Specifications:

Regions Supported: Available in all AWS Regions.
Data Durability: Enhanced when paired with Amazon S3 (11 9’s durability).
Cluster Scaling: Supports manual and automatic scaling.
Instance Types: Compatible with a wide range of EC2 instance types, including compute-optimized, memory-optimized, and storage-optimized instances.

3. Use Cases

Real-Life Applications:

Data Processing Pipelines: Perform ETL operations on massive datasets stored in Amazon S3 or other data sources.
Data Warehousing: Run Apache Hive on Amazon EMR for querying structured data and building analytics dashboards.
Machine Learning Workflows: Leverage Apache Spark on EMR for training and deploying machine learning models.
Log Analysis: Analyze server logs at scale for performance monitoring, error tracking, and business insights.
Genomics Data Analysis: Process and analyze genome sequencing data efficiently.

4. Pricing Model

Amazon EMR uses a pay-as-you-go pricing model. Key pricing factors include:

EC2 Instances: Costs depend on the instance type and number of instances in your cluster.
EMR Charges: An hourly fee is charged per instance in the cluster.
Spot Instances: Reduce costs by using Spot Instances for non-critical workloads.
Data Transfer: Costs for data transfer between AWS services or to/from the internet.

For detailed pricing, visit the Amazon EMR Pricing Page.

5. Comparison with Similar Services

Amazon EMR competes with Google Dataproc and Azure HDInsight. EMR integrates well with AWS services and offers flexibility with features like Auto Scaling and Spot Instances. Google Dataproc is known for its fast cluster setup and tight integration with Google Cloud. Azure HDInsight works best for users already using Microsoft’s ecosystem. Amazon EMR is ideal for large-scale, cost-sensitive workloads thanks to its scalability and cost-optimization options.

6. Benefits and Challenges

Advantages:

High Scalability: Dynamically scale your clusters to meet workload demands.
Cost Efficiency: Leverage Spot Instances and Auto Scaling to minimize costs.
Wide Tool Support: Run a broad range of big data frameworks.
Global Availability: Operates in multiple AWS regions with robust performance.

Limitations or Challenges:

Learning Curve: New users may need time to understand Hadoop and related tools.
Initial Setup Costs: Custom configurations may require additional effort.

7. Real-World Example or Case Study

Case Study: Airbnb

Airbnb uses Amazon EMR to process large volumes of data for machine learning and business intelligence. By leveraging EMR's scalability and integration with Amazon S3, Airbnb can run complex queries and machine learning workflows efficiently. This allows them to improve search recommendations and optimize pricing strategies.

Static Webpage Hosting (S3 + Cloud front) - React app

Harshad D — Thu, 23 Jan 2025 09:41:57 +0000

Hosting a static React application on AWS using S3 and CloudFront is an excellent way to deploy a performant, globally distributed web application. This guide will cover the steps up to creating an S3 bucket.

Step 1: Create a React App
Before we deploy, we need to create a React app. To do this, ensure Node.js and npm (Node Package Manager) are installed on your system.

1.1 Install Node.js and npm

Node.js is a runtime environment for executing JavaScript code, and npm
is its accompanying package manager.

Download and install Node.js from the official website.
Verify installation:
```
       node -v
       npm -v
```

1.2 Create the React App

With Node.js installed, you can use the npx command to create a new React app.

Open a terminal and run the following command:

      npx create-react-app my-react-app

Navigate to the project directory:
```
      cd my-react-app
```

1.3 Customize the App (Add Host Name and Roll Number)

Open the project in your favorite code editor (e.g., VS Code).
Navigate to the src/App.js file.

Replace the content with the following:

      import './App.css';
      function App() {
        return (
          <div style={{ textAlign: 'center', marginTop: '20%' }}>
            <h1>Harshad D</h1>
            <h2>Roll Number: 714022202023</h2>
          </div>
        ); 
      } 
      export default App;

Save the file.

1.4 Test the App

Run the app locally:
```
   npm start
```
Open your browser and navigate to http://localhost:5173/. You should
see your name and roll number displayed on the page.

Step 2: Create an S3 Bucket

AWS S3 (Simple Storage Service) is an object storage service that can be used to host static websites. Follow these steps to create an S3 bucket.

2.1 Log in to AWS

Go to the AWS Management Console.
Log in with your AWS account credentials.

2.2 Create a Bucket

Navigate to the S3 service.
Click the Create bucket button.
Fill in the following details:
- Bucket Name: Enter a unique name (e.g., my-react-app-bucket).
- AWS Region: Choose the region closest to your target audience.
Uncheck the Block all public access option to allow public access
(required for hosting).
Confirm the change and proceed.
Click Create bucket.

2.3 Configure the Bucket for Static Hosting

After the bucket is created, click on its name to access the bucket settings.
Go to the Properties tab.
Scroll down to the Static website hosting section and click Edit.
Enable static website hosting and specify the following:
- Index document: index.html
- (Optional) Error document: error.html
Save the changes.

Step 3: Build and Upload the React App to S3

3.1 Build the React App

To prepare your React app for production, run the following command in the project directory:

       npm run build

This will generate a build folder containing optimized files for deployment.

3.2 Upload Files to the S3 Bucket

In the AWS Management Console, go to your S3 bucket.
Click the Upload button.
Drag and drop all the files from the build folder into the upload area.
Click Upload to start the process.

3.3 Test the S3 Bucket

Once the files are uploaded, go to the Properties tab of your S3 bucket.
Under Static website hosting, you will see a Bucket website endpoint. Open this URL in your browser to view your React app.

Step 4: Configure CloudFront for S3 Hosting

To enhance performance and provide a secure, globally distributed setup, integrate your S3 bucket with CloudFront.

4.1 Create a CloudFront Distribution

Navigate to the CloudFront service in the AWS Management Console.
Click Create Distribution.
Select Web and configure the following:

- Origin Domain Name: Choose your S3 bucket from the dropdown.
- Viewer Protocol Policy: Redirect HTTP to HTTPS.

4.2 Update S3 Bucket Permissions

Ensure your S3 bucket allows access from CloudFront by setting up the appropriate bucket policy.

4.3 Test CloudFront

Once the distribution status changes to Deployed, copy the Domain Name from the CloudFront dashboard.
Open this URL in your browser to view your React app served via CloudFront.

Step 5: Deleting CloudFront to Avoid Charges

If you no longer need the CloudFront distribution, it is crucial to delete it to avoid unnecessary charges.

5.1 Disable and Delete the CloudFront Distribution

Navigate to the CloudFront service in the AWS Management Console.
Select your distribution and click Disable. Wait until the status changes to Disabled.
Once disabled, select the distribution again and click Delete.

5.2 Verify Deletion

Ensure the distribution is no longer listed in the CloudFront dashboard.