Production-Ready Web Application Deployment on AWS

TL;DR

I deployed a secure, resilient, and highly available web application on AWS using EC2 using a VPC, auto load-balancer, auto scaling group across 2 availability zones.

The stack:

• One VPC web-app-vpc (10.0.0.0/16), 2 AZ (us-east-1a, us-east-1b), 4 subnets (2 public, 2 private) - Internet Gateways and 1 NAT per AZ - Application load balancer web-app-alb in the public subnets, and an auto scaling group launching instances in the private subnets (one private subnet per AZ) - Launch template with user-data boot strapped to it. No manual configs for each new instance launched by the scaling group. - Instance profile LabInstanceProfile ( LabRole ), which lets instances use SSM and read Parameter Store.

Architecture

• Network isolation: instances only in private subnets.
• Least privilege perimeter: app-sg ONLY takes port 80 from alb-sg.
• No key pair on any instances: Only SSM.
• No hard coded things: nothing stored in user data of instances (all are read from a single source of truth that can be changed without editing instances).
• Self Healing: ALB health checks re-routes traffic of failed instances to other instances. Auto scaling group replaces failed instances automatically.
• Identical provisions: every instance boots with the same launch template.

Network design

The only things in public subnets are the load balancer and the NAT gateways. EC2 instances sit on the private subnets and don't have public IP.

Subnet	AZ	CIDR	Route 0.0.0.0/0 ->	Holds
web-app-subnet-public1	us-east-1a	10.0.1.0/24	Internet Gateway	ALB node, NAT Gateway 1
web-app-subnet-public2	us-east-1b	10.0.2.0/24	Internet Gateway	ALB node, NAT Gateway 2
web-app-subnet-private1	us-east-1a	10.0.11.0/24	NAT Gateway 1	App instances
web-app-subnet-private2	us-east-1b	10.0.12.0/24	NAT Gateway 2	App instances

Security

The security group of instances only allow traffic from the load balancer

Security group	Direction	Port	Source / destination
alb-sg	Inbound	80	0.0.0.0/0
alb-sg	Outbound	80	app-sg
app-sg	Inbound	80	alb-sg
app-sg	Outbound	443	0.0.0.0/0
app-sg	Outbound	80	0.0.0.0/0

Identity, access and secrets

Instances only get creds from an IAM instance profile, and those creds are temporary and rotate automatically. No key pairs in the launch template and admin access is only through SSM so port 22 is never open.

Note: Since this was done in an AWS academy learner lab, I was blocked from creating IAM roles, so I defaulted to the "LabRole" that was provided. LabRole is broader than needed, so a real prod app would use a new role with just the Session Manager permissions and ssm:GetParameter scoped to /webapp/*. AmazonSSMManagedInstanceCore on its own isn't enough, since it lets the instance read every parameter in the account.

A parameter called /webapp/app_message is stored in Parameter Store. The instances' user-data fetches it at boot using the instance profile's creds. If the web app had secrets like API keys, they would go in Parameter Store as SecureString, so nothing is hardcoded in the user data of the launch template.

Self Healing

Every instance boots ready to serve with the setup instructions in the user-data of launch templates, so the auto scaling group can deploy new ones without human intervention.

Launch Template: web-app-launch-template uses Amazon Linux 2023 x86, t3.micro, app-sg security group (the group that only allows traffic from the load balancer), LabInstanceProfile, IMDSv2 required so instances can know about themselves at runtime instead of needing to hard code credentials. Hop limit is set to 1 so it only reaches the instance itself and not any further.

Target group: HTTP 80, health check at GET /health, every 10 seconds, unhealthy threshold at 2 times. So dead instances are removed in about 20 seconds.

Auto scaling group:

Setting	Value
Subnets	the two private subnets from each AZ in the VPC
Min / desired / max	2 / 2 / 4
Health check type	EC2 + ELB
Scaling policy	Target tracking, average CPU 50%

Trade-offs

I went with one NAT gateway per AZ instead of a single shared one. It costs about double, but if one AZ goes down the private subnet in the other AZ still has internet, so the NAT isn't a single point of failure. For admin access I used SSM Session Manager instead of a bastion host, so there's no port 22, no SSH keys to lose, and no extra public instance, and the only catch is it depends on the SSM agent working. The instances use LabRole because the learner lab won't let me create roles, which means they have more permissions than they need. In a real account I'd make a role scoped to just SSM and /webapp/*. I kept config in Parameter Store instead of Secrets Manager since standard parameters are free, and Secrets Manager mostly adds auto rotation for about $0.40 per secret a month. The app only runs on HTTP because HTTPS needs a domain name for an ACM cert, which I didn't have for this project, so that's the first thing I'd add next. Last, I built everything in the AWS console because it was the fastest way to get it done in a time-limited lab, but the tradeoff is I can't rebuild it with one command. Next time I'd write it in Terraform.

Resilience Demonstration

In this section I will demonstrate the resilience capability of my setup. The demo starts in its ideal healthy state, one instance in each of the two AZs, and the load balancer evenly distributing traffic between every instance. Then I will manually delete one of the instances.

The list of instances in my project in their healthy state as shown in the target group:

In my machine's terminal, I will run a command to continuously ping my project every second while showing which instance in which AZ the request was routed to. The screenshot shows the projects responding healthy 200 codes while being evenly routed between two different instances i-0e6555acb4c09a6d3 and i-0ca5c7c0b17f67601 in us-east-1b and us-east-1a.

Then I will intentionally destroy one of the two instances by navigating to EC2 and deleting the instance i-0e6555acb4c09a6d3 (the instance in us-east-1b). Since this was the only instance in us-east-1b, this also kind of simulates losing a whole AZ, since that zone has no compute left at all.

Checking on my target group again I can see instance i-0e6555acb4c09a6d3 is gone and only one instance remains.

While that was happening, the terminal showed some 000 FAIL lines for about 15 seconds. That's because for a little while the load balancer didn't know the instance was dead yet, so some requests still got sent to it and timed out after 3 seconds. Once the dead instance failed 2 health checks in a row (every 10 seconds), the load balancer pulled it out and sent everything to the healthy instance in us-east-1a, and the FAILs stopped. (Side note: the instance ID printed next to each FAIL is just left over from the previous response, since a failed curl doesn't write a new one.)

2 seconds after the auto scaling group noticed the old instance in us-east-1b was gone, it launched a new one in response to an unhealthy instance needing to be replaced. The launch itself took about 6 seconds (1:44:15 to 1:44:21), and after booting and running the user data the new instance showed up as healthy in the target group at around 1:46, so about 2 minutes from kill to fully recovered, with no clicks from me.

Once the new instance was up the scaling group once again had 1 instance in each availability zone. The terminal now prints traffic being evenly rerouted between the newly created i-0266c702496739b06 in us-east-1b and the old unaffected i-0ca5c7c0b17f67601 in us-east-1a!

From this demo we can see that the load balancer evenly distributes traffic between each instance. The load balancer detects when an instance goes unhealthy, and reroutes traffic to healthy instances gracefully, and the auto-scaling group detects the anomaly (instance count going below my specified minimum of 2) which triggered an immediate provisioning of a new instance at the correct availability zone.

Build Walkthrough

Here's how I implemented the entire secure & highly available web app:

1. Make a VPC with 2 availability zones

Log into your AWS console and enter the VPC page. Click "Create VPC"

Select the same settings as in the image and click "Create". Wait 30 seconds and we've just created a VPC with 2 availability zones, with a public and private subnet in each availability zone.

2. Make a security group for the load balancer, and another for the instances that only take traffic from the load balancer

Then at the top left corner type "EC2" and click on the EC2 icon to jump to the EC2 dashboard.

On the left sidebar go to Network & Security -> Security Groups and click "Create security group".

First we make alb-sg for the load balancer. Give it a name and make sure to switch the VPC to web-app-vpc (it picks the default VPC on its own, which got me the first time). Add one inbound rule with type HTTP and source Anywhere-IPv4 (0.0.0.0/0). This is the only thing in the whole project that the internet can reach. Here's what it looks like once it's made:

Then make another one called app-sg for the instances, also in web-app-vpc. Add an inbound rule with type HTTP, source Custom, and pick alb-sg from the dropdown (it shows up as its sg- ID). Now only the load balancer can talk to the instances. Not the public subnet, not my IP, nothing else. And no SSH rule anywhere.

3. Parameter Store: avoid hardcoding data in instances

Search "Systems Manager" in the top search bar, then on the left sidebar click Parameter Store -> "Create parameter". I named mine /webapp/app_message, left it as Standard tier and String type, and typed in the message I want the web page to show. Every instance reads this when it boots, so I never have to put it in the launch template.

4. IAM role for the instances (blocked in learner lab, so I used LabInstanceProfile)

The plan was to make my own role web-app-instances-role with AmazonSSMManagedInstanceCore and a policy that can only read /webapp/*. So I went to IAM -> Roles -> "Create role", picked AWS service -> EC2, and... the learner lab doesn't let me create roles:

So I just used the LabRole that the lab already gives you (it gets attached to instances through LabInstanceProfile). It can do SSM and read Parameter Store, which is all I need, but it can do a lot more than that too. In a real account I'd use the smaller role I talked about in the Identity section.

5. Launch template for the auto-scaling instances

Go back to EC2 and click Instances -> Launch Templates -> "Create launch template". I named it web-app-launch-template and picked Amazon Linux 2023 (64-bit x86).

Instance type is t3.micro. For key pair pick "Don't include in launch template" since nobody is going to SSH into these. Subnet is also "Don't include" because the auto scaling group picks the subnet later. For security group pick app-sg.

Open up Advanced details and set the IAM instance profile to LabInstanceProfile.

Scroll down a bit and set metadata version to "V2 only (token required)" and hop limit to 1. Then paste the script below into User data. This script is basically the whole deployment. It installs nginx, grabs the instance ID and AZ, reads /webapp/app_message from Parameter Store, writes the web page, and adds a /health page for the load balancer to check. Then click "Create launch template".

#!/bin/bash
set -euo pipefail
dnf install -y nginx

# Get instance ID and AZ via IMDSv2
TOKEN=$(curl -sX PUT http://169.254.169.254/latest/api/token \
  -H "X-aws-ec2-metadata-token-ttl-seconds: 300")
md() { curl -s -H "X-aws-ec2-metadata-token: $TOKEN" \
  http://169.254.169.254/latest/meta-data/$1; }
ID=$(md instance-id)
AZ=$(md placement/availability-zone)
REGION=${AZ%?}

# Read config from Parameter Store (falls back if it fails)
MSG=$(aws ssm get-parameter --region "$REGION" \
  --name /webapp/app_message --query Parameter.Value --output text) || MSG="Hello (param read failed)"

cat > /usr/share/nginx/html/index.html <<EOF
<h1>$MSG</h1>
<p>Instance: $ID</p>
<p>AZ: $AZ</p>
EOF
echo ok > /usr/share/nginx/html/health

systemctl enable --now nginx

6. Target group: health checks that decide which instances get traffic

In EC2 go to Load Balancing -> Target Groups -> "Create target group". Pick Instances, name it web-app-target-group, HTTP on port 80, and VPC web-app-vpc.

Set the health check path to /health. Under Advanced health check settings I set healthy and unhealthy threshold to 2, timeout to 5 seconds, and interval to 10 seconds. So if an instance stops responding, it gets kicked out of rotation in about 20 seconds.

Click next and don't register any targets. The auto scaling group will add the instances by itself. Then click "Create target group".

7. Application Load Balancer in the public subnets

Go to Load Balancing -> Load Balancers -> "Create load balancer" and pick Application Load Balancer. Here's what I set:

• Name web-app-alb, Internet-facing, IPv4
• VPC web-app-vpc, check both AZs and pick the public subnet in each one (web-app-subnet-public1 and web-app-subnet-public2)
• Security group: remove default and add alb-sg
• Listener HTTP:80 -> forward to web-app-target-group

Click "Create load balancer" and wait a few minutes for it to spin up. Once it's active, copy the DNS name. That's the link to the app.

8. Auto Scaling Group in the private subnets

Go to Auto Scaling -> Auto Scaling Groups -> "Create Auto Scaling group". I named it web-app-asg and picked web-app-launch-template.

For network pick web-app-vpc and both private subnets (web-app-subnet-private1 and web-app-subnet-private2). This is the part that keeps the instances hidden (no public IP) and spreads them across both AZs. Leave AZ distribution on "Balanced best effort".

For load balancing pick "Attach to an existing load balancer" -> "Choose from your load balancer target groups" -> web-app-target-group. Then check "Turn on Elastic Load Balancing health checks". Don't skip this one. Without it the scaling group only checks if the machine is on, not if the app is actually working, so if nginx crashed it would never get replaced.

Set desired to 2, min to 2, and max to 4. I also added a target tracking policy on average CPU at 50 so it can add instances if traffic gets heavy. Skip notifications and tags and click "Create Auto Scaling group".

After about 3-5 minutes both instances boot up on their own and show up as healthy in the target group, one in each AZ. I didn't touch either of them:

And curling the load balancer's DNS name gives back a 200 with my Parameter Store message and the instance that answered:

9. Tightening the instances' outbound rules

At this point app-sg still let the instances send any traffic out. They only really need to reach Parameter Store, SSM, and the package repos, so I went back to Security Groups -> app-sg -> Outbound rules -> "Edit outbound rules" and swapped "All traffic" for just two rules: HTTPS 443 and HTTP 80 to 0.0.0.0/0. Security groups remember connections that came in, so traffic from the load balancer still works fine, and the app kept giving back 200s after the change.