DEV Community: Mawuli Denteh

Cross-region data transfer from NFS server to Amazon EFS file system using AWS DataSync

Mawuli Denteh — Tue, 28 Apr 2026 00:08:33 +0000

Overview

This AWS DataSync solution transfers data from an EC2-hosted NFS share in VPC A (us-east-1) to an Amazon EFS file system in VPC B (us-west-2). The two VPCs are connected through a peering connection. This setup simulates a typical on-premises to cloud data migration scenario.

An NFS server uses the NFS protocol to share files and directories over a network.

Amazon EFS is a fully managed, serverless cloud file storage service that provides elastic, shared file storage for Linux-based workloads and applications running on AWS services like EC2.

AWS DataSync, a fully managed, high-speed data transfer service, communicates with both storage locations via VPC endpoints, utilizing a private, fast and secure connection to accomplish the transfer.

VPC Endpoints provide private interface entry points to AWS services from within a VPC. While many AWS services have public endpoints that can be accessed over the internet, it is a best practice to use VPC endpoints to communicate with AWS services securely from within a VPC.

Architecture Diagram

US-EAST-1

VPC A

Create a VPC A in us-east-1 with 10.0.0.0/16 Cidr.
Create two public subnets and an internet gateway.
Create two private subnets and a nat gateway.
Create an SSM Endpoint security group with the following rules:
1. Inbound:
  1. HTTPS from VPC A.
2. Outbound
  1. All traffic to VPC A.
Set up SSM Default Host Management Configuration to enable Session Manager to access the NFS server. You can optionally use a bastion host to access the NFS server.
Use the default Host Management role or create and attach an EC2 role to the NFS server for session manager to work. Attach the following AWS managed policies to the EC2 role:
1. AmazonSSMManagedEC2InstanceDefaultPolicy
2. AmazonSSMManagedInstanceCore

VPC A Endpoints

Create a VPC endpoint each for SSM, EC2-Messages, and SSM-Messages as follows:

Create endpoint.
Optional name.
Choose “AWS services” type.
Type “ssm” and search.
Select the first option com.amazonaws.us-east-1.ssm. The name varies according to the service.
Next, select VPC A.
Check “Enable private DNS name”.
DNS record IP type: IPv4.
Subnets. Check the first two Availability Zones in the list. Under Subnet ID, select both private subnets.
IP address type: IPv4.
Security groups: Use the SSM Endpoint Security Group created earlier. You will use this security group for all VPC endpoints in region A.
Policy: Full access.

NFS Server

Create the NFS server with the following specifications:

Launch instances.
Amazon Linux AMI.
t2.micro instance type.
Proceed without a key pair.
PrivateSubnetA.
Auto-assign public IP disabled.
Create NFS server security group:
1. Inbound:
  1. HTTPS from SSM Endpoint.
  2. NFS from VPC B.
  3. ICMP from VPC B.
2. Outbound
  1. All traffic to VPC A and B.
8 GB gp3 EBS volume.
User data for NFS server. Creates and exports NFS share, generates files for the transfer.


#!/bin/bash
set -e
sleep 180
# Check if the secondary drive exists
if ! lsblk /dev/xvdb &>/dev/null; then
    echo “/dev/xvdb not found. Exiting.”
    exit 1
fi

mkfs.ext4 /dev/xvdb
mkdir -p /mnt/data
mount /dev/xvdb /mnt/data
echo ‘/dev/xvdb /mnt/data ext4 defaults,nofail 0 2’ | sudo tee -a /etc/fstab

# Create and set up NFS share directory
TARGET_DIR=”/mnt/data/nfs_share”
PREFIX=”test”
EXT=”.txt”
mkdir -p $TARGET_DIR
chown -R nobody:nobody $TARGET_DIR
chmod 777 $TARGET_DIR

# Function to generate files
generate_files() {
local start=$1
local end=$2
local size_range=$3
local block_size=$4
local group_label=$5

local start_time=$(date +%s)
echo “Starting generation of $group_label at $(date)”

for i in $(seq -f “%04g” $start $end); do
    FILE=”${TARGET_DIR}/${PREFIX}${i}${EXT}”
    SIZE=$((RANDOM % size_range + 1))
    dd if=/dev/urandom of=”$FILE” bs=$block_size count=$SIZE status=none &
done
wait

local end_time=$(date +%s)
echo “Finished generation of $group_label at $(date)”
local elapsed_time=$((end_time - start_time))
echo “Time taken for $group_label: ${elapsed_time} seconds”
}

# Generate file groups with time tracking
generate_files 1 1000 100 1K “1000 files (1KB to 100KB)” &
generate_files 1001 1100 100 1M “100 files (1MB to 100MB)” &

wait
echo “File generation complete in $TARGET_DIR”

sed -i “\|^$TARGET_DIR |d” /etc/exports
echo “$TARGET_DIR *(rw,sync,no_root_squash,no_all_squash)” | sudo tee -a /etc/exports > /dev/null
systemctl start nfs-server
systemctl enable nfs-server
exportfs -rav

Verify that you can connect to the NFS server via Session Manager.

US-WEST-2

VPC B

Create a target VPC in us-west-2 with 172.31.0.0/16 Cidr.
Create two public subnets and an internet gateway.
Create two private subnets and a NAT gateway.
Follow the same steps as in Region A to enable Session Manager to access the NFS client.
Create and attach an SSM Endpoint Security Group to all the endpoints:
1. Inbound:
  1. HTTPS from DataSync agent.
  2. NFS from VPC B.
  3. 1024 – 1064 from DataSync agent.
2. Outbound
  1. All traffic to VPC B.
Attach the same EC2 role created from VPC A to the NFS client if you are in the same account. For different accounts, you need to create a new EC2 role for the NFS client.
Create and attach an EC2 role to the DataSync agent. Attach the following AWS managed policies to the role:
1. AWSDataSyncFullAccess
2. AmazonSSMManagedEC2InstanceDefaultPolicy
3. AmazonSSMManagedInstanceCore

DataSync Agent

Create the DataSync agent using the specifications below:

DataSync latest AMI.
t3.micro instance type.
Proceed without a key pair.
PrivateSubnetA.
Auto-assign public IP disabled.
Create a DataSync agent security group as follows:
1. Inbound:
  1. HTTPS from VPC B.
  2. HTTP from NFS client.
  3. ICMP from NFS client.
2. Outbound
  1. All traffic to VPC A and B.
20 GB gp3 EBS volume.

VPC B Endpoints

Follow similar procedures from US-EAST-1 to create VPC endpoints for:

SSM.
EC2-Messages.
SSM-Messages.
EFS.
DataSync.

NFS Client

Create the NFS client EC2 as shown below:

Launch instances.
Amazon Linux AMI.
t2.micro instance type.
Proceed without a key pair.
PrivateSubnetA.
Create an NFS client security group:
1. Inbound:
  1. HTTPS from VPC B.
2. Outbound
  1. All traffic to VPC A and B.
8 GB gp3 EBS volume.

EFS file system

Create an EFS Security Group with the following rules:
1. Inbound:
  1. NFS traffic from VPC B.
  2. NFS traffic from NFS Client.
  3. NFS traffic from DataSync agent.
  4. NFS traffic from SSM Endpoint.
  5. HTTPS from VPC B and SSM Endpoint.
  6. ICMP from NFS client.
2. Outbound
  1. All traffic to VPC B.
Create file system.
Select the target VPC.
Keep the recommended settings and Create file system.
Click on the file system ID.
Go to the Network tab.
Manage.
For the two mount points, edit the security groups to use only the earlier created EFS Security Group.

VPC Peering

Create peering connection.
Optional name.
VPC ID (Requester): VPC A ID.
Select another VPC to peer with: My account.
Region: Another Region.
VPC ID (Accepter): VPC B ID.

From Region B (us-west-2) Peering Connections, select the peering connection available.
From the “Actions” menu, Accept request. Confirm.

Update the private route table for VPC A to forward Region B traffic to the peering connection.

Update the private route table for VPC B to forward Region A traffic to the peering connection.

Systems Setup

Mount the NFS share

Connect to the NFS client via Session Manager.
Mount the NFS share from the NFS client over the peering connection.

# Ping the NFS server to verify connection
ping -c 5 10.0.x.x

# Mount NFS server
sudo mkdir -p /mnt/nfs
sudo mount -t nfs 10.0.x.x:/mnt/data/nfs_share /mnt/nfs -o rw,sync

Mount the EFS file system

Mount the EFS file system from the NFS client.

# Mount EFS file system
sudo mkdir -p /mnt/efs

sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-xxxxxxxxxxxxxxx.efs.us-west-2.amazonaws.com:/ /mnt/efs
OR
sudo mount -t nfs4 -o sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport 172.31.x.x:/ /mnt/efs

DataSync agent

Obtain the activation code of the DataSync agent from the NFS client.

# Test connectivity to the agent
nc -vz 172.31.x.x 80 

# Obtain activation code
sudo curl “http://<datasync-agent-ip>/?gatewayType=SYNC&activationRegion=us-west-2&privateLinkEndpoint=<datasync-vpce-ip>&endpointType=PRIVATE_LINK&no_redirect”

# NB: Replace <datasync-agent-ip> and <datasync-vpce-ip> with actual values

Activate the DataSync agent using the activation key.

aws datasync create-agent \
    --agent-name “datasync-agent” \
    --activation-key “xxxxx-xxxxx-xxxxx-xxxxx-xxxxx” \
    --vpc-endpoint-id “vpce-xxxxxxxxxxxxxxxxx” \
    --subnet-arns “arn:aws:ec2:us-west-2:accountId:subnet/subnet-xxxxxxxxxxx” \
    --security-group-arns “arn:aws:ec2:us-west-2:accountId:security-group/sg-xxxxxxxxxxxx”

Create the NFS location.

aws datasync create-location-nfs \
    --server-hostname “10.0.x.x” \
    --subdirectory “/mnt/data/nfs_share” \
    --on-prem-config AgentArns=”arn:aws:datasync:us-west-2:accountId:agent/agent-xxxxxxxxxxx”

Create the EFS location.

aws datasync create-location-efs \
    --efs-filesystem-arn “arn:aws:elasticfilesystem:us-west-2:accountId:file-system/fs-xxxxxxxxxxxxx” \
    --ec2-config SubnetArn=”arn:aws:ec2:us-west-2:accountId:subnet/subnet-xxxxxxxxxxxxx”,SecurityGroupArns=”arn:aws:ec2:us-west-2:accountId:security-group/sg-xxxxxxxxxxxxxxx”

Create a task that connects both locations and start the task.

aws datasync create-task \
    --source-location-arn “arn:aws:datasync:us-west-2:accountId:location/loc-xxxxxxxxxxxxxx” \
    --destination-location-arn “arn:aws:datasync:us-west-2:accountId:location/loc-xxxxxxxxxxxxxx”

YouTube Tutorial

References

Create a cloud-backed IDE for development

Mawuli Denteh — Thu, 23 Oct 2025 23:32:19 +0000

Problem

I find it burdensome creating and configuring my development environment all over again when using a new laptop or system.

The need for speed when installing packages, extensions, and compiling code is also very important, and the network plays a big role in that.

After setting it all up, I also don’t want anything to disrupt the environment. I also want to be able to log in or access it reliably and easily anytime I need from any machine without having to set it all up again.

Having dealt with this scenario many times, I found a way to create this environment, call upon it when I need and put it off when I don’t — and also have the ability to back it up anytime I make progress with new states of the storage/system.

This may not be for everyone, but I think many developers may find value in this setup depending on their situation.

Solution

Create a Linux server in the cloud and use VS Code’s Remote SSH feature to securely access it, with the capability to manage the file system from the editor like you would locally.

I call it “The DevBox”. Sounds cool, doesn’t it? Let’s build it.

Cost Considerations

This solution is going to cost a roughly fixed amount per month for EBS storage and snapshots (minimal) and a variable amount for EC2 compute, depending on how many hours you keep the server up in a month.

Take note and check the cost of what you are going to build (using the AWS Pricing Calculator) based on how you will use it.

Step 1 – Create a Dedicated Network for the DevBox

I chose to create a small VPC with a /24 network, which is just ideal for this case.

You can optionally add an S3 Private Gateway Endpoint for no extra charge. This would be useful for copying files from authorized S3 buckets into your environment when needed.

I chose a simple public subnet design with a restricted instance security group allowing SSH access only from my IP address.

Step 2 – Create the DevBox EC2 Instance

You can go with the latest Amazon Linux AMI or Ubuntu 24.04 LTS, which I used in this setup.

Restrict SSH access to your IP in the security group for security.

Step 3 – Configure SSH Access in VS Code

Right-click the left panel of your VS Code editor and ensure the Remote Explorer is checked.

Right-click on SSH and open the config file.
Edit the config file as shown below:

User: Varies depending on the AMI you use.
IdentityFile: Points to the location of your key file.
HostName: The remote server IP address.

Connect in a new window.

Confirm the step by hitting the Enter key.

It should take a few seconds to connect and configure — and boom! You’re connected via SSH within VS Code, while utilizing cloud storage and networking.

Further Steps

Install your packages and set up your environment as needed.
Expand the storage as you grow.
Optionally add a second EBS volume as the data/working volume.
Connect to an S3 bucket within your account via the S3 Gateway.
Back up the volume, stop, and/or terminate the instance as needed.

Customize VPCs with CloudFormation Conditions

Mawuli Denteh — Mon, 13 Jan 2025 03:29:22 +0000

Using CloudFormation Conditions you can decide which resources to create/configure from your CloudFormation template.

In this post, we are going to build a template that can create a VPC with private subnets and a NAT gateway or only public subnets depending on a parameter when creating the CloudFormation stack.

Steps to create and use Conditions

Define the input parameters you want your condition to evaluate.
Define the condition by using the intrinsic condition functions.
Declare the condition in resources or outputs you want to create.

Let's dive into the example!

Define the parameter and condition

In the first part of our template, we'll define our parameter and condition.

We associate the CreatePrivateResources condition with a value/option from our CreateNatGateway parameter. This enables us to do what has been discussed in the intro above.

Parameters:
  CreateNatGateway:
    Type: String
    Description: "Need a NAT Gateway?"
    AllowedValues:
      - yes
      - no
    Default: yes

Conditions:
  CreatePrivateResources: !Equals
    - !Ref CreateNatGateway
    - yes

Define the public resources

Here, we will define our VPC, internet gateway, public subnets, routes and route tables. These resources will always be created. This is because they will not have any condition associated with them.

Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: "10.0.0.0/16"
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-VPC"

  # Public Resources
  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-InternetGateway"

  AttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref InternetGateway

  PublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: "10.0.1.0/24"
      MapPublicIpOnLaunch: true
      AvailabilityZone: !Select [0, !GetAZs ""]
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-PublicSubnet1"

  PublicSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: "10.0.2.0/24"
      MapPublicIpOnLaunch: true
      AvailabilityZone: !Select [1, !GetAZs ""]
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-PublicSubnet2"

  PublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-PublicRouteTable"

  PublicRoute:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref PublicRouteTable
      DestinationCidrBlock: "0.0.0.0/0"
      GatewayId: !Ref InternetGateway

  PublicSubnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnet1
      RouteTableId: !Ref PublicRouteTable

  PublicSubnet2RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnet2
      RouteTableId: !Ref PublicRouteTable

Use the condition

We now apply our condition to selected resources which creates a private environment for us, i.e. NatGateway, private subnets and routes etc. If we choose no at the prompt, these will not get created.

We also control the display of outputs using the same conditions.

# Private Resources
  NatGateway:
    Condition: CreatePrivateResources
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt EIPNatGateway.AllocationId
      SubnetId: !Ref PublicSubnet1
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-NatGateway"

  EIPNatGateway:
    Condition: CreatePrivateResources
    Type: AWS::EC2::EIP
    Properties:
      Domain: vpc
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-EIPNatGateway"

  PrivateSubnet1:
    Condition: CreatePrivateResources
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: "10.0.3.0/24"
      MapPublicIpOnLaunch: false
      AvailabilityZone: !Select [0, !GetAZs ""]
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-PrivateSubnet1"

  PrivateSubnet2:
    Condition: CreatePrivateResources
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: "10.0.4.0/24"
      MapPublicIpOnLaunch: false
      AvailabilityZone: !Select [1, !GetAZs ""]
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-PrivateSubnet2"

  PrivateRouteTable:
    Condition: CreatePrivateResources
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub "${AWS::StackName}-PrivateRouteTable"

  PrivateRoute:
    Condition: CreatePrivateResources
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref PrivateRouteTable
      DestinationCidrBlock: "0.0.0.0/0"
      NatGatewayId: !Ref NatGateway

  PrivateSubnet1RouteTableAssociation:
    Condition: CreatePrivateResources
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PrivateSubnet1
      RouteTableId: !Ref PrivateRouteTable

  PrivateSubnet2RouteTableAssociation:
    Condition: CreatePrivateResources
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PrivateSubnet2
      RouteTableId: !Ref PrivateRouteTable

Outputs:
  VPCId:
    Description: "VPC ID"
    Value: !Ref VPC
  PublicSubnet1Id:
    Description: "Public Subnet 1 ID"
    Value: !Ref PublicSubnet1
  PublicSubnet2Id:
    Description: "Public Subnet 2 ID"
    Value: !Ref PublicSubnet2
  PrivateSubnet1Id:
    Condition: CreatePrivateResources
    Description: "Private Subnet 1 ID"
    Value: !Ref PrivateSubnet1
  PrivateSubnet2Id:
    Condition: CreatePrivateResources
    Description: "Private Subnet 2 ID"
    Value: !Ref PrivateSubnet2
  NatGatewayId:
    Condition: CreatePrivateResources
    Description: "NAT Gateway ID"
    Value: !Ref NatGateway

You can now customize your VPC creation just by changing a parameter.
Thanks for reading. Happy Building!

Create a load-balanced web server with auto-scaling

Mawuli Denteh — Wed, 06 Mar 2024 00:00:00 +0000

Overview

This tutorial walks you through the process of creating web servers, managed by an auto scaling group. The auto scaling group uses a launch template to create the servers. The auto scaling group launches the servers into a target group. An internet-facing load balancer directs traffic to the target group.

Architecture Diagram

1. Create a VPC

A VPC is an isolated, private network you can create to run your workloads. You have complete control over your VPC when you create one.

Click on Create VPC from the VPC Dashboard to create a new VPC.
Select VPC and more.
Enter a name under Auto-generate.
Choose a 10.0.0.0/16 IPV4 Cidr block.
Number of Availability Zones (AZs) = 2.
Number of public subnets = 2.
Number of private subnets = 0.
NAT gateways ($): None.
VPC endpoints = None.
Leave all other settings as default.
Click Create VPC.

Demo

2. Create a Launch Template

The launch template will serve as the blueprint for creating the exact type of server we need to meet our web server demands. A launch template can be modified to create new versions when you need to change a configuration.

Click on Create launch template from the EC2 console to create a new launch template.
Launch template name - required.
Check the Provide guidance to help me set up a template that I can use with EC2 Auto Scaling box.
Under Application and OS Images (Amazon Machine Image) - required, choose Amazon Linux 2023 AMI.
Instance type = t2.micro.
Key pair - Proceed without a keypair.
Subnet - Don't include in launch template.
Create security group.
Allow HTTP traffic from 0.0.0.0/0 (Ignore the warning about security group. We will edit it later).
VPC - Select the VPC you created.
Under Advanced network configuration, choose Enable under Auto-assign public IP.
Under Storage, leave all other configuration as default and choose "gp3" for Volume type.
Resource tags: optional.
Under Advanced details, scroll down to the User data section and enter the following lines of code exactly as shown:

#!/bin/bash
set -e
# Install packages and Nginx server
sudo apt-get update
sudo apt-get -y install nginx
sudo service nginx status

# Query the instance metadata service
TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")

# Fetch the hostname
HOSTNAME=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/hostname)

# Add web server content
sudo tee /var/www/html/index.nginx-debian.html > /dev/null << EOF
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>WebServer</title>
    <style>
        body {
            font-family: Arial, sans-serif;
            margin: 20px;
            padding: 0;
            text-align: center;
        }
        .container {
            max-width: 600px;
            margin: 0 auto;
            border: 1px solid #ccc;
            padding: 20px;
            border-radius: 8px;
            box-shadow: 0 0 10px rgba(0,0,0,0.1);
        }
        h1 {
            color: #333;
        }
    </style>
</head>
<body>
    <div class="container">
        <h1>Hello from $HOSTNAME</h1>
    </div>
</body>
</html>
EOF

# Configure stress for autoscaling
sudo apt-get install -y stress
cat << 'EOF' > /home/ubuntu/start_stress.sh
#!/bin/bash
sleep 300
vcpus=$(nproc)
workers=$vcpus
stress --cpu "$workers" --timeout 1200
EOF
chmod +x /home/ubuntu/start_stress.sh
nohup /home/ubuntu/start_stress.sh > /home/ubuntu/start_stress.log 2>&1 &
sudo service nginx restart

Demo

3. Create Target Group

A target group will route requests to the web servers we create. Our load balancer will need this target group to know what set of servers to distribute traffic to. Our auto scaling group will also be associated with this target group so it launches our servers into the target group.

Click on Create target group from the EC2 console to create a target group.
Choose a target type: Instances.
Target group name: Enter a name.
Protocol: HTTP.
Port: 80.
VPC: Select the VPC you created.
Leave every other value on this page as default. Next
Register Targets: Leave as is.
Click Create target group.

Demo

4. Create Load Balancer

An application load balancer acts as the entry point for traffic to our web servers. Instead of allowing users to access our application directly, we will use the load balancer to distribute traffic equally among our autoscaling group of web servers. This is better for load management, security and reliability of our application.

Click on Create load balancer from the EC2 console to create a load balancer
Type: Application Load Balancer.
Scheme: Internet-facing.
IP address type: IPV4.
VPC: Select the VPC you created.
Mappings: Check the box beside the two AZs listed.
Subnet: For each AZ selected, choose the public subnet in the dropdown menu.
At this point, go to the Security groups console and create a new security group for the load balancer. The inbound rule should allow HTTP traffic from anywhere.
Select this security group as the load balancer security group.
Listeners and routing: Leave protocol and port as HTTP:80.
Select the target group you created as target group.
Leave every other config as default and click Create load balancer.

Demo

5. Create Auto Scaling Group

The auto scaling group configures and controls how your application scales automatically in response to varying traffic situations.

Click on Create Auto Scaling group from the EC2 console to create an auto scaling group.
Enter a name.
Choose the launch template you created. Click Next.
Select your webserver VPC created from the VPC step.
Under Availability Zones and subnets, select the two public subnets in your VPC, in different AZs. Click Next.

NB: Note that you can use the auto scaling group to override your instance type config from the launch template*.

Under Load balancing, choose the option Attach to an existing load balancer.
Select Choose from your load balancer target groups.
Select the target group you created.
Select VPC Lattice service to attach: No VPC Lattice service.
Additional health check types - optional: Turn on Elastic Load Balancing health checks.
Leave every other config as default. Next.
Group size: Desired: 2, Minimum: 1, Maximum: 4.
Scaling policies: Target Tracking Policy.
Metric type: Average CPU Utilization.
Target Value: 50%.
Create Auto Scaling Group.

Demo

Once you successfully create your autoscaling group, you should see two new instances in the EC2 console. This is because we specified a desired count of 2. Also note that they are automatically placed one in each AZ to support high availability.

Restrict web traffic to servers

With the current design, users can directly access our web server using its IP address. We don't want that. That is why we created a load balancer.

To ensure all incoming HTTP traffic goes through the load balancer, we will update the webserver security group to accept HTTP traffic only from our application load balancer security group.

Go to the autoscale-webserver-sg security group and click on Edit inbound rules.
Delete the existing HTTP rule.
Add a new HTTP rule. In the Source box, scroll down to select the security group of the load balancer. Save rules.
You have successfully restricted traffic going to the servers to the load balancer.
You should no longer be able to access your web server using the server IPs or DNS names. You should now be able to use the load balancer DNS name to access the servers. Test this out.

Observations

From the details tab of the load balancer, copy the DNS name and paste in a new tab. Refresh the tab and note the changing hostnames with every refresh. That's the load balancer alternating traffic between its target web servers.

After launching the desired number of instances, a bash script will run in the background to increase CPU utilization to 50%. This will trigger a scale out action.

This should lead to a maximum number of 4 instances being available to serve web traffic.

When the script stops running, a scale in action should equally be triggered to reduce the instances back to the desired number.

It takes about an hour to observe this.

Conclusion

You have built a load-balanced and highly available web application that auto-scales based on a target of CPU utilization.

Delete the autoscaling group and load balancer after your tests to prevent unwanted charges.

If you made it this far, here's a little reward:

The CloudFormation stack below will deploy exactly what you have built in the demo. Try it out.

Download IAC template