Using CloudFormation to deploy a web app with HA

INTRODUCTION

In this post, I am thrilled to share an exciting project I had the opportunity to work on. I embarked on a journey that not only expanded my knowledge but also empowered me to apply cutting-edge cloud computing and DevOps practices. Specifically, this project delved into Infrastructure as Code with AWS CloudFormation, providing me with invaluable hands-on experience in building scalable cloud infrastructure.

STATEMENT OF PROBLEM

Scenario
Your company is creating an Instagram clone.

Developers want to deploy a new application to the AWS infrastructure.

You have been tasked with provisioning the required infrastructure and deploying a dummy application, along with the necessary supporting software.

This needs to be automated so that the infrastructure can be discarded as soon as the testing team finishes their tests and gathers their results.

Optional - To add more challenge to the project, once the project is completed, you can try deploying sample website files located in a public S3 Bucket to the Apache Web Server running on an EC2 instance.

---

Server specs

Launch Configuration was created for application servers in order to deploy four servers, two located in each of your private subnets. The launch configuration was used by an auto-scaling group. Two vCPUs were used with 4GB of RAM. The Operating System used is Ubuntu 18. An Instance size and Machine Image (AMI) that best fits this spec was chosen.

---

MY SOLUTION:

Architecture Diagram

Below is the content of the parameter files and configuration files for the network infrastructure, s3 buckets, and servers(EC2 instances)

Network Parameters

network.json

[
    {
    "ParameterKey": "EnvironmentName",
    "ParameterValue": "UdacityProject"
    },
    {
    "ParameterKey": "VPCCIDR",
    "ParameterValue": "10.0.0.0/16"
    },
    {
    "ParameterKey": "PubSubnet1CIDR",
    "ParameterValue": "10.0.1.0/24"
    },
    {
    "ParameterKey": "PubSubnet2CIDR",
    "ParameterValue": "10.0.2.0/24"
    },
    {
    "ParameterKey": "PrivSubnet1CIDR",
    "ParameterValue": "10.0.3.0/24"
    },
    {
    "ParameterKey": "PrivSubnet2CIDR",
    "ParameterValue": "10.0.4.0/24"
    }

    ]

S3 bucket Parameters

s3bucket.json

[{
    "ParameterKey": "EnvironmentName",
    "ParameterValue": "UdacityProject"
},
{
    "ParameterKey": "S3BucketName",
    "ParameterValue": "udacityprojects3webserverbucket"
}
]

Server (EC2 Instance) Parameters

servers.json

[
    {
    "ParameterKey": "EnvironmentName",
    "ParameterValue": "UdacityProject"
    }
]

Network Configuration

The network.yaml file below creates a network infrastructure with public and private subnets, routing, and internet access. It includes parameters for customizing the environment, VPC CIDR, and subnet CIDR blocks. The code creates resources such as VPC, internet gateway, subnets (public and private), NAT gateways, and route tables. Outputs are defined to export important values like VPC ID, route table IDs, and subnet IDs. This CloudFormation template enables the creation of a network setup suitable for routing internet traffic to both public and private subnets.

network.yaml

AWSTemplateFormatVersion: "2010-09-09"
Description: Creates the required network infrastructure for public and private routing with internet access
Parameters: 
  EnvironmentName: 
    Description: An Environment name that will be prefixed to resources
    Type: String
  VPCCIDR:
    Type: String
  PrivSubnet1CIDR:
    Type: String
  PrivSubnet2CIDR:
    Type: String
  PubSubnet1CIDR:
    Type: String
  PubSubnet2CIDR:
    Type: String

Resources:
  myVPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VPCCIDR
      EnableDnsHostnames: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: "MainVPC"

# Create Internet Gateway
  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Ref EnvironmentName

# Attached the internet Gateway to myVPC
  InternetGatewayAttached:    
    Type: AWS::EC2::VPCGatewayAttachment
    Properties: 
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref myVPC

# Creating Public and Private Subnets in the same availability zone(us-east-1a).
  PublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: "us-east-1a"
      CidrBlock: !Ref PubSubnet1CIDR
      VpcId:
        Ref: myVPC
      MapPublicIpOnLaunch: true
      Tags:
      - Key: Name
        Value: !Sub ${EnvironmentName} Public Subnet (AZ1)

  PrivateSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId:
        Ref: myVPC
      CidrBlock: !Ref PrivSubnet1CIDR
      MapPublicIpOnLaunch: false
      AvailabilityZone: "us-east-1a"
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName} Private Subnet (AZ1)


# Creating Public and Private Subnets in the same availability zone(us-east-1b).
  PublicSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: "us-east-1b"
      CidrBlock: !Ref PubSubnet2CIDR
      VpcId:
        Ref: myVPC
      MapPublicIpOnLaunch: true
      Tags:
      - Key: Name
        Value: !Sub ${EnvironmentName} Public Subnet (AZ2)

  PrivateSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId:
        Ref: myVPC
      CidrBlock: !Ref PrivSubnet2CIDR
      MapPublicIpOnLaunch: false
      AvailabilityZone: "us-east-1b"
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName} Private Subnet (AZ2)

# Elastic IP for the NATGateway in Subnet1
  EIP1:
    Type: AWS::EC2::EIP
    DependsOn: InternetGatewayAttached
    Properties:
        Domain: myVPC
        Tags:
        - Key: Name
          Value: "Elastic IP for our NATGateway1"
  EIP2:
    Type: AWS::EC2::EIP
    DependsOn: InternetGatewayAttached
    Properties:
        Domain: myVPC
        Tags:
        - Key: Name
          Value: "Elastic IP for our NATGateway2"

# Creating NAT gateway in publicsubnet1
  NAT1:
    Type: AWS::EC2::NatGateway
    Properties:
        AllocationId:
          Fn::GetAtt:
          - EIP1
          - AllocationId
        SubnetId: !Ref PublicSubnet1
        Tags:
        - Key: Name
          Value: "NAT to be used by servers in the private subnet"
  NAT2:
    Type: AWS::EC2::NatGateway
    Properties:
        AllocationId:
          Fn::GetAtt:
          - EIP2
          - AllocationId
        SubnetId: !Ref PublicSubnet2
        Tags:
        - Key: Name
          Value: "NAT to be used by servers in the private subnet"



#_______PUBLIC SUBNET________
# # Route Table for Public subnet
  PublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref myVPC
      Tags:
      - Key: Name
        Value: !Sub ${EnvironmentName} Public Routes

# Create Route for public Subnet 1 & 2 
  PublicRoute:
    Type: AWS::EC2::Route
    DependsOn: InternetGatewayAttached
    Properties:
        RouteTableId: !Ref PublicRouteTable
        DestinationCidrBlock: 0.0.0.0/0
        GatewayId:
          Ref: InternetGateway

# Associate Route Table to Public subnet 1 & 2
  AssociatePublicRoute:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties: 
      RouteTableId: 
        Ref: PublicRouteTable
      SubnetId: !Ref PublicSubnet1

  AssociatePublicRoute:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties: 
      RouteTableId: 
        Ref: PublicRouteTable
      SubnetId: !Ref PublicSubnet2


# ______PRIVATE SUBNET 1______
# Route Table for Private subnet1

  PrivateRouteTable1:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref myVPC
      Tags:
      - Key: Name
        Value: !Sub ${EnvironmentName} Private Routes (AZ1)

# Create Route for Private subnet1
  PrivateRoute1:
    Type: AWS::EC2::Route
    Properties:
        RouteTableId: !Ref PrivateRouteTable1
        DestinationCidrBlock: 0.0.0.0/0
        NatGatewayId:
          Ref: NAT1

# Private Route Table1 Association to Private Subnet1
  AssociatePrivateRoute:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties: 
      RouteTableId: !Ref PrivateRouteTable1
      SubnetId: !Ref PrivateSubnet1



# _____PRIVATE SUBNET 2_____
# Route Table for Private subnet2
  PrivateRouteTable2:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref myVPC
      Tags:
      - Key: Name
        Value: !Sub ${EnvironmentName} Private Routes (AZ2)

# Create Route for Private subnet2
  PrivateRoute2:
    Type: AWS::EC2::Route
    Properties:
        RouteTableId: !Ref PrivateRouteTable2
        DestinationCidrBlock: 0.0.0.0/0
        NatGatewayId:
          Ref: NAT2

# Private Route Table2 Association to Private Subnet2
  AssociatePrivateRoute:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties: 
      RouteTableId: !Ref PrivateRouteTable2
      SubnetId: !Ref PrivateSubnet2


Outputs:
  myVPC:
    Description: The VPC created for this project
    Value: !Ref myVPC
    Export:
      Name: !Sub ${EnvironmentName}-VPCID

  PublicRouteTable: 
        Description: Public Route Table
        Value: !Ref PublicRouteTable
        Export:
          Name: !Sub ${EnvironmentName}-PUB-RT

  PrivateRouteTable1: 
        Description: Private route Table1
        Value: !Ref PrivateRouteTable1
        Export:
          Name: !Sub ${EnvironmentName}-PRI-RT1

  PrivateRouteTable2: 
        Description: Private route Table2
        Value: !Ref PrivateRouteTable2
        Export:
          Name: !Sub ${EnvironmentName}-PRI-RT2

  PublicSubnets:
        Description: A list of the public subnets
        Value: !Join [ ",", [ !Ref PublicSubnet1, !Ref PublicSubnet2 ]]
        Export:
          Name: !Sub ${EnvironmentName}-PUB-SUBNETS

  PublicSubnet1:
        Description: public subnet 1 in "us-east-1a"
        Value: !Ref PublicSubnet1
        Export:
          Name: !Sub ${EnvironmentName}-PUB-SUB1

  PublicSubnet2:
        Description: public subnet 2 in us-east-1b
        Value: !Ref PublicSubnet2
        Export:
          Name: !Sub ${EnvironmentName}-PUB-SUB2

  PrivateSubnets:
        Description: A list of the private subnets
        Value: !Join [ ",", [ !Ref PrivateSubnet1, !Ref PrivateSubnet2 ]]
        Export:
          Name: !Sub ${EnvironmentName}-PRIV-SUBNETS

  PrivateSubnet1:
        Description: private subnet 1 in us-east-1a
        Value: !Ref PrivateSubnet1
        Export:
          Name: !Sub ${EnvironmentName}-PRIV-SUB1

  PrivateSubnet2:
        Description: private subnet 1 in us-east-1b
        Value: !Ref PrivateSubnet2
        Export:
          Name: !Sub ${EnvironmentName}-PRIV-SUB2

  VPCdefaultSecurityGroup:
        Description: Returns the default security group of the created VPC
        Value: !GetAtt myVPC.DefaultSecurityGroup
        Export:
          Name: !Sub ${EnvironmentName}-myVPC-SG

S3 Bucket Configuration

The s3bucket.yaml file below creates an S3 bucket for deploying a high-availability web app. The bucket is configured with public read access, an index document, and an error document. A bucket policy allows all actions on the bucket, and an IAM role with AmazonS3FullAccess policy is created to enable EC2 instances to manage the web app. Outputs include the IAM role, website URL, and secure website URL. This CloudFormation template facilitates the setup of an S3 bucket for hosting a high-availability web app with appropriate permissions and URL accessibility.

s3bucket.yaml

Description: >
  Create an S3 bucket for deploying a high-availability web-app.


Parameters:
  EnvironmentName:
    Description: An environment name that will be prefixed to resource names.
    Type: String

  S3BucketName:
    Description: S3 bucket name.
    Type: String


Resources:
  S3WebServer:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref S3BucketName
      AccessControl: PublicRead
      WebsiteConfiguration:
        IndexDocument: index.html
        ErrorDocument: error.html
      Tags: 
        - Key: Name
          Value: !Sub ${EnvironmentName} s3webserver bucket
    DeletionPolicy: Delete

  S3WebAppPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref S3WebServer
      PolicyDocument:
        Statement:
          - Effect: Allow
            Action: s3:*
            Resource: !Join ['', ['arn:aws:s3:::', !Ref 'S3WebServer', '/*']]
            Principal:
              AWS: '*'

  WebServerIAMRole:
    Type: 'AWS::IAM::Role'
    Properties:
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/AmazonS3FullAccess'
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: 'Allow'
            Principal:
              Service:
                - 'ec2.amazonaws.com'
            Action:
              - 'sts:AssumeRole'
      Path: '/'

  MyInstanceProfile: 
    Type: "AWS::IAM::InstanceProfile"
    Properties: 
      Path: "/"
      Roles: 
        - 
          Ref: "WebServerIAMRole"



Outputs:

  WebServerIAMRole:
    Description: 'Allow EC2 instances to manage Web App S3'
    Value: !Ref MyInstanceProfile
    Export:
      Name: !Sub ${EnvironmentName}-IAM-NAME

  # WebServerIAMRole:
  #   Description: Iam Instance Profile Arn
  #   Value: !GetAtt WebServerIAMRole.Arn
  #   Export:
  #     Name: !Sub ${EnvironmentName}-IAM-ARN

  WebsiteURL:
    Value: !GetAtt [S3WebServer, WebsiteURL]
    Description: URL for website hosted on S3
  WebsiteSecureURL:
    Value: !Join ['', ['https://', !GetAtt [S3WebServer, DomainName]]]
    Description: Secure URL for website hosted on S3

Server (EC2 instance) Configuration

The servers.yaml file below creates a network infrastructure with servers for hosting a high-availability web app.
The code defines several resources, including security groups, launch configuration, auto scaling group, load balancer, listener, target group, scaling policies, and outputs. These resources enable the setup of a load-balanced environment for the web app.
The code provides an output called LoadBalancerEndpoint, which represents the endpoint for reaching the load balancer.
Overall, this CloudFormation template facilitates the creation of a load-balanced infrastructure with auto scaling capabilities for hosting web servers.

servers.yaml

AWSTemplateFormatVersion: "2010-09-09"
Description: Creates the required servers in the network infrastructure defined by "network.yml"
Parameters: 
  EnvironmentName: 
    Description: An Environment name that will be prefixed to resources
    Type: String

  InstanceType:
    Description: Amazon EC2 instance type for the instances
    Type: String
    AllowedValues:
      - t2.micro
      - t3.micro
      - t3.small
      - t3.medium
    Default: t2.micro

Mappings:
      WebServerRegion:
        us-east-1:
          HVM64: ami-052efd3df9dad4825


Resources:

#Loadbalancer security group.
  LBSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow http request to loadbalancer
      VpcId: 
        Fn::ImportValue:
          !Sub "${EnvironmentName}-VPCID"
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
      SecurityGroupEgress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0

# Web Server security group.
  InstanceSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow http to Webserver
      VpcId: 
        Fn::ImportValue:
          !Sub "${EnvironmentName}-VPCID"
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: 0.0.0.0/0
      SecurityGroupEgress:
        - IpProtocol: tcp
          FromPort: 0
          ToPort: 65535
          CidrIp: 0.0.0.0/0

  WebServerLaunchConfig:
    Type: AWS::AutoScaling::LaunchConfiguration
    Properties:
      IamInstanceProfile:
        Fn::ImportValue: !Sub '${EnvironmentName}-IAM-NAME'
      UserData: 
          # wget -P /var/www/html https://project2udacity.s3-us-west-2.amazonaws.com/index.html
        Fn::Base64: !Sub |
          #!/bin/bash
          apt-get update -y
          apt-get install unzip awscli -y
          apt-get install apache2 -y
          systemctl start apache2.service
          sudo rm /var/www/html/index.html
          sudo aws s3 cp s3://udacityprojects3webserverbucket/udagram.zip /var/www/html
          sudo unzip /var/www/html/udagram.zip -d /var/www/html
          sudo rm /var/www/html/udagram.zip 
          systemctl restart apache2.service
      ImageId: !FindInMap [WebServerRegion, !Ref 'AWS::Region', HVM64]
      SecurityGroups: 
        - !Ref InstanceSecurityGroup
      InstanceType: 
        !Ref InstanceType
      BlockDeviceMappings: 
        - DeviceName: /dev/sda1
          Ebs: 
            VolumeSize: '10'
            VolumeType: 'gp2'

  WebServerASG:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      VPCZoneIdentifier:
      - Fn::ImportValue: !Sub "${EnvironmentName}-PRIV-SUB1"
      - Fn::ImportValue: !Sub "${EnvironmentName}-PRIV-SUB2"
      LaunchConfigurationName: !Ref WebServerLaunchConfig
      MaxSize: '4'
      MinSize: '4'
      DesiredCapacity: '4'
      TargetGroupARNs:
      - Ref: WebServerTargetGroup

  WebServerloadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Subnets:
        - Fn::ImportValue: !Sub "${EnvironmentName}-PUB-SUB1"
        - Fn::ImportValue: !Sub "${EnvironmentName}-PUB-SUB2"
      SecurityGroups:
      - Ref: LBSecurityGroup

  Listener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
      - Type: forward
        TargetGroupArn:
          Ref: WebServerTargetGroup
      LoadBalancerArn:
        Ref: WebServerloadBalancer
      Port: '80'
      Protocol: HTTP

  ALBListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties:
      Actions:
      - Type: forward
        TargetGroupArn:
          Ref: WebServerTargetGroup
      Conditions:
      - Field: path-pattern
        Values: [/]
      ListenerArn:
        Ref: Listener
      Priority: 1

  WebServerTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      HealthCheckIntervalSeconds: 5
      HealthCheckPath: /
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 4
      HealthyThresholdCount: 3
      Port: 80
      Protocol: HTTP
      UnhealthyThresholdCount: 3
      VpcId:
        Fn::ImportValue:
          Fn::Sub: "${EnvironmentName}-VPCID"

# Scaling Policy Description (Up)
  WebServerScaleDown:
    Type: AWS::AutoScaling::ScalingPolicy
    Properties:
      AdjustmentType: ChangeInCapacity
      AutoScalingGroupName: !Ref WebServerASG
      Cooldown: 300
      ScalingAdjustment: 1

# Scaling Policy Description (Down)
  WebServerScaleDown:
    Type: AWS::AutoScaling::ScalingPolicy
    Properties:
      AdjustmentType: ChangeInCapacity
      AutoScalingGroupName: !Ref WebServerASG
      Cooldown: 300
      ScalingAdjustment: -1

Outputs:
    LoadBanlancerEndpoint: 
        Description: this endpoint is used to reach the loadbalancer.
        Value: !Join [ "", [ 'http://', !GetAtt WebServerloadBalancer.DNSName  ]]
        Export:
          Name: !Sub ${EnvironmentName}-LBENDPOINT

---

Script Usage
This Repository contains some scripts that will be used to create the CloudFormation stack.

Usage:
Create:
./create.sh (stackName) (script.yml) (parameters.json) (profile)
Example:

./create.sh Udagram infrastructure/network.yaml parameters/network.json udacity_user

Below is the content of the create.sh script

aws cloudformation create-stack --stack-name $1 --template-body file://$2 --parameters file://$3 --capabilities "CAPABILITY_IAM" "CAPABILITY_NAMED_IAM" --region=us-east-1 --profile=$4

---

Update:
./update.sh (stackName) (script.yml) (parameters.json) (profile)
Example:

./update.sh Udagram infrastructure/network.yaml parameters/network.json udacity_user

Below is the content of the update.sh script

aws cloudformation update-stack --stack-name $1 --template-body file://$2 --parameters file://$3 --capabilities "CAPABILITY_IAM" "CAPABILITY_NAMED_IAM" --region=us-east-1 --profile=$4