DEV Community: Samira Yusifova

AWS project - Module 4. Use CloudFront distribution to serve a Static Website Hosted on AWS S3 via CloudFormation

Samira Yusifova — Thu, 22 Jun 2023 02:26:56 +0000

Overview

In the past three modules, we successfully created a basic static web app and deployed it on an S3 bucket. We also set up a CodeBuild project to automate the website's build process on AWS S3. Additionally, we configured Route 53 to enable the use of a custom domain for our S3-hosted static website. Now, let's continue enhancing our architecture further.

In this module we are going to optimize our application’s performance and security while effectively managing cost by setting up Amazon CloudFront to work with your S3 bucket to serve and protect the content.

Agenda

In this module I'll show you how to configure CloudFront distribution to serve a simple static website in ten steps via CloudFormation:
✳️ Step 1. Register a custom domain name
✳️ Step 2. Create S3 buckets for your root domain and subdomain
✳️ Step 3. Create Route 53 hosted zone and configure your domain
✳️ Step 4: Request public SSL/TLS certificate from AWS Certificate Manager (ACM) for our domain name and all its subdomains
✳️ Step 5. Create an Origin Access Identity (OAI) as special CloudFront user
✳️ Step 6. Create CloudFront distribution for subdomain that contains static website
✳️ Step 7. Create a policy for S3 bucket for subdomain to allow OAI to access bucket content
✳️ Step 8. Create another CloudFront distribution for root domain that redirects requests to S3 bucket for subdomain
✳️ Step 9. Create a Route 53 record set to route DNS traffic for root domain and subdomain to CloudFront domain
✳️ Step 10. Test your static website

Architecture

The high-level architecture for our project is illustrated in the diagram below:

Source code

Source code for this project is available on my GitHub profile in a public repository named StaticWebsiteHostingToS3 (check it here)
👉 Frontend source code either simple project or Angular project
👉 Module 4 CloudFormation templates link

List of CloudFormation templates:
1️⃣ Template for S3 buckets link
2️⃣ Template for Route 53 hosted zone link
3️⃣ Template for ACM certification link
4️⃣ Template for CloudFront distribution and OAI link
5️⃣ Template for Route 53 record set link

Initial Setup

Create required accounts:
👉 AWS account
👉 GitHub account

Install required tools locally:
👉 any IDE (personally I prefer Visual Studio Code) or any text editor
👉 git

AWS Resources

Here is the list of AWS resources that we are going to provision with CloudFormation:
👉 S3 buckets
👉 S3 bucket policy
👉 ACM certificate
👉 CloudFront OAI
👉 CloudFront distributions
👉 Route53 hosted zone
👉 Route53 record set group

Why use CloudFront?

Amazon CloudFront is a CDN (Content Delivery Network) provided by AWS team. CDNs are primarily used for caching, and many customers also use AWS CloudFront as a security layer, or use it to handle network spikes.

With CloudFront, when a user requests a webpage or an image, the request is routed to one of Amazon’s 400+ edge locations (caching servers). If the edge server already has the resource cached, it’s served to the client. If the resource isn’t on the edge server, it makes a request to an origin server that hosts your content (S3 bucket in our case).

The edge server then saves a copy of the response locally so it can handle the next request without pestering the origin server.

This reduces load on the origin server, helping you keep the instance housing your application small, and is able to reduce latency for clients by moving commonly requested resources closer to the requestor.

"Amazon S3 and CloudFront — is a match made in the cloud!" it said. Using CloudFront for our project offers three primary benefits: enhanced security, improved performance, and cost efficiency.

✳️ Enhanced security

If you followed Module 3, you may have noticed that in order to access our website hosted on S3 bucket, we granted public read access to it:



 # Create a policy for myS3BucketForRootDomain to let Route53 access website content
  myPolicyForS3BucketForRootDomain:
    Type: 'AWS::S3::BucketPolicy'
    Properties:
      Bucket: !Ref myS3BucketForRootDomain
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Sid: PublicReadForGetBucketObjects
            Effect: Allow
            Principal: '*'
            Action: 's3:GetObject'
            Resource: !Sub "${myS3BucketForRootDomain.Arn}/*"

(check the whole template here)

However it's always a best practice to keep your S3 buckets private to ensure security. So, a question arises: How can you host the static content of your website from a private S3 bucket? Well, CloudFront comes to the rescue. Later on in this module you can witness firsthand that with a CloudFront Distribution it is possible to serve content from a private S3 bucket.

Moreover, by configuring HTTPS with CloudFront, you establish secure end-to-end connections to your origin servers. For this purpose we will request public SSL/TLS certificate from AWS Certificate Manager (ACM) for our domain name and all its subdomains.

✳️ Improved performance

The broad network of edge locations and CloudFront caches copies of content close to the end users that results in lowering latency, high data transfer rates and low network traffic. All these make CloudFront fast.

Here is how it's going to work in our case:

A user requests website URL such as www.example.com which we will host on S3 bucket. The request will be routed to our CloudFront Distribution by Route 53. Now, if the requested content can be served from the cache it will be delivered immediately from the edge location closest to the end users. If the content is not cached (in case of initial request or when cache is expired), CloudFront requests the content directly from S3 bucket.

✳️ Cost efficiency

For small data volumes, using S3 is cost-effective due to the free monthly transfer of the first GB. However, as data usage increases, CloudFront becomes more cost-efficient. The pricing difference for data transfer costs widens significantly when transferring terabytes of data, making CloudFront a more favorable option for content delivery compared to serving content directly from S3.

There is an excellent article "AWS CloudFront Pricing and Cost Optimization Guide" I highly recommend reading (link).

Step 1. Register a custom domain name

In Module 3 we covered the topic of domain name and Route 53. If you haven't had the chance to go through Module 3 and confused why use domain name, how to register it, and why use Route 53 with Amazon S3 website endpoint, please refer to Steps 1 and 4 in the following link.

Step 2. Create S3 buckets for your root domain and subdomain

📌 When you configure an Amazon S3 bucket for website hosting, you must give the bucket the same name as root domain/subdomain name that you want to use to route traffic to the bucket. For example, if you want to route traffic for www.example.com to an S3 bucket that is configured for website hosting, the name of the bucket must be www.example.com.

Once you registered a new domain name, we can use it to provision an S3 bucket with the same name.

Get the full CloudFormation template for S3 buckets from here

1️⃣ The following piece of code helps to create a new S3 bucket for subdomain (such as www.example.com) and configure it to host a static website:



Resources:
  myS3BucketForSubdomain:
    Type: 'AWS::S3::Bucket'
    DeletionPolicy: Retain # keep S3 bucket when its stack is deleted
    Properties:
      BucketName: my-subdomain.my-domain-name # use the name of subdomain with domain, such as www.example.com
      WebsiteConfiguration:
        IndexDocument: index.html
        ErrorDocument: error.html
      VersioningConfiguration: # turn versioning on in case we need to rollback newly built files to older version
        Status: Enabled
      AccessControl: BucketOwnerFullControl

This S3 bucket for subdomain will contain our static website files. Don't forget to replace my-subdomain.my-domain-name with your root domain name with subdomain (such as www.example.com).

2️⃣ If you also want your users to be able to use root domain (such as example.com), to access your static website, create a second S3 bucket.

📌 Our second (root domain) bucket will not host a static website. We will keep it empty and configure it to redirect the request to the first bucket.

The following piece of code helps to create the second S3 bucket for root domain (such as example.com) and set it up to redirect requests to S3 bucket for subdomain (such as from example.com to www.example.com):



Resources:
  myS3BucketForRootDomain:
    Type: 'AWS::S3::Bucket'
    DeletionPolicy: Retain # keep S3 bucket when its stack is deleted
    Properties:
      BucketName: my-domain-name # use the name of your domain, such as example.com
      WebsiteConfiguration:
        RedirectAllRequestsTo: # Configure the bucket to route traffic to the subdomain bucket
          HostName: !Ref myS3BucketForSubdomain
          Protocol: https
      AccessControl: BucketOwnerFullControl

Don't forget to replace my-domain-name with your root domain name (such as example.com).

📌 Note: keep both buckets private. Lets leverage CloudFront to secure our S3 buckets and let an Origin Access Identity (OAI) - which is a special CloudFront user - to access S3 buckets safely. A new policy for S3 bucket (for subdomain only) will be created later on, in CloudFormation template for CloudFront (link) to let CloudFront OAI access content from a private S3 bucket.

3️⃣ We are done with Resources, now let's output Amazon S3 website endpoint and regional domain names for our buckets - we are going to use them to configure CloudFront later in this module.



Outputs:
  outputS3WebsiteURLForRootDomain:
    Description: Amazon S3 website endpoint for root domain
    Value: !GetAtt myS3BucketForRootDomain.WebsiteURL
  outputS3RegionalDomainNameForRootDomain:
    Description:  Regional domain name of S3 bucket for root domain
    Value: !GetAtt myS3BucketForRootDomain.RegionalDomainName 
  outputS3WebsiteURLForSubdomain:
    Description: Amazon S3 website endpoint for subdomain
    Value: !GetAtt myS3BucketForSubdomain.WebsiteURL
  outputS3RegionalDomainNameForSubdomain:
    Description:  Regional domain name of S3 bucket for subdomain
    Value: !GetAtt myS3BucketForSubdomain.RegionalDomainName

4️⃣ We are ready to create and run CloudFormation stack based on our template.

If you need to understand how to create a stack in AWS Console, please read Hands-on AWS CloudFormation - Part 1. It All Starts Here.

Upload our template file for S3 buckets (link) to AWS CloudFormation to create a stack. Specify your domain name and subdomain as parameters values. (I'm going to use domain name I've already registered, which is tiamatt.com.) Run the stack.

Once the stack built, you should see two newly created buckets and the policy under Resources tab:

Under Outputs tab you can find website endpoint and regional domain names for both domain and subdomain:

5️⃣ Currently, if you navigate to http://my-subdomain.my-domain-name.com.s3-website-us-east-1.amazonaws.com using Amazon S3 website endpoint for subdomain then you should get 403 Forbidden page as both our S3 buckets are private.

If you navigate to http://my-domain-name.com.s3-website-us-east-1.amazonaws.com using Amazon S3 website endpoint for root domain then it should redirect your request to Amazon S3 website endpoint for subdomain.

6️⃣ To deploy the static content to our S3 bucket for subdomain you can:
👉 either open S3 Console and manually upload both index.html and error.html files
👉 or automate the build of a static website (Angular project) on AWS S3 using CodeBuild (please follow the steps outlined in Module 2)

Step 3. Create Route 53 hosted zone and configure your domain

In Module 3 we also covered the topic of Route 53 hosted zone. Please follow Step 4 in the following link to configure a hosted zone for your domains.

Get the full CloudFormation template for Route 53 hosted zone from here

Step 4. Request public SSL/TLS certificate from AWS Certificate Manager (ACM) for our domain name and all its subdomains

In the previous three modules, you may have observed that both our Amazon S3 website endpoint (e.g., http://example.s3-website-us-east-1.amazonaws.com) and the custom domain name URL (e.g., http://example.com) were using an unsecured HTTP protocol. However it's essential to always protect your websites with HTTPS, even if they don't handle sensitive information.

📌 Note: HTTPS is crucial for protecting the communication between your websites and users' browsers. It prevents intruders, including malicious attackers and intrusive companies, from tampering with the data. Intruders can exploit unprotected communications to deceive users, steal sensitive information, install malware, or inject unwanted ads. They can exploit any resource, such as images, cookies, scripts, or HTML, that travels between your websites and users. These intrusions can happen anywhere in the network, like a user's device, a Wi-Fi hotspot, or a compromised ISP.

Basically HTTPS (Hypertext Transfer Protocol Secure) is a combination of the HTTP with the SSL/TLS (Secure Socket Layer / Transport Layer Security) protocol.

AWS provides AWS Certificate Manager (ACM) service that handles the complexity of creating, storing, and renewing public and private SSL/TLS certificates and keys that protect your AWS websites and applications. You can provide certificates for your integrated AWS services either by issuing them directly with ACM or by importing third-party certificates into the ACM management system.

Source: IT Cooking website

Now, let's request public SSL/TLS certificate from AWS Certificate Manager (ACM) for our domain name and all its subdomains to use to secure network communications and establish the identity of websites over the Internet.

Get the full CloudFormation template for ACM certification from here

1️⃣ The following piece of code helps to create a request:



Resources:
  mySSLCertificate:
    Type: 'AWS::CertificateManager::Certificate'
    Properties:
      DomainName: example.com
      SubjectAlternativeNames:
        - *.example.com # request a wildcard certificate for all subdomains
      DomainValidationOptions:
        - DomainName: example.com # DNS record for the root domain
          HostedZoneId: your-HostedZoneId
      ValidationMethod: DNS

Before the ACM can issue a certificate for your site, it must verify that you own or control all of the domain names that you specified in your request. You can choose either email validation or DNS validation when you request a certificate. Let's go with DNS validation.

2️⃣ We need to specify certificate ARN as output as we are going to use it during creation of CloudFront distributions (as will be mentioned in Step 8 of this module):



Outputs: 
  outputCertificateArn:
    Description: Issued SSL certificate Arn
    Value: !Ref mySSLCertificate

3️⃣ Upload our template file for ACM certification link to AWS CloudFormation to create a stack.

Once the stack built (you need to wait for a while to let ACM verify your domain/subdomain names), under Resources tab you should see your SSL certificate:

Under Outputs tab you can find certificate ARN:

If you navigate to AWS Certificate Manager (ACM), then go to List certificates, you will see your newly created certificate and domains it covers - example.com and *.example.com:

Step 5. Create an Origin Access Identity (OAI) as special CloudFront user

As we mentioned before, we kept our S3 buckets private which means buckets don't have any public access. Thus in order to let CloudFront access static content from S3 bucket, we must either set an Origin Access Identity (OAI) or permit public access to S3 bucket. However, the latter choice poses a significant security risk, which is why we prefer to opt for the OAI approach.

Basically, Origin Access Identity (OAI) as special CloudFront user that helps to prevent others from viewing your S3 content by simply using the direct URL for the content. By using OAI, we can prevent others from accessing the files using Amazon S3 URLs.

Get the full CloudFormation template for CloudFront from here

The following piece of code helps to create an OIA:



Resources:
  myCloudFrontOAI:
    Type: 'AWS::CloudFront::CloudFrontOriginAccessIdentity'
    Properties:
      CloudFrontOriginAccessIdentityConfig:
        Comment: 'OAI for S3 origins'  # a comment to describe the origin access identity

Easy peasy!

Step 6. Create CloudFront distribution for subdomain that contains static website

Now buckle up for creating CloudFront distribution!

Get the full CloudFormation template for CloudFront from here

The following piece of code helps to create a CloudFront distribution for subdomain:



Resources:
  myCloudFrontDistributionForSubdomain:
    Type: 'AWS::CloudFront::Distribution'
    Properties:
      DistributionConfig:
        Comment: CloudFront distribution points to S3 bucket for subdomain
        Origins: # info about origins for this distribution
          - DomainName: my-subdomain.my-domain-name.s3.${AWS::Region}.amazonaws.com' # Regional domain name of S3 bucket for subdomain (outputS3RegionalDomainNameForSubomain)
            Id: 'S3Origin-my-subdomain-my-domain-name' # unique identifier of an origin access control for this origin
            S3OriginConfig:
              OriginAccessIdentity: !Sub 'origin-access-identity/cloudfront/${myCloudFrontOAI}'
        Aliases: # info about CNAMEs (alternate domain names), if any, for this distribution
          - my-subdomain.my-domain-name
        # let CloudFront replace HTTP status codes in the 4xx and 5xx range with custom error messages before returning the response to the viewer
        CustomErrorResponses:
          - ErrorCode: 403 # 403 from S3 indicates that the file does not exists
            ResponseCode: 404 # HTTP status code that you want CloudFront to return to the viewer along with the custom error pag
            ResponsePagePath: '/error.html' # path to the custom error page that you want CloudFront to return to a viewer when your origin returns the HTTP status code specified by ErrorCode, for example, /4xx-errors/403-forbidden.html
            ErrorCachingMinTTL: 60 # minimum amount of time, in seconds, that you want CloudFront to cache the HTTP status code specified in ErrorCode
        DefaultCacheBehavior:
          AllowedMethods:
            - GET
            - HEAD
            - OPTIONS
          CachedMethods:
            - GET
            - HEAD
            - OPTIONS
          Compress: true
          DefaultTTL: 3600 # in seconds, 1 hour
          ForwardedValues:
            QueryString: false
            Cookies:
              Forward: none
          MaxTTL: 86400 # in seconds, 24 hours
          MinTTL: 60 # in seconds, 1 min
          TargetOriginId: 'S3Origin-my-subdomain-my-domain-name'
          ViewerProtocolPolicy: 'redirect-to-https' # 'allow-all'
        DefaultRootObject: 'index.html' 
        Enabled: true # enable distribution
        HttpVersion: http2 # the maximum HTTP version(s) that you want viewers to use to communicate with CloudFront
        PriceClass: PriceClass_All # allowed values: PriceClass_100 | PriceClass_200 | PriceClass_All
        ViewerCertificate:
          AcmCertificateArn: my-ACM-certificate-Arn
          SslSupportMethod: sni-only

Let me clarify a few things.

We pointed Origins to S3 bucket for subdomain (that contains static website, such as www.example.com) from which CloudFront should get the files to distribute. Here we also specified newly created CloudFront OAI (see Step 5) under S3OriginConfig.

In CustomErrorResponses section we set up CloudFront to replace 403 HTTP status code (that indicates the file does not exists) with 404 code along with the custom error message specified in /error.html page before returning the response to the viewer.

In DefaultCacheBehavior section we set up some cache parameters such as AllowedMethods that controls which HTTP methods CloudFront processes and forwards to our S3 bucket; CachedMethods that caches the responses to GET, HEAD, and OPTIONS requests; ViewerProtocolPolicy makes sure a viewer submits an HTTP request, CloudFront redirects the viewer to the new URL with HTTPS protocol.

Under ViewerCertificate property please use your ACM certificate ARN (as mentioned in Step 4).

Step 7. Create a policy for S3 bucket for subdomain to allow OAI to access bucket content

While provisioning CloudFront distribution we associated our CloudFront OAI (as mentioned in Step 5) with the origin so that viewers can only access objects in our S3 bucket for subdomain (that contains static website, such as www.example.com) through CloudFront. However it alone is insufficient. To ensure proper access, we must also provide permission for CloudFront OAI to read the files stored in our S3 bucket for subdomain. This can be achieved by creating a bucket policy.

Additionally, for the sake of security, let's restrict any access to S3 bucket for non-SSL requests. This is an additional precautionary step as we have already configured the CloudFront distribution to automatically redirect non-SSL requests to HTTPS.

Get the full CloudFormation template for CloudFront from here

The following piece of code helps to create a policy for S3 bucket for subdomain:



Resources:
  myPolicyForS3BucketForSubdomain:
    Type: 'AWS::S3::BucketPolicy'
    Properties:
      Bucket: my-subdomain.my-domain-name
      PolicyDocument:
        Statement:
        - Action: 's3:GetObject'
          Effect: Allow
          Resource: 'arn:aws:s3:::my-subdomain.my-domain-name/*'
          Principal:
            CanonicalUser: !GetAtt myCloudFrontOAI.S3CanonicalUserId
        # deny access for non SSL access to S3 bucket
        - Sid: AllowSSLRequestsOnly 
          Effect: Deny
          Principal: '*'
          Action: 's3:*'
          Resource:
          - !Sub 'arn:aws:s3:::my-subdomain.my-domain-name'
          - !Sub 'arn:aws:s3:::my-subdomain.my-domain-name/*'
          Condition:
            Bool:
              'aws:SecureTransport': false

📌 Note, we don't need to create a policy for S3 bucket for root domain as we keep it empty and has already configured it to redirect the request to S3 bucket for subdomain.

Step 8. Create another CloudFront distribution for root domain that redirects requests to S3 bucket for subdomain

This step is similar to Step 6 for creating a CloudFront distribution for a subdomain, but there is one important difference. It is important to note that there is a known issue with CloudFront to S3 redirects that can result in an "Access denied" error.

📌 Important! If you have configured S3 bucket to redirect the request to another bucket, avoid using the regional domain name of the bucket as the value of DomainName under Origins when configuring your CloudFront distribution. Instead, use S3 website endpoint, as it will save you a significant amount of time and effort.



Resources:
  myCloudFrontDistributionForRootDomain:
    Type: 'AWS::CloudFront::Distribution'
    Properties:
      DistributionConfig:
        Origins:
          # - DomainName: my-domain-name.s3.${AWS::Region}.amazonaws.com # regional domain name of S3 bucket for root domain (outputS3RegionalDomainNameForRootDomain)
          - DomainName: my-domain-name.s3-website-${AWS::Region}.amazonaws.com' # use Amazon S3 website endpoint for root domain (outputS3WebsiteURLForRootDomain)

📌 Important! If you are using S3 website endpoint as the value of DomainName under Origins then make sure to set up CustomOriginConfig property instead of S3OriginConfig. It resolves the following issue: "Resource handler returned message: Invalid request provided: The parameter Origin DomainName does not refer to a valid S3 bucket."



Resources:
  myCloudFrontDistributionForRootDomain:
    Type: 'AWS::CloudFront::Distribution'
    Properties:
      DistributionConfig:
        Origins:
         - DomainName: ....
           CustomOriginConfig:
              HTTPPort: 80 # required
              HTTPSPort: 443 # required
              OriginProtocolPolicy: 'http-only'

Get the full CloudFormation template for CloudFront from here

The following piece of code helps to create create a CloudFront distribution for root domain:



Resources:
  myCloudFrontDistributionForRootDomain:
    Type: 'AWS::CloudFront::Distribution'
    Properties:
      DistributionConfig:
        Comment: CloudFront distribution points to S3 bucket for root domain
        Origins: # info about origins for this distribution
          # Important! Known issue with Cloudfront to s3 redirect giving Access denied error
          # see https://stackoverflow.com/questions/22740084/amazon-s3-redirect-and-cloudfront 
          # - DomainName: my-domain-name.s3.${AWS::Region}.amazonaws.com # Regional domain name of S3 bucket for root domain (outputS3RegionalDomainNameForRootDomain)
          - DomainName: my-domain-name.s3-website-${AWS::Region}.amazonaws.com # use Amazon S3 website endpoint for root domain (outputS3WebsiteURLForRootDomain)
            Id: 'RedirectS3Origin-my-domain-name' # unique identifier of an origin access control for this origin
            # Important, for Amazon S3 website endpoint as DomainName need to be configured as CustomOriginConfig and NOT as S3OriginConfig
            # It resolves the following issue: "Resource handler returned message: "Invalid request provided: The parameter Origin DomainName does not refer to a valid S3 bucket.""
            # see https://stackoverflow.com/questions/40095803/how-do-you-create-an-aws-cloudfront-distribution-that-points-to-an-s3-static-ho 
            # see https://github.com/hashicorp/terraform-provider-aws/issues/7847 
            CustomOriginConfig:
              HTTPPort: 80 # required
              HTTPSPort: 443 # required
              OriginProtocolPolicy: 'http-only' 
        Aliases: # info about CNAMEs (alternate domain names), if any, for this distribution
          - 'my-domain-name'
        # let CloudFront replace HTTP status codes in the 4xx and 5xx range with custom error messages before returning the response to the viewer
        CustomErrorResponses:
          - ErrorCode: 403 # 403 from S3 indicates that the file does not exists
            ResponseCode: 404 # HTTP status code that you want CloudFront to return to the viewer along with the custom error pag
            ResponsePagePath: '/error.html' # path to the custom error page that you want CloudFront to return to a viewer when your origin returns the HTTP status code specified by ErrorCode, for example, /4xx-errors/403-forbidden.html
            ErrorCachingMinTTL: 60 # minimum amount of time, in seconds, that you want CloudFront to cache the HTTP status code specified in ErrorCode
        DefaultCacheBehavior:
          AllowedMethods:
            - GET
            - HEAD
            - OPTIONS
          CachedMethods:
            - GET
            - HEAD
            - OPTIONS
          Compress: true
          DefaultTTL: 3600 # in seconds, 1 hour
          ForwardedValues:
            QueryString: false
            Cookies:
              Forward: none
          MaxTTL: 86400 # in seconds, 24 hours
          MinTTL: 60 # in seconds, 1 min
          TargetOriginId: 'RedirectS3Origin-my-domain-name'
          ViewerProtocolPolicy: 'redirect-to-https' # 'allow-all'
        DefaultRootObject: 'index.html' 
        Enabled: true # enable distribution
        HttpVersion: http2 # the maximum HTTP version(s) that you want viewers to use to communicate with CloudFront
        PriceClass: PriceClass_All # allowed values: PriceClass_100 | PriceClass_200 | PriceClass_All
        ViewerCertificate:
          AcmCertificateArn: !Ref paramACMCertificateArn
          SslSupportMethod: sni-only

The rest configurations are similar to configurations we applied for CloudFront distribution for subdomain (as mentioned in Step 6).

So far we have added the following resources to our CloudFront template (link):
👉 a CloudFront Origin Access Identity (OAI)
👉 a CloudFront distribution for subdomain
👉 a policy for S3 bucket for subdomain
👉 a CloudFront distribution for root domain

We are done with Resources, now let's output CloudFront distribution domain name for both root domain and subdomain that we are going to use as Route 53 targets (as will be mentioned in Step 9 of this module):



Outputs:  
  outputCloudFrontDistributionForSubdomainId:
    Description: CloudFront distribution ID for subdomain
    Value: !Ref myCloudFrontDistributionForSubdomain
  outputCloudFrontDistributionDomainNameForSubdomain:
    Description: CloudFront distribution domain name for subdomain
    Value: !GetAtt myCloudFrontDistributionForSubdomain.DomainName
  outputCloudFrontDistributionForRootDomainId:
    Description: CloudFront distribution ID for root domain
    Value: !Ref myCloudFrontDistributionForRootDomain
  outputCloudFrontDistributionDomainNameForRootDomain:
    Description: CloudFront distribution domain name for root domain
    Value: !GetAtt myCloudFrontDistributionForRootDomain.DomainName

We are ready to create and run CloudFormation stack based on our template.

If you need to understand how to create a stack in AWS Console, please read Hands-on AWS CloudFormation - Part 1. It All Starts Here.

Upload our template file for CloudFront (link) to AWS CloudFormation to create a stack. Specify your ACM certificate ARN as a parameter value. Also specify your domain name and subdomain as parameters values. (I'm going to use domain name I've already registered, which is tiamatt.com.) Run the stack.

Once the stack built, you should see two newly created CloudFront OAI and distributions, and the policy for S3 bucket for subdomain under Resources tab:

Under Outputs tab you can find two CloudFront distributions - for domain and subdomain:

We are done with configuring CloudFront resources, now let's move on Route 53 resources.

Step 9. Create a Route 53 record set to route DNS traffic for root domain and subdomain to CloudFront domain

After you create a hosted zone for your domain, such as example.com (as mentioned in Step 3), you need to create records to tell the Domain Name System (DNS) how you want traffic to be routed for that domain.

Get the full CloudFormation template for Route 53 record set from here

1️⃣ The following piece of code helps to create a Route 53 record set group to route DNS traffic to CloudFront domain for both - domain and subdomain:



Resources:
  # create a Route 53 record set group to route DNS traffic to CloudFront domain for both - domain and subdomain
  myRoute53RecordSetGroup:
    Type: 'AWS::Route53::RecordSetGroup'
    Properties:
      Comment: Route53 record for CloudFront distributions for root domain and subdomain
      HostedZoneId: my-HostedZoneId
      RecordSets:
        # for CloudFront distributions subdomain (such as www.example.com)
        - Name: 'my-subdomain.my-domain-name.'  # keep the dot
          Type: A # 'A' routes traffic to an IPv4 address and some AWS resources. E.g. if the name of the hosted zone is 'example.com' and you want to use www.example.com to route traffic to your distribution, enter 'www.'
          AliasTarget:
              DNSName: !Ref paramCloudFrontDistributionDomainNameForSubdomain
              HostedZoneId: Z2FDTNDATAQYW2 # DONT change! It is a magical alphanumeric ID provided by AWS team for CloudFront distribution 
        - Name: 'my-domain-name.'  # keep the dot
          Type: A # 'A' routes traffic to an IPv4 address and some AWS resources. E.g. if the name of the hosted zone is 'example.com' and you want to use www.example.com to route traffic to your distribution, enter 'www.'
          AliasTarget:
              DNSName: !Ref paramCloudFrontDistributionDomainNameForRootDomain
              HostedZoneId: Z2FDTNDATAQYW2 # DONT change! It is a magical alphanumeric ID provided by AWS team for CloudFront distribution

📌 Important! HostedZoneId specified under AliasTarget is different from hosted zone id that we have created in Route 53 for our domain (as mentioned in Step 3). Hosted zone id for CloudFront is a magical alphanumeric ID provided by AWS team.

See the whole list of hosted zone ids for different AWS services here.

2️⃣ Upload our template file for Route 53 record sets (link) to AWS CloudFormation to create a stack.

Once the stack built, you should see a newly created Route 53 record set group under Resources tab:

Under Outputs tab you can find two URLs to our website - with domain name and subdomain:

Now we are done with provisioning of AWS resources. Our infrastructure is ready. Let's test our website.

Step 10. Test your static website

To verify that the website is working correctly, open a web browser and browse to http://my-domain-name (such as example.com). I'm using my custom domain name I've already registered, which is tiamatt.com:

Please take note of the following:
👉 Your request was redirected from root domain (example.com) to subdomain (www.example.com)
👉 the website protocol is changed from HTTP to HTTPS
👉 index.html was added at the end (if you remove it, it still going to show index page by default)

Also if you try to use either Amazon S3 website endpoint for root domain (such as http://example.com.s3-website-us-east-1.amazonaws.com) or CloudFront distribution domain name for root domain (such as abcde12345.cloudfront.net) then again you will be redirected to https://www.example.com/.

If you try to use Amazon S3 website endpoint for subdomain (such as http://www.example.com.s3-website-us-east-1.amazonaws.com), you should be redirected to 403 Forbidden page.

For CloudFront distribution domain name for subdomain (such as abcde67890.cloudfront.net) you should see the index page with CloudFront url:

Cleanup

You might be charged for running resources. That is why it is important to clean all provisioned resources once you are done with the stack. By deleting a stack, all its resources will be deleted as well.

📌 Note: CloudFormation won’t delete an S3 bucket that contains objects. First make sure to empty the bucket before deleting the stack.

Summary

Congratulations on reaching this milestone!

In this module, we optimized our application's performance and security while managing costs. Using an Infrastructure as Code approach through CloudFormation, we provisioned all the necessary AWS resources. This streamlined and automated the deployment process, ensuring consistency and ease of management. We set up Amazon CloudFront to work with our S3 bucket for content delivery and protection. The steps involved registering a custom domain, creating S3 buckets, configuring Route 53, obtaining SSL/TLS certificate, creating an Origin Access Identity (OAI), setting up CloudFront distributions, and routing DNS traffic using Route 53. These actions helped us achieve a more efficient and secure application setup.

AWS project - Module 3. Use Your Custom Domain for Static Website Hosted on AWS S3 via Route 53 and CloudFormation

Samira Yusifova — Tue, 30 May 2023 02:04:37 +0000

Overview

In Module 1, we have created a simple static website and hosted it on S3 bucket. To access our website we used Amazon S3 website endpoint which looked like http://example.s3-website-us-east-1.amazonaws.com. However for users the link is too long and hard to remember. If you want your website appear legitimate to visitors you should use a custom domain name such as http://example.com.

In this module I'll show you how to configure Route 53 to use your domain to access the website on S3 bucket in six simple steps via CloudFormation:
✳️ Step 1. Register a custom domain name
✳️ Step 2: Create an S3 bucket for your root domain and (optionally) for subdomain
✳️ Step 3. Build and deploy static files to S3 bucket automatically using CodeBuild
✳️ Step 4. Create Route 53 hosted zone and configure your domain
✳️ Step 5. Create Route 53 record set to route your traffic for your domain to S3 bucket
✳️ Step 6. Test your static website

Architecture

The high-level architecture for our project is illustrated in the diagram below:

In this module we will focus on configuring S3 buckets and Route 53 records so that:
✳️ your users will be able to use domain name (such as example.com) to access your static website hosted on S3 bucket for root domain

✳️ (optionally) your users will be able to use subdomain (such as www.example.com) to access your static website hosted on S3 bucket for root domain by redirecting here from S3 bucket for subdomain

Source code

Source code for this project is available on my GitHub profile in a public repository named StaticWebsiteHostingToS3 (check it here)
👉 Frontend source code link
👉 Module 3 CloudFormation templates link

List of CloudFormation templates:
1️⃣ Template for S3 buckets link
2️⃣ Template for CodeBuild project link
3️⃣ Template for Route 53 hosted zone link
4️⃣ Template for Route 53 record set here

Initial Setup

Create required accounts:
👉 AWS account
👉 GitHub account

Install required tools locally:
👉 any IDE (personally I prefer Visual Studio Code) or any text editor
👉 git
❎ no need to install any package locally for our Angular project

AWS Resources

Here is the list of AWS resources that we are going to provision with CloudFormation:
👉 S3 bucket(s)
👉 S3 bucket policy
👉 CodeBuild project
👉 IAM role for CodeBuild project
👉 GitHub Source Credential for CodeBuild project
👉 Route53 hosted zone
👉 Route53 record set group(s)

Why use domain name?

Why should you get a custom domain? Let's break it down!

☑️ Brand Superpowers: A custom domain name represents your brand and helps people easily recognize and remember you. Keep it consistent and unleash your brand's power.

☑️ Memorable Magic: The secret to success is having a domain name that sticks in people's minds like a catchy tune. The easier it is to remember, the more people will visit your site.

☑️ Credibility Boost: With a custom domain, your website becomes the ultimate credibility booster. It shows visitors that you mean business and adds a professional touch.

☑️ Authority Quest: As your domain ages, it gains super search engine authority! By creating quality content and building links, you'll level up your domain's power over time.

☑️ Email Address: The bonus perk of having a custom domain — your very own personalized email address!

☑️ Portability: With a custom domain, you can seamlessly move to different web hosting providers or website platforms while keeping the same web address. It's the ultimate flexibility that lets you adapt and expand your online presence without losing your trusted brand identity.

Few things you should consider when coming up with a domain name:
👉 Keep it short and sweet
👉 Make it unique
👉 Use easy to spell and pronounce name

Step 1. Register a custom domain name

So you've decided to pay for a custom domain name. Congratulations on taking a significant first step towards establishing search authority, credibility, and trust!

To use a domain name (such as example.com), you must find a domain name that isn't already in use and register it. When you register a domain name, you reserve it for your exclusive use everywhere on the internet, typically for one year. (from AWS Developer Guide)

Buying a new domain generally costs between $10 and $20 a year. Price differences depend on which registrar you buy your domain name from, and what kind of domain you're buying. Different registrars offer different packages, so it's worth shopping around to find your best fit.

📌 Note: you can register a new domain with AWS, Google Domains, NameCheap, GoDaddy, HostGator, or any other registrar. Here is the list of Best Domain Registrar 2023 according to Forbes.

Once you choose a domain registrar, all you need is simply to follow 3 general steps:

✳️ Step 1. Choose a unique domain name
✳️ Step 2. Choose a domain name registrar
✳️ Step 3. Purchase and register your domain name

Follow How To Register A Domain Name (2023 Guide) link for general instructions.

Here are the links to some registrars:

👉 AWS using Amazon Route 53 link
👉 Google Domains link (my personal choice, I'm paying $12 annually)
👉 GoDaddy link
👉 HostGator link
👉 NameCheap link

Step 2: Create an S3 bucket for your root domain and (optionally) for subdomain

Let's take a step back to quickly recap domain name structure.

Domain names have a specific structure that consists of several parts. Let's break it down:

1️⃣ Top-Level Domain (TLD): This is the rightmost part of a domain name and represents the highest level in the domain name system hierarchy. Common examples include .com, .org, .net, .edu, .gov, etc.

2️⃣ Second-Level Domain (SLD): The SLD is the part that comes before the TLD and is the main identifier of your website or brand. It typically represents the name of your organization, business, or the purpose of your website.

3️⃣ Subdomain: A subdomain is an optional part that appears before the SLD. It allows you to organize and categorize different sections or services of your website. For example, in blog.example.com, "blog" is a subdomain.

4️⃣ Third-Level Domain and beyond: It is possible to have additional levels of subdomains. For instance, support.blog.example.com has two levels of subdomains ("support" and "blog") before the SLD ("example") and the TLD (".com").

📌 Note: A domain name isn’t the same thing as a uniform resource locator (URL). A URL is the full web address of a site, and while it does contain the domain name, it contains other information, too. Each URL includes the internet protocol (most commonly HTTP or HTTPS) being used to call up the page. URLs can also help point browsers to a specific file or folder being hosted on a web server (for example https://www.my-site.com/blog?page=1).

Now enough theory, let's talk about implementation!

In Module 1 we have discussed how to provision S3 bucket to host our static website. To configure it with Route 53 we need to make some adjustments.

📌 When you configure an Amazon S3 bucket for website hosting, you must give the bucket the same name as the record that you want to use to route traffic to the bucket. For example, if you want to route traffic for example.com to an S3 bucket that is configured for website hosting, the name of the bucket must be example.com.

Once you registered a new domain name, we can use it to provision an S3 bucket with the same name.

Get the full CloudFormation template for S3 buckets from here.

1️⃣ The following piece of code helps to create a new S3 bucket and configure it to host a static website:

Resources:
  myS3BucketForRootDomain:
    Type: 'AWS::S3::Bucket'
    DeletionPolicy: Retain # keep S3 bucket when its stack is deleted
    Properties:
      BucketName: my-domain-name # use the name of your domain, such as example.com
      WebsiteConfiguration:
        IndexDocument: index.html
        ErrorDocument: error.html
      VersioningConfiguration: # turn versioning on in case we need to rollback newly built files to older version
        Status: Enabled
      # AccessControl: PublicRead # throws an error: Bucket cannot have public ACLs set with BlockPublicAccess enabled
      OwnershipControls:
        Rules:
          - ObjectOwnership: ObjectWriter
      PublicAccessBlockConfiguration:
        BlockPublicAcls: false
        BlockPublicPolicy: false
        IgnorePublicAcls: false
        RestrictPublicBuckets: false

This S3 bucket for root domain will contain our static website files. Don't forget to replace my-domain-name with your domain name (such as example.com).

We will push static files to S3 bucket for root domain with the help of CodeBuild in Step 3.

Once we configure Route 53 in Step 4 and Step 5, your users will be able to use my-domain-name (such as example.com) to access your static website hosted on S3 bucket for root domain.

2️⃣ If you also want your users to be able to use subdomain (such as www.example.com), to access your static website, create a second S3 bucket.

📌 Our second (subdomain) bucket will not host a static website. We will keep it empty and configure it to route traffic to the first bucket.

The following piece of code helps to create the second S3 bucket to redirect the traffic to the first bucket:

Resources:
  myS3BucketForSubdomain:
    Type: 'AWS::S3::Bucket'
    DeletionPolicy: Retain # keep S3 bucket when its stack is deleted
    Properties:
      BucketName:  my-subdomain.my-domain-name # use the name of subdomain with domain, such as www.example.com
      WebsiteConfiguration:
        RedirectAllRequestsTo: # Configure the bucket to route traffic to the S3 bucket for root domain
          HostName: !Ref myS3BucketForRootDomain
          Protocol: http
      AccessControl: BucketOwnerFullControl

📌 Note: RedirectAllRequestsTo property of S3 bucket helps to configure second (subdomain) bucket to route traffic to the first (root domain) bucket.

You can use CloudFormation Condition function to provision second (subdomain) bucket only if a subdomain was specified as a parameter:

Parameters:
  paramSubdomain:
    Description: OPTIONAL. Specify a subdomain (such as 'www' or 'apex' for www.example.com or apex.example.com). You can leave it empty to skip.
    Type: String
    Default: www

Conditions:
  # HasSubdomainName is false if paramSubdomain value is empty string
  HasSubdomainName: !Not [!Equals [!Ref paramSubdomain, '']] 

Resources:
 myS3BucketForSubdomain:
    Condition: HasSubdomainName # skip this resource if paramSubdomain value is empty string
    Type: 'AWS::S3::Bucket'

3️⃣ The next step is to create a bucket policy for the first (root domain) bucket.

We need to give a public read access to S3 bucket objects. Here is the policy:

Resources:
  myPolicyForS3BucketForRootDomain:
    Type: 'AWS::S3::BucketPolicy'
    Properties:
      Bucket: !Ref myS3BucketForRootDomain
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Sid: PublicReadForGetBucketObjects
            Effect: Allow
            Principal: '*'
            Action: 's3:GetObject'
            Resource: !Sub "${myS3BucketForRootDomain.Arn}/*"

4️⃣ We are done with Resources, now let's output Amazon S3 website endpoint for both root domain and subdomain to make it easier to navigate using the link once the stack is built.

Outputs:
  outputS3WebsiteURLForRootDomain:
    Description: Amazon S3 website endpoint for root domain
    Value: !GetAtt myS3BucketForRootDomain.WebsiteURL
  outputS3WebsiteURLForSubomain:
    Condition: HasSubdomainName
    Description: Amazon S3 website endpoint for subdomain
    Value: !GetAtt myS3BucketForSubdomain.WebsiteURL

5️⃣ We are ready to create and run CloudFormation stack based on our template.

If you need to understand how to create a stack in AWS Console, please read Hands-on AWS CloudFormation - Part 1. It All Starts Here.

Once the stack built, you should see two newly created buckets and the policy under Resources tab:

Under Outputs tab you can find two URLs to our website - for domain and subdomain:

Currently, if you navigate to http://my-domain-name.com.s3-website-us-east-1.amazonaws.com using Amazon S3 website endpoint for root domain then you should get 404 error as we haven't deployed static content to S3 bucket yet.

To resolve this issue we need to deploy the static content to our S3 bucket for root domain (see Step 3).

If you navigate to http://www.my-domain-name.com.s3-website-us-east-1.amazonaws.com using Amazon S3 website endpoint for subdomain, then your link will be changed to http://my-domain-name.com with This site can’t be reached error. To resolve this issue we need to configure Route 53 (see Step 4 and Step 5).

Step 3. Build and deploy static files to S3 bucket automatically using CodeBuild

In Module 2 we have discussed how to automate the build of a static website on AWS S3 via CodeBuild step by step. Please follow the guide to provision CodeBuild project and run CloudFormation stack for CodeBuild.

Get the full CloudFormation template for CodeBuild project from here.

📌 Note: you don't need to know anything about Angular or install NodeJS, NPM and Angular CLI locally. CodeBuild project will automate the build of Angular project and safe built files for a static website on S3 bucket for root domain.

1️⃣ When you run the stack, change the value of paramS3BucketNameForRootDomain input parameter to name that you used building a stack for S3 bucket for root domain (such as example.com). Then run the stack. And don't forget to pass your GitHub access token as value of paramPersonalGitHubAccessToken input parameter.

Once the stack built, you should see a newly created CodeBuild project and its role under Resources tab:

You can find our newly create CodeBuild project under CodeBuild -> Build projects:

2️⃣ Update frontend source code and watch how it will be built automatically.

Open any IDE or notepad on your machine and make some changes in frontend\app-for-aws\src\app\app.component.html file (it's totally up to you). I want to change the title to

<!-- Resources -->
<h2>Good news, everyone! Testing Module 3!</h2>

Push the changes to remote repo. Then navigate to our CodeBuild project on AWS Console, you should see that build was triggered automatically and its status is in progress. Wait till the status has changed to succeeded:

Go to the browser and use Amazon S3 website endpoint for root domain to navigate to the website:

📌 Note: Amazon S3 website endpoint for subdomain will not work as we haven't configured Route 53 yet (see Step 4 and Step 5).

Step 4. Create Route 53 hosted zone and configure your domain

📌 Note: When you register your domain using AWS registrar, Route 53 automatically creates a hosted zone with the same name. Skip this step if you have already registered your domain with AWS.

Let's take a step back to quickly recap what Route 53, hosted zones and records are.

Route 53 is Domain Name System (DNS) web service provided by AWS team. It offers domain registration, DNS routing, and health checking services (from AWS FAQs):

👉 With Route 53, you can create and manage your public DNS records. Like a phone book, Route 53 lets you manage the IP addresses listed for your domain names in the Internet’s DNS phone book.

👉 Route 53 also answers requests to translate specific domain names like into their corresponding IP addresses like 192.0.2.1. You can use Route 53 to create DNS records for a new domain or transfer DNS records for an existing domain. The simple, standards-based REST API for Route 53 allows you to easily create, update and manage DNS records.

👉 Route 53 additionally offers health checks to monitor the health and performance of your application as well as your web servers and other resources. You can also register new domain names or transfer in existing domain names to be managed by Route 53.

In our case we need to configure DNS routing to transfer internet traffic from our domain (such as example.com) to Amazon S3 website endpoint (such as http://example.s3-website-us-east-1.amazonaws.com) to access the website on S3.

In order to configure our domain name to point to Amazon S3 website endpoint, we need to create a public hosted zone and add routing records there.

Hosted zone is an Amazon Route 53 concept. A hosted zone is analogous to a traditional DNS zone file; it represents a collection of records that can be managed together, belonging to a single parent domain name. All resource record sets within a hosted zone must have the hosted zone's domain name as a suffix. For example, the example.com hosted zone may contain records named www.example.com, and www.blog.example.com, but not a record named www.amazon.com (from AWS FAQs).

Each record contains information about how you want Route 53 to route traffic for a specific domain, and its subdomains. For example, you can specify a record to route your traffic for example.com to S3 bucket, or API Gateway, or Load Balancer, or another AWS resources.

Get the full CloudFormation template for Route 53 hosted zone from here

1️⃣ The following piece of code helps to create a public hosted zone:

Resources:
  myRoute53HostedZone:
    Type: 'AWS::Route53::HostedZone'
    Properties:
      HostedZoneConfig: 
        Comment: My public hosted zone for example.com
      Name: example.com

📌 Note: A hosted zone and the corresponding domain should have the same name.

2️⃣ Let's output hosted zone ID that we will use for Route 53 record set configuration. Also we can get the list of all name servers (NS) where the traffic will be routed to:

Outputs:
  outputRoute53HostedZoneId:
    Description: Public hosted zone ID (such as Z23ABC4XYZL05B)
    Value: !GetAtt myRoute53HostedZone.Id # the same as !Ref myRoute53HostedZone  
  outputRoute53HostedZoneNameServers:
    Description: List of name servers for newly created public hosted zone
    Value: !Join [', ', !GetAtt myRoute53HostedZone.NameServers]

3️⃣ Upload our template file for hosted zone (link) to AWS CloudFormation to create a stack.

Once the stack built, a new public hosted zone will be provisioned.

Under Resources tab you can find the link to hosted zone:

Under Outputs tab you can find the hosted zone ID for our domain and the list of all name servers (NS):

If you navigate to Route 53 -> Hosted zones, you will see your newly created hosted zone (such as example.com) with two records of type NS and SOA:

Both a name server (NS) record and a start of authority (SOA) record are created by Route 53 automatically while provisioning a new public hosted zone. You rarely need to change these records.

✳️ Name server (NS) record lists the four name servers that are the authoritative name servers for your hosted zone.

✳️ Start of authority (SOA) record identifies the base DNS information about the domain.

📌 Note: When you register your domain outside of AWS Route 53, you need to connect your DNS to Route 53.

As I registered my domain on Google Domains, I'll show you how I connected my Google domain to Route 53. Similar steps can be applied for other registrar platforms.

1️⃣ First, go to your Route 53 hosted zone and copy all four NS records from there:

2️⃣ Now, log into your Google Domains account. Then click on My domains -> select DNS from menu -> click on Custom name servers (Active) tab. Click on Manage name servers.

Click on Add another name server and paste four NS records from the Route 53 Record Sets panel one by one. Don't forget to Save changes:

Ta-da! Now our domain has been connected to Route 53.

Step 5. Create Route 53 record set to route your traffic for your domain to S3 bucket

After you create a hosted zone for your domain, such as example.com, you need to create records to tell the Domain Name System (DNS) how you want traffic to be routed for that domain.

Get the full CloudFormation template for Route 53 record set from here

1️⃣ In our case we need to create a record to route internet traffic from root domain (such as example.com) to S3 bucket that hosts static website.

The following piece of code helps to create a new record for S3 bucket for root domain:

Resources:
  myRoute53RecordSetGroupForRootDomain:
    Type: 'AWS::Route53::RecordSetGroup'
    Properties:
      HostedZoneId: !Ref paramHostedZoneId
      RecordSets:
        - Name: example.com # point to an S3 bucket with root domain in the same account (such as example.com)
          Type: A # 'A' routes traffic to an IPv4 address and some AWS resources
          AliasTarget:
              DNSName: !Sub s3-website-${AWS::Region}.amazonaws.com
              HostedZoneId: !FindInMap # note, that it is different from paramHostedZoneId - this hosted zone is for region that you created the bucket in!
                - RegionMap
                - !Ref 'AWS::Region'
                - S3HostedZoneId

📌 Note: HostedZoneId specified under AliasTarget is different from hosted zone id that we have created in Route 53 for our domain in Step 4. Hosted zone id for S3 is a magical alphanumeric ID provided by AWS team. It varies base on the region that you created the bucket in. That's why we used mapping to map S3 hosted zone id to region:

Mappings:
  RegionMap: # based on https://docs.aws.amazon.com/general/latest/gr/s3.html#s3_website_region_endpoints
    us-east-1:
      S3HostedZoneId: Z3AQBSTGFYJSTF
    us-west-1:
      S3HostedZoneId: Z2F56UZL2M1ACD
    us-west-2:
      S3HostedZoneId: Z3BJ6K6RIION7M
    eu-central-1:
      S3HostedZoneId: Z21DNDUVLTQW6Q
    eu-west-1:
      S3HostedZoneId: Z1BKCTXD74EZPE
    ap-southeast-1:
      S3HostedZoneId: Z3O0J2DXBE1FTB
    ap-southeast-2:
      S3HostedZoneId: Z1WCIGYICN2BYD
    ap-northeast-1:
      S3HostedZoneId: Z2M4EHUR26P7ZW
    sa-east-1:
      S3HostedZoneId: Z31GFT0UA1I2HV

See the whole list of hosted zone ids for S3 here.

📌 Also note for future reference that hosted zone id depends on your alias target. For example:

for Global Accelerator accelerator specify Z2BJ6XQ5FK7U4H
for CloudFront distribution specify Z2FDTNDATAQYW2
etc

See the whole list of hosted zone ids for different AWS services here.

2️⃣ Optionally, we can create a new record for S3 subdomain bucket which will redirect traffic to S3 bucket for root domain.

The following piece of code helps to create a new record for S3 bucket for subdomain:

Resources:
  myRoute53RecordSetGroupForSubdomain:
    Condition: HasSubdomainName # skip this resource if paramSubdomain value is empty string
    Type: 'AWS::Route53::RecordSetGroup'
    Properties:
      HostedZoneId: !Ref paramHostedZoneId
      RecordSets:
        - Name: www.example.com # point to an S3 bucket with subdomain in the same account (such as www.example.com)
          Type: A # 'A' routes traffic to an IPv4 address and some AWS resources
          AliasTarget:
              DNSName: !Sub s3-website-${AWS::Region}.amazonaws.com 
              HostedZoneId: !FindInMap # note, that it is different from paramHostedZoneId - this hosted zone is for region that you created the bucket in!
                - RegionMap
                - !Ref 'AWS::Region'
                - S3HostedZoneId

3️⃣ Upload our template file for Route 53 record sets (link) to AWS CloudFormation to create a stack.

Once the stack built, you should see two newly created Route 53 record set groups under Resources tab:

Under Outputs tab you can find two URLs to our website - with domain name and subdomain:

Now we are done with provisioning of AWS resources. Our infrastructure is ready. Let's test our website.

Step 6. Test your static website

Now, let's check our subdomain. Browse to http://your-subdomain.my-domain-name (such as www.example.com). You should see that it will redirect your request to your http://my-domain-name domain (such as example.com).

Cleanup

📌 Note: CloudFormation won’t delete an S3 bucket that contains objects. First make sure to empty the bucket before deleting the stack.

Summary

Well done on reaching this point!

In this module we explained how to configure Route 53 to use a custom domain for your S3-hosted static website. Say goodbye to long, hard-to-remember links! In six simple steps, we registered the domain, set up S3 buckets, deployed files with CodeBuild, configured Route 53, and tested website.

By using a custom domain, we can boost professionalism and make our site more accessible.

AWS project - Module 2. Automate the build of a Static Website Hosted on AWS S3 via CodeBuild and CloudFormation

Samira Yusifova — Thu, 18 May 2023 01:27:39 +0000

Overview

In Module 1, we have created a simple static web app and hosted it on S3 bucket. However we took baby steps to deploy our static content to S3 bucket manually. Ideally we want to use a tool that would rebuild the source code every time a code change is pushed to the repository and deploy built files to S3 bucket automatically.

In this module I'll show you how to automate the build and deployment via AWS CodeBuild in six simple steps:
👉 Step 1. Create buildspec YAML file
👉 Step 2. Provide CodeBuild with access to GitHub repo
👉 Step 3. Configure how AWS CodeBuild builds your source code
👉 Step 4. Create IAM role for CodeBuild project
👉 Step 5. Run the CloudFormation stack
👉 Step 6. Update frontend source code and watch how it will be built automatically

Architecture

The high-level architecture for our project is illustrated in the diagram below:

Source code

Source code for this project is available on GitHub in a public StaticWebsiteHostingToS3 repository.
👉 Check for frontend source code here
👉 Check for Module 2 CloudFormation templates here

Initial Setup

Install required tools:
👉 any IDE (personally I prefer Visual Studio Code) or text editor
👉 git

Note, you don't need to know anything about Angular or install NodeJS, NPM and Angular CLI locally. We will configure CodeBuild project with all necessary installation packages.

AWS Resources

Here is the list of AWS resources that we are going to create:
👉 CodeBuild project
👉 IAM role for CodeBuild project
👉 GitHub Source Credential for CodeBuild project

Step 1. Create buildspec YAML file

Buildspec is a collection of build commands and related settings, in YAML format, that CodeBuild uses to run a build. We need to create and include a buildspec YAML file as part of the source code of frontend app. In our case the file is stored in the root directory of Angular app, which is frontend/app-for-aws folder.

Shortcut: get buildspec YAML file here.

1️⃣ Install phase of buildspec file

Use the install phase only for installing packages in the build environment. In order to run the production build of our Angular web app, CodeBuild server needs Angular installation. Meanwhile Angular depends on NodeJS and NPM. Thus we need to specify NodeJS installation as a runtime and add commands to install Angular CLI and node modules based on dependencies specified in package.json (link):



phases:
  install:
    runtime-versions:
       nodejs: 18
    commands:
      - echo Install Angular CLI and all node_modules 
      - cd $CODEBUILD_SRC_DIR/frontend/app-for-aws
      - npm install && npm install -g @angular/cli

2️⃣ Build phase of buildspec file

Use the build phase for commands that CodeBuild runs during the build. In our case we need to navigate to our frontend folder and run the production build:



  build:
    commands:
      - echo Build process started now
      - cd $CODEBUILD_SRC_DIR/frontend/app-for-aws
      - ng build --configuration=production

3️⃣ Post build phase of buildspec file

Use the post_build phase for commands that CodeBuild runs after the build. Here we can list all the files of newly created dist/app-for-aws folder for easy debugging:



post_build:
    commands:
      - echo Build process finished, upload artifacts to S3 bucket
      - cd dist/app-for-aws
      - ls -la

4️⃣ Artifacts of buildspec file

Artifacts represent information about where CodeBuild can find the build output and how CodeBuild prepares it for uploading to the S3 output bucket. Here we need to specify the path to dist folder so that CodeBuild won't deploy a whole frontend project to S3 bucket but files for prod build only:



 artifacts:
  base-directory: 'frontend/app-for-aws/dist*'
  discard-paths: yes
  files:
    - '**/*'

discard-paths helps to make sure to put all the files in the root folder instead of subfolder. In our case it will take all the files (including index.hmtl file) from dist/app-for-aws folder and put them in the root of S3 bucket.

Step 2. Provide CodeBuild with access to GitHub repo

1️⃣ In order to access source code for frontend located on your GitHub account, CodeBuild needs some access privilege. The easiest and safest way to grand CodeBuild an access to GitHub is to create a personal access token.

Navigate to your GitHub and click your profile photo, then click Settings.

In the left sidebar, scroll down and click Developer settings.

In the left sidebar, under Personal access tokens, click Tokens (classic) (since Fine-grained tokens is in beta and might not be compatible with AWS CodeBuild yet), then click Generate new token.

Select the scopes you'd like to grant this token. Note, as I've already generated a token, my screenshots shows Edit. Please ignore.

Click Generate token. Copy the new token - we will need it while running a CloudFormation stack.

2️⃣ In CloudFormation template we are going to create a new AWS::CodeBuild::SourceCredential resource. Here we provide information about the credentials for a GitHub.

It is recommend to use AWS Secrets Manager to store our credentials. But for the sake of simplicity (let's take baby steps) we will pass a newly generated GitHub access token as a CloudFormation parameter for now.



Parameters:
  paramPersonalGitHubAccessToken:
    Type: String
    MinLength: 10
    ConstraintDescription: Personal GitHub access token is missing
    Description: Provide your personal GitHub access token for 
CodeBuild to access your GitHub repo

Resources:
  myCodeBuildSourceCredential:
    Type: AWS::CodeBuild::SourceCredential
    Properties:
      AuthType: PERSONAL_ACCESS_TOKEN
      ServerType: GITHUB
      Token: !Ref paramPersonalGitHubAccessToken

Step 3. Configure how AWS CodeBuild builds your source code

AWS::CodeBuild::Project resource configures how AWS CodeBuild builds your source code, such as where to get the source code and which build environment to use.

First, let's give a friendly name and add description for our CodeBuild project:



Resources:
  myCodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      Name: westworld-codebuild-for-website-hosting
      Description: CodeBuild project for automatically build of static website hosted on s3

In Source code settings for the CodeBuild project we need to specify the GITHUB as source code's repository type, then add repo location, BuildSpec location and authorization settings for AWS CodeBuild to access the source code to be built. Remember, in Step 2 of this module we have created myCodeBuildSourceCredential Source Credential resource - we need to use it under Auth Resource so that CodeBuild can access the specified GitHub repo.



Resources:
  myCodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      ...
      Source:
        Type: GITHUB
        Location: https://github.com/your-account/StaticWebsiteHostingToS3.git
        GitCloneDepth: 1
        BuildSpec: frontend/app-for-aws/buildspec.yml
        Auth:
          Resource: !Ref myCodeBuildSourceCredential
          Type: OAUTH

In Triggers code settings for the CodeBuild project we want to enable AWS CodeBuild to begin automatically rebuilding the source code every time a code change is pushed to the repository. Basically we configured CodeBuild to listen to any git pushes to main branch to trigger the build.



Resources:
  myCodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      ...
      Triggers:
        Webhook: true
        FilterGroups:
          - - Type: EVENT
              Pattern: PUSH
            - Type: HEAD_REF
              Pattern: ^refs/heads/main # for feature branches use: ^refs/heads/feature/.*

In Environment code settings for the CodeBuild project we basically configured VM where CodeBuild is going to build the source code.



Resources:
  myCodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      ...
      Environment: # use Ubuntu standard v7
        Type: LINUX_CONTAINER
        ComputeType: BUILD_GENERAL1_SMALL
        Image: aws/codebuild/standard:7.0

But how do you know which container, computer and image you need? Well, based on the code source programming language and framework you can check for Docker images here and Available runtimes here.

As we have specified NodeJS v18 in our buildspec file, we can use either Amazon Linux 2 AArch64 standard:3.0 or Ubuntu standard:7.0 image.

Based on selected image I can get Image identifier from this list.

In Artifacts code settings for the CodeBuild project we want to specify what to do with build files. In our case we want to put them to our S3 bucket that hosts the static website. Note, it's very important to disable encryption for our website files.



Resources:
  myCodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      ...
      Artifacts: # drop the build artifacts of S3 bucket that hosts static website
        Type: S3
        Name: '/' # store the artifact in the root of the output bucket
        Location: !Sub arn:aws:s3:::westworld-codebuild-for-website-hosting
        EncryptionDisabled: True #disable the encryption of artifacts in a build to see html pages

In LogsConfig code settings for the CodeBuild project we want to configure CloudWatch logs.



Resources:
  myCodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      ...
      LogsConfig:
        CloudWatchLogs:
          Status: ENABLED
          GroupName: westworld-codebuild-for-website-hosting-CloudWatchLogs

In ServiceRole code settings for the CodeBuild project we need to create a new role with proper access - see Step 4.



Resources:
  myCodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      ...
      ServiceRole: !Ref myCodeBuildProjectRole

Step 4. Create IAM role for CodeBuild project

Final step for our CloudFormation template is to create an appropriate IAM role for CodeBuild project to get an access to S3 bucket where static website is hosted and to CloudWatch logs to stream the logs while building the project.



Resources:
  myCodeBuildProjectRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: role-for-westworld-codebuild-for-website-hosting
      Path: /
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Action:
              - 'sts:AssumeRole'
            Effect: Allow
            Principal:
              Service:
                - codebuild.amazonaws.com
      Policies:
        - PolicyName: policy-for-westworld-codebuild-for-website-hosting
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              # statement to create/stream CloudWatch
              - Action:
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Effect: Allow
                Resource:
                  - !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:westworld-codebuild-for-website-hosting-CloudWatchLogs
                  - !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:westworld-codebuild-for-website-hosting-CloudWatchLogs:*
              # statement to access S3 bucket that hosts static website (CodeBuild will save Artifacts there)
              - Action:
                  - s3:PutObject
                  - s3:GetObject
                  - s3:GetObjectVersion
                  - s3:GetBucketAcl
                  - s3:GetBucketLocation
                Effect: Allow
                Resource:
                  - arn:aws:s3:::your-bucket-name
                  - arn:aws:s3:::your-bucket-name/*

Step 5. Run the CloudFormation stack

Now we are ready to create and run CloudFormation stack based on our template for CodeBuild (see the whole template here).

Note, we have already created and run CloudFormation stack for static website and S3 - check this template for provisioning S3 bucket and step by step guide in Module 1.

I prefer to run the stack from AWS Console as it provides an end-to-end overview of a process flow. But you can always run a stack using the AWS CLI (here is how).

Upload our template file to create a stack.

Change the value of paramStaticWebsiteHostingBucketName input parameter to name that you used building a stack for S3 bucket. Then run the stack. And dont forget to pass your GitHub access token as value of paramPersonalGitHubAccessToken input parameter.

Once the stack built, you should see a newly created CodeBuild project and its role under Resources tab:

You can find our newly create CodeBuild project under CodeBuild -> Build projects:

Let's navigate to our website. Remember, we have created our stack for S3 bucket - under Outputs tab you can find the link to our website (note, your link might be different from mine):

Voila! Our static website is up and running! It was built and deployed to S3 with the initial provisioning of our CodeBuild project.

Step 6. Update frontend source code and watch how it will be built automatically

Now it's time to check out build automation.

1️⃣ Let's make a small change (it's totally up to you) in our source code for static website.

You need to clone frontend project (unless you want to use your own project built in React, Vue or vanilla JavaScript) - check for frontend source code here.

Let's make some changes in frontend\app-for-aws\src\app\app.component.html file. I want to change the title from



<!-- Resources -->
<h2>Good news, everyone!</h2>



<!-- Resources -->
<h2>Good news, everyone! CodeBuild project is up and running!</h2>

Open html file in any IDE or text editor. Make and save the changes.

Now let's push the changes to remote repo. Open your terminal and run the following commands:



# navigate to the project
cd StaticWebsiteHostingToS3
# stage the changes
git add .
# commit the changes
git commit -m"Changed the title of static website"
# push the changes to remote repo
git push

2️⃣ Once you pushed your changes to GitHub and navigate to our CodeBuild project on AWS Console, you should see that build was triggered automatically and its status is in progress:

Click on Build run and you will see the logs:

You can navigate to CloudWatch logs by clicking on View entire log under Build log tab:

Note, that all our logs for build were stored under newly created westworld-codebuild-for-website-hosting-CloudWatchLogs log group.

Also you can check our S3 bucket to see if the files were updated (look at Last modified date and time).

3️⃣ Once build status has changed to Succeeded, go ahead and refresh your website link. You should the code changes.

Note, if you don't see the changes, probably the website was cached. You can either clean the cache or open the website in incognito mode.

Cleanup

Note: CloudFormation won’t delete an S3 bucket that contains objects. First make sure to empty the bucket before deleting the stack. Of course, you can automate the process by creating a Lambda function which deletes all object versions and the objects themselves. I'll try to cover that part in further article.

Summary

Congratulations on getting this far!

In this module, we have provisioned CodeBuild project that automate the build of a static website on AWS S3. We took Infrastructure as Code approach to provisioning and managing our AWS resources by writing a template file and running stacks using AWS CloudFormation service.

Don't forget to do your happy dance!

AWS project - Module 1. Host a Static Website on AWS S3 via CloudFormation

Samira Yusifova — Thu, 18 May 2023 01:25:52 +0000

Overview

This series of articles provide a step-by-step guide on how to deploy a basic static website to an Amazon S3 (Simple Storage Service) bucket using AWS CloudFormation. Starting with a simple architecture, each module identifies gaps and upcoming issues, and gradually introduces additional layers of complexity to resolve those issues. The steps involved registering a custom domain, creating S3 buckets, configuring Route 53, obtaining SSL/TLS certificate, creating an Origin Access Identity (OAI), setting up CloudFront distributions, and routing DNS traffic using Route 53. By leveraging the power of CloudFormation, developers can automate the infrastructure setup and deployment process, making it easier and more efficient to launch static websites.

In this module, we are going to deploy a simple static website to S3 bucket via AWS CloudFormation in three simple steps:
👉 Step 1. Create a simple static web app via Angular
👉 Step 2. Create S3 bucket and its policy, and configure it to host a static website
👉 Step 3. Build and deploy static website manually to newly created S3 bucket (optional, you can skip it and go to Module 2 for build automation)

In next module we will automate the deployment via CodeBuild.

As I’m strongly against managing environments manually and take Infrastructure as Code for granted (if we are not on the same page, I’d suggest you to read this article first), AWS CloudFormation can be a great choice to provision and manage the complete infrastructure and AWS resources in a text file.

Architecture

The high-level architecture for our project is illustrated in the diagram below:

Source code

Source code for this project is available on GitHub in a public StaticWebsiteHostingToS3 repository.
👉 Check for frontend source code here
👉 Check for CloudFormation template here

Initial Setup

Install required tools:
👉 any IDE (personally I prefer Visual Studio Code)
👉 git

Optional:
I’m going to create a simple static website using Angular. You can use any other framework/library if you want.
👉 Angular, Node.js and NPM - v16 or greater
👉 Angular CLI



    # run the following command to install Angular CLI globally
    npm install -g @angular/cli

AWS Resources

Here is the list of AWS resources that we are going to create:
👉 S3 bucket
👉 S3 bucket policy

Why host static website on S3?

Why should you host your static content on S3? Let's break it down!

☑️ Firstly, you no longer need to allocate a specific amount of storage space or plan for it, as S3 buckets automatically scale to accommodate your needs.

☑️ Moreover, since S3 operates as a serverless service, you are relieved of the burden of managing and patching servers that store your files; you simply upload and retrieve your content.

☑️ Additionally, even if your application requires a server (such as a dynamic application), it can be smaller because it doesn't need to handle requests for static content.

Step 1. Create a simple static web app

Shortcut: get the source code for frontend here (but you still need to install Angular to run it locally unless you want to skip manual build and go to Module 2 for build automation).

Let’s create a static Angular app from scratch.

1️⃣ Firstly, create a new folder and name it StaticWebsiteHostingToS3 which will store two folders:

cloudformation folder (stores CloudFormation templates)
frontend folder (stores static website source code)



# create a parent folder 
mkdir StaticWebsiteHostingToS3
cd StaticWebsiteHostingToS3

# initialize a repository
git init

# create a folder that stores CloudFormation templates
mkdir cloudformation 

# create a folder that stores static website source code
mkdir frontend

2️⃣ Secondly, create an initial Angular project by simply running the following commands in terminal:



# navigate to folder that should store frontend 
cd frontend

# create a new Angular project
ng new app-for-aws
# check ‘y’ and ‘SCSS’ to prompt questions such as:
# - Would you like to add Angular routing? (y/N) y
# - Which stylesheet format would you like to use? SCSS

# run the app locally
cd app-for-aws
ng serve

3️⃣ Navigate to http://localhost:4200/

You can customize the website as you wish. Now, let's provision some resources in AWS to host our website.

Step 2. Create S3 bucket and its policy, and configure it to host a static website

Amazon S3 is a perfect AWS service to host a static website.

In order to host our website on S3 bucket, let’s create and configure all required resources via CloudFormation.

Shortcut: get the full CloudFormation template here.

1️⃣ The following piece of code helps to create a new S3 bucket and configure it to host a static website:



Resources:
  myStaticWebsiteHostingBucket:
    Type: 'AWS::S3::Bucket'
    Properties:
      BucketName: your-bucket-name
      WebsiteConfiguration:
        IndexDocument: index.html
        ErrorDocument: error.html

2️⃣ Next step is to set S3 bucket permissions for website access.

When you configure a bucket as a static website, if you want your website to be public, you must disable block public access settings for the bucket and write a bucket policy that grants public read access.

That is why we are going to disable ACLs for our bucket under PublicAccessBlockConfiguration property and set the ownership of S3 bucket content to Object writer (i.e. AWS account that uploads an object owns the object and has full control over it).



      OwnershipControls:
        Rules:
          - ObjectOwnership: ObjectWriter
      PublicAccessBlockConfiguration:
        BlockPublicAcls: false
        BlockPublicPolicy: false
        IgnorePublicAcls: false
        RestrictPublicBuckets: false

Basically we are disabling S3 Block Public Access settings to let users get a public access to the website.

One issue I faced with while configuring access settings and building the stack was the following:

Bucket cannot have public ACLs set with BlockPublicAccess enabled (Service: Amazon S3; Status Code: 400; Error Code: InvalidBucketAclWithBlockPublicAccessError)

The root of the issue was in using AccessControl property with PublicAccessBlockConfiguration property in the template. It should work with existing buckets and fail with all new buckets.

FYI, AWS has changed the default settings of ACLs of S3 bucket:

Starting in April 2023, Amazon S3 will introduce two new default bucket security settings by automatically enabling S3 Block Public Access and disabling S3 access control lists (ACLs) for all new S3 buckets. Once complete, these defaults will apply to all new buckets regardless of how they are created, including AWS CLI, APIs, SDKs, and AWS CloudFormation. These defaults have been in place for buckets created in the S3 management console since the two features became available in 2018 and 2021, respectively, and are recommended security best practices. There is no change for existing buckets. (source)

To resolve the issue, I've commented out AccessControl property:



      # AccessControl: PublicRead # throws an error: Bucket cannot
# have public ACLs set with BlockPublicAccess enabled

3️⃣ Next step is to create an S3 bucket policy

We need to give a public read access to S3 bucket objects. Here is the policy:



   myStaticWebsiteHostingBucketPolicy:
    Type: 'AWS::S3::BucketPolicy'
    Properties:
      Bucket: !Ref myStaticWebsiteHostingBucket
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Sid: PublicReadForGetBucketObjects
            Effect: Allow
            Principal: '*'
            Action: 's3:GetObject'
            Resource: !Join 
              - ''
              - - 'arn:aws:s3:::'
                - !Ref myStaticWebsiteHostingBucket
                - /*

We can turn versioning on for our S3 bucket so that in case of bugs in new version we can easily roll back the deployment to previous version. Simply add VersioningConfiguration under bucket's property:



Resources:  
  myStaticWebsiteHostingBucket:
    Type: 'AWS::S3::Bucket'
    .... 
    Properties:
      VersioningConfiguration:
        Status: Enabled

4️⃣ We are done with Resources, now let's output the website URL to make it easier to navigate using the link once the stack is built.



Outputs:
  outputWebsiteURL:
    Value: !GetAtt 
      - myStaticWebsiteHostingBucket
      - WebsiteURL
    Description: Static website URL

5️⃣ We are ready to create and run CloudFormation stack based on our template.

If you need to understand how to create a stack in AWS Console, please read Hands-on AWS CloudFormation - Part 1. It All Starts Here.

I prefer to run the stack from AWS Console as it provides an end-to-end overview of a process flow. But you can always run a stack using the AWS CLI (here is how).

Upload our template file to create a stack. Change the value of paramStaticWebsiteHostingBucketName input parameter to something unique. Then run the stack.

Once the stack built, you should see a newly created bucket and it's policy under Resources tab:

Under Outputs tab you can find the link to our website.

Currently you should get 404 error as we haven't deployed static content to S3 bucket yet.

Step 3. Build and deploy static website manually to newly created S3 bucket

This step is optional - you can skip it and go to Module 2 for build automation.

1️⃣ First, we need to build the production version of our Angular app locally.

Open terminal for our StaticWebsiteHostingToS3 app. By default, ng build command uses the production build configuration:



cd frontend/app-for-aws
ng build

A new dist folder should be autogenerated with static content.

2️⃣ In AWS Console navigate to our S3 bucket and upload all the files from dist/app-for-aws folder:

Don't forget to click on "Upload" button:

3️⃣ Now refresh the website link - our Angular app should be up and running:

Cleanup

Summary

In this module, we have created a simple static web app and hosted it on S3 bucket. However we took baby steps to deploy our static content to S3 bucket manually. Ideally we want to use a tool that would rebuild the source code every time a code change is pushed to the repository and deploy built files to S3 bucket automatically. In Module 2 I'll show you step by step how to automate the build and deployment via AWS CodeBuild.

AWS project - Building a serverless microservice with Lambda

Samira Yusifova — Thu, 28 Jan 2021 21:14:05 +0000

Overview

In this article, we are going to build a simple serverless microservice on AWS that enables users to create and manage movies data. We will set up an entire backend using API Gateway, Lambda and DynamoDB. Then we will use SwaggerHub as an integrated design environment for our APIs:

The goal of this project is to learn how to create an application composed of small, easily deployable, loosely coupled, independently scalable, serverless components.

As I’m strongly against managing environments manually and take Infrastructure as Code for granted (if we are not on the same page, I’d suggest you to read this article first), AWS SAM will be a great fit for our serverless application. As a result, the entire application should be deployed in any AWS account with a single CloudFormation template.

Architecture

1️⃣ A user requests to the server by calling APIs from SwaggerHub UI. User's request which includes all necessary information is sent to Amazon API Gateway restful service.
2️⃣ API Gateway transfers the collected user information to a particular AWS Lambda function based on user's request.
3️⃣ AWS Lambda function executes event-based logic calling DynamoDB database.
4️⃣ DynamoDB provides a persistence layer where data can be stored/retrieved by the API's Lambda function.

The high-level architecture for the serverless microservice is illustrated in the diagram below:

Source code

All source code for this project is available on GitHub in a public AwsServerlessMicroserviceWithLambda repository.

To clone the GitHub repository, execute the following command:

cd my-folder
git clone https://github.com/Tiamatt/AwsServerlessMicroserviceWithLambda.git

Initial Setup

To codify, build, package, deploy, and manage our AWS resources in a fully automated fashion, we will use:

👉 AWS SAM
👉 AWS CloudFormation
👉 AWS CLI
👉 AWS SDK for Python (boto3)
👉 Docker

If you don't want to install or maintain a local IDE, use AWS Cloud9 instead (you can find how to set up AWS Cloud9 here). Otherwise you might need to install the latest AWS CLI, AWS SAM and Python on your development machine.

AWS Resources

Here is the list of AWS resources that we are going to create:

✔️ AWS Lambda
✔️ Amazon DynamoDB
✔️ Amazon API Gateway
✔️ AWS IAM
✔️ Amazon S3 (that is where your CloudFormation template will be stored)

In this project, we will be building Lambda functions with Python 3.8.

Step 1. Generate an AWS SAM template

Download AWS SAM Hello World template which implements a basic API backend based on Amazon API Gateway endpoint and AWS Lambda function:

cd my-folder
sam init

If you're feeling lost, please, follow Step 2. Create a SAM application described in my previous article.

AWS SAM should create a directory with all necessary files and folders:

Step 2. Create a DynamoDb table

Rewrite pre-generated "template.yaml" with the following code:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: 'SAM Template for ServerlessMicroserviceApp'

# ======================== RESOURCES ======================== #
Resources:
  MoviesTable:
    Type: AWS::Serverless::SimpleTable
    Properties:
      TableName: 'movies-table'
      PrimaryKey:
        Name: uuid
        Type: String

AWS::Serverless::SimpleTable is AWS SAM syntax for DynamoDB table creation with a single attribute primary key named uuid. Note, if you want to add sort key or use other advanced functionality of DynamoDB, then you may want to consider using AWS CloudFormations's AWS::DynamoDB::Table resource instead (yes, you can use AWS CloudFormation syntax in AWS SAM template, combining the best of both worlds 💙).

Step 3. Create an API Gateway

Next step is to define API Gateway in "template.yaml":

MoviesApi: 
    Type: AWS::Serverless::Api
    Properties:
      StageName: Prod
      # DefinitionUri: ./swagger.json

AWS::Serverless::Api is AWS SAM syntax for a collection of Amazon API Gateway resources. FYI, there is no need to explicitly add Api to the template - AWS SAM can implicitly create it based on fromRestApiId attribute defined on AWS::Serverless::Function resources (see Step 4 below). However I prefer an explicit definition as AWS::Serverless::Api has a lot of useful properties I might use in the future (without actially refactoring the template).

Amazon API Gateway acts as the interface layer between the frontend (SwaggerHub) and AWS Lambda, which calls the backend (DynamoDB database). Below are the different APIs that we are about to define in Events attribute of each Lambda function in Step 4:
👉 POST /movie (CreateMovie)
👉 GET /movie/{uuid} (GetMovie)
👉 GET /movie/list (GetMovies)
👉 PUT /movie (UpdateMovie)
👉 DELETE /movie/{uuid} (DeleteMovie)

Step 4. Create Lambda functions

We are going to create five lambda functions:

CreateMovieFunction (create_movie/app.py)
- triggered by POST /movie API
GetMovieFunction (get_movie/app.py)
- triggered by GET /movie/{uuid} API
GetMoviesFunction (get_movies/app.py)
- triggered by GET /movie/list API
UpdateMovieFunction (update_movie/app.py)
- triggered by PUT /movie API
DeleteMovieFunction (delete_movie/app.py)
- triggered by DELETE /movie/{uuid} API

Let's start with CreateMovieFunction:

1️⃣ Rename "hello_world" folder to "create_movie".

2️⃣ Clear the content of "create_movie/requirements.txt" file (no need in any Python dependency).

3️⃣ Open "create_movie/app.py" file and add the following code:

import json
import boto3
import uuid
from botocore.exceptions import ClientError

# Extract movie object from request body and insert it into DynamoDB table
def lambda_handler(event, context):

    if ('body' not in event or event['httpMethod'] != 'POST' or 'title' not in event['body']):
        return {
            'statusCode': 400,
            'body': json.dumps({'message': 'Bad Request'})
        }

    new_movie = json.loads(event['body'])
    response = add_movie_to_db(new_movie)

    return {
        'statusCode': 200,
        'body': json.dumps({'message': 'A new movie was saved successfully in database'})
    }

# Insert a new item into DynamoDB table
def add_movie_to_db(new_movie):

    new_movie['uuid'] = str(uuid.uuid1())

    dynamodb = boto3.resource('dynamodb')
    table = dynamodb.Table('movies-table')

    try:
        response = table.put_item(Item=new_movie)
    except ClientError as e:
        print(e.response['Error']['Message'])
    else:
        return response

This function helps to add a new item to the DynamoDB's movies-table. title field is the only required property of body object passed by a user. uuid field, as a primary key that uniquely identifies each movie, should be automatically generated by uuid.uuid1() Python function.

If you want to debug the code, please follow Step 3. Debug your function locally described in my previous article.

4️⃣ Add lambda function to "template.yaml":

# ======================== GLOBAL ======================== #
Globals:
  Function:
    Runtime: python3.8
    Handler: app.lambda_handler
    Timeout: 60 # default is 3 seconds the function can run before it is stopped
    Environment:
      Variables:
        TABLE_NAME: !Ref MoviesTable

# ======================== RESOURCES ======================== #
Resources:
  # ... other resources ...
  CreateMovieFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: 'CreateMovieFunction'
      CodeUri: create_movie/
      Policies: # follow the principle of least privilege
        - DynamoDBCrudPolicy: # AWS SAM policy
            TableName: !Ref MoviesTable
        - AWSLambdaBasicExecutionRole
      Events:
        CreateMovieApi:
          Type: Api
          Properties:
            RestApiId: !Ref MoviesApi
            Path: /movie
            Method: post

Globals - resources that have common configurations (think of it as a variable in programming). As we are going to create four more functions with the same Runtime, Handler, Timeout and Environment values, we can put all these attributes into Global section following a good old DRY (Don't Repeat Yourself) principle.
CodeUri - the Lambda function's path (might be Amazon S3 URI, local file path, or FunctionCode object). Here we are adding a reference to a create_movie folder. Note, in Global section's Handler attribute we have already specified app file name (without py file extension) and lambda_handler function name.
Policies - one or more policies that will be appended to the default role (a new IAM role will be created) for a Lambda function. Here we are giving our function permission to access our DynamoDB table.
Events - events that trigger the Lambda function. In our case it is POST /movie API event.

In the same way as above, we need to create four more Lambda functions:

GetMovieFunction - get the code from GitHub here
GetMoviesFunction - get the code from GitHub here
UpdateMovieFunction - get the code from GitHub here
DeleteMovieFunction - get the code from GitHub here
The final version of the CloudFormation template - get the code from GitHub here

At the end we should get the following files and folders:

Step 5. Build and deploy the project to AWS Cloud

Build the project inside a Docker container:

cd my-folder
sam build --use-container

AWS SAM builds any dependencies that your application has, and copies your application source code to folders under .aws-sam/build to be zipped and uploaded to Lambda.

Then deploy your application using the following AWS SAM command:

sam deploy --guided

Please follow Step 5. Deploy the project to the AWS Cloud described in my previous article for a detailed explanation.

AWS SAM deploys the application using AWS CloudFormation. You can navigate to AWS Management Console -> CloudFormation to view the list of newly generated resources:

Step 6. SwaggerHub integration

Instead of logging into AWS Management Console and navigating from one API Gateway endpoint to another, we will use SwaggerHub to manage all your APIs in one place and to document them all.

SwaggerHub is an integrated API development platform that brings together all the core capabilities of the open source Swagger framework, along with additional advanced capabilities to build, document, manage, and deploy your APIs.

All you need is to create a free account on Swagger, then navigate to Create New -> Create New API:

Give a name to your template:

Next, modify information of Petstore APIs with our own APIs. For example, for POST /movie/{uuid} API the template should look like this one below:

swagger: '2.0'
info:
  description: |
    This is  a swagger for an AWS serverless microservice built with API Gateway, Lambda and DynamoDB.
  version: 1.0.0
  title: Swagger for AWS serverless microservice
  termsOfService: http://swagger.io/terms/
  license:
    name: Apache 2.0
    url: http://www.apache.org/licenses/LICENSE-2.0.html
tags:
- name: movie
  description: CRUD operations for movie domain
  externalDocs:
    description: Find out about the project
    url: https://github.com/Tiamatt/AWSServerlessMicroserviceWithLambda
paths:
  /movie/{uuid}:
    get:
      tags:
      - movie
      summary: Find a movie by uuid
      description: Returns a single movie
      operationId: getMovieByUuid
      produces:
      - application/json
      - application/xml
      parameters:
      - name: uuid
        in: path
        description: UUID of movie to return
        required: true
        type: string
      responses:
        200:
          description: successful operation
          schema:
            $ref: '#/definitions/Movie'
        400:
          description: Invalid UUID supplied
        404:
          description: Movie not found
definitions:
  Director:
    type: object
    properties:
      firstname:
        type: string
        example: Francis
      lastname:
        type: string
        example: Coppola
    xml:
      name: Director
  Movie:
    type: object
    required:
    - title
    - year
    properties:
      title:
        type: string
        example: Godfather
      year:
        type: integer
        format: int32
        example: 1972
      director:
        $ref: '#/definitions/Director'
      country:
        type: string
        example: United States
    xml:
      name: Movie   
externalDocs:
  description: Find out more about Swagger
  url: http://swagger.io
schemes:
 - https
# Added by API Auto Mocking Plugin
host: {my-id}.execute-api.us-east-1.amazonaws.com
basePath: /Prod

Get the final version of the Swagger template from GitHub here.

You need to replace host value with your API Gateway invoke url, which you can find in Outputs section of AWS CloudFormation Clonsole:

And don't forget to save your changes! Then click on View Documentation icon:

Step 7. Results

Finally, it's time to play with our serverless microservice using beautiful SwaggerHub UI:

🎉 Now, let's create a new movie:

Feel free to add some more movies.

🎉 Let's get a list of all movie:

Copy uuid of 'Godfather' movie for our next steps!

🎉 Get a movie by uuid:

🎉 Delete a movie by uuid:

🎉 And finally, let's update an existing movie:

Step 8. Cleanup

Use the AWS CloudFormation command to delete the stack along with all the resources it created:

aws cloudformation delete-stack --stack-name aws-serverless-microservice-app-stack

Conclusion

Congratulations on getting this far!

Now you know how to create a simple CRUD (create, read, update, delete) app, and to set the foundational services, components, and plumbing needed to get a basic AWS serverless microservice up and running!

Serverless - Create, debug and deploy Lambda and API Gateway via AWS SAM and AWS Cloud9

Samira Yusifova — Wed, 20 Jan 2021 20:14:59 +0000

Let’s say you want to create and deploy a very basic serverless architecture - an API backend that consists of an Amazon API Gateway endpoint and an AWS Lambda function.

For that you need to provision the following resources:
👉 API Gateway
👉 Lambda function
👉 IAM role

What is the fastest and the most convenient way to provision and deploy these serverless resources using AWS native framework?

AWS Serverless Application Model

And the answer is AWS SAM.

AWS SAM (Serverless Application Model) is meant to build serverless applications. Built on AWS CloudFormation, AWS SAM provides shorthand syntax to declare serverless resources.

These 20 lines of code turn into the following serverless architecture:

I took these screenshots from Eric Johnson’s awesome tech talk
which I would highly recommend to watch.

During deployment, AWS SAM transforms the serverless resources into CloudFormation syntax, enabling you to build serverless applications faster.

With that being said, let’s get our hands dirty!

Step 1. Set up AWS Cloud9 environment

In order to write, run, and debug applications with just a browser, without needing to install or maintain a local IDE, we’ll go with AWS Cloud9 IDE. It preconfigures the development environment with all the SDKs, libraries, and plug-ins needed for serverless development. It also provides an AWS Toolkit extension for locally testing and debugging AWS Lambda functions and API Gateway.

First step is to set up a new AWS Cloud9 environment. Sign in to AWS Management Console, go to AWS Cloud9 and click on Create environment:

Give a name to your new environment (you can optionally add description) and go to the next step:

I’ve selected t2.micro instance type as it is covered by the AWS Free Tier.

Once you are done with environment configurations, review your settings and submit. It should take a couple of minutes to launch a new EC2 instance for a new Cloud9 environment. As you can check, there are a lot of preinstalled runtimes and package managers, such as Docker, AWS CLI, pip:

You can run the following command to get a list of all available packages (it’s a pretty long list!):

yum list installed

Step 2. Create a SAM application

On the upper left hand side of the screen you should see AWS Toolkit's AWS icon. Click on it to open AWS:Explorer window. Expand your region and right-click on Lambda, then Create new SAM Application:

Cloud9 should walk you through the following steps:

Voila! AWS SAM creates a directory with the name that you provided as the project name with all necessary files and folders:

hello_world - code for the application's Lambda function
events - invocation events that you can use to invoke the function
tests - unit tests for the application code
template.yaml - a template that defines the application's AWS resources

Step 3. Debug your function locally

Navigate to hello_world/app.py to see AWS Lambda handler logic. We can make a little clean up and uncomment location:

Use Ctrl + S button to save changes in app.py file.

Make sure that you've specified requests dependency in requirements.txt (if you use Node.js functions then check package.json):

Let’s leverage AWS Toolkit to make debugging easier! AWS Toolkit is a very powerful tool that will invoke SAM CLI to build the serverless project, deploy the code to a local Docker instance, invoke the Lambda in Docker, allowing you to step-through your Lambda code. To run the function AWS Toolkit needs launch configuration file. It should be auto-generated during a new SAM app creation - check launch.json file under the .c9 folder.

By the way, if you don't see .c9 folder and some other folders/files then you might need to turn on Show Hidden Files:

Ok, back to launch configuration that gives us more flexibility to configure launchers for our SAM template. As it was mentioned before, launch.json file can be found under the .c9 folder. You can also open the file by Editing Launch Configurations:

In case you are missing launch.json file or if it is empty, open your template file, and click on Add Debug Configuration link. It should automatically set launch configurations for you.

Once launch configuration file is set you should see your function in debugging dropdown:

Set some breakpoints in source code (hello_world/app.py), then select MyFirstSamApp:HelloWorldFunction option from dropdown and click on Run:

Note, in case AWS Toolkit fails to run the function, throwing RuntimeError: Container does not exist, you would need to increase the size of your EBS volume. See how to fix the issue here.

Step-through debugging makes it easier to understand what the code is doing. Thanks to AWS Toolkits it's easy to set breakpoints, execute code line by line, and inspect the values of variables:

After debugging you should get the final response in AWS Toolkit window:

Step 4. Build the project

Building your serverless application involves taking your AWS SAM template file, application code, and any applicable language-specific files and dependencies, and placing all build artifacts in the proper format and location for subsquent steps in your workflow.

First, change into the project directory, where the template.yaml file for the sample application is located.

cd MyFirstSamApp

Then build the project inside a Docker container:

sam build --use-container

AWS SAM builds any dependencies that your application has, and copies your application source code to folders under .aws-sam/build to be zipped and uploaded to Lambda.

Step 5. Deploy the project to the AWS Cloud

After you develop and build your serverless application, you can deploy your application using the following AWS SAM command:

sam deploy --guided

When you specify the --guided flag, the AWS SAM zips your application artifacts, uploads them to Amazon S3, and then deploys your application to the AWS Cloud.

AWS SAM deploys the application using AWS CloudFormation. In the output you can see the changes being made to your AWS CloudFormation stack.

After you've created an AWS CloudFormation stack, you can use the AWS Management Console to view its data and resources:

Results

Time to call our API Gateway endpoint! Go to Outputs sections and click on the link for HelloWorldApi:

When you send a GET request to the API Gateway endpoint, the Lambda function is invoked. This function should return a hello world message with the public IP address:

So, we successfully invoke the function and got the expected response with status code 200.

Now, you know how to configure and use AWS SAM and Cloud9 to build the serverless applications for AWS. Congratulations!

Clean Up

You might be charged for running resources. That is why it is important to clean all provisioned resources once you are done with the stack. Use the AWS CloudFormation command to delete the stack along with all the resources it created:

aws cloudformation delete-stack --stack-name my-first-sam-apps-stack

Summary

Using AWS SAM for serverless application development saves a lot of time by eliminating much of the boilerplate that AWS CloudFormation templates require. It extends AWS CloudFormation with new resource types that follow AWS best practices and are more comfortable to use.

Troubleshooting AWS Cloud9 - RuntimeError: Container does not exist / No space left on device

Samira Yusifova — Tue, 19 Jan 2021 04:58:01 +0000

Issue description

Recently I got a runtime error in AWS Cloud9 while trying to build my serverless application as a Docker container image. I used AWS SAM build command:

sam build --use-container

As a result I got the following error issue:

RuntimeError: Container does not exist. Cannot get logs for this container

It turned out, AWS tried to create amazon/aws-sam-cli-build-image-python3.8 Docker container image:

but failed:

Then I tried to pull the image manually using docker command:

docker pull amazon/aws-sam-cli-build-image-python3.8

but it looks like there is not enough disk space:

You can use the following command to verify that the root volume mounted under "/" is full:

df -h

Here is an output with 95% out of 10 GiB of EBS storage (/dev/xvda1) is used:

To check whether the EBS volume has a partition that must be extended, use the following command:

lsblk

It displays information about the block devices attached to your instance.

Solution

Once the problem is understood, improvement comes naturally. All I need is to increase the size of my EBS volume.

👉 Step 1. Open a new file in Cloud9

👉 Step 2. Add the script below to the file:

#!/bin/bash

# Specify the desired volume size in GiB as a command-line argument. If not specified, default to 20 GiB.
SIZE=${1:-20}

# Get the ID of the environment host Amazon EC2 instance.
INSTANCEID=$(curl http://169.254.169.254/latest/meta-data/instance-id)

# Get the ID of the Amazon EBS volume associated with the instance.
VOLUMEID=$(aws ec2 describe-instances \
  --instance-id $INSTANCEID \
  --query "Reservations[0].Instances[0].BlockDeviceMappings[0].Ebs.VolumeId" \
  --output text)

# Resize the EBS volume.
aws ec2 modify-volume --volume-id $VOLUMEID --size $SIZE

# Wait for the resize to finish.
while [ \
  "$(aws ec2 describe-volumes-modifications \
    --volume-id $VOLUMEID \
    --filters Name=modification-state,Values="optimizing","completed" \
    --query "length(VolumesModifications)"\
    --output text)" != "1" ]; do
sleep 1
done

#Check if we're on an NVMe filesystem
if [ $(readlink -f /dev/xvda) = "/dev/xvda" ]
then
  # Rewrite the partition table so that the partition takes up all the space that it can.
  sudo growpart /dev/xvda 1

  # Expand the size of the file system.
  # Check if we are on AL2
  STR=$(cat /etc/os-release)
  SUB="VERSION_ID=\"2\""
  if [[ "$STR" == *"$SUB"* ]]
  then
    sudo xfs_growfs -d /
  else
    sudo resize2fs /dev/xvda1
  fi

else
  # Rewrite the partition table so that the partition takes up all the space that it can.
  sudo growpart /dev/nvme0n1 1

  # Expand the size of the file system.
  # Check if we're on AL2
  STR=$(cat /etc/os-release)
  SUB="VERSION_ID=\"2\""
  if [[ "$STR" == *"$SUB"* ]]
  then
    sudo xfs_growfs -d /
  else
    sudo resize2fs /dev/nvme0n1p1
  fi
fi

👉 Step 3. Save the file as resize.sh (press Ctrl + S)

👉 Step 4. Run sh-file (shell script):

sh resize.sh 20

You should get detailed information about data block changes:

Here is the link to AWS instuction.

Results

To verify a new EBS volume size, run df -h command again:

Once there is enough disk space for Docker, my container issue is resolved:

Hopefully, this solution could save you both some time and headaches.

Quick guide to pass AWS Certified Cloud Practitioner exam

Samira Yusifova — Tue, 12 Jan 2021 05:33:26 +0000

Recently, I passed the AWS Certified Cloud Practitioner exam with a score of 905/1000. With limited cloud experience, I spent 3 weeks studying but there are some shortcuts I wish I knew. In this article, I would like to share with you my exam prep experience. If you have limited study time and are looking for an efficient study plan, this article is for you.

Why start with AWS Certified Cloud Practitioner?

👉 Of course, you can skip it. But IMHO, it is a must. It gives you a high-level introduction to AWS. You don't need to dive into any particular services, instead all you need is a foundational understanding of cloud principles and AWS services and a general overview of how AWS is structured.

👉 It is the easiest of all the AWS certification exams.

👉 It is a good place to start if you are a cloud newbie. There are tons of new terminology, approaches, abbreviations, services that you need to get used to. By building a basic AWS knowledge layer you can speed up your next AWS exam preparation.

👉 It focuses on billing and pricing which is very important to keep in mind while provisioning services. Otherwise, you might spend a fortune on running services. I launched AWS Cloud9 and forgot about it for a couple of days before I realized that AWS kept charging me for the service I didn’t use any more.

👉 Passing this exam will help you to grow in confidence and familiarize yourself with the general process of passing AWS exams, setting you up for a more successful exam day experience.

👉 And finally, hey, who doesn't like an extra certification!

Exam Details

you need to score at least 700 out of 1000 (that’s 70%) to pass the exam
you have 90 minutes to complete all the questions
the exam contains 65 questions including
- multiple-choice questions: 1 correct response and 3 incorrect responses (distractors)
- multiple-response questions: 2+ correct responses out of 5+ options
the exam costs $100 (when you pass it, you will get some benefits, including $75 off on your next exam purchase)
the certificate is valid for 3 years
you can take a test at the testing center or from home
download the exam guideline from the official AWS website

Where to start

It is easy to get overwhelmed due to the number of online resources and exam preparation guides. For the sake of saving time and money, here are my suggestions:

1️⃣ There is a great "AWS Certified Cloud Practitioner" course by Andrew Brown (ExamPro). It is well organized and very informative, giving you just enough to understand the concepts clearly for the exam. You have two options:

free version on YouTube
$21 course on ExamPro website that includes:
- notes and cheat sheets
- flashcards (a perfect tool for learning faster and reviewing for the exam)
- a lot of quizzes on every lecture
- 4 practice exams (55 questions each)
- note, once you’ve purchased the course you have only 1-year access :-(

2️⃣ A free 6-hour "AWS Cloud Practitioner Essentials" course by AWS official website is also a good source of fundamental knowledge.

3️⃣ Tests, tests, tests! By solving practice tests you develop a needed speed and accuracy. Try to solve as many practice tests as possible! "AWS Certified Cloud Practitioner: 6 Full Practice Exams 2020" by Mozdora Education on Udemy is a very useful one! Worth every penny! It contains 6 practice tests, 65 questions each (mocks a real exam). It costs $9.99-$11.99 when goes on sale (wait for Udemy's sale campaigns).

Optional sources

4️⃣ (extra) If you need more tests, you can try "AWS Certified Cloud Practitioner 500 Practice Exam Questions" by Neal Davis on Udemy. It contains 6 practice tests, 65 questions each (mocks a real exam). And it costs $9.99-$11.99 when goes on sale.

5️⃣ (extra) You might find useful some cheat sheets by DigitalCloud.

6️⃣ (extra) AWS advises a list of white papers to read but I skipped them all as I found them boring. You can find these papers under the "Foundational-level AWS Certification" section here:

Studying tips

🎯 First, focus on minimum valuable knowledge, i.e. the most important contents required by the exam as soon as possible, so you can prioritize your study time to focus on your weakest areas.

🎯 Once you get a high-level understanding of the major services it is time to switch to practice tests. Start with your first exam, and don’t worry if your score isn’t high. The goal is to learn which areas you should spend more time focusing on.

🎯 Solve in a real exam-like environment which means put your mobile away, don’t take any break (remember, you are not allowed to leave a real exam to go to the bathroom!), don’t put your test on pause (a real exam doesn’t have a pause button).

🎯 Revisit both correct and incorrect questions to understand the logic behind answers.

🎯 Repeat all the tests until you get 90% or more on each one

🎯 It’s important to create a detailed plan for what you’re going to study and for how long. It’s up to you how many hours you are ready to spend studying (you may also have to juggle other responsibilities such as family, work, sports, etc). But you have to be persistently consistent. Set some short-term goals for studying like "learn 10 lectures of ExamPro course per day" or "take two practice exams per day with detailed analysis of your results".

🎯 Avoid the trap of perfectionism when studying. If you wait to feel 100% confident before your exam, you’ll never get it done. Create a good knowledge base of AWS, summarise, then practice. Once you get 90% or more on each practice test, go ahead and schedule your exam.

🎯 Remember, tomorrow never comes, start studying today!

What to Focus On

Here are my suggestions to focus your attention on:

AWS Well-Architected Framework (must know!)
- 5 Cloud Architecture technologies: Availability, Scalability, Elasticity, Fault Tolerance, Disaster Recovery
- 5 pillars of AWS Well-Architected Framework, see the link
Concepts:
- Six Advantages of Cloud Computing (Global reach, economy of scale, agility and elasticity, etc)
- Shared responsibility model
Storages:
- S3, Glacier, S3 classes (3-4 questions)
- EBS (1-2 questions)
- Storage Gateway (1-2 questions)
- EFS and FSx
- Snow Family (Snowmobile, Snowball, Snowcone)
Database:
- RDS (including Aurora)
- DynamoDB
- ElastiCache
Compute:
- EC2
- EC2 pricing models (3-4 questions)
- Lambda
- Elastic Beanstalk
Management and Governance:
- CloudWatch
- CloudTrail
- Control Tower (1 question)
- Trusted Advisor
- Personal Health Dashboard
- Well-Architectured Tool (1 question)
App Integration:
- SNS
- SES
- SQS
Cost Management:
- Cost Explorer
- Cost and Usage Report
- Budgets
- Organization and consolidation bills (2-3 questions)
Network and Content Delivery:
- CloudFront (1-2 questions)
- DirectConnect, AWS VPN, PrivateLink (3-4 questions)
- NACL and security groups
- Global Accelerator
- ELB
- Route53
Security, Identity & Compliance:
- IAM (3-4 questions), roles, groups
- Inspector
- Artifacts and compliance
- Cognito
- GuardDuty
- Macie
- Shield
- WAF
- AWS Penetration Testing
Also:
- Amazon Polly (Text-to-Speech Service) (1 question)
- CodeDeploy (1 question)

Good luck!

Hands-on AWS CloudFormation - Part 5. IAM users, groups and roles

Samira Yusifova — Mon, 04 Jan 2021 07:02:14 +0000

In the “Hands-on AWS CloudFormation” series we continue to create small templates by provisioning different types of AWS resources with AWS CloudFormation. In the end of this series we can turn the small templates into building blocks for full stack templates. For example, in Part 4 we’ve learned how to create a VPC with private and public subnets - ultimately, it will help us to create a secure, highly reliable and fault-tolerant system using multiple EC2 instances in a private network and ancillary services such as Auto Scaling and Elastic Load Balancing. But to get to the final results we need to take a few baby steps at a time. That being said, for today’s lesson we will cover the IAM part of AWS using CloudFormation.

Don’t ignore IAM

AWS IAM is something you need to take seriously if you are working in the AWS space. Think of it as an essential gate-keeper of AWS. This is the place where you would administer authentication and authorization for AWS’s environments and services.

It’s admirable that AWS IAM follows an incredibly granular approach in providing permissions and access control within your environments. With that, let's take advantage of the great cloud platform and create the basic components of IAM with CloudFormation, such as:

Policies
Users
Groups
Roles

Creating IAM policies

We manage access in AWS by creating policies and attaching them to IAM identities (users, groups of users, and roles) or AWS resources. Policies specify a set of permissions. Permissions in the policies determine whether the request is allowed or denied.

There are three types of IAM policies:

AWS Managed Policy
Customer Managed Policy
Inline Policy

AWS Managed Policy

AWS Managed Policy is a standalone policy that is created and administered by AWS. AWS managed policies could be reused between IAM entities (users, groups, or roles) and cannot be modified.

Here is an example of how you can attach AWS managed policy to a new role:



Resources:
  myRole: # a new role
    Type: 'AWS::IAM::Role'
    Properties: 
      # ... some code here ...
      ManagedPolicyArns: # list of ARNs of IAM managed policies that you want to attach to the role
        - arn:aws:iam::aws:policy/AWSCloud9Administrator # provides administrator access to AWS Cloud9

You can get a needed AWS managed policy via AWS Management Console by navigating to IAM -> Policies, then filter by Policy type checking ‘AWS managed’ checkbox:

or you can use AWS CLI command:



aws iam list-policies --scope AWS

Check out this link if you need to install AWS CLI.

Customer Managed Policy

Customer Managed Policy is a standalone policy that is created by a user. Customer managed policies could be reused between IAM entities and can be modified.

Let’s create a customer managed policy that gives read only access to EC2 instance:



Resources:
  myCustomerManagedPolicyForEC2:
    Type: 'AWS::IAM::ManagedPolicy'
    Properties:
      ManagedPolicyName: customerManagedEC2ReadOnlyPolicy # give a name to this policy
      Description: Customer managed policy for read only access to EC2 instance
      Path: '/'
      PolicyDocument: # (required) JSON policy document 
        Version: '2012-10-17'
        Statement: # allow read only access to EC2 instance
          - Effect: Allow
            Action: 
              - 'ec2:Describe'
            Resource: '*'
      # IAM entities (Groups, Roles, and Users) are optional properties
      Users: # attach this policy to the list of existing users
          - userA
          - userB
      Groups: # attach this policy to the list of existing groups
          - groupA
          - groupB
      Roles: # attach this policy to the list of existing roles
          - roleA
          - roleB

Take a look at this GitHub ‘CustomerManagedPolicy’ template. Also check out AWS link for documentation.

Note, PolicyDocument is the only required field in Properties. It is simply a policy (a JSON document). Here is my favorite link to the great list of example policies. Usually you need to provide policies in JSON format in IAM. However, for AWS CloudFormation templates formatted in YAML, you can provide the policy in JSON or YAML format. AWS CloudFormation always converts a YAML policy to JSON format before submitting it to IAM.

What is the Path? If you are using the IAM API or AWS CLI to create IAM resources, you can also give some resources an optional path, e.g. ‘/companyA/departmentB/projectC/’ to match your company's organizational structure. You could then create a policy to allow all users in that path to access the policy simulator API.

Inline Policy

Inline Policy is a policy that is created by a user and embedded directly to IAM entities. Inline policies cannot be reused in different IAM entities as it emphasizes direct one-to-one relationship between entity and the policy itself. Once the entity is deleted, inline policies attached to it get removed as well.

Let’s create an inline policy that gives read only access to all S3 buckets:



Resources:
  myInlinePolicyForS3ReadOnly:
    Type: 'AWS::IAM::Policy'
    Properties:
      PolicyName: inlineS3ReadOnlyPolicy # (required) give a name to this policy
      PolicyDocument: # (required) JSON policy document
        Version: '2012-10-17'
        Statement: # allow read only access to all S3 buckets
          - Effect: Allow
            Action:
              - 's3:Get*'
              - 's3:List*'
            Resource: '*'
      # Note, Groups, Roles, and Users fields are optional. However, you must specify at least one of these fields
      Users: # attach this policy to the list of existing users
          - userA
          - userB
      Groups: # attach this policy to the list of existing groups
          - groupA
          - groupB
      Roles: # attach this policy to the list of existing roles
          - roleA
          - roleB

Take a look at this GitHub ‘InlinePolicy’ template. Also check out AWS link for documentation.

Note, PolicyName and PolicyDocument are the only required fields in Properties. Roles, Users, and Groups fields are optional. But you must specify at least one of these fields. Thus, if you run the stack ignoring all three fields (Groups, Roles, and Users), CloudFormation will roll it back notifying of an error:

Useful links

Check out the library of IAM identity-based policies here - it can help you to find the JSON policy document as a template for your own policies.
Check out IAM Policy Simulator which can be used to test and troubleshoot IAM and resource based policies.
Take a look at a formal Grammar for the language used to create JSON policies in IAM.
If you need to use the guide for a visual editor to create and modify your IAM policies, check out this article.

Creating IAM users and groups

IAM user is a person that needs to interact with your AWS resources or services either from the AWS Console or with the AWS CLI. When you create a new user, no credentials are assigned, and the user does not have any permission to access your AWS resources.

Now, let’s create some IAM users with AWS Cloudformation:



Resources:
  myUser:
    Type: 'AWS::IAM::User'
    Properties:
      UserName: userA # give a name to this user
      LoginProfile: # specify a password for this user
        Password: pa$$w0rd
        PasswordResetRequired: true # make this user to set a new password on next sign-in
      Path: '/'
      Groups: # attach this user to the list of existing groups
          - groupA
          - groupB

Take a look at this GitHub User template. Also check out AWS link for documentation.

LoginProfile contains the user's password. You can also use PasswordResetRequired to specify whether the user is required to set a new password on next sign-in.

As mentioned before, by default any new IAM user is created with no access to any AWS services (non-explicit deny). You can set permissions by adding needed policies to the user (please make sure to follow the standard security advice of granting least privilege).

In order to attach IAM managed policies to a user, use ManagedPolicyArns field:



Resources:
  myUser:
    Type: 'AWS::IAM::User'
    Properties:
      UserName: userB # give a name to this user
      ManagedPolicyArns: # list of ARNs of IAM managed policies that you want to attach to the user
        - arn:aws:iam::aws:policy/AWSCloud9Administrator # provides administrator access to AWS Cloud9
        - arn:aws:iam::111111111111:policy/myCustomerManagedPolicy # use your own customer managed policy

Also you can add an inline policy document that is embedded in the specified IAM user, using Policies field:



Resources:
  myUser:
    Type: 'AWS::IAM::User'
    Properties:
      UserName: userC # give a name to this user
      Policies: # list of inline policy documents that are embedded in the user
        - PolicyName: inlineS3ReadOnlyPolicy # give a unique name to this policy
          PolicyDocument: # JSON policy document
            Version: '2012-10-17'
            Statement: # allow read only access to all S3 buckets
              - Effect: Allow
                Action:
                  - 's3:Get*'
                  - 's3:List*'
                Resource: '*'

Take a look at this GitHub UserWithPolicies template.

But what if you need to give Admin permissions to a hundred of users? Are you going to attach a new policy to each and every user? If so, what if tomorrow you want to change Admin permissions to something else? The easiest way to do that is to create an IAM group called Admins and give that group the types of permissions that administrators typically need. Then add users to that group. Those added users will then automatically have all of their group’s permissions. Without a group, listing permissions for every single user will be a huge hassle.

Group is a collection of IAM users. Groups are useful when we want to manage permissions for a group of users.

If you already have some groups, you can attach them to userA specifying a list of groups in Groups field.



Resources:
  myUser:
    Type: 'AWS::IAM::User'
    Properties:
      UserName: userD # give a name to this user
      Groups: # attach this user to the list of existing groups
          - groupA
          - groupB

If not, then let’s create a new group:



Resources:
  myGroup:
    Type: 'AWS::IAM::Group'
    Properties:
      GroupName: ApiDevelopers # give a name to this group
      Path: '/'
      ManagedPolicyArns: # list of ARNs of IAM managed policies that you want to attach to the group
        - arn:aws:iam::aws:policy/AWSCloud9Administrator # provide administrator access to AWS Cloud9
        # - arn:aws:iam::111111111111:policy/customerManagedBlahBlahPolicy # use your own customer managed policy specifying its ARN
      Policies: # list of inline policy documents that are embedded in the group
        - PolicyName: inlineCloudWatchLogsPolicy # give a unique name to this policy
          PolicyDocument: # JSON policy document
            Version: '2012-10-17'
            Statement: # provide write permissions to CloudWatch Logs
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream*'
                  - 'logs:PutLogEvents'
                Resource: '*'

Take a look at this GitHub ‘Group’ template. Also check out AWS link for documentation.

You can specify IAM managed policies using ManagedPolicyArns field and inline policies using Policies field.

The last step is to learn how to attach multiple existing users to an existing group. For that you might need to use AWS::IAM::UserToGroupAddition:



Resources:
  myUserToGroupAddition:
    Type: 'AWS::IAM::UserToGroupAddition'
    Properties:
      GroupName: groupB # existing group name
      Users: # list of existing user names
          - userA
          - userB

Take a look at this GitHub ‘AttachUsersToGroup’ template. Also check out AWS link for documentation.

Note, a group does not have security credentials as well as cannot access and manage AWS’s resources - it just helps to manage user permissions.

Creating IAM roles

IAM roles allow you to delegate access to users or services that normally don't have access to your organization's AWS resources. IAM users or AWS services can assume a role to obtain temporary security credentials that can be used to make AWS API calls. Consequently, you don't have to share long-term credentials or define permissions for each entity that requires access to a resource.

Creating a new role is similar to delegate access permissions to those trusted entities without having to share access keys. A role cannot make direct requests to AWS services, but the entity it attached to.

Let’s create a simple role for an EC2 instance:



Resources:
  myRole:
    Type: 'AWS::IAM::Role'
    Properties: 
      RoleName: roleA # give a name to this role
      Description: IAM role for EC2 instance
      AssumeRolePolicyDocument: # (required) only one trust policy with a role
        Version: '2012-10-17'
        Statement: 
          - 
            Effect: Allow
            Principal: 
              Service: 
                - 'ec2.amazonaws.com'
            Action: 
              - 'sts:AssumeRole'
      MaxSessionDuration: 3600 # in seconds, 1 hour
      Path: '/'

Take a look at this GitHub ‘Role’ template. Also check out AWS link for documentation.

Note, AssumeRolePolicyDocument is the only required field in Properties. It is the trust policy that is associated with this role. Trust policies define which entities can assume the role. But you can associate only one trust policy with a role.

MaxSessionDuration (in seconds) helps you to set the maximum session duration for a new role. If you do not specify a value for this setting, the default maximum of one hour (3600 seconds) is applied. This setting can have a value from 1 hour to 12 hours.

Similar to users, you can set permissions for the role by adding needed policies to it (and again, please follow the standard security advice of granting least privilege).

In order to attach IAM managed policies to the role, use ManagedPolicyArns field:



myRole:
    Type: 'AWS::IAM::Role'
    Properties: 
      RoleName: roleB # give a name to this role
      AssumeRolePolicyDocument: # (required) only one trust policy with a role
      # ... some code here ...
      ManagedPolicyArns: # list of ARNs of IAM managed policies that you want to attach to the role
        - arn:aws:iam::aws:policy/AWSCloud9Administrator # provides administrator access to AWS Cloud9
        - arn:aws:iam::111111111111:policy/myCustomerManagedPolicy # use your own customer managed policy

Also you can add an inline policy document that is embedded in the role, using Policies field:



myRole:
    Type: 'AWS::IAM::Role'
    Properties: 
      RoleName: roleB # give a name to this role
      AssumeRolePolicyDocument: # (required) only one trust policy with a role
      # ... some code here …
      Policies: # list of inline policy documents that are embedded in the role
        - PolicyName: inlineS3ReadOnlyPolicy # give a unique name to this policy
          PolicyDocument: # JSON policy document
            Version: '2012-10-17'
            Statement: # allow read only access to all S3 buckets
              - Effect: Allow
                Action:
                  - 's3:Get*'
                  - 's3:List*'
                Resource: '*'

Take a look at this GitHub ‘RoleWithPolicies’ template.

Creating Instance Profile

You should use an Instance Profile to pass an IAM role to an EC2 instance. But what is the difference between an AWS role and an instance profile?

Roles are designed to be 'assumed' by other principals which do define 'who am I?', such as users, AWS services, and EC2 instances.
Instance profile, on the other hand, defines 'who am I?' Just like an IAM user represents a person, an instance profile represents EC2 instances. The only permissions an EC2 instance profile has is the power to assume a role.

Here is how you can create a new instance profile in order to attach your role to EC2 instance:



Resources: 
  myInstanceProfile: 
    Type: 'AWS::IAM::InstanceProfile'
    Properties: 
      InstanceProfileName: instanceProfileA
      Roles: # (required) existing role name to associate with the instance profile 
      # Note: only one role can be assigned to an EC2 instance at a time, but type is 'List of String'
        - roleD
      Path: '/'

Take a look at this GitHub ‘InstanceProfile’ template. Also check out AWS link for documentation.

Note, there are some limitations - an instance profile can contain only one IAM role, although a role can be included in multiple instance profiles. This limit of one role per instance profile cannot be increased. You can remove the existing role and then add a different role to an instance profile.

Review

AWS IAM provides a number of security features to consider as you develop and implement your own security policies.

In this article, we walked through how to start creating IAM policies and IAM identities (users, groups, and roles) using AWS CloudFormation. You can download and use IAM templates from my GitHub account and use them in stacks. Hopefully this hands-on guide will help you excel in the field of AWS CloudFormation.

Quick guide to start using AWS CodeCommit via AWS CLI

Samira Yusifova — Mon, 28 Dec 2020 01:41:35 +0000

I was looking for AWS CodeCommit quick tutorial and ended up watching ACloudGuru’s “Manage and Deploy Code with AWS Developer Tools” 8 hour course (BTW, a great one!), reading some articles and AWS documentations. Based on that I’ve decided to create a quick guide to start using AWS CodeCommit via AWS CLI.

What is AWS CodeCommit?

Think of it as an AWS version of GitHub. In other words, AWS CodeCommit is a managed source control service that hosts private Git repositories. You can use CodeCommit to store and manage documents, source code, and binary files in the cloud.

CodeCommit is integrated with a number of AWS services such as CodeBuild, CodePipeline, Lambda, SNS, Amplify, CloudFormation, etc.

AWS CodeCommit vs GitHub

Both can be primarily classified as "Code Collaboration & Version Control" tools.

Similarities
Support code review
Use two methods of authentication, SSH and HTTPS
Use Git repositories

Based on comparison provided by CompareCamp, here are some cons and pros of both services:

AWS CodeCommit Pros	AWS CodeCommit Cons
free private repos (for up to 5 users)	complex menu options
faster deployments using AWS solutions	resubmit flow and code review unavailable
encrypted repo data	limited triggers
issue tracker	lack of CI system integrations

On the other hand,

GitHub Pros	GitHub Cons
ease of use	notifications are hard to configure
easy integration with third-party tools	lack of first-party support for mobile
code highlighting for ObjectScript	lack of command-line configuration options
comprehensive issue tracking	security scanning problems

Prerequisites and setup (using HTTPS)

Of course, you can use AWS CodeCommit console with friendly UI to manage files and repositories directly. But in order to work with multiple files, files across branches, and also to control multiple AWS services from the command line and automate them through scripts consider setting up your local computer to work with AWS CodeCommit via AWS CLI.

Step 1. Install Git

Check installation by running the following command:

git --version

Step 2. Install AWS CLI

Check installation by running the following command:

aws --version

Step 3. Create Git credentials for IAM user

Firstly, create a new AWS IAM user and give it a Programmatic access

Then attach AWSCodeCommitFullAccess policy to the user.

Store a user's credentials in a safe place

Step 4. Configure AWS CLI

Use the following command and enter newly created Access key ID and Secret access key:

aws configure

AWS CLI will also ask you to enter the Region whose servers you want to send your requests to. By default it is typically the region closest to you, but you can enter any other region.

And finally, you might want to specify Output format how the results are formatted. json is used as the default, but you can use yaml, yaml-stream (streaming allows for faster handling of large data types), text, and table formats.

Alternatively, you can set up AWS CodeCommit using SSH connections for Windows and for Linux, macOS, or Unix.

Pricing

First 5 active users	Each additional active user beyond the first 5
$0.00	$1.00 per month
Unlimited repositories	Unlimited repositories
50 GB-month of storage	10 GB-month of storage per active user
10,000 Git requests/month	2,000 Git requests/month per active user

Check the latest prices here.

AWS CodeCommit commands

Here is a full list of AWS CodeCommit commands. In this article, we are going to understand how to use the top command. Just follow along the guide to get a basic experience with AWS CodeCommit.

To create a new repository, run:

aws codecommit create-repository --repository-name MyDemoRepo --repository-description "This is a demo repo"

Output:

You should find a new repo on AWS CodeCommit Console:

To get a list of all existing repositories, run:

aws codecommit list-repositories

To get information about specific repository, run:

aws codecommit get-repository --repository-name MyDemoRepo

To get information about multiple specific repositories, run:

aws codecommit batch-get-repositories --repository-names MyDemoRepo MyBestRepo

To update repository’s name, run:

aws codecommit update-repository-name --old-name MyDemoRepo --new-name MyDemo2Repo

To update repository’s description, run:

aws codecommit update-repository-description --repository-name  MyDemo2Repo --repository-description "This is updated description"

To clone repository, you need to get a url from AWS CodeCommit console

then run the following commands:

# navigate to your local directory
cd "C:\Users\{userName}\{folder}\{nested-folder}"
# # clone a repository from AWS CodeCommit to your local machine
git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/MyDemo2Repo localMyDemo2Repo

Important! If a Windows Security is popped up asking for username and password, then cancel it

In my case I still got the warning about an empty repository with a successfully copied repository on my local machine.

The issue has gone once I've uninstalled and reinstalled Git for Windows with cleared the check box for the option to install the Git Credential Manager utility (details) because the credential manager is not compatible with the credential helper for AWS CodeCommit.

If you got different kind of issues while attempting to clone a repository, see AWS's Troubleshooting section.

To commit and push changes to AWS CodeCommit, run Git commands:

# navigate to your local repository
cd localMyDemo2Repo
# create a sample text file
echo "Greeting, AWS CodeCommit nerds! Let's rock it!" > hello.txt
# stage the file for commit to your local repository
git add .
# commit the file that you've staged
git commit -m"Add a new hello file"
# push the changes in your local repository to AWS CodeCommit
git push origin master

As a result, a new "hello.txt" file should be added to remote repository:

To manage branches, merges, tags, logs, etc, use Git commands. You can find the full list of Git commands here and one of my favorite Git cheat sheet here.

To view information about current repository, run:

git remote show origin

Output:

Also, it you want to know how to migrating a repository from GitHub into AWS CodeCommit via AWS CLI, check this article.

Create AWS CodeCommit trigger to send notification via Amazon SNS

You can configure a AWS CodeCommit repository so that code pushes or other events trigger notification from Amazon SNS. Beware of limitations, as AWS allows up to 10 triggers for each CodeCommit repository.

Let's configure "MyDemo2Repo" repository in AWS CodeCommit so that any event triggers Amazon SNS to send a notification to email.

Step 1. Give an IAM user some permissions to manage Amazon SNS

An IAM user named 'code-commit-user' doesn't have enough permissions to manage an Amazon SNS. In addition to AWSCodeCommitFullAccess policy you might want to attach AmazonSNSFullAccess policy to that user:

Step 2. Create an SNS topic

To create a topic, use the aws sns create-topic command and specify the name to assign to the topic:

aws sns create-topic --name my-codecommit-trigger

You should get a TopicArn as a response - write it down for step 3:

Step 3. Subscribe to a topic

Create a subscription and subscribe to a topic specifying TopicArn:

aws sns subscribe --topic-arn arn:aws:sns:{your-region}:{your-accountId}:my-codecommit-trigger --protocol email --notification-endpoint {your-email@blah.blah}

Output:

AWS immediately sends a confirmation message by email to your address. You need to confirm the subscription.

Once confirmed, your browser should display a notification message with information similar to the following:

You can find a newly created topic on Amazon SNS Console:

Step 4. Create a JSON file with trigger description

Next step is to create a trigger for a CodeCommit repository so that events in that repository trigger notifications from SNS topic.

Firstly, create a JSON file that specifies SNS topic name, the repository and branches you want to monitor with this trigger, and the events that activate this trigger.

For example, to create a trigger for a repository named MyDemo2Repo that publishes all repository events to an Amazon SNS topic named my-codecommit-trigger for a master branch, save the following code in 'my-first-codecommit-trigger.json' file:

{
    "repositoryName": "MyDemo2Repo",
    "triggers": [
        {
            "name": "AllEventsTriggerForMyDemo2Repo",
            "destinationArn": "arn:aws:sns:{your-region}:{your-accountId}:my-codecommit-trigger",
            "customData": "",
            "branches": [
                "master"
            ],
            "events": [
                "all"
            ]
        }
    ]
}

In our example we used all event type but here is the full list of event types you can create triggers for:

Event	Description
all	for all events in the specified repository and branches
updateReference	for when commits are pushed to the specified repository and branches
createReference	for when a new branch or tag is created in the specified repository
deleteReference	for when a branch or tag is deleted in the specified repository

Let's test our json file:

 aws codecommit test-repository-triggers --cli-input-json "file://C:\Users\{your-folders}\my-first-codecommit-trigger.json"

Note, you have to use file:// before the json name.

If successful, this command returns information similar to the following:

Step 5. Create a trigger

Now it's time to create a trigger in AWS CodeCommit that is described in json file. The command is similar to the previous one but instead of 'test' use 'put':

 aws codecommit put-repository-triggers --cli-input-json "file://C:\Users\{your-folders}\my-first-codecommit-trigger.json"

This command returns a configuration ID:

Check your email, you should get a very first notification from AWS:

Step 6. View trigger configuration

If you want to view the configuration of the trigger for target repository, run:

aws codecommit get-repository-triggers --repository-name MyDemo2Repo

Output:

Step 7. Make some repository changes

To test the functionality of the trigger itself, make and push a commit to the repository where you configured the trigger. You should see a response from the Amazon SNS topic.

# navigate to your local repository
cd localMyDemo2Repo
# create a text file
echo "This is a new file for MyDemo2Repo repository that should trigger a notification from AWS SNS topic" > test-sns.txt
# stage the file for commit to your local repository
git add .
# commit the file that you've staged
git commit -m"Add a new test-sns file"
# push the changes in your local repository to AWS CodeCommit
git push origin master

As a result, a new "test-sns.txt" file should be added to remote repository and a new notification from AWS should be sent to your email:

Clean up

To avoid ongoing charges for resources you created to complete this guide, let's delete our repository and sns topic:

# get the list of all subscriptions
aws sns list-subscriptions
# from the list above get a target SubscriptionArn and use it in command below to unsubscribe from a topic and stop receiving messages published to that topic
aws sns unsubscribe --subscription-arn {your-SubscriptionArn }
# get the list of all sns topics
aws sns list-topics
# from the list above get a target TopicArn and use it in command below to delete SNS topic
aws sns delete-topic --topic-arn {your-TopicArn}
# get the list of all repositories
aws codecommit list-repositories
# from the list above get a target repositoryName and use it in command below to delete repository 
aws codecommit delete-repository --repository-name MyDemo2Repo

All done! Till the next time.

Migrating a repository from GitHub into AWS CodeCommit via AWS CLI

Samira Yusifova — Sun, 27 Dec 2020 22:41:31 +0000

Actually, you can migrate your repository not only from GitHub but also from any git-based source control service such as GitLab, BitBucket, Beanstalk (don’t confuse it with AWS Elastic Beanstalk). In order to migrate repository from non git-based source control service (such as Subversion, TFS, Perforce) you will have to migrate to a git-based service first. Also you can choose to migrate the whole repository or just some of the branches.

In this article I'll show how to migrate one of GitHub's AWS Samples repositories to AWS CodeCommit.

Step 1. Create a new repository in AWS CodeCommit

You need to create an empty repository in AWS CodeCommit where you want to migrate a GitHub repository:

aws codecommit create-repository --repository-name MyMigratedRepo --repository-description "This is a repo that was migrated from GitHub"

Step 2. Clone a target repository from GitHub to your local machine

First you need to get a target repository’s url from GitHub:

Then run the following commands:

# navigate to a directory where you want to save a new repository
cd "{my-full-path-to-destination-folder}"
# clone a repository from GitHub to your local machine
git clone --mirror https://github.com/aws-samples/aws-glue-samples.git myMigrationRepo

Step 3. Push a cloned repository to AWS CodeCommit

First you need to get a destination repository’s url from AWS CodeCommit:

Then run the following commands

git push https://git-codecommit.{my-region}.amazonaws.com/v1/repos/MyMigratedRepo --all

Step 4. Enjoy the results

Voila! We have just migrated a repository from GitHub into AWS CodeCommit:

Hands-on AWS CloudFormation - Part 4. Create VPC with private and public subnets

Samira Yusifova — Tue, 22 Dec 2020 09:19:09 +0000

Based on the knowledge from all previous parts of Hands-on AWS CloudFormation series let's create something simple, essential and cool!

A virtual network in the AWS cloud

When you provision a service within a public cloud, it is effectively open to the world and can be at risk of attacks from the internet. In order to secure resources against attacks from the outside, you lock them within a VPC. VPC restricts what sort of traffic, IP addresses and also the users that can access your resources. In other words, VPC is your network but in the cloud.

Just to be clear, AWS has already created a default VPC for you to use so that you don't have to create and configure your own VPC. But, by default its subnet in each AZ is a public subnet, because the main route table sends the subnet's traffic that is destined for the internet to the internet gateway.

This tutorial walks through how to create a fully functional custom VPC from scratch with a pair of public and private subnets spread across two AZs using AWS CloudFormation. Here are core components that we are going to create:

a VPC
an Internet Gateway
a custom route table for all public subnets
two public subnets spread across two AZs
two NAT Gateways attached to public subnets
two custom route tables for each of private subnets
two private subnets spread across two AZs

At the end we will get the following architecture:

Let’s get started!

Step 1. Create VPC and Internet Gateway

Firstly, we start by creating a VPC with a size /16 IPv4 CIDR block. This provides up to 65,536 private IPv4 addresses. We might hard code the CIDR block for VPC but it’s better to define it as an input parameter so a user can either customize the IP ranges or use a default value.



Parameters:
  paramVpcCIDR:
    Description: Enter the IP range (CIDR notation) for VPC
    Type: String
    Default: 10.192.0.0/16
Resources:
  # a) Create a VPC
  myVPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref paramVpcCIDR
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
      - Key: CloudFormationLab
        Value:  !Ref paramUniqueName

CidrBlock - the IP address range available to our VPC
EnableDnsSupport - if set to true, AWS will resolve DNS hostnames to any instance within the VPC’s IP address
EnableDnsHostnames - if set to true, instances get allocated DNS hostnames by default

Secondly, we need to create an Internet Gateway. An Internet Gateway is a logical connection between a VPC and the Internet. If there is no Internet Gateway, then there is no connection between the VPC and the Internet.



  # b) Create a Internet Gateway
  myInternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
      - Key: CloudFormationLab
        Value:  !Ref paramUniqueName

And finally, let’s attach the Internet Gateway to the VPC



  # c) Attach the Internet Gateway to the VPC
  myVPCGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref myVPC
      InternetGatewayId: !Ref myInternetGateway

With that, we have already built a very basic VPC.

Step 2. Create a public route table and public subnets across two AZs

Placing our instances in multiple AZs increase the availability of our resources and improves the fault tolerance in our applications. If one AZ experiences an outage, you might want to redirect the traffic to the other AZ. Having said that, we are going to follow AWS best practices to create subnets - each in a different AZ.

Firstly, let’s create a custom route table for all public subnets and name it public route table. We need it to control the routing for the public subnets which we are about to create.



  # a) Create a public route table for the VPC (will be public once it is associated with the Internet Gateway)
  myPublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref myVPC
      Tags:
        - Key: Name
          Value: !Ref paramUniqueName

Secondly, we need to add a new route to the public route table that points all traffic (0.0.0.0/0) to the Internet Gateway.



  # b) Associate the public route table with the Internet Gateway
  myPublicRoute:
    Type: AWS::EC2::Route
    DependsOn: myVPCGatewayAttachment
    Properties:
      RouteTableId: !Ref myPublicRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref myInternetGateway

DestinationCidrBlock - it specifies which traffic we want this route to be applied to. In this case, we apply it to all traffic using the 0.0.0.0/0 CIDR block
GatewayId - it specifies where traffic matching the CIDR block should be directed

Note, when you add a DependsOn attribute to a resource, that resource is created only after the creation of the resource specified in the DependsOn attribute.

Thirdly, once we are done with the public route table, time to create two public subnets with a size /24 IPv4 CIDR block in each of two AZs. This provides up to 256 addresses per subnet, a few of which are reserved for AWS use.

It’s better to define CIDR blocks for both subnets as input parameters so a user can either customize the IP ranges or use a default range.



Parameters:
  # etc
  paramPublicSubnet1CIDR:
    Description: Enter the IP range (CIDR notation)  for the public subnet in AZ A
    Type: String
    Default: 10.192.10.0/24
  paramPublicSubnet2CIDR:
    Description: Enter the IP range (CIDR notation)  for the public subnet in AZ B
    Type: String
    Default: 10.192.11.0/24

Important! The CIDR blocks of two subnets must not overlap with the CIDR block that's associated with the VPC.

Subnets must exist within a VPC, so this is how we associate these two subnets within our VPC:



   # c) Create a public subnet in AZ 1 (will be public once it is associated with public route table)
  myPublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref myVPC
      AvailabilityZone: !Select [ 0, !GetAZs '' ] # AZ 1
      CidrBlock: !Ref paramPublicSubnet1CIDR
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: !Ref paramUniqueName
  # Create a public subnet in AZ 2 (will be public once it is associated with public route table)
  myPublicSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref myVPC
      AvailabilityZone: !Select [ 1, !GetAZs  '' ] # AZ 2
      CidrBlock: !Ref paramPublicSubnet2CIDR
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: !Ref paramUniqueName

By setting MapPublicIpOnLaunch to true instances launched into the subnet will be allocated a public IP address by default. This means that any instances in this subnet will be reachable from the Internet via the Internet Gateway attached to the VPC.

And finally, it’s important to associate the route table with both public subnets:



  # d) Associate the public route table with the public subnet in AZ 1
  myPublicSubnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref myPublicRouteTable
      SubnetId: !Ref myPublicSubnet1
  # Associate the public route table with the public subnet in AZ 2
  myPublicSubnet2RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref myPublicRouteTable
      SubnetId: !Ref myPublicSubnet2

At that point, we are done with public subnets, now let’s create private subnets

Step 3. Create NAT Gateways, private route tables and private subnets across two AZs

We don't want instances within our private subnets to be reachable from the public internet. But we do want these instances to be able to initiate outbound connections. Also we want them to be able to do this without having public IP addresses. That is when NAT gateway comes in handy.

NAT gateway enables instances in a private subnet to connect to the internet or other AWS services, but prevent the internet from initiating a connection with those instances. It will have a public IP address and will be associated with a public subnet. Private instances in private subnets will be able to use this to initiate outbound connections. But the NAT will not allow the reverse, a party on the public internet cannot use the NAT to connect to our private instances.

Thus, we need an Elastic IP (EIP) address and a NAT gateway. Not even one, but two! Because a single NAT Gateway in a single AZ has redundancy within that AZ only, so if there are any zonal issues then instances in other AZs would have no route to the internet. To remain highly available, we need a NAT Gateway in each AZ and a different route table for each AZ.



# a) Specify an Elastic IP (EIP) address for a NAT Gateway in AZ 1
  myEIPforNatGateway1:
    Type: AWS::EC2::EIP
    DependsOn: myVPCGatewayAttachment
    Properties:
      Domain: vpc
  # Specify an Elastic IP (EIP) address for a NAT Gateway in AZ 2
  myEIPforNatGateway2:
    Type: AWS::EC2::EIP
    DependsOn: myVPCGatewayAttachment
    Properties:
      Domain: vpc

  # b) Create a NAT Gateway in the public subnet for AZ 1
  myNatGateway1:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt myEIPforNatGateway1.AllocationId
      SubnetId: !Ref myPublicSubnet1
   # Create a NAT Gateway in the public subnet for AZ 2
  myNatGateway2:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt myEIPforNatGateway2.AllocationId
      SubnetId: !Ref myPublicSubnet2

  # c) Create a private route table for AZ 1
  myPrivateRouteTable1:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref myVPC
      Tags:
        - Key: Name
          Value: !Ref paramUniqueName
  # Create a private route table for AZ 2
  myPrivateRouteTable2:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref myVPC
      Tags:
        - Key: Name
          Value: !Ref paramUniqueName

  # d) Associate the private route table with the Nat Gateway in AZ 1
  myPrivateRouteForAz1:
    Type: AWS::EC2::Route
    DependsOn: myVPCGatewayAttachment
    Properties:
      RouteTableId: !Ref myPrivateRouteTable1
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref myNatGateway1       
  #  Associate the private route table with the Nat Gateway in AZ 2
  myPrivateRouteForAz2:
    Type: AWS::EC2::Route
    DependsOn: myVPCGatewayAttachment
    Properties:
      RouteTableId: !Ref myPrivateRouteTable2
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref myNatGateway2

In the sample above, we created and specified an Elastic IP address to associate with the NAT gateway for each AZ. After creating a NAT gateway, we created and updated the route table associated with each private subnet to point internet-bound traffic to the NAT gateway. This enables instances in our private subnets to communicate with the internet.

And our final step is to create two private subnets with a size /24 IPv4 CIDR block in each of two AZs. And then associate the private route table with the private subnet for each AZ.



Parameters:
  # etc
  paramPrivateSubnet1CIDR:
    Description: Enter the IP range (CIDR notation)  for the private subnet in AZ A
    Type: String
    Default: 10.192.20.0/24
  paramPrivateSubnet2CIDR:
    Description: Enter the IP range (CIDR notation)  for the private subnet in AZ B
    Type: String
    Default: 10.192.21.0/24
  # e) Create a private subnet in AZ 1
  myPrivateSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref myVPC
      AvailabilityZone: !Select [ 0, !GetAZs '' ] # AZ 1
      CidrBlock: !Ref paramPrivateSubnet1CIDR
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: !Ref paramUniqueName
  # Create a private subnet in AZ 2
  myPrivateSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref myVPC
      AvailabilityZone: !Select [ 1, !GetAZs  '' ] # AZ 2
      CidrBlock: !Ref paramPrivateSubnet2CIDR
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: !Ref paramUniqueName

  # f) Associate the private route table with the private subnet in AZ 1
  myPrivateSubnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref myPrivateRouteTable1
      SubnetId: !Ref myPrivateSubnet1
  #  Associate the private route table with the private subnet in AZ 2
  myPrivateSubnet2RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref myPrivateRouteTable2
      SubnetId: !Ref myPrivateSubnet2

Optionally, we can add an Output section to get the list of newly created private and public subnets



Outputs:
  outputVPC:
    Description: A reference to the created VPC
    Value: !Ref myVPC
  outputPublicSubnets:
    Description: A list of the public subnets
    Value: !Join [ ",", [ !Ref myPublicSubnet1, !Ref myPublicSubnet2 ]]
  outputPrivateSubnets:
    Description: A list of the private subnets
    Value: !Join [ ",", [ !Ref myPrivateSubnet1, !Ref myPrivateSubnet2 ]]

Here's the GitHub link to the whole template.

Below, we see our AWS CloudFormation stack with the list of newly created resources:

If you need to understand how to create a stack in AWS Console, please see Part 1 of this series.

Look! We have a stack. At the end we got four subnets, including two public and two private within a newly created VPC:

Summary

If you made it all the way to the end, congrats, and happy CloudFormation construction!

Our VPC template allows to create two public and two private subnets, in different AZs for redundancy using AWS CloudFormation. It builds a private networking environment in which you can securely run AWS resources, along with related networking resources.

The ability to create, modify, and delete a stack of resources through AWS CloudFormation is a powerful example of the Infrastructure as Code concept.