Working with AWS CloudFormation - Part 2

Let us continue from where we left. In our previous article, we went through the basics of CloudFormation, understood the structure of CloudFormation, modules and had a Todo backend example.

In our next part, we want to go through the following:-

• Working with stack and stack sets

• Advance with modules

• Using CloudFormation registry

• Complete the section with another example.

In this article, we are going to see how you can use nested stacks and create the todo backend. The only difference is we are going to use Postrge Database. With Postgre Database we can use Hasura and Apollo Client to connect and use graphql instead of REST API. Nested stacks allow you to build templates that can be reused to build infrastructure. Modules and nested stacks decouple our main stack and allow reusability.

Designing a CloudFormation template is an art and needs you to understand the best practices to follow and how to connect each service. You also need to keep track of the quotas of the services you are provisioning because with CloudFormation you are only charged for the services you deploy on AWS.

Working with stack and stack sets

Assuming you provisioned the Todo backend we went through in the first article we will use this created stack to go through this section. AWS CloudFormation provides two methods for updating stacks:

• Direct update - This will update the existing stack and make the changes immediately.

• Executing change sets - With this, you can preview the changes AWS CloudFormation will make to your stack and decide. This is the ideal approach.

When updating your resources you should keep in mind that the following can take place with your resources replaced, interrupted, or no interruption.

This command will update our template directly. This is not ideal when you are updating an application that is already in production.

aws cloudformation update-stack --stack-name todobackend --template-body file://backend.yml --parameters file://backend-parameters.json --capabilities CAPABILITY_NAMED_IAM

Change sets

Change sets don’t indicate whether CloudFormation will successfully update a stack. For example, a change set doesn’t check if you will surpass an account limit, if you’re updating a resource that doesn’t support updates, or if you have insufficient permissions to modify a resource, all of which can cause a stack update to fail. If an update fails, CloudFormation attempts to roll back your resources to their original state.

Assume we have made changes in our backend.yml we can create a changeset as follows:

aws cloudformation create-change-set --stack-name todobackend  
    --change-set-name todobackendchangeset --template-body file://backend.yml --parameters file://backend-parameters.json --capabilities CAPABILITY_NAMED_IAM

In case we have only updated the parameters then we can create a new changeset with the previous template as follows:

aws cloudformation create-change-set --stack-name todobackend --change-set-name todobackendchangeset --use-previous-template --parameters file://backend-parameters.json

Once we have created all our change sets we can list them as well as describe them.

// list change sets aws cloudformation list-change-sets --stack-name todobackend

// describe change sets
aws cloudformation describe-change-set --stack-name todobackend  --change-set-name todobackendchangeset

OR use change set ARN
aws cloudformation describe-change-set --change-set-name arn:aws:cloudformation:us-east-1:440343172651:changeSet/todobackendchangeset/f48c2397-4da9-4751-8d3f-3a047161ebb1

This is a great way because it will assist when working as a team. Once a change sets has been approved you can execute it as follows:

aws cloudformation execute-change-set --change-set-name arn:aws:cloudformation:us-east-1:440343172651:changeSet/todobackendchangeset/f48c2397-4da9-4751-8d3f-3a047161ebb1

Note that once you execute your change sets then AWS automatically deletes the change sets.

To manually delete a change sets use the following command:

aws cloudformation delete-change-set --change-set-name arn:aws:cloudformation:us-east-1:440343172651:changeSet/todobackendchangeset/f48c2397-4da9-4751-8d3f-3a047161ebb1

You can cancel a stack update when the status is UPDATE_IN_PROGRESS using the following command:

aws cloudformation cancel-update-stack --stack-name todobackend

In case you want to prevent any update from your stack you can update the stack policy as follows

aws cloudformation set-stack-policy --stack-name todobackend --stack-policy-body file://policy.json

// this will deny any update to the stack policy.json
    {
      "Statement" : [
        {
          "Effect" : "Deny",
          "Action" : "Update:*",
          "Principal": "*",
          "Resource" : "*"
        }  
      ]
    }

Sometimes we edit some resources outside the CloudFormation which might bring a problem when rolling back. For example, deleting a bucket through the console but forgetting we provisioned the bucket through AWS CloudFormation.

To detect changes that have happened in our stack we are going to use drift detection.

Drift detection enables you to detect whether a stack's actual configuration differs, or has drifted, from its expected configuration

To detect drift on an entire stack using the AWS CLI, use the following aws cloudformation commands:

• detect-stack-drift to initiate a drift detection operation on a stack.

• describe-stack-drift-detection-status to monitor the status of the stack drift detection operation.

• describe-stack-resource-drifts to review the details of the stack drift detection operation.
  

aws cloudformation describe-stack-resource-drifts --stack-name todobackend --stack-resource-drift-status-filters MODIFIED DELETED

Importing existing resources to a stack

You can add already existing resources into a stack and manage them.

Let us first run a summary of our backend.yml

aws cloudformation get-template-summary --template-body file://backend.yml

I will create a new table called testthrough the console and later we add it to our stack.

Let us create a file called changes.txt and add the following:

[
 {
  "ResourceType": "AWS::DynamoDB::Table",
   "LogicalResourceId": "TestTable",
   "ResourceIdentifier": {"TableName": "test"}
  }
]

Then we can update our backend.yml file. Note that we have a logical resource id that we have indicated in the changes.txt but we currently do not have it in the backend.yml

Add the following in your backend.yml file and with that, you will have referenced the logical id.

Now create a changeset and execute the changeset once you have confirmed all the changes that will take place.

// create a change set as follows
aws cloudformation create-change-set --stack-name todobackend --change-set-name newchangeset --change-set-type IMPORT --resources-to-import file://changes.txt  --template-body file://backend.yml --parameters file://backend-parameters.json --capabilities CAPABILITY_NAMED_IAM

Note that we have indicated change-set-type IMPORT . Once ready run the following command:

aws cloudformation execute-change-set --change-set-name ImportChangeSet --stack-name todobackend

Working with nested stacks

Nested stacks are stacks created as part of other stacks. You create a nested stack within another stack by using the AWS::CloudFormation::Stack resource.

Read more about the properties that are required when creating nested stacks https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-stack.html

    Type: AWS::CloudFormation::Stack
    Properties: 
      NotificationARNs: 
        - String
      Parameters: 
        Key : Value
      Tags: 
        - Tag
      TemplateURL: String
      TimeoutInMinutes: Integer

Stack sets

A stack set lets you create stacks in AWS accounts across regions by using a single CloudFormation template. All the resources included in each stack are defined by the stack set's CloudFormation template. As you create the stack set, you specify the template to use, in addition to any parameters and capabilities that template requires.

With stack sets, you can deploy multiple stacks to different regions using a single CloudFormation template. You can set up either self-managed or service-managed permission.

This will be advantageous if you are deploying in multiple accounts and regions. You can manage all the resources using a single template.

Here is a command for use in creating a stackset.

aws cloudformation create-stack-set --stack-set-name todoapp-stackset --template-body file://backend.yml

Just keep in mind that when you are dealing with multiple regions use stacksets to manage the stacks in all regions.

CloudFormation Registry

The CloudFormation registry lets you manage the extensions, both public and private, that are available for use in your CloudFormation account.

Public extensions are accessible to everybody and there are two kinds:

• Amazon public extensions

• Third-party public extensions - These ones have to be activated into your account before usage.

There are also two kinds of private extensions:

• Activated private extensions

• Registered private extensions

You can view the CloudFormation registry https://console.aws.amazon.com/cloudformation/home

Registration. Usage and deregistration of modules

Let us go through how to create, register, use and deregister modules. You can use the following command cfn init to scaffold a quick sample. So we will create a simple s3 module.

cfn init
> MODULE // select module
> Name::S3::Bucket::MODULE // module name

Once you have let us remove versioning and encryption so we have a basic s3 module. This should be done in the following folder fragments/sample.json

To deploy the module to the CloudFormation registry run the following command:

cfn submit

This is how the module that we have submitted should be. We can reuse this module to create an S3 bucket.

How to use the module in your template is simple.

...
  Type: Name::S3::Bucket::MODULE

By referencing the module CloudFormation will use it to create our bucket. Run the following command:

// create stack
aws cloudformation create-stack --stack-name s3-module --template-body file://module-stack.yml --parameters file://parameters.json --capabilities CAPABILITY_AUTO_EXPAND

// describe mnodule
aws cloudformation describe-type --type MODULE --type-name Name::S3::Bucket::MODULE

To deregister a module use the following command:

aws cloudformation deregister-type --type MODULE --type-name Name::S3::Bucket::MODULE

Nested stacks vs Modules

In my personal opinion modules and stacks have some similarities. They allow you to reuse templates. With nested stack, your templates have to be uploaded to s3 buckets on the other hand with modules you have to upload to the cloud formation registry.

EXAMPLE SECTION

As explained above I noted down that we are trying to recreate todo backend. To begin here are the links to our nested module templates.

https://github.com/kevinodongo/cloudformation/tree/main/nested-example

Before using the templates we need to upload the following to the s3 bucket.

• cognito-template.yml

• rds-database-template.yml

• s3-storage-template.yml

You can use the following command to upload your files to s3 bucket.

// example of uploading cognito template
aws s3api put-object --bucket nested-templates --key auth-cognito --region us-east-1 --body ./cognito-template.yml

The template that we are going to use to deploy the nested stack is nested-template.yml.

If you compare nested-template.yml and backend.yml you will note that with nested we have reduced the code in our template and everything looks neat and simple.

We can reuse the templates and recreate the same backend in any region of our choice. We only need to provide the parameters.

This truly reduces our working load and ensures we have the correct and similar deployment.

This is truly awesome and I believe you are getting to the logic of how to work with nested templates.

To deploy the nested template just run the following command:

aws cloudformation create-stack --stack-name nestedbacked --template-body file://nested-template.yml --capabilities CAPABILITY_NAMED_IAM

Once deployed when you log in to the Cloudformation dashboard you should be able to see the following:

This indicates that all our resources have been provisioned. Now we can use Hasura and connect to our database and create our users and tasks tables.

// Get hasura docker compose file
curl https://raw.githubusercontent.com/hasura/graphql-engine/stable/install-manifests/docker-compose/docker-compose.yaml -o docker-compose.yml

// run the following command
docker-compose up -d

Once your docker is running connect to the Hasura dashboard through http://localhost:8080/console and connect to the Postgre database. You can get the outputs of the nested rds database you will find the postgre link which you can use to connect.

In this example, once you have Hasura then you can use ApolloClient and get started with graphql and complete your todo app.

Hey, that is all we needed to go through in this article. In our final article, we are going to go through the following examples:

• Networking - Deploy EC2, VPC, etc

• ECS and ECR

• Continuos delivery

Thank you for going through this article.