5 Ways to Prevent Accidentally Deleting Your CloudFormation Resources

CloudFormation is an AWS service that allows you to maintain Infrastructure as Code (IaC). Whether you are using it natively (with JSON or YML) or through a third-party service such as the Serverless Framework, AWS CDK or SAM, it is a great way to make your infrastructure reproducible across various stages. It also makes the deployment process easily automatable through CI/CD pipelines. In other words, it makes managing your infrastructure less prone to human errors.

Although automating things sounds like a good idea, one of the downsides of CloudFormation is that it is hard to understand what is going on under the hood and what exactly is going to happen to your stack during the process, turning every single deployment into a potential 7 minutes of terror story. Imagine that an entire resource gets deleted and all its data with it. Make just one mistake and you will only find out when it's too late. What if it is a production database? It happens more than you think. 😱😱😱

To avoid this kind of disasters, I will show you 5 ways to protect your resources from deletion with CloudFormation.

---

1. Review the Changeset

The first technique is to understand which actions will effectively be executed during the update before they happen. CloudFormation offers a tool that lets you pre-visualize all the modifications that would be applied by a change in your template.

To use it, follow these simple steps:

1. go to your CloudFormation console and select the stack that you want to update
2. click the Stack actions button and then select Create change set for current stack.
3. Choose Replace current template and upload your new template, or enter an S3 path to the file.
4. From there, just follow the guide in order to create the changeset

It might take a few seconds for the changeset to be generated. Once it is done, the console will show you a detailed summary of what actions would be executed if you decided to proceed with the update.

Example:

As you can see, you can easily spot what resources will be Modified, or Removed and if they require replacement. Once you are confident enough that this is what you intend to do, you can hit the Execute button with a certain peace of mind 🧘

This method is useful when you want to visually confirm a change that you are unsure about. However, it is not always convenient. Let's explore other solutions.

2. Retain Specific Resources

With the DeletionPolicy attribute, you can control what CloudFormation should do with a resource in the event of it being removed from the template, or if the stack is deleted altogether. The default value is Delete which is probably not what you want in some cases. By changing the value to Retain, you are telling CloudFormation to keep the resource instead.

Example:

Resources:
  MyTable:
    Type: AWS::DynamoDB::Table
    DeletionPolicy: Retain
    Properties:
      TableName: mytable

One thing to notice here is that this method will not make your deployment fail. CloudFormation will execute all your changes. The difference is that any instruction to delete a resource with a Retain policy will be ignored and the resource will be "detached" from the stack instead. This also means that if you try to add the resource back to the stack, any subsequent deployment might fail because CloudFormation will try to re-create the resource that already exists (e.g: the DynamoDB table already exists with that name). If that happens, you can check this guide for Importing Existing Resources into a CloudFormation Stack.

⚠️ Attention!

This capability doesn't apply to resources whose physical instance is replaced during stack update operations. For example, if you edit a resource's properties such that CloudFormation replaces that resource during a stack update.

This extract from the official documentation is very important. What it means is that if you change a property of a resource that requires replacement (e.g.: changing a DynamoDB table's name), the deletion policy will not apply, and it would still be deleted and re-created. Before you change a property, you should pay attention to the Update requires section of the CloudFormation documentation for that resource's attribute.

With certain types of resources, like EC2 volumes or RDS instances, you can also use Snapshot. In that case, the asset would still be deleted but a backup would be executed first.
You can read more about this strategy by reading the official documentation.

3. Define a Stack Policy

A more advanced way of protecting your resources is through Stack Policies. With Stack Policies, you can constraint what actions are allowed to be executed or not according to specific rules that you define. When you add a policy, all resources are protected by default. You need to explicitly Allow the changes on the resources that you want to update. You can think of it as an IAM policy, but the difference here is that it only applies during stack updates.

Example:

The following policy allows any change on all resources, except for the resource whose id is MyDynamoDBTable. By explicitly denying Update:Delete and Update:Replace, the resource is protected against deletion and replacement. On the other hand, modifications are still allowed (e.g.: Add a Global Secondary Index).

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["Update:*"],
      "Principal": "*",
      "Resource": "*"
    },
    {
      "Effect": "Deny",
      "Action": ["Update:Delete", "Update:Replace"],
      "Principal": "*",
      "Resource": ["LogicalResourceId/MyDynamoDBTable"]
    }
  ]
}

To learn how to write custom stack policies, refer to the documentation

4. Enable Stack Termination Protection

If all you worry about is someone (or a process) tearing down a whole stack by mistake, what you need is Stack termination protection. When enabled, CloudFormation will reject any attempt of deleting the stack.

To enable termination protection:

1. Go to CloudFormation and select the stack that you want to protect.
2. Chose Stack actions followed by Edit termination protection
3. Chose Enabled and hit Save

5. Place Sensitive Resources in Different Stacks

Last but not least, if you are too paranoid about deleting precious resources and all the data they contain, the best thing you can do is isolate them into their own stack. Place each one of them in a dedicated template and touch them only if and when you need to. By doing so, you will not risk destroying them while deploying other stacks that change more often.

---

Which One Should You Use?

Each solution has its own pros and cons. They also behave differently in different situations. To help you better understand the differences, I created a simple cheat sheet.

Will my resource be protected if	It is removed from the stack	It requires replacement	The stack is deleted
Changeset review	Manual (1)	Manual (1)	No
DeletionPolicy	Yes	No	Yes
Stack Policy	Yes (2)	Yes (2)	No
Stack Termination Protection	No	No	Yes
Resource Isolation	No (3)	No (3)	No (3)

(1) You will need to manually review and approve the changes.

(2) Provided you configure the policy properly

(3) On its own, resource isolation will not protect any resource. You'll need to combine it with other solutions

As you can see, there is no one-fits-all solution (none of the rows has all Yeses). You will need to use more than one if you want full protection.

---

Conclusion

I just showed you 5 ways to avoid accidental deletion of CloudFormation resources:

• Review the changeset is good if you want to sporadically review changes manually before applying some important changes.
• The DeletionPolicy attribute will save your data in the event of a resource removal or stack deletion, but it won't help against resource replacement.
• Stack Policies will save you from accidentally removing a resource from the stack and changes that force a replacement. On the other hand, it won't be of any help if the stack is deleted altogether.
• Stack termination protection will only prevent accidental deletion of the stack.
• Placing sensitive resources in isolation will help against some human mistakes, but on its own, it will not protect your data.

Use the one that best fits your needs and your particular use-cases. If you need complete protection, you can combine them together and benefit from several safety nets at the same time.

Hopefully, these measures will help you and your team sleep better at night 😴.

---

If you would like to read more content like this, follow me on Twitter and subscribe to my brand new newsletter on Hashnode.