DEV Community

Cover image for Migrating CloudFormation to TF
6 2 3 2 2

Migrating CloudFormation to TF

One day you might find yourself in the unfortunate position of wanting to migrate away from CloudFormation (CFN). While some may say that CFN is bad and should never be used. I can confirm that it is still better than:

  • CloudFormation CDK
  • AWS SAM
  • Serverless - Not "serverless", but the company that is abusing this name.
  • SST
  • And many others

The truth is: CloudFormation isn't bad, however like most things, it is bad when you find out your current solution doesn't support the thing that you want it to support.

So back to the problem...You want to migrate from CloudFormation to OpenTofu (since no one uses Terraform anymore after their legal scandal), and part of that problem involves a migration.

The Migration

Migrations are technically easy. Monolith to microservices, event buses to REST, MSSQL to NoSQL DynamoDB. The hard part is always the non-technical part. The part where you figure out what you want, now that's the problem. Unless of course you have a monolith, because you should just give up now. No one successfully converts from a Monolith to microservices. They write some code, complain a lot, then apply for a new job at a new company telling their would-be manager "Look how I helped this company migrate to microservices. I'm Great!"

But this isn't a story about how monoliths are bad, it is about how to migrate your Infrastructure as Code (IaC) solution.

Realistically, you have to painstakingly generate the new IaC HCL files for OpenTofu. You have existing CloudFormation as well as the real live version of your infrastructure currently supporting an massive business. And if you are like us at Authress, you might also have a 99.999% uptime SLA you need to account for.

If you have 100+ CFN stacks, you probably don't want to import these resources in OpenTofu by hand. Instead, you'll want some sort of tool to do this, and there are a bunch:

  • Former2 - Export from AWS to HCL.
  • Firefly.ai - AI in the company name, yuck
  • CF2TF - Open source converter
  • Doing it by hand to verify you have everything you need.

And there are still more... You could even try one of the LLMs out there.

Generating the configuration

Terraform and OpenTofu actually support configuration generation out of the gate as well, so we will use their strategy here, and if you want to use one of the less great ones from above, you do you!

1. Add the import block:

import {
  to = aws_instance.example
  id = "foo"
}
Enter fullscreen mode Exit fullscreen mode

2. Run the configuration generation command:

tf plan -generate-config-out=generated.tf
Enter fullscreen mode Exit fullscreen mode

Output:

resource "aws_instance" "example" {
  arn = "arn:aws:ec2:eu-west-1:1234567890:instance/i-00deadc0de"
  ami = "ami-000a4d9c6067d5d0d"
  instance_type = "t3.micro"
  ...   
}
Enter fullscreen mode Exit fullscreen mode

3. Commit then new configuration:
Add the configuration to your files, and git commit to your IaC repository.

Running the migration

Once we have all of those generated we just need to run tf plan, tf apply, and then delete the import statements.

And you are done!

Cleanup

The one thing that no one tells you at this point is that you aren't done. Importing the resources and having the committed IaC HCL does not mean you are done. If you are like me, then you care that you still have 100s of CFN stacks deployed in your AWS accounts. Maybe these stacks all have CFN Drift and don't even represent the current state of the world anymore.

However, even if they do represent the current state, you probably don't want someone going into your account and accidentally updating or deleting those. Or your desire to have a pristine account compels you to delete these stacks. You probably wouldn't be someone working on this problem in the first place if you didn't care that these old stacks are still here.

The problem is that there is no way to delete a stack without also deleting the resources in that stack. And of course, you want to keep the resources in those stacks, so that's a conundrum. Thankfully, I've figured out a hack to get around this.

Warren disappears due to the creation of a hack

The involves utilizing three features:

  • the delete_failed status
  • FORCE_DELETION action flag
  • CloudFormation execution Role ARN

The delete_failed status occurs whenever CFN tries to delete a resource that it believes is no longer necessary, but the resource is either in use OR CFN doesn't have access to delete the resource. Take note of this second one.

Second, when a stack is in the delete_failed status, you are allowed to force delete the stack and retain explicit resources that you might still be using.

So all we need to do is get the stack into the delete_failed state, and then ask CFN to retain all the resources.

CloudFormation allows, for "security reasons", you to specify a role ARN to execute CFN with. When you do that the CFN stack changes will only be executed with that role. So we'll define a new role that does not have access to anything. We'll abuse the Role ARN property to force CFN to fail to delete any resources and thus fail to delete stack.

Cleanup Execution

Create the Role:

CfnDeleteStackRole:
     Type: AWS::IAM::Role
     Properties:
       RoleName: cfn-delete-stack-role
       AssumeRolePolicyDocument:
         Version: '2012-10-17'
         Statement:
           - Effect: Allow
             Principal:
               Service: cloudformation.amazonaws.com
             Action: sts:AssumeRole
Enter fullscreen mode Exit fullscreen mode

With that role, we'll call the Delete Stack:

aws cloudformation delete-stack \
    --stack-name my-stack
    --role-arn arn:aws:iam::account:role/cfn-delete-stack-role
Enter fullscreen mode Exit fullscreen mode

This execution call will fail, but we knew that was going to happen. Now, it will put the stack in the status delete_failed.

Finally, we can execute the delete again, utilizing the force deletion parameters:

aws cloudformation delete-stack \
    --stack-name my-stack
    --role-arn arn:aws:iam::account:role/cfn-delete-stack-role
    --deletion_mode FORCE_DELETE_STACK
Enter fullscreen mode Exit fullscreen mode

Depending on the resources you have in your stack you or if you want extra security to prevent deleting your precious resources, you can add the flag --retain-resources to the CLI command:

aws cloudformation delete-stack \
    --stack-name my-stack
    --role-arn arn:aws:iam::account:role/cfn-delete-stack-role
    --deletion_mode FORCE_DELETE_STACK
    --retain-resources $LOGICAL_RESOURCES_LIST
Enter fullscreen mode Exit fullscreen mode

With $LOGICAL_RESOURCES_LIST value set as the string list of CFN resources.

const cfnTemplateFile = await fs.readFile('./cfn-template.json');
const cfnTemplate = JSON.parse(cfnTemplateFile);
const resourceKeys = Object.keys(cfnTemplate.Resources).join(',')
return resourceKeys;

// Use resourceKeys as $LOGICAL_RESOURCES_LIST
Enter fullscreen mode Exit fullscreen mode

Repeat this for every CFN stack in every region in every AWS account in your org, and everything will be cleaned up, just the way you wanted it to.


Curious about this and worth discussing more, join my community and chat with me:

Join the community

👋 While you are here

Reinvent your career. Join DEV.

It takes one minute and is worth it for your career.

Get started

Top comments (2)

Collapse
 
drjoanneskiles profile image
Joanne Skiles •

This is a relevant topic, especially for teams looking to standardize infrastructure-as-code workflows. I appreciate how you outlined the process and challenges of migrating from CloudFormation to Terraform.

Thanks for sharing these insights—great read!

Collapse
 
nedtechie profile image
Nedim Hadzimahmutovic •

Recently I was wondering if I am missing out by sticking to Terraform. I guess not. :)

Best Practices for Running  Container WordPress on AWS (ECS, EFS, RDS, ELB) using CDK cover image

Best Practices for Running Container WordPress on AWS (ECS, EFS, RDS, ELB) using CDK

This post discusses the process of migrating a growing WordPress eShop business to AWS using AWS CDK for an easily scalable, high availability architecture. The detailed structure encompasses several pillars: Compute, Storage, Database, Cache, CDN, DNS, Security, and Backup.

Read full post

👋 Kindness is contagious

Explore a sea of insights with this enlightening post, highly esteemed within the nurturing DEV Community. Coders of all stripes are invited to participate and contribute to our shared knowledge.

Expressing gratitude with a simple "thank you" can make a big impact. Leave your thanks in the comments!

On DEV, exchanging ideas smooths our way and strengthens our community bonds. Found this useful? A quick note of thanks to the author can mean a lot.

Okay