Importing existing AWS resources using AWS CDK

Cover photo from Gavin Allanwood on Unsplash.

I like infrastructure and I advocate AWS CDK for one reason: it allows us to use Typescript (or other languages) which is less of a blocker for new contributors in frontend. It is verbose but it does a lot of the weight lifting for you and is extremely flexible as well.

I've always been curious about the following situation: most companies start setting up their infrastructure through AWS Console. At some point Infrastructure as Code is introduced. How do we close the gap between them? Shall we code everything from scratch?

For a while (end of 2019) we can import existing AWS resources into a CloudFormation Stack. The blog post from AWS does a good job explaining how it works. There is another blog post that explains how to do it with AWS CDK.

My goal here is to make it visual and with code examples. If you follow the steps from both blog posts you are good to go but I found myself confused with some names in between and decided to summarize it for myself (as I'm pretty sure I won't remember how to do it next time) and maybe help you as well.

I'm assuming that you're familiar with AWS CDK and also with CloudFormation. If not, don't hesitate to ask questions and I can try to answer them or guide you to other resources.

AWS CDK output

AWS CDK "simply" translates some code we write into a CloudFormation template in this case, a .json file by the end. It is this file that is used by AWS CloudFormation to setup our infrastructure.

The output from AWS CDK can be found inside the cdk.out/ folder. The CloudFormation template from your stack is in the file MyStackName.template.json.

AWS CDK creates this file whenever we run synth or deploy (which runs synth beforehand). The main difference is that deploy uploads this file to AWS CloudFormation, while synth "only" creates it.

Importing existing AWS resources

I will manually create a bucket in my account and run a small example (the steps are similar for more resources but it takes more time). I called it viniciuskneves-aws-cdk-real-bucket and you can see it in the image below:

In order to import an existing resource to our stack, we first need to create the stack. We can create a stack with a dummy resource as follows:

import { RemovalPolicy, Stack, StackProps } from "aws-cdk-lib";
import { Construct } from "constructs";
import { Bucket } from "aws-cdk-lib/aws-s3";

export class AwsCdkTypescriptStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const DummyBucket = new Bucket(this, "DummyBucket", {
      removalPolicy: RemovalPolicy.DESTROY, // Easier to remove later
    });
  }
}

The bucket removal policy was set to delete as this bucket will be deleted later and it is not our real bucket, which will be imported later on.

Now we can run deploy and, after deploying our Stack, we should be able to see it in AWS CloudFormation console:

Following the instructions from AWS, we need to create a template with the resource we would like to import in this case, a S3 bucket. We can rely on AWS CDK to do the job for us. To create a S3 bucket, we need the following piece of code:

import { RemovalPolicy, Stack, StackProps } from "aws-cdk-lib";
import { Construct } from "constructs";
import { Bucket } from "aws-cdk-lib/aws-s3";

export class AwsCdkTypescriptStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const DummyBucket = new Bucket(this, "DummyBucket", {
      removalPolicy: RemovalPolicy.DESTROY,
    });
    const RealBucket = new Bucket(this, "RealBucket", {
      removalPolicy: RemovalPolicy.RETAIN, // Recommended in order to avoid data loss
    });
  }
}

Now we can run synth to obtain our CloudFormation template. This template contains a bucket but it is not yet deployed, we will use it to manually import the resource through the AWS console.

We can go to our stack in AWS CloudFormation console, click Stack Actions and Import resources into stack:

There we select the file created by synth in the step above (cdk.out/MyStackName.template.json):

This step tells CloudFormation which new template to use.
The next step is how to link the CloudFormation template with an existing resource. AWS S3 uses the bucket name for linking, so we need to input the bucket name (from the bucket created earlier) here:

In the last step you will see the applied changes and our imported bucket will be there:

After a while (in our example it happens almost immediately as we're talking about a single resource) you will find the resource imported in your stack:

Now we can try to run deploy again and we will get a message that no changes have been found in the stack, which means success to us:

From now on we can make changes in the stack and it will be applied as expected, as if the resource had been created by us, in AWS CDK, since the beginning:

const RealBucket = new Bucket(this, "RealBucket", {
  websiteIndexDocument: "index.html",
  removalPolicy: RemovalPolicy.RETAIN,
});

After deploying, we've:

We can finally remove the DummyBucket and as it had a destroy policy, the bucket will be removed as expected (no need to manually delete it):

import { RemovalPolicy, Stack, StackProps } from "aws-cdk-lib";
import { Construct } from "constructs";
import { Bucket } from "aws-cdk-lib/aws-s3";

export class AwsCdkTypescriptStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const RealBucket = new Bucket(this, "RealBucket", {
      websiteIndexDocument: "index.html",
      removalPolicy: RemovalPolicy.RETAIN,
    });
  }
}

After deploying, we've:

Importing resources that don't follow defaults

Our example above followed a clean approach: the real bucket had no extra configuration, it was the default, the same setup from a bucket created from AWS CDK without any configuration.

What would happen if the bucket had, for example, static webhosting setup?

It will allow us to import the resource as we did in the example above. One curiosity: the defaults, from AWS CDK, won't override already set configuration.

If we deploy the stack, it won't disable static webhosting, for example. The only way to disable it (overwrite it), is to enable it through CDK, deploy, and then remove it from CDK and deploy again.

Even if you run cdk diff you won't get this as a diff as it will compare the output of synth with the template that has been uploaded to CloudFormation. Stack drift won't highlight it as well which is weird.

It is a good approach as it won't magically overwrite everything that has been working. It could be tricky though if you don't know that a resource had been imported and some defaults don't apply to it.

What would happen if the template had some setup, like static webhosting, when importing an existing resource?

It is the opposite of the situation above. The resource we create in AWS CDK to be imported has some setup and is not the default resource.

There are two cases:

1. Imported resource and template have the same setup (static webhosting enabled): no problem. The resource is imported and changes made in CDK will apply once you deploy.
2. Imported resource and template have a different setup (static webhosting enabled in template): the difference won't be applied but drift will warn you that there is a difference.

---

I hope it helps you, as it helped me, to visualize the process described in both blog posts mentioned in the introduction. I also went a bit further to explore drifts in the CloudFormation stack but it is a bit weird (check the examples above).

If you've further insights/questions, feel free to drop them in the comments!

Comments

[Vinicius Kiatkoski Neves]

Ehhh, that happens in general. Try to focus on what you like first and pair as much as possible with people to help you. Reading docs, going through tutorials and so on might be overwhelming sometimes, so someone to help is always good!
You don't need to master all the topics as well =]

[Vinicius Kiatkoski Neves]

Hey Sebastian,

I would say in general "it doesn't matter" because you should learn most of the concepts. I know that in reality it is a bit trickier as there are specific things for each provider.

I've always worked with AWS so I would recommend that but if you know people that work with GCP and could help you, go for it. My point is: start learning, no matter the tool, focus on the concepts and later on you will go deeper in the one that your company uses.

I hope it helps!