How to become an infrastructure-as-code ninja, using AWS CDK - part 1

Do you want your Amazon Web Services (AWS) cloud infrastructure to be resilient and consistent?

Do you want quickly repeatable and automated AWS infrastructure set-up?

Tired of messy and time-consuming infrastructure management through AWS Console?

Do you use AWS CloudFormation, and find it verbose, clunky to manage?

If you answer yes to one or more of these questions, and want to find a better approach, then this article series may be for you!

We are going to use one of the newer tools for automating infrastructure management from AWS, the AWS Cloud Development Kit, also known as AWS CDK. It is a way to describe the desired infrastructure state through the use of programming languages.

You can read the articles and follow along writing your own infrastructure code. Step by step you will build up a consistent and repeatable infrastructure solution.

Feel free to also just read the areas that you find interesting. I would recommend skimming through each article before diving in deep to get an idea of the scope of the article.

Also, please read the goals and requirements sections below, so that you see what to expect. I hope you will get good value from these articles!

Goals

The aim of this series of articles is for you up to speed with using the Amazon Web Services Cloud Development Kit (AWS CDK). The AWS CDK is a tool to define AWS cloud infrastructure using programming languages.

Thus, you will define, provide, and update multiple types of infrastructure, including virtual machines, containers and service functions.

By the end of this series of articles, you will have

• Set up virtual machines (EC2) in an existing network infrastructure
• Provide secure access to these virtual machines without internet exposure
• Set up a container-based services, with load balancer
• Include basic monitoring and logging
• Defined unit tests for your infrastructure definitions

This series of articles does not cover migration from the existing usage of CloudFormation to AWS CDK. There will be separate articles to cover that area.

Update 2022-03-07: Originally, the list of goals above included setting up a simple CI/CD-type pipeline as well, which would have used AWS CDK's own pipelines module. I have removed that, because I think that the choice of pipeline solution is a very different topic from getting started with AWS CDK.

Requirements

For you to get the most of this series of articles, there are a few requirements

• You should have some level of programming experience, but do not need to be an expert developer. The article series will cover multiple programming languages - pick the one that fits you best! However, the initial version of the series will cover Typescript, with other languages to come later.

• You need access to an AWS account with administrative privileges. In this AWS account, you should be able to use either temporary (preferred) or permanent AWS programmatic credentials (e.g. Access Key Id, Secret access key, session token (if temporary credentials)).

• You should be able to install software on the computer you use.

• An installation of your programming language of choice, that is also supported by AWS CDK.

• You also need some basic knowledge of a few common AWS services, like IAM, EC2.

We do not require you to know a lot about AWS CloudFormation, but some basic understanding will be beneficial.

Introduction

What is AWS CDK?

The AWS Cloud Development Kit (AWS CDK) is a framework, a set of libraries, and a command-line tool to develop, provide, and manage AWS Cloud infrastructure solutions. The CDK team build it on top of the AWS service CloudFormation.

AWS CloudFormation is a mature and capable tool to define, provide, and manage AWS cloud infrastructure. Yet, it has been cumbersome to use, in particular for larger and more complex infrastructure solutions. The tools to support using CloudFormation have also been a bit lacking.

What AWS CloudFormation has done right is to provide a declarative way to describe the desired state of your infrastructure. You tell CloudFormation what you want, and it tries to make that happen. You do not need to specify the details of what resources to create, update, or delete - CloudFormation will figure that out.

A good thing here is that you can just describe exactly what you want. It is reasonably clear what settings are in place for each resource. A bad thing here is that you must describe exactly what you want. There can be too much detail and it becomes hard to grasp, anyway. Here is a small example of a CloudFormation definition (YAML format) of an instance of an EC2 virtual machine, configured to use the latest Amazon Linux, and run on a machine of type and size t3.micro. Do not worry if you do not understand it, it is simply for comparison purposes.

Resources:
  myEc2:
    Type: AWS::EC2::Instance
    Properties:
      AvailabilityZone: eu-west-1a
      ImageId:
        Ref: SSMImageParameter 
      InstanceType: t3.micro
      SecurityGroupIds:
      - Fn::GetAtt:
        - mySecurityGroup
        - GroupId
      SubnetId: subnet-12345678
      Tags:
      - Key: Name
        Value: my-stack/my-ec2

  mySecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: my-stack/my-ec2/InstanceSecurityGroup
      SecurityGroupEgress:
      - CidrIp: 0.0.0.0/0
        Description: Allow all outbound traffic by default
        IpProtocol: "-1"
      Tags:
      - Key: Name
        Value: my-stack/my-ec2
      VpcId: vpc-12345678

Parameters:
  SSMImageParameter:
    Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
    Default: "/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2"

The AWS CDK tries to provide the expressive power of programming languages to make it possible to describe the cloud infrastructure at a level that is easier to grasp, even if it becomes more complex. Under the hood, it will still generate the CloudFormation descriptions, but make it possible to hide much of the complexity. To illustrate this, below is a code snippet written in Typescript, that defines the same thing as the CloudFormation definition above. In fact, it is essentially the CloudFormation generated by this code, except the CloudFormation is a bit stripped down and with a bit nicer names.

const vpc = Vpc.fromLookup(stack, 'my-vpc', {isDefault: true });

new Instance(stack, 'my-ec2', {
  vpc,
  instanceType: InstanceType.of(InstanceClass.T3, InstanceSize.MICRO),
  machineImage: MachineImage.latestAmazonLinux(),
});

There are simply less details that you have to bother with - you can if you want, but the general idea is that the AWS CDK should provide sane defaults. In this case, you can focus on your intention (latest Amazon Linux, using default VPC), and do not need to sort out all low level details.

Just as with any program you write, it is possible to write code for AWS CDK that is difficult to understand and maintain, as well as code that code for infrastructure that is straightforward to maintain and understand. You have that power. You choose what you will do with it. After this article series, you should be well equipped to handle that power.

There are a few key concepts to keep in mind when using the AWS CDK:

• An App. This the top level container, which contains everything that you can provision in the solution or project you are working with.
• A Stack. This is a collection of AWS resources that you provision and manage as a unit. It could be your whole solution, or it could be part of a solution.
• A Construct. This is a logical grouping of one or more AWS resources and may also encapsulate certain logic, practices and conventions.
• A Resource. A unit of infrastructure in AWS, which is available via CloudFormation and AWS APIs primarily.
• An Environment. A specific combination of an AWS account and an AWS region.
• Synthesize. The process of executing CDK-based code to generate the corresponding CloudFormation representation.
• Deploy. The process of taking the underlying CloudFormation representation that has been generated in the synthesize step to actually provision (create, update or delete) AWS resources.
• Bootstrap. The process of preparing an environment for deployment with AWS CDK. This is typically a one-time process for each environment.

A CDK App will consist of a number of stacks, which in turn consists of one or more constructs. Each construct may encapsulate one or more resources.

It is also helpful to know a few CloudFormation concepts:

• A template. This a textual description of a collection of resources to manage as a unit, in either JSON or YAML format.
• A Stack. This is representation of the provisioned set of resources described by a CloudFormation template, and managed by CloudFormation itself.
• A Change Set. This is a representation of potential change of resources that CloudFormation will perform, to reach a desired target state from an existing state.

A CloudFormation template has a 1-to-1 mapping to a CloudFormation stack. These have a 1-to-1 mapping to a CDK Stack.

The process to provision from a set of CDK-based code to actual resources consists of

1. Compile the code (if using a compiled programming language)
2. Run the compiled result (Synthesize process), which builds the resource structures, and generates CloudFormation template(s)
3. Create a CloudFormation change set (Deploy process), with potential approval step
4. Provision changes defined in the change set (Deploy process)

Prior to running this provisioning cycle, you need to bootstrap each target environment. This essentially sets up a few resources that AWS CDK needs for deployment, and is a one-time process per environment.

The AWS CDK command-line tool is the primary tool to perform these process steps:

• cdk synth
• cdk deploy
• cdk bootstrap

We will expand on the usage of these commands in later sections.

Installation

AWS CDK Installation

Before building something with the AWS CDK, and pick a language to use, we need to install the AWS CDK command-line tool. The tool itself is written in Typescript, and uses the Node.js runtime.
If you already have Node.js installed on your computer, then that is great! You can proceed to the actual CDK installation below. If you do not have Node.js you will need to install its runtime.
You can download the Node.js runtime from the Node.js website. The long-term support (LTS) release is the recommended version to use. However, newer versions and some older versions are ok to use as well.
You can also install Node.js via different package managers, depending on the computer environment you are working with. See the the Node.js package manager page for suggestions. You can also download tools there etc manage multiple separate versions of Node.js, should you ned that.

Once you have installed Node.js, it is time to install the AWS CDK command-line tool. If you use any of the supported programming languages that is not Typescript, you want to do a global install of the command-line tool (not a project-local install), since you would likely not use Node.js in the project itself. If you intend to use Typescript as a language, both a global install and a project-local install would be valid options.

To install the AWS CDK command-line tool globally, run the following command in a shell:

npm install -g aws-cdk@next

The command npm is the Node.js package manager, which installs the package aws-cdk. In this case, we have also specified @next after the package name. The reason for this is that we want to use Aws CDK version 2, which at the time of this writing is not has not become the active release yet. It is however the major version that I would recommend to use if you are new to AWS CDK and are just starting out.

When you have installed the aws-cdk package, you will have the command cdk available through the command-line. You can check that it is working by running cdk version:

❯ cdk version
2.0.0-rc.23 (build 1e54fb9)

If you have this working, you are now all set to take the next step and set up a project in your programming language of choice!

This ends the AWS CDK command-line tool installation, and also the part of the series that is the same for all languages. When we start creating actual infrastructure-as-code projects, we will do that with specific target languages. The actual infrastructure will be the same regardless of programming language - you just pick the one you prefer!

Let us get started with our infrastructure! Jump to part 2 to get started.

A note about AWS credentials

In all the following articles in this series, you will interact with an AWS environment. It is up to you to supply this environment.

All examples of running commands with the AWS CDK command-line tool assumes that your default/current AWS credentials are the ones you will be using. There are a few ways to set up your credentials that way:

1. If you have an AWS credentials profile that is the default profile. You are good to go, nothing else to do.
2. You have explicit and permanent AWS access key id and secret access key values. Set the AWS environment variables AWS_ACCESSS_KEY_ID, and AWS_SECRET_ACCESS_KEY with these values, in your command-line session.
3. You have explicit and temporary AWS access key id and secret access key values. Set the AWS environment variables AWS_ACCESSS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN with these values, in your command-line session.
4. You have an AWS credential profile, which is not the default profile. Set the environment variable AWS_PROFILE to the name of that profile.

Also remember to set the AWS_DEFAULT_REGION environment variable with your AWS region of choice, unless that is set explicitly in the AWS credential profile you are using.

Note that the AWS CDK command-line tool also supports the --profile option to specify the AWS credential profile, and you are free to use that one instead.