DEV Community: Erik Lundevall Zara

Quick blog set-up with Quarto and Fly.io

Erik Lundevall Zara — Mon, 24 Mar 2025 09:30:51 +0000

This is a quick tip to get started to set up your own blog website, similar to my blog website https://eriklz.online.
I have set up a couple of websites with blogs using this approach, and I find it quite nice - and I hope you will find it useful as well!

The set-up here is pretty much the same as I use with my blog, and it is fairly straightforward to do.

There are a few key components needed:

Docker Desktop, or something that is a compatible alternative
Quarto, a technical publication system to create websites, books, documents etc from Markdown
Fly.io, a developer-centric and friendly cloud platform
Optional: GitHub, for version control and automating publishing of updates

Installation

Follow the installation instructions for your platform for Docker Desktop and Quarto, if you do not have them installed already.

Also, create a personal account on Fly.io, and install their flyctl command-line tool.

Set-up of website/blog content

First, if you are going to use GitHub for version control, create a repository there, let us assume it is called my-awesome-blog. It is fine to have a private repository.
You can then clone it to your local computer from the command-line, e.g.

git clone git@github.com/myuser/my-awesome-blog.git

Change to this repository and set up a blog project using Quarto:

cd my-awesome-blog
quarto create project blog .

The quarto command will ask for the title of the blog, and then generate some initial files for a blog set-up.
You can view what the current setup looks like in the browser by running quarto preview.
This command will generate the website content, start a local web server and open up the home page in a browser.

For updating the actual content, you will need to look a bit further in the quarto documentation.
For general text and image content, it should be pretty ok to learn - quarto uses Markdown files, but has also a lot of extensions on top of the regular Markdown.

The _quarto.yml file is the file that will control the overall layout of the webpage, and individual .qmd files are essentially the Quarto Markdown files that provide content for each page.

To generate the web site content for "real" you would rather use the quarto render command.
This is what you use to generate the content to be used with further steps to deploy the blog, so you can run that and verify that it works.

The generated content will be in the directory _site.

Set-up of webserver

Next step is to set up a web server to serve the content to the real world.
For this, we use fly.io.

In the root of your project/repository, create a file Dockerfile with the following content:

FROM nginx:alpine

COPY _site /usr/share/nginx/html

We use a container image for nginx web server, and we copy the generated output from quarto, which will be in directory _site, to where web pages will be served from in nginx.

Now on the command-line, in the root directory of your project, run the command

flyctl launch

This will start a wizard to set up a server to run nginx.
If it asks to include the contents of the .gitignore file into .dockerignore, you can also answer no.
When it asks if you want to modify the configuration in fly.toml you can answer yes, and at least change the port number from 8080 to 80.

You can also change the app name, since it will generate a hostname by default that is my-awesome-blog.fly.dev for you.
(The app name will be my-awesome-blog unless that is already in use, and fly.io will generate a different app name for you)

Once the wizard has created the configuration, it will also try to deploy the web server and start it.
If that works ok, you will be able to access your blog web site on my-awesome-blog.fly.dev.

If you need to tweak the configuration, you can do that in the generated fly.toml file and run flyctl deploy to re-deploy the web server.

By default, the generated configuration will scale down to 0 machines after some idle time, and automatically start up a new machine if there is incoming traffic.
So if you are not getting much traffic, the costs will be quite low.
Fly.io even waives your bill if it is lower than $5 in a month.

So I would say it is pretty safe to start experimenting this way without worrying about runaway costs.

Version control tweaks

At this stage, you may want to commit your updates to your git repository.
Before you do that, you can add _site to your .gitignore file.

Thus you will only keep the source data in version control, not the generated result.

Optional - automate deployment on change through GitHub actions

If you have a GitHub repository, the flyctl wizard will have generated a file .github/workflows/fly-deploy.yml for you.
This include enough to essentially run flyctl deploy through GitHub actions.

However, you also want to let quarto render the web site content before deploying.
You can replace the content of this workflow file with the following content to also invoke quarto render before deployment:

name: Fly Deploy
on:
  push:
    branches:
      - main
jobs:
  deploy:
    name: Deploy app
    runs-on: ubuntu-latest
    concurrency: deploy-group    # optional: ensure only one action runs at a time
    steps:
      - uses: actions/checkout@v4
      - uses: quarto-dev/quarto-actions/setup@v2
      - uses: quarto-dev/quarto-actions/render@v2
      - uses: superfly/flyctl-actions/setup-flyctl@master
      - run: flyctl deploy --remote-only
        env:
          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

In the GitHub repository, you must also add a secret in the settings for the repository, FLY_API_TOKEN.
On the command-line in the project, run the following command:

flyctl tokens create deploy

Copy all the content in the output to the FLY_API_TOKEN secret in the GitHub repository.

Now any update to the main branch should trigger the workflow and re-deploy the updated website.

Custom domain and certificates

If you want to set up your own custom domain and create certificates for your awesome blog, then you need to register your domain name of choice with a domain registrar.
After that, you can go through this guide at fly.io to set up your custom domain for your website.

Final remarks

I have set up multiple websites with this approach, including blogs and online books.
For this purpose, Quarto and Fly.io have worked really well, I think.
Since the cost will essentially be 0 for low traffic, you can experiment with this type of setup without worrying too much about the cost.

You can of course publish Quarto-based material through other means as well, including GitHub pages, Netlify, etc.
For me, using fly.io has suited me very well, and their developer experience is quite nice.

I hope this information has been useful to you, and inspire to using Quarto and/or fly.io!

An opinionated AWS CDK project template for Python

Erik Lundevall Zara — Tue, 18 Jul 2023 08:30:00 +0000

AWS Cloud Development Kit (AWS CDK) is a nice framework to use if you build infrastructure for AWS. It can be relatively quick to define various types of cloud infrastructure for AWS.

However, I have found the experience to set up a new project to be lacking. This project template, found at https://github.com/cloudgnosis/awscdk-python-template, addresses some of these issues, specifically for Python users..

Template summary

The summary here of what is included you can see here, to read a bit more about the reasoning, look further below:

Adding config to CDK context automatically, from TOML files.
Encourage a flatter, non-inheritance approach to resource composition to facilitate refactoring
Simplified stack creation - automatic if single stack project, and always assign AWS account and region to stacks
Simplified stack-level tagging of resources
Modern package manager usage option - PDM
Python development tools include:
- black - code formatting
- ruff - fast linter
- pre-commit - Pre-commit hooks for formatting and linting
- pytest - testing framework
Logging and print debugging
- structlog
- icecream
Extendable model and behaviour processing of project setup options
Requires Python 3.10 or higher currently

Usage

This is a GitHub template repository, so if you re using the GitHub GUI, you can click on Use this template and select Create a new repository.

https://github.com/cloudgnosis/awscdk-python-template

Or just copy the files, or use the parts that fit your use cases. Either way, I hope you find some parts of it useful.

Reasoning behind the template

The core of the template comes from a frustration with the regular cdk init templates, which is not that flexible. I rip out or change a lot of the content - to where I might have set it up from scratch instead.

There is also a certain amount of boilerplate work for handling config data, stacks, tags, and AWS environment setup. I'm a fan of black (code formatting), ruff (code linting), pytest (testing) and pre-commit (run checks before git commit). So these are included by default.

AWS CDK examples encourage building solutions with inheritance, and push most logic into constructors. Grouping resources through inheritance leads to (too) long resource identifiers. Keeping resources after refactoring is difficult because of CloudFormation's restrictions.

Also, I think inheritance is overused. It might be useful for building specific reusable components, but not for everything in a CDK-based solution.

There are multiple project scaffolding solutions that work for Python, but not so much aimed at issues with AWS CDK and Python.

The intention is to provide a rather lightweight template for setting up AWS CDK projects, which may cover some common use cases. It is extendable and many behaviours can be replaced easily, if desired.

Examples

All examples below call a function add_sqs_sns_example() to add an SQS queue and an SNS topic, with the SQS as a subscriber to that topic. This is very much like the default AWS CDK template generated, except that the stack to add them to is passed as a parameter to this function. The same pattern can be used for any resources.

Here is a rather simple example, which will have a single stack, and all resources tagged with Environment = dev:

def main_example():
    model = ias.init_ias_model(
        {
            "deployment_name": "examples",
            "tags": {
                "Environment": "dev",
            },
        }
    )
    stack = model["stacks"]["default"]
    add_sqs_sns_example(stack)
    ias.generate_from_model(model)

This will set up a CDK App, and a single stack with id default and name examples. All resources will be tagged with tag Environment with value dev. The model object is a dictionary that contains the app, stack, and environment resources.

You get the stack from the model, and can call functions to add resources to that stack. When this is done, the generate_from_model() function will create the CloudFormation, the same as calling app.synth() on the CDK App object.

AWS account and region will be determined based on the currently active credentials, in this case.

A slightly more complicated example may look like this:

def main_example():
    model = ias.init_ias_model(
        {
            "deployment_name": "examples",
            "stacks": [{"id": "main"}, {"id": "other"}],
            "tags": {"Environment": "dev"},
        },
        option_processors=[ias.simple_config_defaults],
    )
    main_stack = model["stacks"]["main"]
    other_stack = model["stacks"]["other"]

    add_sqs_sns_example(main_stack)
    add_sqs_sns_example(other_stack)
    ias.generate_from_model(model)

Here there are two stacks, with ids main and other and stack names examples-main and examples-other . There is a different option processor, which in this case includes reading config from TOML files. All config data is available though CDK context, so it can be retrieved via any CDK Construct-type object (Construct, Stack, Stage, App).

The option processors are simply functions that update the input options and return a new set of input options. When all these processors have been executed, the actual model initialisation takes place.

The example resource function that works for the main example functions above look like this:

def add_sqs_sns_example(scope: cdk.Stack):
    config_group = scope.node.id # Use CDK stack id as config group name
    queue = sqs.Queue(scope, "queue", visibility_timeout=cdk.Duration.seconds(300))
    topic = sns.Topic(
        scope,
        "topic",
        topic_name=ias.get_context_data(scope, [config_group, "topic_name"]),
    )
    topic.add_subscription(subs.SqsSubscription(queue))

and the example TOML config file:

[main]
topic_name = "my-topic"

[other]
topic_name = "other-topic"

Design considerations

All options and the model structure use the TypedDict feature. This is like a regular dictionary, but also includes typing information for the defined fields. Thus you can get type hints in your editor of choice for these structures, plus it allows for you to extend them yourself as well. It also uses structural typing, meaning that as long as the dictionary includes the required fields, it is a valid data structure. No need to create class objects explicitly. It is kind of similar to the typing experience found in, for example, TypeScript.

Rather than loading resources into the stack constructor, I suggest using regular functions to add them. This should help with building a bit more modular code without too much boilerplate.

Things to do

Add handling for stack dependencies
Generate API docs
- Issues with getting pdoc to work when JSII is involved
Figure out reasonable scope without making it too heavy, for example
- Wrappers for sharing data between stacks? (Parameter store, etc)
- Add (optional) default monitoring (e.g. via cdk-monitoring-constructs)?
- Add (optional) default security/policy processing (e.g. via cdk-nag)?
Get more feedback

Summary

This template is an attempt to make some nicer defaults for people developing AWS CDK solutions with Python. Check out https://github.com/cloudgnosis/awscdk-python-template and tell me what you think of it!

How to become an infrastructure-as-code ninja, using AWS CDK - part 8

Erik Lundevall Zara — Tue, 04 Oct 2022 09:44:18 +0000

In this article, we are going to wrap up this article series on using AWS CDK for managing infrastructure as code. We will cover two topics in this article:

Monitoring infrastructure
Finding and using 3rd party libraries for AWS CDK

By the end of this article, you will know more about setting up monitoring and alarms for your infrastructure, and you will know more about finding useful libraries, not only from the core AWS CDK team.

The infrastructure we will monitor is based on what has been defined from previous articles in this series, so please have a look at earlier articles to see what the infrastructure we will manage is.

This article series uses Typescript as an example language. However, there are repositories with example code for multiple languages.

The repositories will contain all the code examples from the articles series, implemented in different languages with the AWS CDK. You can view the code from specific parts and stages in the series by checking out the code tagged with a specific part and step in that part. See the README file in each repository for more details.

Note: At the time of this writing, the code from this article will only be available for Typescript and Python. This is because of the 3rd party library we will use is not yet published for Go. When the library will become available for Go, the example code above will be updated accordingly.

Finding and using 3rd party libraries for AWS CDK

Currently, one of the best places to find libraries and resources for use with AWS CDK, or any other CDK (CDKTF, CDK8s) is Construct Hub.

It is a website which contains information about many CDK construct libraries, including the standard ones from AWS, 3rd party libraries and generated constructs for public CloudFormation registry modules.

Since we are going to set up some monitoring for our infrastructure, let us search for monitoring and see what we get...

The first one in the search result is a library called cdk-monitoring-constructs, which sounds pretty promising. Other results include something for DataDog. There are many other hits for the term monitoring as well. However, for now, let us look at cdk-monitoring-constructs.

We can see from the symbols that this library is available for Typescript, Python, Java and .NET (C#). No Go though (yet).

We can click on the entry to get into the documentation, read about installation and how to use this construct library. Good stuff!

This will be the starting point for our work to set up some monitoring for our solution.

Set up our solution monitoring

Recap of solution setup

If you have read the previous articles in this series, you know that we have defined an infrastructure running a containerized application in an AWS ECS cluster, behind a load balancer and with auto-scaling set up, to increase or decrease the number of container instances based on various performance characteristics (CPU and memory).

The application we have used is the Apache httpd web server.

We currently have our infrastructure code divided up into 3 different files:

bin/my-container-infrastructure.ts - the main program
lib/containers/container-management.ts - support functions for container infrastructure
test/containers/container-management.test.ts- test code for support functions

The current code can be seen here, and is also available in the GitHub repositories mentioned at the beginning of this article.

bin/my-container-infrastructure.ts

import { App, Stack } from 'aws-cdk-lib';
import { IVpc, Vpc } from 'aws-cdk-lib/aws-ec2';
import { 
  addCluster, 
  addLoadBalancedService,
  addTaskDefinitionWithContainer, 
  ContainerConfig, 
  setServiceScaling, 
  TaskConfig 
} from '../lib/containers/container-management';

const app = new App();
const stack = new Stack(app, 'my-container-infrastructure', {
  env: {
    account: process.env.CDK_DEFAULT_ACCOUNT,
    region: process.env.CDK_DEFAULT_REGION,
  },
});

let vpc: IVpc;

let vpcName = app.node.tryGetContext('vpcname');
if (vpcName) {
  vpc = Vpc.fromLookup(stack, 'vpc', {
    vpcName,
  });
} else {
  vpc = new Vpc(stack, 'vpc', {
    vpcName: 'my-vpc',
    natGateways: 1,
    maxAzs: 2,
  });
}

const id = 'my-test-cluster';
const cluster = addCluster(stack, id, vpc);

const taskConfig: TaskConfig = { cpu: 512, memoryLimitMB: 1024, family: 'webserver' };
const containerConfig: ContainerConfig = { dockerHubImage: 'httpd', tcpPorts: [80] };
const taskdef = addTaskDefinitionWithContainer(stack, `taskdef-${taskConfig.family}`, taskConfig, containerConfig);
const service = addLoadBalancedService(stack, `service-${taskConfig.family}`, cluster, taskdef, 80, 2, true);
setServiceScaling(service.service, {
  minCount: 1,
  maxCount: 4,
  scaleCpuTarget: { percent: 50 },
  scaleMemoryTarget: { percent: 70 },
});

lib/containers/container-management.ts

import { CfnOutput } from 'aws-cdk-lib';
import { IVpc, Peer, Port, SecurityGroup } from 'aws-cdk-lib/aws-ec2';
import { Cluster, ContainerImage, FargateService, FargateTaskDefinition, LogDriver, IService, TaskDefinition, Protocol } from 'aws-cdk-lib/aws-ecs';
import { ApplicationLoadBalancedFargateService } from 'aws-cdk-lib/aws-ecs-patterns';
import { RetentionDays } from 'aws-cdk-lib/aws-logs';
import { Construct } from 'constructs';

export const addCluster = function(scope: Construct, id: string, vpc: IVpc): Cluster {
    return new Cluster(scope, id, {
        vpc,
    });
}

export interface TaskConfig {
    readonly cpu: 256 | 512 | 1024 | 2048 | 4096;
    readonly memoryLimitMB: number;
    readonly family: string;
}

export interface ContainerConfig {
    readonly dockerHubImage: string;
    readonly tcpPorts: number[];
}

export const addTaskDefinitionWithContainer = 
function(scope: Construct, id: string, taskConfig: TaskConfig, containerConfig: ContainerConfig): TaskDefinition {
    const taskdef = new FargateTaskDefinition(scope, id, {
        cpu: taskConfig.cpu,
        memoryLimitMiB: taskConfig.memoryLimitMB,
        family: taskConfig.family,
    });

    const image = ContainerImage.fromRegistry(containerConfig.dockerHubImage);
    const logdriver = LogDriver.awsLogs({ 
        streamPrefix: taskConfig.family,
        logRetention: RetentionDays.ONE_DAY,
    });
    const containerDef = taskdef.addContainer(`container-${containerConfig.dockerHubImage}`, { image, logging: logdriver });
    for (const port of containerConfig.tcpPorts) {
        containerDef.addPortMappings({ containerPort: port, protocol: Protocol.TCP });
    }

    return taskdef;
};

export const addLoadBalancedService = 
function(scope: Construct, 
         id: string, 
         cluster: Cluster, 
         taskDef: FargateTaskDefinition, 
         port: number, 
         desiredCount: number, 
         publicEndpoint?: boolean,
         serviceName?: string): ApplicationLoadBalancedFargateService {
    // const sg = new SecurityGroup(scope, `${id}-security-group`, {
    //     description: `Security group for service ${serviceName ?? ''}`,
    //     vpc: cluster.vpc,
    // });
    // sg.addIngressRule(Peer.anyIpv4(), Port.tcp(port));

    const service = new ApplicationLoadBalancedFargateService(scope, id, {
        cluster,
        taskDefinition: taskDef,
        desiredCount,
        serviceName,
        //securityGroups: [sg],
        circuitBreaker: {
            rollback: true,
        },
        publicLoadBalancer: publicEndpoint,
        listenerPort: port,
    });

    return service;
};


export interface ScalingThreshold {
    percent: number;
}
export interface ServiceScalingConfig {
    minCount: number;
    maxCount: number;
    scaleCpuTarget: ScalingThreshold;
    scaleMemoryTarget: ScalingThreshold;

}


export const setServiceScaling = function(service: FargateService, config: ServiceScalingConfig) {
    const scaling = service.autoScaleTaskCount({
        maxCapacity: config.maxCount,
        minCapacity: config.minCount,
    });

    scaling.scaleOnCpuUtilization('CpuScaling', {
        targetUtilizationPercent: config.scaleCpuTarget.percent,
    });

    scaling.scaleOnMemoryUtilization('MemoryScaling', {
        targetUtilizationPercent: config.scaleMemoryTarget.percent,
    });

}

test/containers/container-management.test.ts

import { Stack } from 'aws-cdk-lib';
import { Vpc } from 'aws-cdk-lib/aws-ec2';
import { Capture, Match, Template } from 'aws-cdk-lib/assertions';
import {
    addCluster,
    addLoadBalancedService,
    addTaskDefinitionWithContainer,
    ContainerConfig,
    setServiceScaling,
    TaskConfig
} from '../../lib/containers/container-management';
import { Cluster, TaskDefinition } from 'aws-cdk-lib/aws-ecs';

test('ECS cluster is defined with existing vpc', () => {
    // Test setup
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');

    // Test code
    const cluster = addCluster(stack, 'test-cluster', vpc);

    // Check result
    const template = Template.fromStack(stack);

    template.resourceCountIs('AWS::ECS::Cluster', 1);

    expect(cluster.vpc).toEqual(vpc);
});


test('ECS Fargate task definition defined', () => {
    // Test setup
    const stack = new Stack();
    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName, tcpPorts: [80] };

    // Test code
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    // Check result
    const template = Template.fromStack(stack);

    expect(taskdef.isFargateCompatible).toBeTruthy();
    expect(stack.node.children.includes(taskdef)).toBeTruthy();

    template.resourceCountIs('AWS::ECS::TaskDefinition', 1);
    template.hasResourceProperties('AWS::ECS::TaskDefinition', {
        RequiresCompatibilities: [ 'FARGATE' ],
        Cpu: cpuval.toString(),
        Memory: memval.toString(),
        Family: familyval,
    });

});

test('Container definition added to task definition', () => {
    // Test setup
    const stack = new Stack();
    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName, tcpPorts: [80] };

    // Test code
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    // Check result
    const template = Template.fromStack(stack);
    const containerDef = taskdef.defaultContainer;

    expect(taskdef.defaultContainer).toBeDefined();
    expect(containerDef?.imageName).toEqual(imageName); // Works from v2.11 of aws-cdk-lib
    template.hasResourceProperties('AWS::ECS::TaskDefinition', {
        ContainerDefinitions: Match.arrayWith([
            Match.objectLike({
                Image: imageName,
            }),
        ]),
    });
});

describe('Test service creation options', () => {
    let stack: Stack;
    let cluster: Cluster;
    let taskdef: TaskDefinition;

    beforeEach(() => {
        // Test setup
        stack = new Stack();
        const vpc = new Vpc(stack, 'vpc');
        cluster = addCluster(stack, 'test-cluster', vpc);

        const cpuval = 512;
        const memval = 1024;
        const familyval = 'test';
        const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
        const imageName = 'httpd';
        const containerCfg: ContainerConfig = { dockerHubImage: imageName, tcpPorts: [80] };
        taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);
    });

    test('Fargate load-balanced service created, with provided mandatory properties only', () => {    
        const port = 80;
        const desiredCount = 1;

        // Test code
        const service = addLoadBalancedService(stack, 'test-service', cluster, taskdef, port, desiredCount);

        // Check result
        const sgCapture = new Capture();
        const template = Template.fromStack(stack);

        expect(service.cluster).toEqual(cluster);
        expect(service.taskDefinition).toEqual(taskdef);

        template.resourceCountIs('AWS::ECS::Service', 1);
        template.hasResourceProperties('AWS::ECS::Service', {
            DesiredCount: desiredCount,
            LaunchType: 'FARGATE',
            NetworkConfiguration: Match.objectLike({
                AwsvpcConfiguration: Match.objectLike({
                    AssignPublicIp: 'DISABLED',
                    SecurityGroups: Match.arrayWith([sgCapture]),
                }),
            }),
        });

        template.resourceCountIs('AWS::ElasticLoadBalancingV2::LoadBalancer', 1);
        template.hasResourceProperties('AWS::ElasticLoadBalancingV2::LoadBalancer', {
            Type: 'application',
            Scheme: 'internet-facing',
        });

        //template.resourceCountIs('AWS::EC2::SecurityGroup', 1);
        template.hasResourceProperties('AWS::EC2::SecurityGroup', {
            SecurityGroupIngress: Match.arrayWith([
                Match.objectLike({
                    CidrIp: '0.0.0.0/0',
                    FromPort: port,
                    IpProtocol: 'tcp',
                }),
            ]),
        });
    }); 

    test('Fargate load-balanced service created, without public access', () => {    
        const port = 80;
        const desiredCount = 1;

        // Test code
        const service = addLoadBalancedService(stack, 'test-service', cluster, taskdef, port, desiredCount, false);

        // Check result
        const template = Template.fromStack(stack);

        template.resourceCountIs('AWS::ElasticLoadBalancingV2::LoadBalancer', 1);
        template.hasResourceProperties('AWS::ElasticLoadBalancingV2::LoadBalancer', {
            Type: 'application',
            Scheme: 'internal',
        });
    }); 

    test('Scaling settings of load balancer', () => {
        const port = 80;
        const desiredCount = 2;
        const service = addLoadBalancedService(stack, 'test-service', cluster, taskdef, port, desiredCount, false);

        // Test code
        const config = {
            minCount: 1,
            maxCount: 5,
            scaleCpuTarget: { percent: 50 },
            scaleMemoryTarget: { percent: 50 },
        };

        setServiceScaling(service.service, config);


        // Check result
        const scaleResource = new Capture();
        const template = Template.fromStack(stack);
        template.resourceCountIs('AWS::ApplicationAutoScaling::ScalableTarget', 1);
        template.hasResourceProperties('AWS::ApplicationAutoScaling::ScalableTarget', {
            MaxCapacity: config.maxCount,
            MinCapacity: config.minCount,
            ResourceId: scaleResource,
            ScalableDimension: 'ecs:service:DesiredCount',
            ServiceNamespace: 'ecs',
        });

        template.resourceCountIs('AWS::ApplicationAutoScaling::ScalingPolicy', 2);
        template.hasResourceProperties('AWS::ApplicationAutoScaling::ScalingPolicy', {
            PolicyType: 'TargetTrackingScaling',
            TargetTrackingScalingPolicyConfiguration: Match.objectLike({
                PredefinedMetricSpecification: Match.objectEquals({
                    PredefinedMetricType: 'ECSServiceAverageCPUUtilization',
                }),
                TargetValue: config.scaleCpuTarget.percent,
            }),
        });

        template.hasResourceProperties('AWS::ApplicationAutoScaling::ScalingPolicy', {
            PolicyType: 'TargetTrackingScaling',
            TargetTrackingScalingPolicyConfiguration: Match.objectLike({
                PredefinedMetricSpecification: Match.objectEquals({
                    PredefinedMetricType: 'ECSServiceAverageMemoryUtilization',
                }),
                TargetValue: config.scaleMemoryTarget.percent,
            }),
        });
    });

});

Where to start with the monitoring?

So where do we start? We have found a monitoring library for AWS CDK that may make our lives easier perhaps, but we do not know that much about it yet.

If we have some monitoring, we want to see how our solution is doing, based on some kind of metrics, and we may want alerts when things may go bad. It would be good with some visualisation of this.

You may already have some corporate solution you want to hook up your monitoring to, but for this article series, we are just going to stick within the AWS services. One option available to us in this case is to set up monitoring dashboards in CloudWatch.

Thus, our infrastructure code should set up a dashboard at least, and then we have to sort out what we may want to put on that dashboard and how.

Let us explore that! **We will start by writing a test.

Add monitoring foundation

Our initial idea here is that if we should add monitoring to our solution, we should also have at least one dashboard that can visualize information for us. We may change our minds later about this, but it is a starting point.

For this purpose, we will create a file to include tests for our monitoring - test/monitoring.test.ts. We add a test for a function called initMonitoring, which we can apply to our stack, and provide a configuration input. This should return data or handle that we can use for handling the monitoring we want to add. At the very least, we should have an empty CloudWatch dashboard set up. It seems appropriate to include the dashboard name then in the config.

How do we know if we will have a dashboard? The documentation for cdk-monitoring-constructs is not entirely clear from initial view, but it seems it may add a dashboard implicitly. If we look at the CloudFormation documentation, we also see that there is an AWS::CloudWatch::Dashboard resource.

So our test can make a call to an initMonitoring function and this should cause that a CloudWatch dashboard being added to the generated CloudFormation. Let us also now initially log what the generated CloudFormation will look like to see what we actually get.

import { Stack } from 'aws-cdk-lib';
import { Template } from 'aws-cdk-lib/assertions';
import { initMonitoring, MonitoringConfig } from '../lib/monitoring';

test('Init monitoring of stack, with only defaults', () => {
    const stack = new Stack();

    const monitoringConfig: MonitoringConfig = {
        dashboardName: 'test-monitoring',
    }
    const monitoring = initMonitoring(stack, monitoringConfig);

    const template = Template.fromStack(stack);
    console.log(JSON.stringify(template.toJSON(), null, 2));
    template.resourceCountIs('AWS::CloudWatch::Dashboard', 1);
    template.hasResourceProperties('AWS::CloudWatch::Dashboard', {
        DashboardName: monitoringConfig.dashboardName
    });
});

For implementing initMonitoring, we can try to use the MonitoringFacade from cdk-monitoring-constructs. The function can simply return a structure which includes the MonitoringFacade object, which seems suitable for our purposes.

import { Construct } from 'constructs';
import { MonitoringFacade } from 'cdk-monitoring-constructs';

export interface MonitoringConfig {
    readonly dashboardName: string;
}

export interface MonitoringContext {
    readonly handler: MonitoringFacade;
}

export const initMonitoring = function(scope: Construct, config: MonitoringConfig): MonitoringContext {

    return {
        handler: new MonitoringFacade(scope, config.dashboardName),
    }
}

Running the test now when the code compiles, we can see both from our test and the log output, that our guess was correct, there will be a dashboard created when we create the MonitoringFacade!

We can add a call in our main program also to initialize the monitoring. The docs for the monitoring library also show that we can add header info to the dashboard, so we can include that as well.

const monitoring = initMonitoring(stack, {
  dashboardName: 'monitoring',
});

monitoring.handler.addMediumHeader('Test App monitoring');

You can deploy the updated AWS CDK code if you want and verify that there will be an actual dashboard created. It would not contain anything, though.

Add monitoring of actual resources

Our next step is to add some actual monitoring of resources. We can look at what cdk-monitoring-constructs provides for us. From the documentation, we can see that are many functions available from the MonitoringFacade, that start with monitor in the name and refer to different resources.

Since our solution sets up a Fargate Service in an ECS cluster with an application load balancer in front of the containers we run, the functions monitorFargateApplicationLoadBalancer and monitorFargateService seems relevant in this case. There is also a more generic monitorScope, which could also apply. Reading the somewhat sparse docs a bit more, monitorFargateService may be more appropriate if you use the AWS CDK ApplicationLoadBalancedFargateService, which is what we use in our solution.

So let us add monitoring using this function! There is only one required property for configuring monitoring, which is the ApplicationLoadBalancedFargateService object we have created. An optional field we may want to add also is the humanReadableName property as well.

After we add this call to our code, we can deploy the solution and see what we get.

const service = addLoadBalancedService(stack, `service-${taskConfig.family}`, cluster, taskdef, 80, 2, true);
setServiceScaling(service.service, {
  minCount: 1,
  maxCount: 4,
  scaleCpuTarget: { percent: 50 },
  scaleMemoryTarget: { percent: 70 },
});

const monitoring = initMonitoring(stack, {
  dashboardName: 'monitoring',
});

monitoring.handler.addMediumHeader('Test App monitoring');
monitoring.handler.monitorFargateService({
  fargateService: service,
  humanReadableName: 'My test service',
});

After deployment, we can check out in CloudWatch to see what has been added.

We have a few widgets here for CPU and memory, TCP traffic and task health. That is a good start! The task health view actually shows a property of our configuration. We have set the desired task count for our service to 2, but we have also set up our auto-scaling to have a minimum task count of 1 and a maximum of 4. So the service initially started with 2 tasks running and then scaled down to a single task.

Note: I also tried to use the **monitorFargateApplicationLoadBalancer* function, and the result was the same dashboard. The difference was the parameters that you were required to provide in the call*.

Setting up alarms - what do we need?

Now that we have some visuals in dashboard for our monitoring, let us try to set up some kind of alarm as well. For easy testing, let us set some alarm on the number of tasks running in our solution.

Reading the docs for cdk-monitoring-constructs, there is an option for us to add an alarm for running task count, e.g. if the number of running tasks go below a certain threshold for some time.

Reading through the docs, one can see that for this type of alarm to work, we need to enable container insights on the ECS Cluster. By default, this is disabled. The aws-ecs Cluster construct in AWS CDK allows us to set this properly, but does not allow us to check the state on the created cluster. This means that in order to test this, we need to check the generated CloudFormation.

Also, if we set an alarm, we need to send a notification about the alarm somewhere. One common approach is to send notifications on an SNS topic. Thus, we need to make sure we have an SNS topic that alarms will go to.

So we have 3 things to develop here that we can immediately think of:

The running task alarm itself
The SNS topic to send alarms to
Enabling container insights on the ECS Cluster

Container insights should be in place before the alarm itself, so let us start there. The SNS topic does not have to be in place before the alarm, since notification is optional and SNS is not the only way to send notifications.

Enabling container insights

Let us start by adding a new test for creating an ECS cluster. Right now we have the addCluster function, which we pass in a construct scope, and id and a VPC. If we are going to pass in more properties to this function, we can add more function parameters. We can also use the same pattern as CDK constructs and pass in a set of properties as a single parameter.

I like the latter better, because it becomes more clear what each input is with named properties - at least in Typescript.

So let us refactor the current addCluster function to pass in a set of properties as a single parameter, and then add a new property to enable container insights.

export interface ClusterConfig {
    readonly vpc: IVpc;
    readonly enableContainerInsights?: true;
}

export const addCluster = function(scope: Construct, id: string, config: ClusterConfig): Cluster {
    return new Cluster(scope, id, {
        vpc: config.vpc,
        containerInsights: config.enableContainerInsights ?? false,
    });
}

In our new test to check the container insights checking, we test the generated CloudFormation for the setting.

test('Check that container insights will be enabled when that option is set', () => {
    // Test setup
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');
    const config: ClusterConfig = {
        vpc,
        enableContainerInsights: true
    };

    // Test code
    const cluster = addCluster(stack, 'test-cluster', config);

    // Check result
    const template = Template.fromStack(stack);
    template.hasResourceProperties('AWS::ECS::Cluster', {
        ClusterSettings: Match.arrayWith([
            Match.objectEquals({
                Name: 'containerInsights',
                Value: 'enabled',
            }),
        ]),
    });
});

We can re-deploy the cluster with the new setting if we want.

Adding an alarm

Next step is to add an alarm. Here, the logic lives with the cdk-monitoring-constructs library itself. So there is not much point in making sure the alarm is there, if we are just using the functions in the library itself.

The alarm to set up for testing this feature will be to trigger an alarm if the number of running tasks is less than 2 for 10 minutes or more.

The alarm configuration will use 5-minute periods and trigger an alarm if 2 evaluation periods have passed and 2 data points fulfill the condition for the alarm. Why not just a single 10-minute period? The reason here is that sometimes data may be delayed or simply missing in CloudWatch. To avoid false positives, we set the evaluation to include multiple periods and data points.

We can provide alarm information in our call to monitorFargateService, which we set up for an alarm if the number of tasks go below 2. We know this will happen since our minimal task count in the auto-scaling is set to 1.

We add the alarm to our code and redeploy to see what the effect is on our monitoring deployment. The dashboard has an update, and there is a new alarm in place. We can see that the alarm has no action associated with it.

monitoring.handler.addMediumHeader('Test App monitoring');
monitoring.handler.monitorFargateService({
  fargateService: service,
  humanReadableName: 'My test service',

  addRunningTaskCountAlarm: {
    alarm1: {
      maxRunningTasks: 2,
      comparisonOperatorOverride: ComparisonOperator.LESS_THAN_THRESHOLD,
      evaluationPeriods: 2,
      datapointsToAlarm: 2,
      period: Duration.minutes(5),
    }
  }
});

Thus, the next step for us is to associate the alarm with an SNS topic.

Add an alarm notification topic

Our next consideration is how this SNS topic should be added, and how to test that.

We can add an alarm action for each alarm we define, which includes an SNS topic. However, it may be cumbersome to add this for every single alarm we define - especially if we decide to use the same topic for all or most alarms.

Fortunately, cdk-monitoring-constructs allows us to define a default action for alarms, when we create the MonitoringFacade. In that way, we can define the topic once only, and in one place.

Let us add an optional property to the configuration passed to initMonitoring, which is an SNS topic construct, and set that up as the default action. We can refactor this function to include a topic that will be set as a default action.

But how do we test this? Unfortunately, there is not any easy test, since we cannot directly extract that information from the created MonitoringFacade object. We essentially would need to create an alarm on some resource that we also created and then examine the created alarm if it has an action which includes the default SNS topic we have set. To do this, we would also need to examine the generated CloudFormation to see the details there.

Technically, we can certainly build such a test to check that this is generated properly. But then we also would mainly test the cdk-monitoring-constructs library, and not that much of our own code. That is wasteful. And possibly also brittle, since we cannot be 100% sure that our test code would work if the underlying implementation changed.

So we will relax on the test coverage here a bit for now.

test('Init monitoring of stack, with SNS topic for alarms', () => {
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');
    const cluster = addCluster(stack, 'test-cluster', {vpc});
    const alarmTopic = new Topic(stack, 'alarm-topic');

    const dashboardName = 'test-monitoring';
    const monitoringConfig: MonitoringConfig = {
        dashboardName,
        defaultAlarmTopic: alarmTopic,
    };

    const monitoring = initMonitoring(stack, monitoringConfig);

    expect(monitoring.defaultAlarmTopic).toEqual(alarmTopic);
});

Add an alarm notification

We can deploy the infrastructure updates and see that we have alarm information in place as well now, which has an action to send to our SNS topic.

const alarmTopic = new Topic(stack, 'alarm-topic', {
  displayName: 'Alarm topic',
});
const monitoring = initMonitoring(stack, {
  dashboardName: 'monitoring',
  defaultAlarmTopic: alarmTopic,
});

If you want to check that the notification is sent via SNS, you can add an email subscriber to the topic and check that way.

const alarmEmail = 'hello@example.com';
alarmTopic.addSubscription(new EmailSubscription(alarmEmail));

Alarm severity and category

When we send the alarm to SNS, we have nothing right now to show the severity of the alarm, nor any categorization of the alarm beside the name of the alarm.

This is often handled by external solutions. It is also possible to use AWS services for this, like AWS Systems Manager OpsCenter. We can add some code to include sending alarm info to OpsCenter also, besides the SNS topic, with an override on the default alarm strategy on our alarm.

Deploying this code will allow the alarm to be visible at the OpsCenter dashboard as well!

const alarmActions: IAlarmActionStrategy[] = [
  new OpsItemAlarmActionStrategy(OpsItemSeverity.MEDIUM, OpsItemCategory.PERFORMANCE),
];
if (monitoring.defaultAlarmTopic) {
  alarmActions.push(new SnsAlarmActionStrategy({
    onAlarmTopic: monitoring.defaultAlarmTopic,
    onOkTopic: monitoring.defaultAlarmTopic,
  }));
}

monitoring.handler.addMediumHeader('Test App monitoring');
monitoring.handler.monitorFargateService({
  fargateService: service,
  humanReadableName: 'My test service',

  addRunningTaskCountAlarm: {
    alarm1: {
      maxRunningTasks: 2,
      comparisonOperatorOverride: ComparisonOperator.LESS_THAN_THRESHOLD,
      evaluationPeriods: 2,
      datapointsToAlarm: 2,
      period: Duration.minutes(5),
      actionOverride: new MultipleAlarmActionStrategy(alarmActions),
    }
  }
});

Summary and final words

In this article, we took an add-on library for AWS CDK to facilitate monitoring of our solution infrastructure. With that, we set up a dashboard with a few widgets for monitoring visualisation.

We also added an alarm with notification via SNS topic and to AWS Systems Manager OpsCenter.

We have kept the solution small and simple in this article series, and kept all infrastructure in the same stack. In a real-world setting, we may have multiple stacks, each dedicated to a specific group of resources.

We would also likely add some automation for the provisioning of the infrastructure. This is, however, beyond this article series.

This is the final part of this article series. However, it is not the end of this material. It will be refactored into a new form and with more material to come.

I hope you have enjoyed this article series, and it has provided some value to you. If it has, I would be happy to know more! If it has not, I would be happy to know about that as well! We need feedback to improve.

Thank you for your time!

/Erik

Appendix: Code re-cap

Our final code will now look like this:

bin/my-container-infrastructure.ts

import { App, Duration, Stack } from 'aws-cdk-lib';
import { ComparisonOperator } from 'aws-cdk-lib/aws-cloudwatch';
import { OpsItemCategory, OpsItemSeverity } from 'aws-cdk-lib/aws-cloudwatch-actions';
import { IVpc, Vpc } from 'aws-cdk-lib/aws-ec2';
import { Topic } from 'aws-cdk-lib/aws-sns';
import { EmailSubscription } from 'aws-cdk-lib/aws-sns-subscriptions';
import { IAlarmActionStrategy, MultipleAlarmActionStrategy, OpsItemAlarmActionStrategy, SnsAlarmActionStrategy } from 'cdk-monitoring-constructs';
import { 
  addCluster, 
  addLoadBalancedService,
  addTaskDefinitionWithContainer, 
  ClusterConfig,
  ContainerConfig, 
  setServiceScaling, 
  TaskConfig 
} from '../lib/containers/container-management';
import { initMonitoring, MonitoringConfig } from '../lib/monitoring';

const app = new App();
const stack = new Stack(app, 'my-container-infrastructure', {
  env: {
    account: process.env.CDK_DEFAULT_ACCOUNT,
    region: process.env.CDK_DEFAULT_REGION,
  },
});

let vpc: IVpc;

let vpcName = app.node.tryGetContext('vpcname');
if (vpcName) {
  vpc = Vpc.fromLookup(stack, 'vpc', {
    vpcName,
  });
} else {
  vpc = new Vpc(stack, 'vpc', {
    vpcName: 'my-vpc',
    natGateways: 1,
    maxAzs: 2,
  });
}


const id = 'my-test-cluster';
const clusterConfig: ClusterConfig = { vpc, enableContainerInsights: true };
const cluster = addCluster(stack, id, clusterConfig);

const taskConfig: TaskConfig = { cpu: 512, memoryLimitMB: 1024, family: 'webserver' };
const containerConfig: ContainerConfig = { dockerHubImage: 'httpd', tcpPorts: [80] };
const taskdef = addTaskDefinitionWithContainer(stack, `taskdef-${taskConfig.family}`, taskConfig, containerConfig);
const service = addLoadBalancedService(stack, `service-${taskConfig.family}`, cluster, taskdef, 80, 2, true);
setServiceScaling(service.service, {
  minCount: 1,
  maxCount: 4,
  scaleCpuTarget: { percent: 50 },
  scaleMemoryTarget: { percent: 70 },
});

const alarmTopic = new Topic(stack, 'alarm-topic', {
  displayName: 'Alarm topic',
});
const monitoring = initMonitoring(stack, {
  dashboardName: 'monitoring',
  defaultAlarmTopic: alarmTopic,
});

const alarmActions: IAlarmActionStrategy[] = [
  new OpsItemAlarmActionStrategy(OpsItemSeverity.MEDIUM, OpsItemCategory.PERFORMANCE),
];
if (monitoring.defaultAlarmTopic) {
  alarmActions.push(new SnsAlarmActionStrategy({
    onAlarmTopic: monitoring.defaultAlarmTopic,
    onOkTopic: monitoring.defaultAlarmTopic,
  }));
}

monitoring.handler.addMediumHeader('Test App monitoring');
monitoring.handler.monitorFargateService({
  fargateService: service,
  humanReadableName: 'My test service',

  addRunningTaskCountAlarm: {
    alarm1: {
      maxRunningTasks: 2,
      comparisonOperatorOverride: ComparisonOperator.LESS_THAN_THRESHOLD,
      evaluationPeriods: 2,
      datapointsToAlarm: 2,
      period: Duration.minutes(5),
      actionOverride: new MultipleAlarmActionStrategy(alarmActions),
    }
  }
});

const alarmEmail = 'hello@example.com';
alarmTopic.addSubscription(new EmailSubscription(alarmEmail));

lib/containers/container-management.ts

import { CfnOutput } from 'aws-cdk-lib';
import { IVpc, Peer, Port, SecurityGroup } from 'aws-cdk-lib/aws-ec2';
import { Cluster, ContainerImage, FargateService, FargateTaskDefinition, LogDriver, IService, TaskDefinition, Protocol } from 'aws-cdk-lib/aws-ecs';
import { ApplicationLoadBalancedFargateService } from 'aws-cdk-lib/aws-ecs-patterns';
import { RetentionDays } from 'aws-cdk-lib/aws-logs';
import { Construct } from 'constructs';

export interface ClusterConfig {
    readonly vpc: IVpc;
    readonly enableContainerInsights?: boolean;
}

export const addCluster = function(scope: Construct, id: string, config: ClusterConfig): Cluster {
    return new Cluster(scope, id, {
        vpc: config.vpc,
        containerInsights: config.enableContainerInsights ?? false,
    });
}

export interface TaskConfig {
    readonly cpu: 256 | 512 | 1024 | 2048 | 4096;
    readonly memoryLimitMB: number;
    readonly family: string;
}

export interface ContainerConfig {
    readonly dockerHubImage: string;
    readonly tcpPorts: number[];
}

export const addTaskDefinitionWithContainer = 
function(scope: Construct, id: string, taskConfig: TaskConfig, containerConfig: ContainerConfig): TaskDefinition {
    const taskdef = new FargateTaskDefinition(scope, id, {
        cpu: taskConfig.cpu,
        memoryLimitMiB: taskConfig.memoryLimitMB,
        family: taskConfig.family,
    });

    const image = ContainerImage.fromRegistry(containerConfig.dockerHubImage);
    const logdriver = LogDriver.awsLogs({ 
        streamPrefix: taskConfig.family,
        logRetention: RetentionDays.ONE_DAY,
    });
    const containerDef = taskdef.addContainer(`container-${containerConfig.dockerHubImage}`, { image, logging: logdriver });
    for (const port of containerConfig.tcpPorts) {
        containerDef.addPortMappings({ containerPort: port, protocol: Protocol.TCP });
    }

    return taskdef;
};

export const addLoadBalancedService = 
function(scope: Construct, 
         id: string, 
         cluster: Cluster, 
         taskDef: FargateTaskDefinition, 
         port: number, 
         desiredCount: number, 
         publicEndpoint?: boolean,
         serviceName?: string): ApplicationLoadBalancedFargateService {
    // const sg = new SecurityGroup(scope, `${id}-security-group`, {
    //     description: `Security group for service ${serviceName ?? ''}`,
    //     vpc: cluster.vpc,
    // });
    // sg.addIngressRule(Peer.anyIpv4(), Port.tcp(port));

    const service = new ApplicationLoadBalancedFargateService(scope, id, {
        cluster,
        taskDefinition: taskDef,
        desiredCount,
        serviceName,
        //securityGroups: [sg],
        circuitBreaker: {
            rollback: true,
        },
        publicLoadBalancer: publicEndpoint,
        listenerPort: port,
    });

    return service;
};


export interface ScalingThreshold {
    percent: number;
}
export interface ServiceScalingConfig {
    minCount: number;
    maxCount: number;
    scaleCpuTarget: ScalingThreshold;
    scaleMemoryTarget: ScalingThreshold;

}


export const setServiceScaling = function(service: FargateService, config: ServiceScalingConfig) {
    const scaling = service.autoScaleTaskCount({
        maxCapacity: config.maxCount,
        minCapacity: config.minCount,
    });

    scaling.scaleOnCpuUtilization('CpuScaling', {
        targetUtilizationPercent: config.scaleCpuTarget.percent,
    });

    scaling.scaleOnMemoryUtilization('MemoryScaling', {
        targetUtilizationPercent: config.scaleMemoryTarget.percent,
    });

}

lib/monitoring/index.ts

import { Construct } from 'constructs';
import {
    IAlarmActionStrategy,
    MonitoringFacade,
    NoopAlarmActionStrategy,
    SnsAlarmActionStrategy
} from 'cdk-monitoring-constructs';
import { ITopic } from 'aws-cdk-lib/aws-sns';

export interface MonitoringConfig {
    readonly dashboardName: string;
    readonly defaultAlarmNamePrefix?: string;
    readonly defaultAlarmTopic?: ITopic;
}

export interface MonitoringContext {
    readonly handler: MonitoringFacade;
    readonly defaultAlarmTopic?: ITopic;
    readonly defaultAlarmNamePrefix?: string;
}

export const initMonitoring = function(scope: Construct, config: MonitoringConfig): MonitoringContext {

    let snsAlarmStrategy: IAlarmActionStrategy = new NoopAlarmActionStrategy;
    if (config.defaultAlarmTopic) {
        snsAlarmStrategy = new SnsAlarmActionStrategy({ onAlarmTopic: config.defaultAlarmTopic });
    }
    const defaultAlarmNamePrefix = config.defaultAlarmNamePrefix ?? config.dashboardName;
    return {
        handler: new MonitoringFacade(scope, config.dashboardName, {
            alarmFactoryDefaults: {
                actionsEnabled: true,
                action: snsAlarmStrategy,
                alarmNamePrefix: defaultAlarmNamePrefix,
            },
        }),
        defaultAlarmTopic: config.defaultAlarmTopic,
        defaultAlarmNamePrefix,
    }
}

test/containers/container-management.test.ts

import { Stack } from 'aws-cdk-lib';
import { Vpc } from 'aws-cdk-lib/aws-ec2';
import { Capture, Match, Template } from 'aws-cdk-lib/assertions';
import {
    addCluster,
    addLoadBalancedService,
    addTaskDefinitionWithContainer,
    ClusterConfig,
    ContainerConfig,
    setServiceScaling,
    TaskConfig
} from '../../lib/containers/container-management';
import { Cluster, TaskDefinition } from 'aws-cdk-lib/aws-ecs';

test('ECS cluster is defined with existing vpc', () => {
    // Test setup
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');

    // Test code
    const cluster = addCluster(stack, 'test-cluster', {vpc});

    // Check result
    const template = Template.fromStack(stack);

    template.resourceCountIs('AWS::ECS::Cluster', 1);

    expect(cluster.vpc).toEqual(vpc);
});

test('Check that container insights will be enabled when that option is set', () => {
    // Test setup
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');
    const config: ClusterConfig = {
        vpc,
        enableContainerInsights: true
    };

    // Test code
    const cluster = addCluster(stack, 'test-cluster', config);

    // Check result
    const template = Template.fromStack(stack);
    template.hasResourceProperties('AWS::ECS::Cluster', {
        ClusterSettings: Match.arrayWith([
            Match.objectEquals({
                Name: 'containerInsights',
                Value: 'enabled',
            }),
        ]),
    });
});


test('ECS Fargate task definition defined', () => {
    // Test setup
    const stack = new Stack();
    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName, tcpPorts: [80] };

    // Test code
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    // Check result
    const template = Template.fromStack(stack);

    expect(taskdef.isFargateCompatible).toBeTruthy();
    expect(stack.node.children.includes(taskdef)).toBeTruthy();

    template.resourceCountIs('AWS::ECS::TaskDefinition', 1);
    template.hasResourceProperties('AWS::ECS::TaskDefinition', {
        RequiresCompatibilities: [ 'FARGATE' ],
        Cpu: cpuval.toString(),
        Memory: memval.toString(),
        Family: familyval,
    });

});

test('Container definition added to task definition', () => {
    // Test setup
    const stack = new Stack();
    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName, tcpPorts: [80] };

    // Test code
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    // Check result
    const template = Template.fromStack(stack);
    const containerDef = taskdef.defaultContainer;

    expect(taskdef.defaultContainer).toBeDefined();
    expect(containerDef?.imageName).toEqual(imageName); // Works from v2.11 of aws-cdk-lib
    template.hasResourceProperties('AWS::ECS::TaskDefinition', {
        ContainerDefinitions: Match.arrayWith([
            Match.objectLike({
                Image: imageName,
            }),
        ]),
    });
});

describe('Test service creation options', () => {
    let stack: Stack;
    let cluster: Cluster;
    let taskdef: TaskDefinition;

    beforeEach(() => {
        // Test setup
        stack = new Stack();
        const vpc = new Vpc(stack, 'vpc');
        cluster = addCluster(stack, 'test-cluster', {vpc});

        const cpuval = 512;
        const memval = 1024;
        const familyval = 'test';
        const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
        const imageName = 'httpd';
        const containerCfg: ContainerConfig = { dockerHubImage: imageName, tcpPorts: [80] };
        taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);
    });

    test('Fargate load-balanced service created, with provided mandatory properties only', () => {    
        const port = 80;
        const desiredCount = 1;

        // Test code
        const service = addLoadBalancedService(stack, 'test-service', cluster, taskdef, port, desiredCount);

        // Check result
        const sgCapture = new Capture();
        const template = Template.fromStack(stack);

        expect(service.cluster).toEqual(cluster);
        expect(service.taskDefinition).toEqual(taskdef);

        template.resourceCountIs('AWS::ECS::Service', 1);
        template.hasResourceProperties('AWS::ECS::Service', {
            DesiredCount: desiredCount,
            LaunchType: 'FARGATE',
            NetworkConfiguration: Match.objectLike({
                AwsvpcConfiguration: Match.objectLike({
                    AssignPublicIp: 'DISABLED',
                    SecurityGroups: Match.arrayWith([sgCapture]),
                }),
            }),
        });

        template.resourceCountIs('AWS::ElasticLoadBalancingV2::LoadBalancer', 1);
        template.hasResourceProperties('AWS::ElasticLoadBalancingV2::LoadBalancer', {
            Type: 'application',
            Scheme: 'internet-facing',
        });

        //template.resourceCountIs('AWS::EC2::SecurityGroup', 1);
        template.hasResourceProperties('AWS::EC2::SecurityGroup', {
            SecurityGroupIngress: Match.arrayWith([
                Match.objectLike({
                    CidrIp: '0.0.0.0/0',
                    FromPort: port,
                    IpProtocol: 'tcp',
                }),
            ]),
        });
    }); 

    test('Fargate load-balanced service created, without public access', () => {    
        const port = 80;
        const desiredCount = 1;

        // Test code
        const service = addLoadBalancedService(stack, 'test-service', cluster, taskdef, port, desiredCount, false);

        // Check result
        const template = Template.fromStack(stack);

        template.resourceCountIs('AWS::ElasticLoadBalancingV2::LoadBalancer', 1);
        template.hasResourceProperties('AWS::ElasticLoadBalancingV2::LoadBalancer', {
            Type: 'application',
            Scheme: 'internal',
        });
    }); 

    test('Scaling settings of load balancer', () => {
        const port = 80;
        const desiredCount = 2;
        const service = addLoadBalancedService(stack, 'test-service', cluster, taskdef, port, desiredCount, false);

        // Test code
        const config = {
            minCount: 1,
            maxCount: 5,
            scaleCpuTarget: { percent: 50 },
            scaleMemoryTarget: { percent: 50 },
        };

        setServiceScaling(service.service, config);


        // Check result
        const scaleResource = new Capture();
        const template = Template.fromStack(stack);
        template.resourceCountIs('AWS::ApplicationAutoScaling::ScalableTarget', 1);
        template.hasResourceProperties('AWS::ApplicationAutoScaling::ScalableTarget', {
            MaxCapacity: config.maxCount,
            MinCapacity: config.minCount,
            ResourceId: scaleResource,
            ScalableDimension: 'ecs:service:DesiredCount',
            ServiceNamespace: 'ecs',
        });

        template.resourceCountIs('AWS::ApplicationAutoScaling::ScalingPolicy', 2);
        template.hasResourceProperties('AWS::ApplicationAutoScaling::ScalingPolicy', {
            PolicyType: 'TargetTrackingScaling',
            TargetTrackingScalingPolicyConfiguration: Match.objectLike({
                PredefinedMetricSpecification: Match.objectEquals({
                    PredefinedMetricType: 'ECSServiceAverageCPUUtilization',
                }),
                TargetValue: config.scaleCpuTarget.percent,
            }),
        });

        template.hasResourceProperties('AWS::ApplicationAutoScaling::ScalingPolicy', {
            PolicyType: 'TargetTrackingScaling',
            TargetTrackingScalingPolicyConfiguration: Match.objectLike({
                PredefinedMetricSpecification: Match.objectEquals({
                    PredefinedMetricType: 'ECSServiceAverageMemoryUtilization',
                }),
                TargetValue: config.scaleMemoryTarget.percent,
            }),
        });
    });

});

test/monitoring.test.ts

import { Stack } from 'aws-cdk-lib';
import { Template } from 'aws-cdk-lib/assertions';
import { Vpc } from 'aws-cdk-lib/aws-ec2';
import { Topic } from 'aws-cdk-lib/aws-sns';
import { initMonitoring, MonitoringConfig } from '../lib/monitoring';
import { addCluster } from '../lib/containers/container-management';

test('Init monitoring of stack, with only defaults', () => {
    const stack = new Stack();

    const monitoringConfig: MonitoringConfig = {
        dashboardName: 'test-monitoring',
    }
    const monitoring = initMonitoring(stack, monitoringConfig);

    const template = Template.fromStack(stack);
    //console.log(JSON.stringify(template.toJSON(), null, 2));
    template.resourceCountIs('AWS::CloudWatch::Dashboard', 1);
    template.hasResourceProperties('AWS::CloudWatch::Dashboard', {
        DashboardName: monitoringConfig.dashboardName
    });
});

test('Init monitoring of stack, with SNS topic for alarms', () => {
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');
    const cluster = addCluster(stack, 'test-cluster', {vpc});
    const alarmTopic = new Topic(stack, 'alarm-topic');

    const dashboardName = 'test-monitoring';
    const monitoringConfig: MonitoringConfig = {
        dashboardName,
        defaultAlarmTopic: alarmTopic,
    };

    const monitoring = initMonitoring(stack, monitoringConfig);

    expect(monitoring.defaultAlarmTopic).toEqual(alarmTopic);
    expect(monitoring.defaultAlarmNamePrefix).toEqual(dashboardName);
});

test('Init monitoring of stack, with SNS topic for alarms and alarm prefix set', () => {
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');
    const cluster = addCluster(stack, 'test-cluster', {vpc});
    const alarmTopic = new Topic(stack, 'alarm-topic');

    const dashboardName = 'test-monitoring';
    const alarmPrefix = 'my-prefix';
    const monitoringConfig: MonitoringConfig = {
        dashboardName,
        defaultAlarmTopic: alarmTopic,
        defaultAlarmNamePrefix: alarmPrefix,
    };

    const monitoring = initMonitoring(stack, monitoringConfig);

    expect(monitoring.defaultAlarmTopic).toEqual(alarmTopic);
    expect(monitoring.defaultAlarmNamePrefix).toEqual(alarmPrefix);
});

How to become an infrastructure-as-code ninja, using AWS CDK - part 7

Erik Lundevall Zara — Mon, 30 May 2022 11:31:18 +0000

In this article, we are going to expand on the service in ECS we have already defined in part 6 and make it a load-balanced service that can automatically scale out and scale in, based on load. We are going to use a test-driven design (TDD) approach to figure out what we want to do here.

A recap

First, a quick re-cap what we have done so far:

We are using AWS CDK to define an infrastructure, using Typescript
This infrastructure uses an existing VPC (Virtual Private Cloud - network infrastructure)
We run a container-based solution using AWS Elastic Container Service (ECS)
We have an ECS task definition that defines the container properties to use
We have an ECS service definition that uses this task definition, plus a desired state description, so ECS will make sure that our solution matches that desired state.

We can expose the container publicly and connect to the Apache web server (httpd) that we have used so far to test the infrastructure building blocks.

So far, we have three code files we have worked with:

bin/my-container-infrastructure.ts - the main program file
lib/containers/container-management.ts - where most of our container handling logic lives
test/containers/container-management.test.ts - The test code we have for our container management logic

Note about code availability

This article series uses Typescript as an example language. However, there are repositories with example code for multiple languages.

The code (in Typescript) to this point is below here:

my-container-infrastructure.ts

import { App, Stack } from 'aws-cdk-lib';
import { Vpc } from 'aws-cdk-lib/aws-ec2';
import {
    addCluster,
    addService,
    addTaskDefinitionWithContainer,
    ContainerConfig,
    TaskConfig
} from '../lib/containers/container-management';

const app = new App();
const stack = new Stack(app, 'my-container-infrastructure', {
    env: {
        account: process.env.CDK_DEFAULT_ACCOUNT,
        region: process.env.CDK_DEFAULT_REGION,
    },
});

const vpc = Vpc.fromLookup(stack, 'vpc', {
    isDefault: true,
});

const id = 'my-test-cluster';
const cluster = addCluster(stack, id, vpc);

const taskConfig: TaskConfig = { cpu: 512, memoryLimitMB: 1024, family: 'webserver' };
const containerConfig: ContainerConfig = { dockerHubImage: 'httpd' };

const taskdef = addTaskDefinitionWithContainer(stack, `taskdef-${taskConfig.family}`, taskConfig, containerConfig);
addService(stack, `service-${taskConfig.family}`, cluster, taskdef, 80, 0, true);

container-management.ts

import { IVpc, Peer, Port, SecurityGroup } from 'aws-cdk-lib/aws-ec2';
import {
    Cluster,
    ContainerImage,
    FargateService,
    FargateTaskDefinition,
    LogDriver,
    TaskDefinition
} from 'aws-cdk-lib/aws-ecs';
import { RetentionDays } from 'aws-cdk-lib/aws-logs';
import { Construct } from 'constructs';

export const addCluster = function(scope: Construct, id: string, vpc: IVpc): Cluster {
    return new Cluster(scope, id, {
        vpc,
    });
}

export interface TaskConfig {
    readonly cpu: 256 | 512 | 1024 | 2048 | 4096;
    readonly memoryLimitMB: number;
    readonly family: string;
}

export interface ContainerConfig {
    readonly dockerHubImage: string;
}

export const addTaskDefinitionWithContainer = 
function(scope: Construct, id: string, taskConfig: TaskConfig, containerConfig: ContainerConfig): TaskDefinition {
    const taskdef = new FargateTaskDefinition(scope, id, {
        cpu: taskConfig.cpu,
        memoryLimitMiB: taskConfig.memoryLimitMB,
        family: taskConfig.family,
    });

    const image = ContainerImage.fromRegistry(containerConfig.dockerHubImage);
    const logdriver = LogDriver.awsLogs({ 
        streamPrefix: taskConfig.family,
        logRetention: RetentionDays.ONE_DAY,
    });
    taskdef.addContainer(`container-${containerConfig.dockerHubImage}`, { image, logging: logdriver });

    return taskdef;
};

export const addService = 
function(scope: Construct, 
         id: string, 
         cluster: Cluster, 
         taskDef: FargateTaskDefinition, 
         port: number, 
         desiredCount: number, 
         assignPublicIp?: boolean,
         serviceName?: string): FargateService {
    const sg = new SecurityGroup(scope, `${id}-security-group`, {
        description: `Security group for service ${serviceName ?? ''}`,
        vpc: cluster.vpc,
    });
    sg.addIngressRule(Peer.anyIpv4(), Port.tcp(port));

    const service = new FargateService(scope, id, {
        cluster,
        taskDefinition: taskDef,
        desiredCount,
        serviceName,
        securityGroups: [sg],
        circuitBreaker: {
            rollback: true,
        },
        assignPublicIp,
    });

    return service;
};

container-management.test.ts

import { Stack } from 'aws-cdk-lib';
import { Vpc } from 'aws-cdk-lib/aws-ec2';
import { Capture, Match, Template } from 'aws-cdk-lib/assertions';
import {
    addCluster,
    addService,
    addTaskDefinitionWithContainer,
    ContainerConfig,
    TaskConfig
} from '../../lib/containers/container-management';

test('ECS cluster is defined with existing vpc', () => {
    // Test setup
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');

    // Test code
    const cluster = addCluster(stack, 'test-cluster', vpc);

    // Check result
    const template = Template.fromStack(stack);

    template.resourceCountIs('AWS::ECS::Cluster', 1);

    expect(cluster.vpc).toEqual(vpc);
});

test('ECS Fargate task definition defined', () => {
    // Test setup
    const stack = new Stack();
    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName };

    // Test code
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    // Check result
    const template = Template.fromStack(stack);

    expect(taskdef.isFargateCompatible).toBeTruthy();
    expect(stack.node.children.includes(taskdef)).toBeTruthy();

    template.resourceCountIs('AWS::ECS::TaskDefinition', 1);
    template.hasResourceProperties('AWS::ECS::TaskDefinition', {
        RequiresCompatibilities: [ 'FARGATE' ],
        Cpu: cpuval.toString(),
        Memory: memval.toString(),
        Family: familyval,
    });

});

test('Fargate service created, with provided mandatory properties only', () => {
    // Test setup
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');
    const cluster = addCluster(stack, 'test-cluster', vpc);

    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName };
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    const port = 80;
    const desiredCount = 1;

    // Test code
    const service = addService(stack, 'test-service', cluster, taskdef, port, desiredCount);

    // Check result
    const sgCapture = new Capture();
    const template = Template.fromStack(stack);

    expect(service.cluster).toEqual(cluster);
    expect(service.taskDefinition).toEqual(taskdef);

    template.resourceCountIs('AWS::ECS::Service', 1);
    template.hasResourceProperties('AWS::ECS::Service', {
        DesiredCount: desiredCount,
        LaunchType: 'FARGATE',
        NetworkConfiguration: Match.objectLike({
            AwsvpcConfiguration: Match.objectLike({
                AssignPublicIp: 'DISABLED',
                SecurityGroups: Match.arrayWith([sgCapture]),
            }),
        }),
    });

    template.resourceCountIs('AWS::EC2::SecurityGroup', 1);
    template.hasResourceProperties('AWS::EC2::SecurityGroup', {
        SecurityGroupIngress: Match.arrayWith([
            Match.objectLike({
                CidrIp: '0.0.0.0/0',
                FromPort: port,
                IpProtocol: 'tcp',
            }),
        ]),
    });
});

test('Container definition added to task definition', () => {
    // Test setup
    const stack = new Stack();
    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName };

    // Test code
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    // Check result
    const template = Template.fromStack(stack);
    const containerDef = taskdef.defaultContainer;

    expect(taskdef.defaultContainer).toBeDefined();
    expect(containerDef?.imageName).toEqual(imageName); // Works from v2.11 of aws-cdk-lib
    template.hasResourceProperties('AWS::ECS::TaskDefinition', {
        ContainerDefinitions: Match.arrayWith([
            Match.objectLike({
                Image: imageName,
            }),
        ]),
    });
});

What we will do now

In the previous part, there were some points about our design that we should address, which include:

Specifying a port number to access the service through addService is right now fine for a single container instance only. For multiple containers (desiredCount > 1) there would need to be a load balancer.
The design opens for traffic from everywhere, regardless of whether it uses public or private IP addresses

We should also add some mechanisms to scale the solution based on some load characteristics.

The points above tie in with each other. If we put the container service behind a load balancer, we want to restrict access so that it only goes via the load balancer. We also want to have a suitable number of containers running, based on the load on the service.

So we have a few tasks:

Add a load balancer in front of a service, when we set up a service in ECS
Make sure access is restricted to go via load balancer
Add scalability configuration for containers behind load balancer

We can break these up in more detailed steps later.

The trouble with default VPCs

Until now, our infrastructure experiments have assumed that there is a VPC in place, and we have also for simplicity used the default VPC, which is something most AWS accounts will have.

A problem with the default VPC is that there are only public subnets in that VPC. The reason for that is backward compatibility with the old EC2 Classic, back in those days when you did not need an explicit VPC to run your virtual machines in AWS. This is my guess anyway.

Either way, it is a shortcoming of this VPC. To enable us to use a different VPC with both private and public subnets, we will add a small code segment to optionally add a VPC in the stack. You can also point to an existing VPC that has both public and private subnets.


let vpc: IVpc;

let vpcName = app.node.tryGetContext('vpcname');
if (vpcName) {
  vpc = Vpc.fromLookup(stack, 'vpc', {
    vpcName,
  });
} else {
  vpc = new Vpc(stack, 'vpc', {
    vpcName: 'my-vpc',
    natGateways: 1,
    maxAzs: 2,
  });
}

Instead of simply looking up the default VPC, we have two options. Option one is to provide the name of an existing VPC, and in that case, we use that. If not VPC name is provided, we create a new VPC.

The created VPC will have both public and private subnets, will use two availability zones and have a shared NAT gateway for outbound traffic from private subnets. This is strictly to keep costs down.

If you have an existing VPC to use, you can provide the name of that VPC through a command line option:

cdk synth -c vpcname=thenameofmyvpc

This approach to sending contextual data to the AWS CDK App can be quite useful. The key takeaway for us here is that we can have a VPC with both private and public subnets.

Starting from the tests - TDD a load-balanced service

Let us start by looking at our tests. We currently have one test for the addService function, see below. There is a fair amount of set-up work here.

test('Fargate service created, with provided mandatory properties only', () => {
    // Test setup
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');
    const cluster = addCluster(stack, 'test-cluster', vpc);

    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName };
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    const port = 80;
    const desiredCount = 1;

    // Test code
    const service = addService(stack, 'test-service', cluster, taskdef, port, desiredCount);

    // Check result
    const sgCapture = new Capture();
    const template = Template.fromStack(stack);

    expect(service.cluster).toEqual(cluster);
    expect(service.taskDefinition).toEqual(taskdef);

    template.resourceCountIs('AWS::ECS::Service', 1);
    template.hasResourceProperties('AWS::ECS::Service', {
        DesiredCount: desiredCount,
        LaunchType: 'FARGATE',
        NetworkConfiguration: Match.objectLike({
            AwsvpcConfiguration: Match.objectLike({
                AssignPublicIp: 'DISABLED',
                SecurityGroups: Match.arrayWith([sgCapture]),
            }),
        }),
    });

    template.resourceCountIs('AWS::EC2::SecurityGroup', 1);
    template.hasResourceProperties('AWS::EC2::SecurityGroup', {
        SecurityGroupIngress: Match.arrayWith([
            Match.objectLike({
                CidrIp: '0.0.0.0/0',
                FromPort: port,
                IpProtocol: 'tcp',
            }),
        ]),
    });
});

My suspicion at this point is that we may need multiple tests for the service part. Each test would have will have had the same setup work to do. With that in mind, I think we could refactor the service test into a group of tests that all run the same setup. Later, if we add more service tests, most of the set-up work will be in place already. Luckily, the Jest test framework we use with our AWS CDK Typescript code has support for that, using the beforeEach function. The refactored code looks like this:

describe('Test service creation options', () => {
    let stack: Stack;
    let cluster: Cluster;
    let taskdef: TaskDefinition;

    beforeEach(() => {
        // Test setup
        stack = new Stack();
        const vpc = new Vpc(stack, 'vpc');
        cluster = addCluster(stack, 'test-cluster', vpc);

        const cpuval = 512;
        const memval = 1024;
        const familyval = 'test';
        const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
        const imageName = 'httpd';
        const containerCfg: ContainerConfig = { dockerHubImage: imageName };
        taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);
    });

    test('Fargate load-balanced service created, with provided mandatory properties only', () => {    
        const port = 80;
        const desiredCount = 1;

        // Test code
        const service = addService(stack, 'test-service', cluster, taskdef, port, desiredCount);

        // Check result
        const sgCapture = new Capture();
        const template = Template.fromStack(stack);

        expect(service.cluster).toEqual(cluster);
        expect(service.taskDefinition).toEqual(taskdef);

        template.resourceCountIs('AWS::ECS::Service', 1);
        template.hasResourceProperties('AWS::ECS::Service', {
            DesiredCount: desiredCount,
            LaunchType: 'FARGATE',
            NetworkConfiguration: Match.objectLike({
                AwsvpcConfiguration: Match.objectLike({
                    AssignPublicIp: 'DISABLED',
                    SecurityGroups: Match.arrayWith([sgCapture]),
                }),
            }),
        });

        template.resourceCountIs('AWS::EC2::SecurityGroup', 1);
        template.hasResourceProperties('AWS::EC2::SecurityGroup', {
            SecurityGroupIngress: Match.arrayWith([
                Match.objectLike({
                    CidrIp: '0.0.0.0/0',
                    FromPort: port,
                    IpProtocol: 'tcp',
                }),
            ]),
        });
    }); 
});

When you change the code, run npm test to make sure that the tests still work.

Next, let us think about what we want to do. We want to add a load balancer in front of our container service, so that we can scale it to a suitable about and set meaningful desired count larger than 1. Maybe the function call should not be called addService then, but addLoadBalancedService?

We can change the name of the function call, and do the corresponding change in the code itself to fix our now broken test, and test that now works again.

Next, since we should have a load balanced service, it would make sense to test that we get a load balancer in place. The underlying CloudFormation would generate a resource called AWS::ElasticLoadBalancerV2::LoadBalancer. So we can add a test that there will be such a resource after we have called our addLoadBalancedService function.

How do we implement this? We are trying to keep things simple, and fortunately AWS CDK has a construct that fits what we need well, called ApplicationLoadBalancedFargateService. This should set up a load balancer and create the service behind it, which is just what we want. We already have a task definition for the container we want to run as a service and this construct can accept that as input, which is just great!

Another change we can make to our function besides the name change is the optional parameter assignPublicIp. This is not relevant for a load balancer, since we will not have a specific IP address for the load balancer. But the underlying purpose of that parameter is that we should be able to tell if the endpoint we expose should be public (available through the internet) or private (only reachable in our own network). So we can also change the name of this parameter to better reflect our intent, instead of what it should do. So we can rename it to publicEndpoint instead, for example.

So now we have a new test and some new code to support this, but if we run npm test now, we get a failure! Our new construct will create security groups under the hood for us, and there will be security groups for the load balancer itself, as well as for the targets it will redirect traffic to. So our previous test for a single security group is no longer valid. We might not even care exactly how many service groups are created, as long as the functionality they are supposed to handle is in place (only allow specific network traffic to selected resources). We can also reasonably assume that those who have built ApplicationLoadBalancedFargateService have tested out that code well enough that we should not test their implementation. So we can skip our test for a single security group we had before.

So we adjust the test and we also adjust the underlying implementation of addLoadBalancedService, since we do not need to care about managing the security groups ourselves, at this point in time at least.

Now if we run npm test, we see the test succeed! Is that good enough? Actually, we should also be clear what ports to map, so we should also include some port mapping information. The load balancer port and the container ports may not be the same. This may be suitable to include in the container configuration for the container itself.

Finally, let us also make sure we have a test for the publicEndpoint parameter. Here, one relatively simple way to check is if the CloudFormation load balancer resource has its scheme set to internal if publicEndpoint is false, and it should be internet-facing if publicEndpoint is true.

Now we have a new set of code, see below:

container-management.test.ts

import { Stack } from 'aws-cdk-lib';
import { Vpc } from 'aws-cdk-lib/aws-ec2';
import { Capture, Match, Template } from 'aws-cdk-lib/assertions';
import {
    addCluster,
    addLoadBalancedService,
    addTaskDefinitionWithContainer,
    ContainerConfig,
    TaskConfig
} from '../../lib/containers/container-management';
import { Cluster, TaskDefinition } from 'aws-cdk-lib/aws-ecs';

test('ECS cluster is defined with existing vpc', () => {
    // Test setup
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');

    // Test code
    const cluster = addCluster(stack, 'test-cluster', vpc);

    // Check result
    const template = Template.fromStack(stack);

    template.resourceCountIs('AWS::ECS::Cluster', 1);

    expect(cluster.vpc).toEqual(vpc);
});


test('ECS Fargate task definition defined', () => {
    // Test setup
    const stack = new Stack();
    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName, tcpPorts: [80] };

    // Test code
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    // Check result
    const template = Template.fromStack(stack);

    expect(taskdef.isFargateCompatible).toBeTruthy();
    expect(stack.node.children.includes(taskdef)).toBeTruthy();

    template.resourceCountIs('AWS::ECS::TaskDefinition', 1);
    template.hasResourceProperties('AWS::ECS::TaskDefinition', {
        RequiresCompatibilities: [ 'FARGATE' ],
        Cpu: cpuval.toString(),
        Memory: memval.toString(),
        Family: familyval,
    });

});

test('Container definition added to task definitio', () => {
    // Test setup
    const stack = new Stack();
    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName, tcpPorts: [80] };

    // Test code
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    // Check result
    const template = Template.fromStack(stack);
    const containerDef = taskdef.defaultContainer;

    expect(taskdef.defaultContainer).toBeDefined();
    expect(containerDef?.imageName).toEqual(imageName); // Works from v2.11 of aws-cdk-lib
    template.hasResourceProperties('AWS::ECS::TaskDefinition', {
        ContainerDefinitions: Match.arrayWith([
            Match.objectLike({
                Image: imageName,
            }),
        ]),
    });
});

describe('Test service creation options', () => {
    let stack: Stack;
    let cluster: Cluster;
    let taskdef: TaskDefinition;

    beforeEach(() => {
        // Test setup
        stack = new Stack();
        const vpc = new Vpc(stack, 'vpc');
        cluster = addCluster(stack, 'test-cluster', vpc);

        const cpuval = 512;
        const memval = 1024;
        const familyval = 'test';
        const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
        const imageName = 'httpd';
        const containerCfg: ContainerConfig = { dockerHubImage: imageName, tcpPorts: [80] };
        taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);
    });

    test('Fargate load-balanced service created, with provided mandatory properties only', () => {    
        const port = 80;
        const desiredCount = 1;

        // Test code
        const service = addLoadBalancedService(stack, 'test-service', cluster, taskdef, port, desiredCount);

        // Check result
        const sgCapture = new Capture();
        const template = Template.fromStack(stack);

        expect(service.cluster).toEqual(cluster);
        expect(service.taskDefinition).toEqual(taskdef);

        template.resourceCountIs('AWS::ECS::Service', 1);
        template.hasResourceProperties('AWS::ECS::Service', {
            DesiredCount: desiredCount,
            LaunchType: 'FARGATE',
            NetworkConfiguration: Match.objectLike({
                AwsvpcConfiguration: Match.objectLike({
                    AssignPublicIp: 'DISABLED',
                    SecurityGroups: Match.arrayWith([sgCapture]),
                }),
            }),
        });

        template.resourceCountIs('AWS::ElasticLoadBalancingV2::LoadBalancer', 1);
        template.hasResourceProperties('AWS::ElasticLoadBalancingV2::LoadBalancer', {
            Type: 'application',
            Scheme: 'internet-facing',
        });

        //template.resourceCountIs('AWS::EC2::SecurityGroup', 1);
        template.hasResourceProperties('AWS::EC2::SecurityGroup', {
            SecurityGroupIngress: Match.arrayWith([
                Match.objectLike({
                    CidrIp: '0.0.0.0/0',
                    FromPort: port,
                    IpProtocol: 'tcp',
                }),
            ]),
        });
    }); 

    test('Fargate load-balanced service created, without public access', () => {    
        const port = 80;
        const desiredCount = 1;

        // Test code
        const service = addLoadBalancedService(stack, 'test-service', cluster, taskdef, port, desiredCount, false);

        // Check result
        const template = Template.fromStack(stack);

        template.resourceCountIs('AWS::ElasticLoadBalancingV2::LoadBalancer', 1);
        template.hasResourceProperties('AWS::ElasticLoadBalancingV2::LoadBalancer', {
            Type: 'application',
            Scheme: 'internal',
        });
    }); 
});

container-management.ts

import { IVpc } from 'aws-cdk-lib/aws-ec2';
import { Cluster, ContainerImage, FargateTaskDefinition, LogDriver, TaskDefinition, Protocol } from 'aws-cdk-lib/aws-ecs';
import { ApplicationLoadBalancedFargateService } from 'aws-cdk-lib/aws-ecs-patterns';
import { RetentionDays } from 'aws-cdk-lib/aws-logs';
import { Construct } from 'constructs';

export const addCluster = function(scope: Construct, id: string, vpc: IVpc): Cluster {
    return new Cluster(scope, id, {
        vpc,
    });
}

export interface TaskConfig {
    readonly cpu: 256 | 512 | 1024 | 2048 | 4096;
    readonly memoryLimitMB: number;
    readonly family: string;
}

export interface ContainerConfig {
    readonly dockerHubImage: string;
    readonly tcpPorts: number[];
}

export const addTaskDefinitionWithContainer = 
function(scope: Construct, id: string, taskConfig: TaskConfig, containerConfig: ContainerConfig): TaskDefinition {
    const taskdef = new FargateTaskDefinition(scope, id, {
        cpu: taskConfig.cpu,
        memoryLimitMiB: taskConfig.memoryLimitMB,
        family: taskConfig.family,
    });

    const image = ContainerImage.fromRegistry(containerConfig.dockerHubImage);
    const logdriver = LogDriver.awsLogs({ 
        streamPrefix: taskConfig.family,
        logRetention: RetentionDays.ONE_DAY,
    });
    const containerDef = taskdef.addContainer(`container-${containerConfig.dockerHubImage}`, { image, logging: logdriver });
    for (const port of containerConfig.tcpPorts) {
        containerDef.addPortMappings({ containerPort: port, protocol: Protocol.TCP });
    }

    return taskdef;
};

export const addLoadBalancedService = 
function(scope: Construct, 
         id: string, 
         cluster: Cluster, 
         taskDef: FargateTaskDefinition, 
         port: number, 
         desiredCount: number, 
         publicEndpoint?: boolean,
         serviceName?: string): ApplicationLoadBalancedFargateService {

    const service = new ApplicationLoadBalancedFargateService(scope, id, {
        cluster,
        taskDefinition: taskDef,
        desiredCount,
        serviceName,
        circuitBreaker: {
            rollback: true,
        },
        publicLoadBalancer: publicEndpoint,
        listenerPort: port,
    });

    return service;
};

my-container-infrastructure.ts

import { App, Stack } from 'aws-cdk-lib';
import { IVpc, Vpc } from 'aws-cdk-lib/aws-ec2';
import { 
  addCluster, 
  addLoadBalancedService,
  addTaskDefinitionWithContainer, 
  ContainerConfig, 
  TaskConfig 
} from '../lib/containers/container-management';

const app = new App();
const stack = new Stack(app, 'my-container-infrastructure', {
  env: {
    account: process.env.CDK_DEFAULT_ACCOUNT,
    region: process.env.CDK_DEFAULT_REGION,
  },
});

let vpc: IVpc;

let vpcName = app.node.tryGetContext('vpcname');
if (vpcName) {
  vpc = Vpc.fromLookup(stack, 'vpc', {
    vpcName,
  });
} else {
  vpc = new Vpc(stack, 'vpc', {
    vpcName: 'my-vpc',
    natGateways: 1,
    maxAzs: 2,
  });
}


const id = 'my-test-cluster';
const cluster = addCluster(stack, id, vpc);

const taskConfig: TaskConfig = { cpu: 512, memoryLimitMB: 1024, family: 'webserver' };
const containerConfig: ContainerConfig = { dockerHubImage: 'httpd', tcpPorts: [80] };
const taskdef = addTaskDefinitionWithContainer(stack, `taskdef-${taskConfig.family}`, taskConfig, containerConfig);
addLoadBalancedService(stack, `service-${taskConfig.family}`, cluster, taskdef, 80, 2, true);

Run npm test and make sure all the tests pass. You can also to deploy the solution now, using cdk deploy. Expect the deployment to take some time, especially if you deploy everything from scratch and if you have it set up to create a new VPC.

You notice that when the deployment is completed, there will be two outputs representing the load balancer endpoint. This results from using ApplicationLoadBalancedFargateService, which adds these outputs by default to the stack.

A bit of auto-scaling

If we have a load balancer in front of our service to distribute the load among multiple containers, it would make sense to scale up and down how many containers will handle that load. Fortunately, after some investigation, we can see that the FargateService class has a function called autoScaleTaskCount, which allows us to set the minimum and maximum number of tasks. It also allows us to specify thresholds for CPU and memory to determine if the service should scale down or up the number of tasks running in the service.

There is no corresponding function on ApplicationLoadBalancedFargateService though, but we can access the underlying FargateService using a service property on ApplicationLoadBalancedFargateService.

Unfortunately, we cannot access all these scaling settings on FargateService once we have set them, so to verify that these settings have been set, we would resort to check CloudFormation resources. In this case, it is resources of the ApplicationAutoScaling that is used by CloudFormation.

Frankly, I am hesitant about the value of some of these tests, since this may be borderline to test the CDK implementation rather than testing our own logic. I kept them here more to be a learning experience. Even with these tests, we would want to do some actual integration tests to see that the real behaviour once deployed is what we expect. There is some early experimental work in AWS CDK for building integration tests, but that is out of scope for this article.

For now, we can at least know that there are some settings in place for the auto-scaling, but more work would be needed to test its actual behaviour.

Extract from container-management.test.ts

    test('Scaling settings of load balancer', () => {
        const port = 80;
        const desiredCount = 2;
        const service = addLoadBalancedService(stack, 'test-service', cluster, taskdef, port, desiredCount, false);

        // Test code
        const config = {
            minCount: 1,
            maxCount: 5,
            scaleCpuTarget: { percent: 50 },
            scaleMemoryTarget: { percent: 50 },
        };

        setServiceScaling(service.service, config);


        // Check result
        const scaleResource = new Capture();
        const template = Template.fromStack(stack);
        template.resourceCountIs('AWS::ApplicationAutoScaling::ScalableTarget', 1);
        template.hasResourceProperties('AWS::ApplicationAutoScaling::ScalableTarget', {
            MaxCapacity: config.maxCount,
            MinCapacity: config.minCount,
            ResourceId: scaleResource,
            ScalableDimension: 'ecs:service:DesiredCount',
            ServiceNamespace: 'ecs',
        });

        template.resourceCountIs('AWS::ApplicationAutoScaling::ScalingPolicy', 2);
        template.hasResourceProperties('AWS::ApplicationAutoScaling::ScalingPolicy', {
            PolicyType: 'TargetTrackingScaling',
            TargetTrackingScalingPolicyConfiguration: Match.objectLike({
                PredefinedMetricSpecification: Match.objectEquals({
                    PredefinedMetricType: 'ECSServiceAverageCPUUtilization',
                }),
                TargetValue: config.scaleCpuTarget.percent,
            }),
        });

        template.hasResourceProperties('AWS::ApplicationAutoScaling::ScalingPolicy', {
            PolicyType: 'TargetTrackingScaling',
            TargetTrackingScalingPolicyConfiguration: Match.objectLike({
                PredefinedMetricSpecification: Match.objectEquals({
                    PredefinedMetricType: 'ECSServiceAverageMemoryUtilization',
                }),
                TargetValue: config.scaleMemoryTarget.percent,
            }),
        });
    });

Extract from container-management.ts


export interface ScalingThreshold {
    percent: number;
}
export interface ServiceScalingConfig {
    minCount: number;
    maxCount: number;
    scaleCpuTarget: ScalingThreshold;
    scaleMemoryTarget: ScalingThreshold;

}


export const setServiceScaling = function(service: FargateService, config: ServiceScalingConfig) {
    const scaling = service.autoScaleTaskCount({
        maxCapacity: config.maxCount,
        minCapacity: config.minCount,
    });

    scaling.scaleOnCpuUtilization('CpuScaling', {
        targetUtilizationPercent: config.scaleCpuTarget.percent,
    });

    scaling.scaleOnMemoryUtilization('MemoryScaling', {
        targetUtilizationPercent: config.scaleMemoryTarget.percent,
    });

}

Extract from my-container-infrastructure.ts

const service = addLoadBalancedService(stack, `service-${taskConfig.family}`, cluster, taskdef, 80, 2, true);
setServiceScaling(service.service, {
  minCount: 1,
  maxCount: 4,
  scaleCpuTarget: { percent: 50 },
  scaleMemoryTarget: { percent: 70 },
});

You can run npm test to make sure all tests pass, and run cdk deploy to see that it also deploys ok. We are at enough lines of code now that I would recommend to review the code at https://github.com/cloudgnosis/iac-ninja-aws-cdk-ts.

This is the end of part 7 of the infrastructure-as-code ninja series. If you enjoyed this material, you can check out the other parts of this series and other articles at Tidy Cloud AWS. Send comments, questions, and suggestions!

Infrastructure-as-code - where is the customer obsession, AWS?

Erik Lundevall Zara — Thu, 12 May 2022 21:23:40 +0000

AWS has a problem with its handling of customers’ adoption of infrastructure as code. This applies to CloudFormation, and by extension, to the AWS Cloud Development Kit (AWS CDK). This is part of a bigger problem with AWS, I think.

I am a fairly happy user of AWS CDK. It is a very nice tool in the toolbox for infrastructure-as-code. However, it also suffers from some drawbacks coming from CloudFormation, some of which will be mentioned here.

Meeting customers where they are

AWS prides themselves with customer obsession, and meeting customers where they are. They have a Well-Architected Framework which points to virtues of adaptability and automating things. Yet, for allowing customers to gradually move to a more automated and adaptable approach, there are sometimes too many hurdles in place.

It is, most times, fairly easy to use the AWS Console (ClickOps) to set up something with a particular AWS Service. There are also tutorials for using the AWS Console for many tasks.

Yet, there is zero support to get a corresponding representation in CloudFormation or AWS CDK from that ClickOps effort, so you can now manage those resources with infrastructure-as-code.

There is seldom any description or example to describe, in an easily digestible way, what you need to do to replicate that ClickOps experience.

I have yet to meet anyone using AWS that got a near-perfect infrastructure-as-code immutable infrastructure solution in place right away. There are always pain points and teachings on the way, and a process of adaption to new ways of working.

In addition, there is also the reality with operational incidents that need to be resolved quickly. This may most times involve some manual and direct updates, if you have not reached automation and immutable infrastructure nirvana yet. That is a reality.## Missing pieces in the puzzleYou cannot easily import existing resources into CloudFormation, and for a long time, it was not possible at all. Nowadays, you can import some existing resources into CloudFormation stacks, but it is difficult. It is also not supported for all CloudFormation resources either. There is a list in the AWS documentation of what resources are supported.It would have been better to have a list of what resources are not supported. This import feature only guarantees that required properties are the same as the actual resource when imported into CloudFormation.

The matter becomes even worse with AWS CDK since the import works on the CloudFormation level, which AWS CDK, to a large extent, abstracts away. It is then also potentially more complicated to replicate these resources through AWS CDK code.

There are tools, not from AWS, like Former 2, which can ease the pain somewhat, but it can only do so much and you get something less nice from a code perspective.If you are happy to replace all the existing resources, then this is less of a problem. But in many cases, this is not a good option if you still have a somewhat long way to go to immutable infrastructure nirvana.

Migration issues

Even migrating from CloudFormation to AWs CDK has its challenges. AWS CDK has some nice support to encapsulate existing CloudFormation in the AWS CDK code, which works pretty nicely. Just getting the AWS CDK command-line tool in the workflow is a nice bonus compared to what tooling is provided by AWS for dealing with CloudFormation directly.

You can keep the resources as-is with this approach, without replacing them. However, it becomes more challenging after that. You cannot easily break out parts of CloudFormation and replace it with AWS CDK code, without re-creating the infrastructure.

Again, not an issue with immutable infrastructure nirvana, but definitely a problem for many others.

How to set up ad hoc (developer) environments

Erik Lundevall Zara — Mon, 04 Apr 2022 10:00:37 +0000

Have you had a task to fix a supposedly simple thing in a project, a code or configuration change, only to spend most of the time to set up the environment so you can do the fix in the first place?

The setup you just did for this now clutter your setup, and conflicts with other projects? In order to get back to your old working environment, you have to do some re-installs of software packages?

One approach is to use containers from Docker images to package and run the specific tools. However, that can sometimes be cumbersome to set up and combine with tools that you already have.

The approach here does not use containers, and it can just be a matter of a one-line command to create a new shell environment. The extra software needed is in place as long as the shell is active only.

First, I will show an example, then look at how to set it up and get started.

For a project, we want to install Node.js 14 + Yarn + git + ripgrep, plus cowsay for good measure. First, just check what is on the machine before. After that, install the required tools and verify that they are available in our new shell. When we are done, we exit the shell and get the environment back to its old state.

bash-4.4# # First let us create a work directory
bash-4.4# mkdir work && cd work
bash-4.4# pwd
/work
bash-4.4# # We want to have Node.js 14.x, yarn, git, ripgrep and cowsay
bash-4.4# which node yarn git rg cowsay
which: no node in (/root/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/nix/var/nix/profiles/default/sbin)
which: no yarn in (/root/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/nix/var/nix/profiles/default/sbin)
/root/.nix-profile/bin/git
which: no rg in (/root/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/nix/var/nix/profiles/default/sbin)
which: no cowsay in (/root/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/nix/var/nix/profiles/default/sbin)
bash-4.4#       
bash-4.4# # None of the tools are installed 
bash-4.4# # Now we will set up the tools in our ad hoc environment
bash-4.4# nix-shell -p nodejs-14_x yarn git ripgrep cowsay
these 25 paths will be fetched (71.04 MiB download, 327.01 MiB unpacked):
  /nix/store/1l6jgfsgjmsrbrciz8r714dnkyngvpkb-zlib-1.2.11-dev
  /nix/store/1v0pmb9f8jmn126nvbs2k2varzqfqczq-icu4c-69.1
  /nix/store/2dd4n8cca16pa8yic1gh079hxb2x0ji4-yarn-1.22.10
  /nix/store/3k69hbxg04sdxlgi1236ddggs346sxf3-stdenv-linux
  /nix/store/4wf2bnddgwg17m465rg66j67gmcwy83q-bash-interactive-4.4-p23-info
  /nix/store/4xkhd51k7y6g9vi2h9kfvcxd67k3v3f1-gawk-5.1.0
  /nix/store/5ddb4j8z84p6sjphr0kh6cbq5jd12ncs-binutils-2.35.1
  /nix/store/5wvf6qy2pjijd93n4dn6vvbn45ij9fdr-linux-headers-5.12
  /nix/store/7al99yyrc2ix9c5idq2kf8zgksscaf9z-bash-interactive-4.4-p23-doc
  /nix/store/88ghxafjpqp5sqpd75r51qqg4q5d95ss-gcc-wrapper-10.3.0
  /nix/store/bjyyiipkwxgn8awrpp5aj3yp75l043i9-nodejs-14.17.6
  /nix/store/cr23xzsbbvjgxaf1mchn2drhh6yvsq0q-cowsay-3.03+dfsg2
  /nix/store/dr5yvz4dv2rq6mqr8pq5ix2g9wn4h5wp-icu4c-69.1-dev
  /nix/store/k6sslkn2f764mgnd2a6q0wfwlnfwpybi-ripgrep-12.1.1
  /nix/store/llywalnma9vv74z0xivwc87kxs6crk9b-binutils-wrapper-2.35.1
  /nix/store/mbc7y2k96nhgazi7w7hls8y4pbma1y82-diffutils-3.7
  /nix/store/p5lnl4zr45n7mf9kz9w8yz3rqh001b5c-bash-interactive-4.4-p23-dev
  /nix/store/q141hd8jl7in5223jmf7kmx9h517km4p-glibc-2.32-54-dev
  /nix/store/sa90ixnafl08k8d9wwmzhr7br2dyjxp2-expand-response-params
  /nix/store/sf6w5cyyksw7kacvhhw0aw0m1x74r524-ed-1.17
  /nix/store/shcm9y3r9xlx6pksg3kkkpi56vm5cgqg-libuv-1.41.0
  /nix/store/sjhz1j2d1ssn59f66kqp92xj9mpsww2d-gcc-10.3.0
  /nix/store/wfzdk9vxayfnw7fqy05s7mmypg5a8lyr-gnumake-4.3
  /nix/store/xfc0ffwz6s87mnz9x2k2qj6ykwxgwjxk-patchelf-0.12
  /nix/store/ywkvpy8va67s7z8i1m32mklar9mii8qd-patch-2.7.6
copying path '/nix/store/7al99yyrc2ix9c5idq2kf8zgksscaf9z-bash-interactive-4.4-p23-doc' from 'https://cache.nixos.org'...
copying path '/nix/store/p5lnl4zr45n7mf9kz9w8yz3rqh001b5c-bash-interactive-4.4-p23-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/4wf2bnddgwg17m465rg66j67gmcwy83q-bash-interactive-4.4-p23-info' from 'https://cache.nixos.org'...
copying path '/nix/store/5ddb4j8z84p6sjphr0kh6cbq5jd12ncs-binutils-2.35.1' from 'https://cache.nixos.org'...
copying path '/nix/store/cr23xzsbbvjgxaf1mchn2drhh6yvsq0q-cowsay-3.03+dfsg2' from 'https://cache.nixos.org'...
copying path '/nix/store/mbc7y2k96nhgazi7w7hls8y4pbma1y82-diffutils-3.7' from 'https://cache.nixos.org'...
copying path '/nix/store/sf6w5cyyksw7kacvhhw0aw0m1x74r524-ed-1.17' from 'https://cache.nixos.org'...
copying path '/nix/store/sa90ixnafl08k8d9wwmzhr7br2dyjxp2-expand-response-params' from 'https://cache.nixos.org'...
copying path '/nix/store/4xkhd51k7y6g9vi2h9kfvcxd67k3v3f1-gawk-5.1.0' from 'https://cache.nixos.org'...
copying path '/nix/store/wfzdk9vxayfnw7fqy05s7mmypg5a8lyr-gnumake-4.3' from 'https://cache.nixos.org'...
copying path '/nix/store/1v0pmb9f8jmn126nvbs2k2varzqfqczq-icu4c-69.1' from 'https://cache.nixos.org'...
copying path '/nix/store/shcm9y3r9xlx6pksg3kkkpi56vm5cgqg-libuv-1.41.0' from 'https://cache.nixos.org'...
copying path '/nix/store/dr5yvz4dv2rq6mqr8pq5ix2g9wn4h5wp-icu4c-69.1-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/5wvf6qy2pjijd93n4dn6vvbn45ij9fdr-linux-headers-5.12' from 'https://cache.nixos.org'...
copying path '/nix/store/ywkvpy8va67s7z8i1m32mklar9mii8qd-patch-2.7.6' from 'https://cache.nixos.org'...
copying path '/nix/store/q141hd8jl7in5223jmf7kmx9h517km4p-glibc-2.32-54-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/xfc0ffwz6s87mnz9x2k2qj6ykwxgwjxk-patchelf-0.12' from 'https://cache.nixos.org'...
copying path '/nix/store/llywalnma9vv74z0xivwc87kxs6crk9b-binutils-wrapper-2.35.1' from 'https://cache.nixos.org'...
copying path '/nix/store/sjhz1j2d1ssn59f66kqp92xj9mpsww2d-gcc-10.3.0' from 'https://cache.nixos.org'...
copying path '/nix/store/k6sslkn2f764mgnd2a6q0wfwlnfwpybi-ripgrep-12.1.1' from 'https://cache.nixos.org'...
copying path '/nix/store/88ghxafjpqp5sqpd75r51qqg4q5d95ss-gcc-wrapper-10.3.0' from 'https://cache.nixos.org'...
copying path '/nix/store/1l6jgfsgjmsrbrciz8r714dnkyngvpkb-zlib-1.2.11-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/3k69hbxg04sdxlgi1236ddggs346sxf3-stdenv-linux' from 'https://cache.nixos.org'...
copying path '/nix/store/bjyyiipkwxgn8awrpp5aj3yp75l043i9-nodejs-14.17.6' from 'https://cache.nixos.org'...
copying path '/nix/store/2dd4n8cca16pa8yic1gh079hxb2x0ji4-yarn-1.22.10' from 'https://cache.nixos.org'...

[nix-shell:/work]# 

[nix-shell:/work]# which node yarn git rg cowsay
/nix/store/bjyyiipkwxgn8awrpp5aj3yp75l043i9-nodejs-14.17.6/bin/node
/nix/store/2dd4n8cca16pa8yic1gh079hxb2x0ji4-yarn-1.22.10/bin/yarn
/nix/store/cpx9hzqcxci468qjwx1yk9k8rwmhlqxl-git-2.31.1/bin/git
/nix/store/k6sslkn2f764mgnd2a6q0wfwlnfwpybi-ripgrep-12.1.1/bin/rg
/nix/store/cr23xzsbbvjgxaf1mchn2drhh6yvsq0q-cowsay-3.03+dfsg2/bin/cowsay

[nix-shell:/work]# git init
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint: 
hint:   git config --global init.defaultBranch <name>
hint: 
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint: 
hint:   git branch -m <name>
Initialized empty Git repository in /work/.git/

[nix-shell:/work]# cowsay Now we have the tools for our environment in place
 _______________________________ 
/ Now we have the tools for our \
\ environment in place          /
 ------------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

[nix-shell:/work]# exit
exit
bash-4.4# which node yarn git rg cowsay
which: no node in (/root/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/nix/var/nix/profiles/default/sbin)
which: no yarn in (/root/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/nix/var/nix/profiles/default/sbin)
/root/.nix-profile/bin/git
which: no rg in (/root/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/nix/var/nix/profiles/default/sbin)
which: no cowsay in (/root/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/nix/var/nix/profiles/default/sbin)
bash-4.4# 
bash-4.4# # We have left the shell we created with nix-shell, and the tools are not there anymore 
bash-4.4# # One more time...
bash-4.4# 
bash-4.4# nix-shell -p nodejs-14_x yarn git ripgrep cowsay

[nix-shell:/work]# cowsay All packages cached locally, so quicker this time
 _________________________________________ 
/ All packages cached locally, so quicker \
\ this time                               /
 ----------------------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

[nix-shell:/work]# exit
exit
bash-4.4#

So what is this nix-shell? It is a command-line tool which is part of a group of tools, under the name Nix. Nix is a few different things:

A package manager
A domain specific language for building, packaging and deploying software
A Linux distribution using the Nix package manager, which is called NixOS.

Nix is very much about creating reproducible and declarative deployments and system configurations, with tooling to support these tasks. One outcome of this focus is to have i*mmutable configurations* - you are not destroying any particular resources, just adding to what is already there.

You do not have to run NixOS to take advantage of other features of Nix. In fact, the tools work on other Linux distributions, on macOS or in WSL 2 (Windows Subsystem for Linux) on Windows machines. You just install Nix to get a couple of command-line tools.

The Nix project has been around for a while, about 19 years. However, a large portion of that time, its usage was quite small. It has only grown more significantly in the last few years.

A good starting point is the NixOS website. The site covers Nix and a tool and you do not need to set up NixOS to get started with it. It outlines some of the use cases for Nix, of which ad hoc environments are just one of them.

You find the installation instructions for Nix here. This also includes running it as a container via Docker, which is a good way to get an idea of what it is all about, before actually installing the software itself. The example above was running in an instance of this docker image, in fact. The recommendation is to do a multi-user installation if you use Linux or macOS, while use a single-user installation under Windows WSL2.

It is essentially running a shell command on your local computer to download and run an installation script:

For macOS:

sh <(curl -L https://nixos.org/nix/install)

For Windows (WSL2), single-user installation:

sh <(curl -L https://nixos.org/nix/install) --no-daemon

For Linux, single user installation is same as above. For multi-user installation:

sh <(curl -L https://nixos.org/nix/install) --daemon

If you want to run the docker image instead and have Docker desktop installed, you can run:

docker run -it nixos/nix

or run it to expose a selected directory (workdir) on your local computer:

docker run -it -v $(pwd)/workdir:/workdir nixos/nix

The first time you run the command it will take a bit of time, because it will download the and setting up the specified packages from NixPkgs, which have over 80000 packages at this point in time. If you re-run the command again later, it will be faster, since the packages have been cached locally.

Tweak and persist the package information

For the simplest use cases, just specifying the package labels works fine. However, tweak it a bit more, depending on which packages to install and any additional settings that may be needed. For this purpose, there is also the Nix domain specific language, which is geared towards repeatable and/or reproducible setups.

Let us create a directory work, and in that directory, create a file mysetup.nix. The content will look like this:

let
    getpkgs = import <nixpkgs>;
    pkgs = getpkgs {};
in
    pkgs.mkShell {
        buildInputs = [
            pkgs.git
            pkgs.nodejs-14_x
            pkgs.yarn
            pkgs.ripgrep
            pkgs.cowsay
        ];
        shellHook = ''
            cowsay "here is your shell environment"
        '';
    }

What this piece of code will do is to retrieve information about all available packages in the latest update of NixPkgs. Using the retrieved data, it will create a shell environment (mkShell), where there packages we want to install/build are inputs. As a bonus here, we also add an execution of cowsay once the environment is ready.

Now we can set up the environment using the command

nix-shell mysetup.nix

In the work directory, we should see something like this:

bash-4.4# nix-shell mysetup.nix 
 ________________________________ 
< here is your shell environment >
 -------------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

[nix-shell:/work]#

This makes for a repeatable setup, but is not reproducible. This is because the latest versions of the different packages might be different if the latest versions of these packages are updated. To get the same versions each time, we need to be specific about which release of NixPkgs we are using. Nix has multiple ways to get a specific release of NixPkgs. Since NixPkgs information is available through a repository in GitHub, we can fetch a specific release and git commit from that repository. In that case, we will always get the same releases for the packages, as well as the same dependencies for all these packages. We cannot choose specific versions for each package, since there is an infinite number of combinations with these packages, including their dependencies in that case. Repeatable and reproducible environments are of higher importance here. We will get the specific versions of the packages and their dependencies that are in that specific release of NixPkgs.

Let us create a new file, this time called shell.nix. The content is adjusted slightly, so that we fetch the package data from the NixPkgs repository in GitHub, with a specific tag or branch and commit that is associated with that. One way to get the specific commit is to use the git ls-remote command:

git ls-remote --tags https://github.com/nixos/nixpkgs/ 21.11
506445d88e183bce80e47fc612c710eb592045ed        refs/tags/21.11

Instead of a general reference to "<nixpkgs>" we make a more specific reference, using the fetchGit function. The other parts of the code stay the same.

let
    nixpkgs = (fetchGit {
        name = "nixos-release-21.11";
        url = "https://github.com/nixos/nixpkgs/";
        ref = "refs/tags/21.11";
        rev = "506445d88e183bce80e47fc612c710eb592045ed";
    });
    getpkgs = import nixpkgs;
    pkgs = getpkgs {};
in
    pkgs.mkShell {
        buildInputs = [
            pkgs.git
            pkgs.nodejs-14_x
            pkgs.yarn
            pkgs.ripgrep
            pkgs.cowsay
        ];
        shellHook = ''
            cowsay "here is your shell environment"
        '';
    }

Now, because we have named the file shell.nix, we do not need to specify the filename when running, we can simply run nix-shell. Again, the first time you run the command, it will take some time to fetch the package information (be patient!), but if that is run again, it will be cached and execution is much faster.

Final words

This was a quick introduction to a specific tool within Nix, the nix-shell. It is a nice way to create ad hoc environments in a repeatable and/or reproducible way. It is only scratching the surface of Nix, though, and there is a lot more to explore there.

I must stay though that the documentation around Nix and its usage leaves a bit to be desired. It can sometimes be a bit confusing to figure out what you can do and how to do it. Right now I hope to make some examples and descriptions as I learn and explore more of Nix, which I think is very promising, despite some challenges.

Take a cue to supercharge your configurations

Erik Lundevall Zara — Fri, 04 Mar 2022 16:51:28 +0000

Do you have a love/hate relationship with YAML or JSON for configurations?

Are your configurations getting messed up because of simple typos, or problems with white space, or confusion between strings and numbers?

Do you try to manage some templating solution on top of YAML?

Is it hard to manage complex overrides across multiple configuration files?

If any of these questions resonate with you, then this article may interest you!

Configuration complexity

In today's configuration landscape, YAML is pretty much everywhere. It is relatively easy for humans to read and also fairly compact format. It is also easy to mess up with some idiosyncrasies and white-space/indentation dependent structure. JSON is a simpler format in its structure, but more limited and not really intended for human consumption. TOML is another format which tries to strike a balance for readability and maintainability.

Neither of them have any built-in capabilities for elaborate validation of data or feature to facilitate organising (complex) configurations. It gets messy, in particular with more complex configurations.

Can we do better? Yes, I believe we can. There is one effort in this space that I think is promising, and that is CUE (Configure Unify Execute), an open-source configuration language.

There is a lot to like about CUE, and I will explore a few bits and pieces of it in this article. It is only scratching the surface of what you can do with CUE though.

We are going to focus on a bit of data validation and simple conversions:

Convert a YAML configuration file to CUE
Step-by-step define schema and validation rules for that configuration file

Install CUE

The CUE website has installation instructions for how to install CUE. There are binary downloads you can download for macOS, Windows, and Linux distributions, or you can install via Homebrew on macOS and Linux.

brew install cue-lang/tap/cue

If you have the Go language installed, you can also download and build CUE directly easily, since CUE is written in Go.

You do not have to install CUE now if you do not want to follow the rest of the article, but if you want to play around yourself, you should do that.

You can also have a look at the CUE Playground. This is only for writing CUE and possibly converting it to JSON or YAML, so that is different use case than what we are covering in this article.

Read and convert YAML configuration

We are going to start with a relatively small configuration file in YAML. This configuration describes a network configuration (VPC - Virtual Private Cloud) in Amazon Web Services (AWS). It is not important exactly what it is though; we are only using this for illustration. Our YAML configuration file has the name network_config.yaml and looks like this:

account: "123456789012"
region: eu-north-1
network:
  vpc_cidr: 10.100.0.0/16
  az_count: 2
  nat_gateway_count: 1
  public_mask: 25
  private_subnets:
    - name: private1
      mask: 21
  isolated_subnets:
    - name: isolated1
      mask: 21

AWS has then notion of organising cloud resources into accounts and regions, so this is something that is relevant for provisioning cloud resources. The rest of the structure has a mix of networking related information, essentially telling us how a network configuration should be set up.

The configuration has a mix of single values, some in a hierarchy, some in lists/arrays. There is a mix of strings and numbers. There are restrictions on what may be valid values for several fields. All of that is invisible to us in the YAML file, though.

Let us start by converting this YAML into CUE. We can do this with the cue import command:

cue import network_config.yaml

This will produce a network_config.cue file, with the same content, which looks like this:

account: "123456789012"
region:  "eu-north-1"
network: {
        vpc_cidr:          "10.100.0.0/16"
        az_count:          2
        nat_gateway_count: 1
        public_mask:       25
        private_subnets: [{
                name: "private1"
                mask: 21
        }]
        isolated_subnets: [{
                name: "isolated1"
                mask: 21
        }]
}

As you can see, it looks fairly similar. CUE is a superset of JSON, any valid JSON is also valid CUE. However, there is a lot more to CUE, and it has a more readable syntax than plain JSON and does not have white-space/indentation formatting of YAML. It does not require commas between key-value pairs either.

One thing to note here also is that string values are quoted, always - like JSON and unlike YAML.

We will use this generated as a starting point to define data validation for our YAML configuration.

Define data validation

The CUE language has an unusual property which sets it apart from many other languages and formats - types and values are the same. Not the same as only same type of structure, like JSON and JSON Schema, but actually the same.

Our CUE file is just as much a schema definition as it is actual data values. The CUE command-line tool has the command cue vet to perform validation. You can provide both CUE and non-CUE files (e.g. JSON or YAML) to validate. So if we want to validate our YAML configuration towards the CUE data, we can run the command.

❯ cue vet network_config.cue network_config.yaml

There is no output, which means it validated ok. You might think, how do I really know it has performed some validation here? Let us just change a value in the CUE file, for example, the "eu-north-1" value to "eu-west-1", then run the cue vet command again.

❯ cue vet network_config.cue network_config.yaml
region: conflicting values "eu-west-1" and "eu-north-1":
    ./network_config.cue:2:10
    ./network_config.yaml:2:10

As you can see, it now reports that there is a conflict, and points out where the data is conflicting in these files. So the validation works.

It is meaningless, though, to only have a hard-coded configuration to validate against. So let us generalise this a bit!

For the region parameter, let us decide that we only allow the values "eu-north-1" and "eu-west-1"
For other values, let us define general string and integer types.

The updated CUE file looks like below then. We have not added string and int types, and we have with the "|" character told CUE that either "eu-west-1" or "eu-north-1" are valid values, but nothing else.

account: string
region:  "eu-west-1" | "eu-north-1"
network: {
    vpc_cidr:          string
    az_count:          int
    nat_gateway_count: int
    public_mask:       int
    private_subnets: [{
        name: string
        mask: int
    }]
    isolated_subnets: [{
        name: string
        mask: int
    }]
}

If we run cue vet on this CUE file and the YAML file, we can see that it is still valid. In CUE, we can mix actual values with type definitions, that is all the same.

Reusable definitions and lists

We can see in the structure that we have two lists/array structures under private_subnets and isolated_subnets. The data structures in each of these are the same for both as well. Right now, if we would add another element in either of these lists in the YAML configuration, the validation would fail. So how can we make a reusable definition for the structure in the lists and also allow an arbitrary number of them in each list?

In CUE, a reusable description of a structure is called a Definition, and it looks like any other structure, only that the name itself is prefixed with "#" (or "_#", but that is out of scope for now). We can in the list use a spread operator "..." to define a variable number of elements in the list. Our new CUE file looks like this now:

account: string
region:  "eu-west-1" | "eu-north-1"
network: {
    vpc_cidr:          string
    az_count:          int
    nat_gateway_count: int
    public_mask:       int
    private_subnets: [...#Subnet]
    isolated_subnets: [...#Subnet]
}

#Subnet: {
    name: string
    mask: int
}

You can run cue vet to check that it is still valid. In this case, we put the #Subnet definition after the rest, we just as well put it first as well. The order does not matter to CUE. This property becomes important when we organise our data in different ways.

Adding constraints

Next step is to add additional constraints on some fields. The integer fields we use cannot have any integer value. We have a few constraints we would like to impose on these fields:

The az_count fields refers to the number of Availability Zones that our AWS network setup should include. In most AWS regions, this would be a number between 1 and 3.
The nat_gateway_count field refers to the number of NAT Gateways provisioned. This is an infrastructure component that may be the same value as the number of availability zones, but could also be fewer. It should not be larger than the number of availability zones.
The mask and public_mask fields can only have integer values between 16 and 28. This is a hard limitation set by AWS.

Besides types and values, we can also add constraints to our fields. The above rules can be described in the example below:

account: string
region:  "eu-west-1" | "eu-north-1"
network: {
    vpc_cidr:          string
    az_count:          int & >=1 & <=3
    nat_gateway_count: int & >=1 & <=az_count
    public_mask:       #Netmask
    private_subnets: [...#Subnet]
    isolated_subnets: [...#Subnet]
}

#Netmask: int & >=16 & <=28

#Subnet: {
    name: string
    mask: #Netmask
}

We added another reusable definition for the mask value, since that is used in two different fields. The constraints can be combined with the type data, or actual values. Multiple types/constraints/values for a field can be combined with the "&" symbol.

The syntax is concise, and also easy to read.

Now let us look at the strings. There are two fields, account and vpc_cidr, which we should impose some constraints on. An AWS account is a string value that always comprises 12 digits. The VPC CIDR value is a combination of an IP address, a slash, and a net mask value. One way to approach the constraints here is to use regular expressions to define what is allowed. If you are not familiar with regular expression syntax, this will be a bit like keyboard noise to you. Fear not, you are not alone! It is useful to define restrictions on string patterns, and we use that to define the constraints on account and vpc_cidr.

account: string & =~"^[0-9]{12}$"
region:  "eu-west-1" | "eu-north-1"
network: {
    vpc_cidr:          #CIDR
    az_count:          int & >=1 & <=3
    nat_gateway_count: int & >=1 & <=az_count
    public_mask:       #Netmask
    private_subnets: [...#Subnet]
    isolated_subnets: [...#Subnet]
}

#CIDR: string & =~"^10\\.(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.){2}(0|16|32|64|128)(\\/(2[0-8]|1[6-9]))$"

#Netmask: int & >=16 & <=28

#Subnet: {
    name: string
    mask: #Netmask
}

You can run cue vet and see that it still validates. Try also to change any of the values that we have put constraints on to see what the result will be if we violate the constraints and run cue vet.

Let us do one more modification here, set a default value. For the vpc_cidr field, if no value is specified, with will use the string "10.0.0.0/16" as the default value. We use an asterisk to denote a default value:

account: string & =~"^[0-9]{12}$"
region:  "eu-west-1" | "eu-north-1"
network: {
    vpc_cidr:          *"10.0.0.0/16" | #CIDR
    az_count:          int & >=1 & <=3
    nat_gateway_count: int & >=1 & <=az_count
    public_mask:       #Netmask
    private_subnets: [...#Subnet]
    isolated_subnets: [...#Subnet]
}

#CIDR: string & =~"^10\\.(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.){2}(0|16|32|64|128)(\\/(2[0-8]|1[6-9]))$"

#Netmask: int & >=16 & <=28

#Subnet: {
    name: string
    mask: #Netmask
}

We have dipped our toes into how to define schemas and constraints with CUE. There is more to explore here, however, these basics can get you a good start.

Enforcing validation

We have validated our YAML configuration, and this has worked fine. However, if we add another dummy entry to our YAML configuration and run cue vet on that, it will still work fine.

account: "123456789012"
region: eu-north-1
stuff: 333  ################### Our extra stuff here ###########
network:
  vpc_cidr: 10.100.0.0/16
  az_count: 2
  nat_gateway_count: 1
  public_mask: 25
  private_subnets:
    - name: private1
      mask: 21
  isolated_subnets:
    - name: isolated1
      mask: 21

❯ cue vet network_config.cue network_config.yaml

As long as CUE finds valid and non-conflicting data structures, it is all good. If we want to restrict the validation to only accept specific fields, we need to be explicit about what to validate. Let us just add a layer in our schema definition that includes the three top level fields we have, account, region and network.

#Config: {
    account: string & =~"^[0-9]{12}$"
    region:  "eu-west-1" | "eu-north-1"
    network: {
        vpc_cidr:          *"10.0.0.0/16" | #CIDR
        az_count:          int & >=1 & <=3
        nat_gateway_count: int & >=1 & <=az_count
        public_mask:       #Netmask
        private_subnets: [...#Subnet]
        isolated_subnets: [...#Subnet]
    }
}

#CIDR: string & =~"^10\\.(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.){2}(0|16|32|64|128)(\\/(2[0-8]|1[6-9]))$"

#Netmask: int & >=16 & <=28

#Subnet: {
    name: string
    mask: #Netmask
}

By defining reusable definition, we can also be specific in what should be included in the non-CUE data we validate when we run cue vet:

❯ cue vet --schema '#Config' network_config.cue network_config.yaml
field not allowed: stuff:
    ./network_config.yaml:3:2
    ./network_config.cue:1:1
    ./network_config.cue:1:10

Now our extra stuff is not accepted. Sometimes, though, we want only partial validation and allow for extra items to be present. If our schema definition work is a work in progress, that may very well be the case. In those cases, we can add "..." to our definition, which makes the structure open to other fields, as opposed to the default closed structure definition.

#Config: {
    account: string & =~"^[0-9]{12}$"
    region:  "eu-west-1" | "eu-north-1"
    network: {
        vpc_cidr:          *"10.0.0.0/16" | #CIDR
        az_count:          int & >=1 & <=3
        nat_gateway_count: int & >=1 & <=az_count
        public_mask:       #Netmask
        private_subnets: [...#Subnet]
        isolated_subnets: [...#Subnet]
    }
    ...
}

#CIDR: string & =~"^10\\.(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.){2}(0|16|32|64|128)(\\/(2[0-8]|1[6-9]))$"

#Netmask: int & >=16 & <=28

#Subnet: {
    name: string
    mask: #Netmask
}

Running cue vet with this schema definition will accept our extra stuff.

Final words

This has been an introduction to CUE, a quite versatile configuration language which also can play well with existing tooling. This has only scratched the surface of CUE though, and there is a lot more to it!

There are some neat material on CUE and some good presentations, at different levels - a few of these can be found here:

https://fosdem.org/2022/schedule/event/cue_pratical_guide/

https://cuelang.org

https://cuetorials.com

How to become an infrastructure-as-code ninja, using AWS CDK - part 6

Erik Lundevall Zara — Tue, 15 Feb 2022 12:22:30 +0000

In this article, we are continuing with building our AWS infrastructure, which is a container-based solution running in AWS Elastic Container Service (AWS ECS).

Back in part 5, we set up an Apache web server-based solution running in a container, in a container cluster using AWS ECS, using Typescript as a language to describe the infrastructure. Building the solution, we divided the solution into two separate files:

bin/my-container-infrastructure.ts - this is the main program for describing our infrastructure
lib/containers/container-management.ts - this contains support functions to describe container-based infrastructure

Before adding new feature to our solution, we are going to take a step back and look at what we can do to organise and test the building blocks in our infrastructure-as-code solution. In particular, testing is something we want to keep in mind right from the start ideally.

We will look at how to incorporate testing into what we already have and then continue to add new infrastructure and testing as we move ahead.

Warning! The built-in testing support libraries in AWS CDK expect you to know AWS CloudFormation. Some familiarity of CloudFormation is recommended if you use these.

Update:

This article series uses Typescript as an example language. However, there are repositories with example code for multiple languages.

The repositories will contain all the code examples from the articles series, implemented in different languages with the AWS CDK. You can view the code from specific parts and stages in the series, by checking out the code tagged with a specific part and step in that part, see the README file in each repository for more details.

Different testing

What do we mean by testing? There are various aspects to consider, which include:

That we get the infrastructure we expect to have
That the solution and its infrastructure adhere to any security and compliance policies in place
That the solution itself works as expected when deployed with the infrastructure

The testing aspect we will focus on here in this article is the first one, that we get the infrastructure we expect to have. This may include both actual resources that will be provisioned, that t*hese resources have expected properties* and that relations between resources are as expected. Also, that we do not introduce unexpected changes.

We will treat these tests as unit tests essentially, so they will run in our (local) development environment, and will run in the order of seconds (or minutes).

Who writes the tests?

It depends a bit on how you have the infrastructure-as-code work organised, but the people that build and maintain (re-usable) infrastructure building blocks should write tests for those building blocks.

That may mean every developer, or specific platform developers or other groups of people. For YAML/JSON-based infrastructure using CloudFormation, there is limited support for testing and validation of the infrastructure. Frankly, the need may sometimes be limited, since you in those cases also just declare what you want to have, there may not be that much logic to test. However, when you get enough logic and conditions included with CloudFormation YAML/JSON, it can get quite messy.

If you use programming languages and AWS CDK, you get a more imperative layer to generate the declarative model. This can both make it easier to make it clear what is intended, but also make it more complex to understand exactly what infrastructure you will get.

Get started with writing tests

Enough preparation talk now, let us get into practical work! If you have followed along in this article series to set up the project using the cdk init command for a Typescript project, you will already have some test tooling in place. AWS CDK includes the Jest test framework by default in the installation.

You can use whichever testing framework you want, though, with the testing support provided in AWS CDK. The examples we will build here will use Jest though, since that it what is set up by default in an AWS CDK project.

If you have set up the AWS CDK project as described in part 4 and part 5 of this series, you will have a directory in your project folder named test. This is where will take place or tests to write. Since we have cheated and not practiced test-driven development (TDD) right from the start, we will build some test for the existing infrastructure we have defined, before moving further with new infrastructure.

Infrastructure recap

Let us first re-cap what we had built so far in the two source files in our project:

bin/my-container-infrastructure.ts

import { App, Stack } from 'aws-cdk-lib';
import { Vpc } from 'aws-cdk-lib/aws-ec2';
import { 
  addCluster, 
  addService,
  addTaskDefinitionWithContainer, 
  ContainerConfig, 
  TaskConfig 
} from '../lib/containers/container-management';

const app = new App();
const stack = new Stack(app, 'my-container-infrastructure', {
  env: {
    account: process.env.CDK_DEFAULT_ACCOUNT,
    region: process.env.CDK_DEFAULT_REGION,
  },
});

const vpc = Vpc.fromLookup(stack, 'vpc', {
  isDefault: true,
});

const id = 'my-test-cluster';
const cluster = addCluster(stack, id, vpc);

const taskConfig: TaskConfig = { cpu: 512, memoryLimitMB: 1024, family: 'webserver' };
const containerConfig: ContainerConfig = { dockerHubImage: 'httpd' };
const taskdef = addTaskDefinitionWithContainer(stack, `taskdef-${taskConfig.family}`, taskConfig, containerConfig);
addService(stack, `service-${taskConfig.family}`, cluster, taskdef, 80, 2, true);

lib/containers/container-management.ts

import { IVpc, Peer, Port, SecurityGroup } from 'aws-cdk-lib/aws-ec2';
import { Cluster, ContainerImage, FargateService, FargateTaskDefinition, LogDriver, TaskDefinition } from 'aws-cdk-lib/aws-ecs';
import { RetentionDays } from 'aws-cdk-lib/aws-logs';
import { Construct } from 'constructs';

export const addCluster = function(scope: Construct, id: string, vpc: IVpc): Cluster {
    return new Cluster(scope, id, {
        vpc,
    });
}

export interface TaskConfig {
    readonly cpu: 256 | 512 | 1024 | 2048 | 4096;
    readonly memoryLimitMB: number;
    readonly family: string;
}

export interface ContainerConfig {
    readonly dockerHubImage: string;
}

export const addTaskDefinitionWithContainer = 
function(scope: Construct, id: string, taskConfig: TaskConfig, containerConfig: ContainerConfig): TaskDefinition {
    const taskdef = new FargateTaskDefinition(scope, id, {
        cpu: taskConfig.cpu,
        memoryLimitMiB: taskConfig.memoryLimitMB,
        family: taskConfig.family,
    });

    const image = ContainerImage.fromRegistry(containerConfig.dockerHubImage);
    const logdriver = LogDriver.awsLogs({ 
        streamPrefix: taskConfig.family,
        logRetention: RetentionDays.ONE_DAY,
    });
    taskdef.addContainer(`container-${containerConfig.dockerHubImage}`, { image, logging: logdriver });

    return taskdef;
};

export const addService = 
function(scope: Construct, 
         id: string, 
         cluster: Cluster, 
         taskDef: FargateTaskDefinition, 
         port: number, 
         desiredCount: number, 
         assignPublicIp?: boolean,
         serviceName?: string): FargateService {
    const sg = new SecurityGroup(scope, `${id}-security-group`, {
        description: `Security group for service ${serviceName ?? ''}`,
        vpc: cluster.vpc,
    });
    sg.addIngressRule(Peer.anyIpv4(), Port.tcp(port));

    const service = new FargateService(scope, id, {
        cluster,
        taskDefinition: taskDef,
        desiredCount,
        serviceName,
        securityGroups: [sg],
        circuitBreaker: {
            rollback: true,
        },
        assignPublicIp,
    });

    return service;
};

We will start with the functions we have defined in lib/containers/container-management.ts, addCluster, addService, and addTaskDefinitionWithContainer. We will use the assertions module provided with AWS CDK for your testing, and use Jest to define the tests.

Let us write the first test

To build our first tests, let us create a new file in the test directory, called /containers/container-management.test.ts and add our test code there. We will start with a single test for the addCluster function and look at how that is built up:

import { Stack } from 'aws-cdk-lib';
import { Vpc } from 'aws-cdk-lib/aws-ec2';
import { Template } from 'aws-cdk-lib/assertions';
import { addCluster } from '../../lib/containers/container-management';

test('ECS cluster is defined with existing vpc', () => {
    // Test setup
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');

    // Test code
    const cluster = addCluster(stack, 'test-cluster', vpc);

    // Check result
    const template = Template.fromStack(stack);

    template.resourceCountIs('AWS::ECS::Cluster', 1);

    expect(cluster.vpc).toEqual(vpc);
});

We include the assertions sub-module from AWS CDK, which has features to generate CloudFormation templates from different sources, and then perform tests on these templates.

To use this, we need to create a stack, so we import that as well. Since our addCluster function requires some kind of construct, an identifier and a reference to a Vpc construct, we will create an provide that. The stack is possible to create with no AWS CDK App, or even an identifier, so we will just create an empty stack.

The actual test code is to call the addCluster function and pick up the resulting cluster object. What are we expecting the result will be?

We expect that an ECS cluster has been added to the stack we supply, and the provided VPC parameter is included with the cluster.

So testing this, we check two things:

There is a CloudFormation AWS::ECS::Cluster resource in the stack
The returned cluster object contains a reference to the provided Vpc object.

Note here that in CloudFormation, the ECS cluster (AWS::ECS::Cluster) does not have a reference to a VPC. This is something we can see if we look in the AWS CloudFormation documentation for the AWS::ECS::Cluster resource.This is purely something that the AWS CDK itself has added, for use later with other constructs. Besides checking that there is an AWS::ECS::Cluster in the stack, we currently do not care more about any details. So for us it suffices to check that the cluster resource is in the stack, and that we have one of it.

The AWS CDK Cluster object should have a reference to a Vpc though and it should be the one we provide to the addCluster function. So we simply test that and use the Jest expect() function to test this.

We can run the test with the command npm test and see what we get:

❯ npm test

> my-container-infrastructure@0.1.0 test
> jest

 PASS  test/containers/container-management.test.ts (13.649 s)
  ✓ ECS cluster is defined with existing vpc (125 ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        13.739 s, estimated 15 s
Ran all test suites.

Success! If this would have been a proper test-driven development cycle, we would go through a different feedback loop here, though. For now, we are mainly catching up a bit, though.

Our first test case involves both a check that goes down into the generated CloudFormation, and another test which checks the state of a higher level construct. These are both valid types of tests to do, and to what extent you do explicit CloudFormation tests depends on the use case. If you build your own high-level construct from direct CloudFormation resources, it makes sense to do a lot of lower-level testing. If you are combining higher-level constructs, then there might not be the same need.

Task definition testing

Our next test target is the addTaskDefinitionWithContainer function, which should create an ECS Fargate task definition and associate a container from DockerHub with it. From that description, we can think about what we could test.

The function signature says it returns a TaskDefinition. We can look for ways that the returned task definition says it will be used with Fargate in its interface.
The task definition should have the family, cpu and memory settings we have provided
We can also check the underlying CloudFormation AWS::ECS::TaskDefinition that is has been created and has the expected properties.
We also need to check that the container we provide has been added to the task definition. We can check if we can use the TaskDefinition object returned to check that.
We can also check the underlying CloudFormation for an appropriate AWS::ECS::ContainerDefinition as well.

We can do testing on the higher level constructs the AWS CDK provides, or we can do more low-level testing on the generated CloudFormation.

Initially, when I first saw the assertions support functions provided for the AWS CDK, my mind was very much set on testing a lot of CloudFormation details. But I have changed my mind there. If I am building higher-level constructs from other high-level constructs in AWS CDK, the need to check explicitly generated CloudFormation is slightly limited. If you build your own constructs which use resources that map directly to CloudFormation resource, then it is very useful to check the generated CloudFormation. In other cases, that is only partially true. Look at what to can check from the construct interfaces first, and if that is not sufficient, then go to the CloudFormation-oriented tests.

I want to consider CloudFormation an implementation detail of the AWS CDK, and preferably not think about it if I can. It cannot be avoided currently though, and in practice you will have to deal with it sometimes.

Test for Fargate Task Definition

Let us take a first stab at the tests for addTaskDefinitionWithContainer and check that we have a task definition that is Fargate compatible.

test('ECS Fargate task definition defined', () => {
    // Test setup
    const stack = new Stack();
    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName };

    // Test code
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    // Check result
    const template = Template.fromStack(stack);

    expect(taskdef.isFargateCompatible).toBeTruthy();
    expect(stack.node.children.includes(taskdef)).toBeTruthy();

    template.resourceCountIs('AWS::ECS::TaskDefinition', 1);
    template.hasResourceProperties('AWS::ECS::TaskDefinition', {
        RequiresCompatibilities: [ 'FARGATE' ],
        Cpu: cpuval.toString(),
        Memory: memval.toString(),
        Family: familyval,
    });

});

Again, in this test, we use both higher-level tests and some low-level CloudFormation tests. We can check directly that the returned task definition is Fargate compatible and we can check that it has been added to the stack without resorting to checking the CloudFormation.

We can also as before, check at CloudFormation level that one Task Definition has been added. The TaskDefinition interface does not allow us to check for the cpu, memory limit and family values, though, so in this case we would need to dive into the actual CloudFormation. The Template.hasResourceProperties() function is quite useful for that. We can specify the properties we expect to find in the resource, and only the properties we are interested in. The other properties we do not need to care about.

So we added a check for the cpu, memory and family settings to verify that those are in place.

Note: If you look at the test code, you see that the Cpu and Memory values are converted to strings. In the CloudFormation documentation examples, these values are numbers. However, according to the CloudFormation specification, the values are strings. The AWS CDK generates the direct CloudFormation resources from the specification. So if there is a discrepancy, the AWS CDK is likely handling it correctly.

Test for container definition

Let us add another test to check that the container definition is added to the task definition. Our function creates a task definition with a single container. The TaskDefinition construct can provide a reference to the default container definition, so it makes sense to check that this is in place - there is at least some container definition in place.

However, the container definition provided by the AWS CDK does not allow us (easily) to check what the image reference is. So in this case, we might complement this with a CloudFormation-oriented test. In this case, we will still check the AWS::ECS::TaskDefinition, but we will look at a different part of the structure.

Update 2022-02-10: From version 2.11 of aws-cdk-lib, the image name is available from the container definition. A check for this is added in the test, without removing the CloudFormation check, as it illustrates nested matching as well.

In the previous test case, we could just enter the properties we wanted to match with. Here, we will go deeper into the CloudFormation resource, so we have to be a bit more explicit about the type of matching to do.

test('Container definition added to task definitio', () => {
    // Test setup
    const stack = new Stack();
    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName };

    // Test code
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    // Check result
    const template = Template.fromStack(stack);
    const containerDef = taskdef.defaultContainer;

    expect(taskdef.defaultContainer).toBeDefined();
    expect(containerDef?.imageName).toEqual(imageName); // Works from v2.11 of aws-cdk-lib
    template.hasResourceProperties('AWS::ECS::TaskDefinition', {
        ContainerDefinitions: Match.arrayWith([
            Match.objectLike({
                Image: imageName,
            }),
        ]),
    });
});

The functions in the Match class provide different features to use. Match.objectLike() is the same as we did implicitly at the top level in the previous test. Match.arrayWith() allows us to check that there is an element in an array that matches what we are looking for. These functions help us check that there is a container definition inside the task definition, and it refers to the DockerHub image we provided.

Test the service

The last function to test here is addService(). This is a function that ties our previously defined resources together and adds something we will spin up in a cluster and actually run. We provide a port that should be available to access our service on, and we provide a desired count for the container to run, plus tie all the pieces together.

Based on this information and what we have implemented, we can create a test like this:

test('Fargate service created, with provided mandatory properties only', () => {
    // Test setup
    const stack = new Stack();
    const vpc = new Vpc(stack, 'vpc');
    const cluster = addCluster(stack, 'test-cluster', vpc);

    const cpuval = 512;
    const memval = 1024;
    const familyval = 'test';
    const taskCfg: TaskConfig = { cpu: cpuval, memoryLimitMB: memval, family: familyval };
    const imageName = 'httpd';
    const containerCfg: ContainerConfig = { dockerHubImage: imageName };
    const taskdef = addTaskDefinitionWithContainer(stack, 'test-taskdef', taskCfg, containerCfg);

    const port = 80;
    const desiredCount = 1;

    // Test code
    const service = addService(stack, 'test-service', cluster, taskdef, port, desiredCount);

    // Check result
    const sgCapture = new Capture();
    const template = Template.fromStack(stack);

    expect(service.cluster).toEqual(cluster);
    expect(service.taskDefinition).toEqual(taskdef);

    template.resourceCountIs('AWS::ECS::Service', 1);
    template.hasResourceProperties('AWS::ECS::Service', {
        DesiredCount: desiredCount,
        LaunchType: 'FARGATE',
        NetworkConfiguration: Match.objectLike({
            AwsvpcConfiguration: Match.objectLike({
                AssignPublicIp: 'DISABLED',
                SecurityGroups: Match.arrayWith([sgCapture]),
            }),
        }),
    });

    template.resourceCountIs('AWS::EC2::SecurityGroup', 1);
    template.hasResourceProperties('AWS::EC2::SecurityGroup', {
        SecurityGroupIngress: Match.arrayWith([
            Match.objectLike({
                CidrIp: '0.0.0.0/0',
                FromPort: port,
                IpProtocol: 'tcp',
            }),
        ]),
    });
});

A new feature added here is the ability to capture values from the generated CloudFormation. This will literally be whatever is at the location where the capture object has been placed. We will just use that as a placeholder for now.

If you look at the test built here, you may spot some concerns and issues with our infrastructure design. While the test passes, there are some design issues here.

Does our design suck? Wrapping up

There are several issues one may spot when testing our infrastructure design, some of which include:

Run things as Fargate is implicit in the interface. Should it be?
Specifying a port number to access the service through addService() is right now fine for a single container instance only. For multiple containers (desiredCount > 1) there would need to be a load balancer.
The design opens for traffic from everywhere, regardless of whether it uses public or private IP addresses
Do we have the right abstraction level for this? If our test cases become too complicated, maybe we need to find a different approach.
No configuration or tweaking of container setup

We skipped some complexities by setting up a container-based environment in a cluster when we did the initial solution. This was a conscious choice then. It is easy to forget about some of these decisions later. Adding tests is a way both to validate that we get what we want, but also that our design choices for how we build our infrastructure are sound, for our use cases.

We will work with the tests and also change the design somewhat, based on what our end goals are.

If you enjoyed this material, you can check out the other parts of this series and other articles at Tidy Cloud AWS. Send comments, questions, and suggestions!

How to migrate from CloudFormation to AWS CDK - part 1

Erik Lundevall Zara — Mon, 07 Feb 2022 10:21:55 +0000

Have you, at least partially, adopted AWS CloudFormation to have better management of your Cloud Infrastructure?

Do you feel that your endeavours with CloudFormation get messy and cumbersome to handle?

Do you want to find a better way to handle cloud infrastructure better, without redoing all your work?

These articles are for you if you answer yes to most of the questions!

---READMORE---

In these articles, we will explore how we can improve our management of Cloud Infrastructure in AWS by migrating workflow and tooling from plain AWS CloudFormation and manual steps to using AWS Cloud Development Kit (AWS CDK).

AWS CDK is a newer generation of cloud infrastructure tools from AWS, which builds upon AWS CloudFormation. It can be adopted to make your Cloud Infrastructure management more enjoyable and less cumbersome.

Read the goals and requirements sections below so that you see what to expect. I hope you will get good value from these articles!

Goals

The aim of this series of articles is to explore how go migrate from AWS CloudFormation to AWS Cloud Development Kit (AWS CDK). This includes workflow and tooling aspects.

The focus is on migration, and to preserve existing infrastructure when possible, required and desirable - while still benefit from improvements in AWS CDK tooling. Your infrastructure should be less cumbersome and more enjoyable to maintain and update, and automated as much as possible - possibly more than you may have today.

By the end of this series of articles, we have explored:

Take an existing AWS CloudFormation solution, preserved, and adapt it to AWS CDK tooling
Handle both simple and nested CloudFormation stack solutions
Step-wise reorganisation of CloudFormation stacks to a more CDK-oriented model
Step-wise replacement of CloudFormation template resources with AWS CDK Constructs
Adapting CloudFormation-oriented tooling usage to AWS CDK, like cfn-lint, cfn-nag, etc.
Change and adapt parameterisation of stacks, from CloudFormation model to AWS CDK model

If you want to build green field solutions with AWS CDK, then the series How to become an infrastructure-as-code ninja, using AWS CDK is what you should look at. There will be a bit of overlap between the two series.

If you already know why you want to use AWS CDK and what you need, you can skip to the installation part, or jump to the first example.

Requirements

There are a few requirements for you to get the most of this series.

You should know how to use AWS CloudFormation. You do not need to be an expert, but you should be familiar about wring, reading and deploying CloudFormation stacks.
You need access to an AWS account where you have administrative privileges, temporary or permanent.
You should be able to install software on the computer you use.
You should be able to write and run some small programs with Typescript or Python.
An installation of your programming language of choice, that you can use with AWS CDK. Use a somewhat recent version of each language, such as recent Python version 3, and Typescript version 4).
You also need some basic knowledge of a few common AWS services, like IAM, EC2.

The AWS CDK supports multiple programming languages. There will be sample code using two languages in this article series:

Typescript - this is the native language of AWS CDK
Python - a common choice of language for many scripting tasks, and supported by AWS CDK

There are other languages supported as well, but for now we use these two.

Later updates of this article series are planned to include additional language examples.

AWS CloudFormation vs AWS Cloud Development Kit

So why would or should you consider migrating to AWS CDK from AWS CloudFormation?

What is the value proposition?

There are a few areas which may provide benefits:

It is easier to build and use interfaces at a suitable level for a use case, rather than being forced into low-level details for pretty much everything with CloudFormation.
Better default tooling. AWS CDK has a command-line tool which makes it rather easy to deploy, update, check for changes and delete Cloud infrastructure stacks. The default tooling using either AWS CLI or AWS Tools for PowerShell frankly sucks in comparison.
Better language tooling. There are more and better tools to support writing (complex) things in programming languages, than YAML or JSON.
Better organisational capabilities in AWS CDK, what to put where and how and when to parameterise the infrastructure.

There are also drawbacks that come with AWS CDK:

Additional complexity. While CloudFormation YAML/JSON suck to build abstractions, it has the benefit that it is more clear what infrastructure will be generated at a micro-level. You are a bit more at risk shooting yourself in the foot with bad code in AWS CDK.
Dependency on Node.js, and NPM. Regardless of which programming language you use, you will need to have a Node.js runtime installed from NPM. This is for the command-line tool.
Leaky abstractions. AWS CDK uses AWS CloudFormation under the hood, and in some areas that show through. This means you need to understand both AWS CDK and some CloudFormation for some scenarios.

You can essentially use AWS CDK command-line tool to make a better workflow and take advantage of benefits 2 and 4 and still write your infrastructure with CloudFormation only, or you can go all-in on programming language code, or somewhere in between. We will explore different options as part of the migration efforts we will look at.

AWS CDK Installation

The first step to get started with using AWS CDK to migrate from AWS CloudFormation is to install AWS CDK command-line tool. The tool itself is written in Typescript, and uses the Node.js runtime.

If you already have Node.js installed on your computer, then that is a great start! You can go on with the actual CDK installation below. If you do not have Node.js you will need to install its runtime.

You can download the Node.js runtime from the Node.js website. The long-term support (LTS) release is the recommended version to use, but you can use some older versions and newer versions as well.

You can also install Node.js via different package managers, depending on the computer environment you are working with. See the Node.js package manager page for suggestions. You can also download tools there to manage multiple separate versions of Node.js, if needed.

If you have Node.js installed, you can install the AWS CDK command-line tool. For any of the supported programming languages that are not Typescript, you want to do a global install of the command-line tool (not a project-local install), since you would likely not use Node.js in the cloud infrastructure project itself. If you intend to use Typescript as a language, both a global install and a project-local install would be valid options.

To install the AWS CDK command-line tool globally, run the following command in a shell:

npm install -g aws-cdk

The command npm is the Node.js package manager, which installs the package aws-cdk. It will install the latest version of AWS CDK, which is 2.10.0 at the time of writing this. It is the major version that I would recommend to use if you are new to AWS CDK and are just starting out. If you have AWS CDK version 1, the code will be slightly different, but will work as well. In this context, the differences are small.

After installation of the aws-cdk package, you will have the command-line tool cdk available. You can test that it is available and with the expected version by running it with the version argument:

❯ cdk version

2.10.0 (build e5b301f)

If you have that in place, we are now ready to get started with the actual first migration project sample! Time to get our hands dirty, and our cloud infrastructure more tidy!

First sample migration

Since our goal if to migrate existing CloudFormation infrastructure to AWS CDK, we will of course start with some CloudFormation infrastructure. I will show our steps here with a sample CloudFormation template from AWS. You are also free to pick your own one to use. I have a few suggestions for what to pick at this point in time if you bring your own template:

It should be a template which creates only a single stack, no nested templates and stacks.
You can use AWS SAM templates, although I would suggest plain CloudFormation templates initially.
It should only have a handful of parameters at most, preferably not over 4-5, but not a strict rule.
You should be ok with deploying that CloudFormation stack first prior to starting the AWS CDK work, or already have that deployed in an environment (preferably non-production).
You can use either YAML or JSON format for your CloudFormation template, both work fine

If you do not have your own template to work with, we will start by using this sample template from AWS: https://s3.eu-west-1.amazonaws.com/cloudformation-templates-eu-west-1/EC2InstanceWithSecurityGroupSample.template

This CloudFormation template will create an EC2 instance with an associated security group. It requires an existing KeyPair name for SSH access, for which there are better alternatives nowadays.

Using this template will insure some cost as well, for as long as the EC2 instance is running. This cost should be small, as long as you remember to clean it up afterwards.

The exact content of the CloudFormation template is not so important, just that we start with a relatively simple template at first. If you use your own template, you can, of course, decide yourself whether it should be cleaned up.

Initalize the project

Before writing any code for our CloudFormation to AWS CDK migration, we should initialise our AWS CDK project to get started.

In a command-line shell, create a directory to contain the project, and go to that directory:

mkdir my-cfn-cdk-migration

cd my-cfn-cdk-migration

In this directory, we will use the CDK command-line tool to initialise a project for Typescript. We use the init app sub-command with the ---language option set to either typescript, python, or go.

❯ cdk init app --language=typescript

Applying project template app for typescript

❯ cdk init app --language=python

Applying project template app for python

Each initialisation will create slightly different files, depending on the language. But before looking at any of these files, let us save the CloudFormation template we choose to use in our project file structure. We can create a directory named cfn and store the template in there as sample.template. If you use Linux, macOS, or Linux on Windows, you could run the command below to use the sample template mentioned:

❯ mkdir cfn

❯ curl --output cfn/sample.template <https://s3.eu-west-1.amazonaws.com/cloudformation-templates-eu-west-1/EC2InstanceWithSecurityGroupSample.template>

For each of the languages chosen, we will initially focus on just a single file to write the AWS CDK code to use our existing CloudFormation template (cfn/sample.template):

Typescript: bin/my-cfn-cdk-migration.ts
Python: app.py

We will just start with the code, and then explain the details step by step, along with testing how that works. You see both Typescript and Python code below, and you will find them reasonably similar. The code comprises essentially five areas:

Import of modules and class/structure definitions
Environment setup
AWS CDK App and Stack initialisation
Inclusion of CloudFormation template
Synthesize CloudFormation

#!/usr/bin/env node
import 'source-map-support/register';
import * as path from 'path';
import { App, Environment, Stack } from 'aws-cdk-lib';
import { CfnInclude } from 'aws-cdk-lib/cloudformation-include';

const env: Environment = {
  account: process.env.CDK_DEFAULT_ACCOUNT,
  region: process.env.CDK_DEFAULT_REGION,
};

const stackName = 'mystack';

const app = new App();
const stack = new Stack(app, 'wrapper-stack', { env, stackName });

new CfnInclude(stack, 'included-template', {
  templateFile: path.join(process.cwd(), 'cfn', 'sample.template'),
});

app.synth();

#!/usr/bin/env python3
import os

from aws_cdk import (
    App,
    Environment,
    Stack
)
from aws_cdk.cloudformation_include import CfnInclude

env = Environment(account=os.getenv('CDK_DEFAULT_ACCOUNT'), region=os.getenv('CDK_DEFAULT_REGION'))

stack_name = 'mystack'
app = App()
stack = Stack(app, 'wrapper-stack', env=env, stack_name=stack_name)
CfnInclude(stack, 'included-template', template_file=os.path.join(os.getcwd(), 'cfn', 'sample.template'))
app.synth()

Let us start with the App and Stack parts. When we use the AWS CDK, all infrastructure we manage as some kind of larger scale unit is grouped into an App. This the top-level container, which contains everything that you can provision in the project or solution you are working with.

Each App may comprise one or more stacks. A Stack is a collection of AWS resources that you provide and manage as a unit. It could be your whole solution, or it could be just one part of a solution. An AWS CDK stack maps 1-to-1 with a CloudFormation stack.

For Stacks, we can in AWS CDK attach Constructs, which are a logical grouping of Resources, which are the same type of resources that you represent in CloudFormation as well. In the code, we have a CfnInclude construct, which performs the inclusion of a CloudFormation template, a Stack and an App in AWS CDK. The only parameter here so far is to point to the CloudFormation template in our project. This is the stuff to include into the AWS CDK App and the related stack.

We also have the concept of an Environment, which in AWS CDK context is a combination of AWS account and AWS region. A CDK App can target multiple accounts and/or multiple regions within the same app.

The environment setup in the code is essentially using two environment variables that the AWS CDK has built in, which captures whatever is the current AWS account id and AWS region, based on the currently active AWS credentials that you use. CDK_DEFAULT_ACCOUNT and CDK_DEFAULT_REGION are the names of these environment variables.

So this approach assumes you have done your work to activate an AWS credentials profile, or set environment variables for credentials or equivalent, and then the CDK code will get the account and region info from that. This can be fine for development environments, but in particular with production environments, you would want some additional guardrails. For now, that is not our focus, though.

The final area is to synthesize to CloudFormation. This the task of the code app.synth(). It generates the underlying CloudFormation.

You can put the code into the program files mentioned above, app.py or bin/my-cfn-cdk-migration.ts.

You can see the hierarchy shown in the picture in the code itself. The App itself is just created, we have an object for that. The line with the Stack creation adds a reference to the App, plus a name which is the name of the stack in this case. After that, there are additional parameters for the environment.

The CfnInclude creation follows the same pattern. It adds a reference to the Stack, plus a name. After that, there are additional parameters, in this case, a reference to the CloudFormation template file.

All logical resource groupings in AWS CDK follow the same pattern - a reference to the parent in a hierarchy and a name. With a Stack, the name is the same as the name of the CloudFormation stack in AWS. Below the stack level, into construct territory, there is less concern with having a specific name. However, a name will need to be unique under a specific parent in the hierarchy.

Note: If you have not already deployed sample.template CloudFormation to an environment without using the AWS CDK, now is the time to do this. We will assume below that the CloudFormation template already has been deployed, and the work we do will be in the light of changes to do when we have an existing (non-CDK) infrastructure already in place.

So deploy it, if you need to. We will wait...

... alright, done now? Great!

Also, note down the name of the CloudFormation stack you have deployed. This is important. If the name is not the same as the name set in the Stack object in the code, change the name of the stack in the code to match the deployed stack.

The AWS CDK initialisation we did before with the cdk init command behaved slightly differently depending on the language we choose to use. For Typescript, the cdk init performed the needed package installation. For Python, we have a bit more to do:

For Python, activate the virtual environment that cdk init created for you. In Linux och macOS, run

source ./.venv/bin/activate

and in Windows

.venv\Scripts\activate.bat

To activate the virtual environment that has been set up. Then run the package installation:

pip3 install -r requirements.txt

This will install the required AWS CDK packages. We have now:

An initialised AWS CDK project
Required AWS CDK packages installed
An AWS CDK app wrapper around our existing CloudFormation template

Now it is time to look closer to the code and actually check that this AWS CDK app actually works!

Checking for CloudFormation changes

Since the plan here is to do a migration from an existing CloudFormation infrastructure deployment to use AWS CDK, we want really everything to be the same when we use AWS CDK. No messing around with our existing infrastructure!

So what we want first is to check that what we have with AWS CDK matches what we have already deployed before. We can use the AWS CDK command-line tool for this. All commands should be executed in the same directory as the cdk.json file is in, which is the root directory of our project.

Note: you have to have your AWS credentials in place when you run the AWS CDK commands, since you will work towards the target AWS environment. It does not matter whether you set AWS regular environment variables (e.g. AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID, AWS_DEFAULT_REGION, etc) or use AWS credentials profiles. If you use AWS credentials profiles, add a --profile profile_name to each command line to specify the name of the profile, if the profile name is not default.

The AWS CDK command-line tool has a sub-command diff, which can compare the under-the-hood generated CloudFormation with what is actually deployed. If you run the cdk diff command and run it with the sample template we have, and the stack name is correct, and you have set the AWS credentials for your environment, see the following output (expect probably to wait 10-15 seconds):

❯ cdk diff       
[Info at /mystack/included-template/$Mappings/AWSInstanceType2Arch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSInstanceType2NATArch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSRegionArch2AMI] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
Stack mystack
Parameters
[+] Parameter BootstrapVersion BootstrapVersion: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"/cdk-bootstrap/hnb659fds/version","Description":"Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"}

Other Changes
[+] Unknown Rules: {"CheckBootstrapVersion":{"Assertions":[{"Assert":{"Fn::Not":[{"Fn::Contains":[["1","2","3","4","5"],{"Ref":"BootstrapVersion"}]}]},"AssertDescription":"CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."}]}}

If you use your own CloudFormation template and stack, the FindInMap warnings will probably not be the same, since these depend on the specific CloudFormation template in use. We can ignore these warnings for now as well. It is only a potential optimisation that could be done.

There are two resources that are shown with plus signs, so if we deploy the stack in the AWS CDK App, these would be added to the stack. There are no resources with minus signs in front of them, which means there are no resources removed. The differences are shown at the CloudFormation level, so it is beneficial to know a bit of CloudFormation for this type of operation.

What are these added entries?

The first entry is an added CloudFormation template parameter, called BootstrapVersion. The value will automatically be retrieved from this parameter in Parameter Store when CloudFormation deploys the stack.

The second entry is a CloudFormation rule, which performs validation of BootstrapVersion. Have a recent enough bootstrap done, otherwise the stack deployment will fail.

We will cover the bootstrapping in the next section below. What we can see here, though, is that, except for this bootstrapping part, there are no changes. As long as we can accept this bootstrap parameter and added rule validation, it will be all the same.

All AWS CDK stacks will have this type of parameter and rule. It is part of the baseline you can expect.

You can see the generated CloudFormation template from AWS CDK by running the cdk synth command. The output should be identical to the original CloudFormation template, with the addition of the parameter and rule for BootstrapVersion.

What is bootstrapping?

To actually deploy stacks in an AWS CDK App, you need to bootstrap the target environment. The bootstrap process will create an S3 bucket, and at least some IAM roles, and the BootstrapVersion parameter in Parameter Store.

The S3 bucket is used to upload things to deploy basically. If you use plain CloudFormation, you know that for many cases you will need to upload CloudFormation template files to some S3 bucket first, before it can be used.

The AWS CDK bootstrap creates such a bucket for you and handles the uploads and references to uploaded data under the hood for you, so you do not need to worry about it. This simplifies workflows with AWS CDK.

The bootstrap process is a onetime task for each target environment (account plus region combination). You do that once, and then do not have to do that again - only if there were some changes in prerequisites for the environment, you may need to perform an update at some point. For all normal interactions, though, once you have bootstrapped you are good to do not have to think about it again.

To bootstrap the current target environment, you simply execute the command cdk bootstrap. It will deploy a CloudFormation stack with the resources that belong to the bootstrapping, with a (default) name of the stack as CDKToolkit.

Please run cdk bootstrap in your target environment. It is needed for us to perform any changes/updates on our target stack. If you prefer not to do any updates, just read about the rest. But in order to perform actual changes with AWS CDK, you need to bootstrap first.

Deploy update to stack

Once we have bootstrapped our target environment, we can deploy this update to our existing stack. This is also something to think about whether you want to adopt AWS CDK as a wrapper around your existing CloudFormation infrastructure. AWS CDK will add this bootstrap information, as well as additional metadata to the CloudFormation stacks it deploys.

You can keep working with your existing CloudFormation templates and edit those. In addition, you also have an AWS CDK wrapper code around your CloudFormation templates, plus use the AWS DK command-line tool. You will get bootstrap and metadata added to the stacks if you choose to use AWS CDK commands for deployment and management. Mixing that with other deployment methods for CloudFormation stacks may not be optimal.

So a strong recommendation here is to experiment with AWS CDK command-line tool and see if you think it brings value, compared to what you use today.

That being said, let us look at deploying these changes to the stack. To perform the update to the stack, we can simply run the cdk deploy command. Since we only have a single stack in the AWS CDK app, it will simply deploy that stack using the current AWS credentials.

❯ cdk deploy
[Info at /mystack/included-template/$Mappings/AWSInstanceType2Arch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSInstanceType2NATArch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSRegionArch2AMI] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap

✨  Synthesis time: 10.36s

mystack: deploying...
[0%] start: Publishing 4dcb1b0af0f37ffd3820e5c6c660615d743078920ae05497462670220e892d74:current_account-eu-west-1
[100%] success: Published 4dcb1b0af0f37ffd3820e5c6c660615d743078920ae05497462670220e892d74:current_account-eu-west-1
mystack: creating CloudFormation changeset...





 ✅  mystack

✨  Deployment time: 25.58s

Outputs:
mystack.AZ = eu-west-1b
mystack.InstanceId = i-08ec677b2ca564471
mystack.PublicDNS = ec2-63-35-162-187.eu-west-1.compute.amazonaws.com
mystack.PublicIP = 63.35.162.187
Stack ARN:
arn:aws:cloudformation:eu-west-1:123456789012:stack/mystack/ffc6cc30-7e98-11ec-8c08-0244eda65617

✨  Total time: 35.94s

After executing the deployment of the stack, you can run the cdk diff command again. You will see that there are no differences between the generated output from the AWS CDK app and the actual deployed stack.

❯ cdk diff  
[Info at /mystack/included-template/$Mappings/AWSInstanceType2Arch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSInstanceType2NATArch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSRegionArch2AMI] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
Stack mystack
There were no differences

The stack generated from our AWS CDK app is now the same as what we have deployed. No real changes, just extra information that AWS CDK itself requires. Next step is to do some real changes, and also look at handling parameters we send in during stack creation/update.

Updating parameters and stacks

Let us now look at how we can update our stacks using AWS CDK. We start with changing the existing parameters. You noticed that by default, we were not required to re-enter any existing parameters if we updated the existing stack. This is normal behaviour for CloudFormation as well.

However, if we want to change an existing parameter when we deploy a stack, we can add that on the command-line, using the ---parameters option. In our sample template, we have a parameter InstanceType, which specifies what type of EC2 instance we shall deploy. By default, it deploys a t2.small instance. We can change that to a “t2.micro" instance, for example, by changing the parameter on the command-line when we deploy it. We need to specify the name of the stack and the name of the parameter. Remember that an AWS CDK app can contain multiple stacks!

❯ cdk deploy --parameters mystack:InstanceType=t2.micro
[Info at /mystack/included-template/$Mappings/AWSInstanceType2Arch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSInstanceType2NATArch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSRegionArch2AMI] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap

✨  Synthesis time: 10.65s

mystack: deploying...
[0%] start: Publishing 4dcb1b0af0f37ffd3820e5c6c660615d743078920ae05497462670220e892d74:current_account-eu-west-1
[100%] success: Published 4dcb1b0af0f37ffd3820e5c6c660615d743078920ae05497462670220e892d74:current_account-eu-west-1
mystack: creating CloudFormation changeset...





 ✅  mystack

✨  Deployment time: 114.52s

Outputs:
mystack.AZ = eu-west-1b
mystack.InstanceId = i-08ec677b2ca564471
mystack.PublicDNS = ec2-34-245-79-51.eu-west-1.compute.amazonaws.com
mystack.PublicIP = 34.245.79.51
Stack ARN:
arn:aws:cloudformation:eu-west-1:123456789012:stack/mystack/ffc6cc30-7e98-11ec-8c08-0244eda65617

✨  Total time: 125.17s

If we try to run cdk diff using ---parameters option, that will not work through. The cdk diff command does not support ---parameters, at least not at the time of writing this. Is there a way to overcome this issue?

AWS CDK only provides support for CloudFormation parameters to be compatible with CloudFormation. You can resolve values when you run the code, and when you deploy the generated stacks, through different means.

We can include the parameters directly in the CfnInclude construct creation. How we get these parameters are up to us - from command-line, from some other file, hard-coded, a database or anything else we can do with some code. In our example here, I have simply added a variable in the code with the desired value.

#!/usr/bin/env node
import 'source-map-support/register';
import * as path from 'path';
import { App, Environment, Stack } from 'aws-cdk-lib';
import { CfnInclude } from 'aws-cdk-lib/cloudformation-include';

const env: Environment = {
  account: process.env.CDK_DEFAULT_ACCOUNT,
  region: process.env.CDK_DEFAULT_REGION,
};

const stackName = 'mystack';

const app = new App();
const stack = new Stack(app, 'wrapper-stack', { env, stackName });

const instanceType = 't2.small';

new CfnInclude(stack, 'included-template', {
  templateFile: path.join(process.cwd(), 'cfn', 'sample.template'),
  parameters: {
    InstanceType: instanceType,
  },
});

app.synth();

#!/usr/bin/env python3
import os

from aws_cdk import (
    App,
    Environment,
    Stack
)
from aws_cdk.cloudformation_include import CfnInclude

env = Environment(account=os.getenv('CDK_DEFAULT_ACCOUNT'), region=os.getenv('CDK_DEFAULT_REGION'))

stack_name = 'mystack'
app = App()
stack = Stack(app, 'wrapper-stack', env=env, stack_name=stack_name)

instance_type = 't2.small'
CfnInclude(stack, 
           'included-template', 
           template_file=os.path.join(os.getcwd(), 'cfn', 'sample.template'),
           parameters={
               'InstanceType' : instance_type,
           })
app.synth()

If you run cdk diff after these code changes, you will notice something interesting! The parameter itself is removed completely from the generated CloudFormation, and the actual instance type value is included directly instead!

❯ cdk diff
[Info at /mystack/included-template/$Mappings/AWSInstanceType2Arch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSInstanceType2NATArch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSRegionArch2AMI] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
Stack mystack
Parameters
[-] Parameter InstanceType: {"Type":"String","Default":"t2.small","AllowedValues":["t1.micro","t2.nano","t2.micro","t2.small","t2.medium","t2.large","m1.small","m1.medium","m1.large","m1.xlarge","m2.xlarge","m2.2xlarge","m2.4xlarge","m3.medium","m3.large","m3.xlarge","m3.2xlarge","m4.large","m4.xlarge","m4.2xlarge","m4.4xlarge","m4.10xlarge","c1.medium","c1.xlarge","c3.large","c3.xlarge","c3.2xlarge","c3.4xlarge","c3.8xlarge","c4.large","c4.xlarge","c4.2xlarge","c4.4xlarge","c4.8xlarge","g2.2xlarge","g2.8xlarge","r3.large","r3.xlarge","r3.2xlarge","r3.4xlarge","r3.8xlarge","i2.xlarge","i2.2xlarge","i2.4xlarge","i2.8xlarge","d2.xlarge","d2.2xlarge","d2.4xlarge","d2.8xlarge","hi1.4xlarge","hs1.8xlarge","cr1.8xlarge","cc2.8xlarge","cg1.4xlarge"],"ConstraintDescription":"must be a valid EC2 instance type.","Description":"WebServer EC2 instance type"}

Resources
[~] AWS::EC2::Instance included-template/EC2Instance EC2Instance replace
 ├─ [~] ImageId (requires replacement)
 │   └─ [~] .Fn::FindInMap:
 │       └─ @@ -6,9 +6,7 @@
 │          [ ] {
 │          [ ]   "Fn::FindInMap": [
 │          [ ]     "AWSInstanceType2Arch",
 │          [-]     {
 │          [-]       "Ref": "InstanceType"
 │          [-]     },
 │          [+]     "t2.small",
 │          [ ]     "Arch"
 │          [ ]   ]
 │          [ ] }
 └─ [~] InstanceType (may cause replacement)
     └─ @@ -1,3 +1,1 @@
        [-] {
        [-]   "Ref": "InstanceType"
        [-] }
        [+] "t2.small"

The reason for this is that AWS CDK can determine the value to set for InstanceType already at the time you run the code. There is no need to have a parameter at all, so the parameter from the original CloudFormation template is eliminated. Run cdk synth and look at the generated CloudFormation output where there is no InstanceType parameter anymore.

This is a different mental model compared to using CloudFormation. AWS CDK resolves everything it can when running the code, and the other things it does later. This happens under the hood to some extent. An AWS CDK app may contain all stacks for all environments. The differences are expressed in the code.

It is, of course, possible to not follow that model. AWS CDK tries to facilitate to use this approach, but does not enforce it. You can keep your existing way of working with parameters if you want to keep using AWS CloudFormation templates as your primary source for infrastructure and only use AWS CDK command-line tool to make the workflow a bit more convenient. If you want to adopt using AWS CDK for the infrastructure, then you should also adapt your approach to handling parameterisation.

You can now run cdk deploy to deploy these changes. AWS CDK will perform the update, which will take a few minutes to complete, since it will replace the EC2 instance with a new one.

Another parameter update - security changes

In the same way that we updated InstanceType in our sample template, we can also update the SSHLocation parameter. We simply add that in the same way in the code and set the default value 0.0.0.0/0 in the code.

#!/usr/bin/env node
import 'source-map-support/register';
import * as path from 'path';
import { App, Environment, Stack } from 'aws-cdk-lib';
import { CfnInclude } from 'aws-cdk-lib/cloudformation-include';

const env: Environment = {
  account: process.env.CDK_DEFAULT_ACCOUNT,
  region: process.env.CDK_DEFAULT_REGION,
};

const stackName = 'mystack';

const app = new App();
const stack = new Stack(app, 'wrapper-stack', { env, stackName });

const instanceType = 't2.small';
const sshLocation = '0.0.0.0/0';

new CfnInclude(stack, 'included-template', {
  templateFile: path.join(process.cwd(), 'cfn', 'sample.template'),
  parameters: {
    InstanceType: instanceType,
    SSHLocation: sshLocation,
  },
});

app.synth();

#!/usr/bin/env python3
import os

from aws_cdk import (
    App,
    Environment,
    Stack
)
from aws_cdk.cloudformation_include import CfnInclude

env = Environment(account=os.getenv('CDK_DEFAULT_ACCOUNT'), region=os.getenv('CDK_DEFAULT_REGION'))

stack_name = 'mystack'
app = App()
stack = Stack(app, 'wrapper-stack', env=env, stack_name=stack_name)

instance_type = 't2.small'
ssh_location = '0.0.0.0/0'
CfnInclude(stack, 
           'included-template', 
           template_file=os.path.join(os.getcwd(), 'cfn', 'sample.template'),
           parameters={
               'InstanceType' : instance_type,
               'SSHLocation' : ssh_location
           })
app.synth()

If you run a cdk diff on the code, and you use the sample template, you will see an output like this:

❯ cdk diff
[Info at /mystack/included-template/$Mappings/AWSInstanceType2Arch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSInstanceType2NATArch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSRegionArch2AMI] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
Stack mystack
Security Group Changes
┌───┬────────────────────────────────────────────────────┬─────┬──────────┬─────────────────┐
│   │ Group                                              │ Dir │ Protocol │ Peer            │
├───┼────────────────────────────────────────────────────┼─────┼──────────┼─────────────────┤
│ - │ ${included-template/InstanceSecurityGroup.GroupId} │ In  │ TCP 22   │ ${SSHLocation}  │
├───┼────────────────────────────────────────────────────┼─────┼──────────┼─────────────────┤
│ + │ ${included-template/InstanceSecurityGroup.GroupId} │ In  │ TCP 22   │ Everyone (IPv4) │
└───┴────────────────────────────────────────────────────┴─────┴──────────┴─────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Parameters
[-] Parameter SSHLocation: {"Type":"String","Default":"0.0.0.0/0","AllowedPattern":"(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})/(\\d{1,2})","ConstraintDescription":"must be a valid IP CIDR range of the form x.x.x.x/x.","Description":"The IP address range that can be used to SSH to the EC2 instances","MaxLength":"18","MinLength":"9"}

Resources
[~] AWS::EC2::SecurityGroup included-template/InstanceSecurityGroup InstanceSecurityGroup 
 └─ [~] SecurityGroupIngress
     └─ @@ -1,8 +1,6 @@
        [ ] [
        [ ]   {
        [-]     "CidrIp": {
        [-]       "Ref": "SSHLocation"
        [-]     },
        [+]     "CidrIp": "0.0.0.0/0",
        [ ]     "FromPort": 22,
        [ ]     "IpProtocol": "tcp",
        [ ]     "ToPort": 22

A security group rule has the source value changed from a variable to an explicit value 0.0.0.0/0. It is the same value provided via the parameter before, but now it is set already when the code runs.

To have the actual value in the template is better, since you can check the value with security tooling before it is deployed. So this is an advantage of the AWS CDK model with trying to resolve parameterisation as early as possible.

If you run cdk deploy to deploy these changes, you will also see a change in the deployment workflow. You will get a prompt asking about security group changes, so that you can do an approval that these changes will be performed.

❯ cdk deploy                                     
[Info at /mystack/included-template/$Mappings/AWSInstanceType2Arch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSInstanceType2NATArch] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap
[Info at /mystack/included-template/$Mappings/AWSRegionArch2AMI] Consider making this CfnMapping a lazy mapping by providing `lazy: true`: either no findInMap was called or every findInMap could be immediately resolved without using Fn::FindInMap

✨  Synthesis time: 10.72s

This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:

Security Group Changes
┌───┬────────────────────────────────────────────────────┬─────┬──────────┬─────────────────┐
│   │ Group                                              │ Dir │ Protocol │ Peer            │
├───┼────────────────────────────────────────────────────┼─────┼──────────┼─────────────────┤
│ - │ ${included-template/InstanceSecurityGroup.GroupId} │ In  │ TCP 22   │ ${SSHLocation}  │
├───┼────────────────────────────────────────────────────┼─────┼──────────┼─────────────────┤
│ + │ ${included-template/InstanceSecurityGroup.GroupId} │ In  │ TCP 22   │ Everyone (IPv4) │
└───┴────────────────────────────────────────────────────┴─────┴──────────┴─────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Do you wish to deploy these changes (y/n)? y
mystack: deploying...
[0%] start: Publishing 573ae98f43ba74567ae46ffe8df6d11132f8ef444d61e22ee031b763e4562c2b:069901141591-eu-west-1
[100%] success: Published 573ae98f43ba74567ae46ffe8df6d11132f8ef444d61e22ee031b763e4562c2b:069901141591-eu-west-1
mystack: creating CloudFormation changeset...

 ✅  mystack (no changes)

✨  Deployment time: 9.83s

Outputs:
mystack.AZ = eu-west-1b
mystack.InstanceId = i-08ec677b2ca564471
mystack.PublicDNS = ec2-34-247-191-237.eu-west-1.compute.amazonaws.com
mystack.PublicIP = 34.247.191.237
Stack ARN:
arn:aws:cloudformation:eu-west-1:123456789012:stack/mystack/ffc6cc30-7e98-11ec-8c08-0244eda65617

✨  Total time: 20.55s

The default check is to prompt you for confirmation if it needs additional access or permissions from before. You can set this behaviour with the ---require-approval option. It can be set to either never, any-change, or to broadening. The default is broadening.

This is a feature that I think is valuable and a minimum sanity check. This is something that adds value if you don't have similar tools before.

Wrapping up

We have the following goals in this series of articles:

Take an existing AWS CloudFormation solution, preserved, and adapt it to AWS CDK tooling
Handle both simple and nested CloudFormation stack solutions
Step-wise reorganisation of CloudFormation stacks to a more CDK-oriented model
Step-wise replacement of CloudFormation template resources with AWS CDK Constructs
Adapting CloudFormation-oriented tooling usage to AWS CDK, like cfn-lint, cfn-nag, etc.
Change and adapt parameterisation of stacks, from CloudFormation model to AWS CDK model

We have started with the first point in this list of goals and have partially done point two as well. Finally, we have also started with the sixth point to change and adapt the parameterisation of stacks.

This is as far as we will go in this article, and in the next article we will dig further into the template and stack handling with nested CloudFormation templates and stacks.

Before signing off in this article, I highly recommend that if you have created new infrastructure with the sample template provided, you clean it up and remove it. This is pretty straightforward with the AWS CDK command-line tool, you simply run the cdk destroy command. It will prompt you to confirm that you want to delete the stack, and then delete it, if you answer yes.

❯ cdk destroy
Are you sure you want to delete: mystack (y/n)? y
mystack: destroying...



 ✅  mystack: destroyed

Do not forget that!

I hope you have found this article useful. Comments, questions and suggestions are welcome! You can also send feedback via Tidy Cloud AWS, or check out other posts there.

How to become an infrastructure-as-code ninja, using AWS CDK - part 5

Erik Lundevall Zara — Mon, 20 Dec 2021 23:48:48 +0000

Now it is time to build a service, running in Elastic Container Service (ECS)!

In part 4 of our series, we set up an ECS cluster to run our containers using Fargate, so that we do not need to bother with underlying server infrastructure for the containers. We also added a Task Definition, so that we could manually start the container and get the Apache web server running.

We had a list of goals, which we could cover partially:

Expose an endpoint for a web server for HTTP traffic from internet.
Web server shall run in a container.
The container itself shall not be directly reachable from internet.
We should be able to have a service set up so that containers will automatically start when needed.
We should be able to build our custom solution for this web server.
We should be able to get container images from DockerHub.
We do not care about managing the underlying server infrastructure that runs the containers. I.e., we will use Fargate.

Let us now address more points in the goal list, one by one. Since we got the Apache web server running, but had to start it manually. Let us remove that manual step. We can accomplish this by setting up an ECS Service for the web server.

Update:

This article series uses Typescript as an example language. However, there are repositories with example code for multiple languages.

The repositories will contain all the code examples from the articles series, implemented in different languages with the AWS CDK. You can view the code from specific parts and stages in the series, by checking out the code tagged with a specific part and step in that part, see the README file in each repository for more details.

Setting up an ECS Service

An ECS Service will allow us to have a container running, and if the container fails, ECS will start up a new instance of a container automatically. Looking at the AWS CDK documentation, we can see that we have a FargateService class we can use. Let us think a bit about what we need to provide:

The cluster the service should run in
The task definition to run as a service
The desired number of task instances we should run
Any port openings to allow traffic to the container
Optionally, a name for the service

To allow traffic to the service, we need a security group. In part 4, when we started a Fargate task, the AWS Console created a security group for us, with an opening for port 80 to the entire world. As a starting point, we aim at re-creating that experience.

So we have 5 pieces of information to include. On top of these, we also should provide a logical id for the service, and we add the scope which the service is added to. Let us create a function skeleton for this:

export const addService = 
function(scope: Construct, 
         id: string, 
         cluster: Cluster, 
         taskDef: FargateTaskDefinition, 
         port: number, 
         desiredCount: number, 
         serviceName?: string): FargateService {
}

In the function body, we add code to create a security group, and add an ingress rule for the port we provide.

We will then also create the service using FargateService, passing in our parameters. Finally, we will return to the service construct. The result looks like this:

export const addService = 
function(scope: Construct, 
         id: string, 
         cluster: Cluster, 
         taskDef: FargateTaskDefinition, 
         port: number, 
         desiredCount: number, 
         serviceName?: string): FargateService {
    const sg = new SecurityGroup(scope, `${id}-security-group`, {
        description: `Security group for service ${serviceName ?? ''}`,
        vpc: cluster.vpc,
    });
    sg.addIngressRule(Peer.anyIpv4(), Port.tcp(port));

    const service = new FargateService(scope, id, {
        cluster,
        taskDefinition: taskDef,
        desiredCount,
        serviceName,
        securityGroups: [sg],
    });

    return service;
};

The function code uses the id provided for the service construct to generate an id for the security group in the function. We make a call to add an ingress rule (incoming traffic) from anywhere using the specified TCP port. This is like what the AWS Console experience generated for us. We essentially replicate this for now, but this is not what we want to have in the end.

Now we need to add a call to our new function in our main program, to add the service to our existing code (since part 4) with the ECS cluster and task definition. The call itself a single line in what we have already written before. Notice here that I picked the family name from the task configuration to generate an id value for the service. The desired count for the service is a single task instance.

addService(stack, `service-${taskConfig.family}`, cluster, taskdef, 80, 1);

Now let us put this into its context and look at the code for the whole main program again. We retrieve data for the existing default VPC, which we use when we set up an ECS cluster. We define an ECS task definition using the standard httpd web service image, and we use this information to define an ECS service that should run with a single instance of that task.

The main program of our code now looks like this:

import { App, Stack } from 'aws-cdk-lib';
import { Vpc } from 'aws-cdk-lib/aws-ec2';
import { 
  addCluster, 
  addService,
  addTaskDefinitionWithContainer, 
  ContainerConfig, 
  TaskConfig 
} from '../lib/containers/container-management';

const app = new App();
const stack = new Stack(app, 'my-container-infrastructure', {
  env: {
    account: process.env.CDK_DEFAULT_ACCOUNT,
    region: process.env.CDK_DEFAULT_REGION,
  },
});

const vpc = Vpc.fromLookup(stack, 'vpc', {
  isDefault: true,
});

const id = 'my-test-cluster';
const cluster = addCluster(stack, id, vpc);

const taskConfig: TaskConfig = { cpu: 512, memoryLimitMB: 1024, family: 'webserver' };
const containerConfig: ContainerConfig = { dockerHubImage: 'httpd' };
const taskdef = addTaskDefinitionWithContainer(stack, `taskdef-${taskConfig.family}`, taskConfig, containerConfig);
addService(stack, `service-${taskConfig.family}`, cluster, taskdef, 80, 1);

We can also look at our container management module and see the functions and data structures we currently have in place there:

import { IVpc, Peer, Port, SecurityGroup } from 'aws-cdk-lib/aws-ec2';
import { Cluster, ContainerImage, FargateService, FargateTaskDefinition, TaskDefinition } from 'aws-cdk-lib/aws-ecs';
import { Construct } from 'constructs';

export const addCluster = function(scope: Construct, id: string, vpc: IVpc): Cluster {
    return new Cluster(scope, id, {
        vpc,
    });
}

export interface TaskConfig {
    readonly cpu: 256 | 512 | 1024 | 2048 | 4096;
    readonly memoryLimitMB: number;
    readonly family: string;
}

export interface ContainerConfig {
    readonly dockerHubImage: string;
}

export const addTaskDefinitionWithContainer = 
function(scope: Construct, id: string, taskConfig: TaskConfig, containerConfig: ContainerConfig): TaskDefinition {
    const taskdef = new FargateTaskDefinition(scope, id, {
        cpu: taskConfig.cpu,
        memoryLimitMiB: taskConfig.memoryLimitMB,
        family: taskConfig.family,
    });

    const image = ContainerImage.fromRegistry(containerConfig.dockerHubImage);
        taskdef.addContainer(`container-${containerConfig.dockerHubImage}`, { image });

    return taskdef;
};

export const addService = 
function(scope: Construct, 
         id: string, 
         cluster: Cluster, 
         taskDef: FargateTaskDefinition, 
         port: number, 
         desiredCount: number, 
         serviceName?: string): FargateService {
    const sg = new SecurityGroup(scope, `${id}-security-group`, {
        description: `Security group for service ${serviceName ?? ''}`,
        vpc: cluster.vpc,
    });
    sg.addIngressRule(Peer.anyIpv4(), Port.tcp(port));

    const service = new FargateService(scope, id, {
        cluster,
        taskDefinition: taskDef,
        desiredCount,
        serviceName,
        securityGroups: [sg],
    });

    return service;
};

Before we try to deploy this solution, we should have a few things in mind:

If you deploy an ECS Service with a desired count > 0, it will try to start the task during the deployment
Deployment is only considered successful if the desired count has been reached and considered being in a healthy state
By default, deployment to ECS can get stuck if the service does not work.

1) reduce potential waiting times if we get any trouble, there are two things we want to do here. First, let us set the desired count for the service to 0. This means that AWS CDK (and ECS) will provision the service, but will not try to start it. We can then first try to start the service manually. Second, we can tell ECS to use a circuit breaker pattern for the deployment. What this means is that it will try a few times to run the service, and if that does not work, it will roll back the deployment. Let us update the code to include both.
2) set the desired count to 0, we simply change the count parameter to the addService() call:

addService(stack, `service-${taskConfig.family}`, cluster, taskdef, 80, 0);

3) apply the circuit breaker pattern, we have a property circuitBreaker on FargateService for this. When we specify this, we can also say that it should roll back the deployment in case there is an error. In this way, we can get back to the previous state of the service if the updated deployment fails. The updated addService() code now looks like this:

export const addService = 
function(scope: Construct, 
         id: string, 
         cluster: Cluster, 
         taskDef: FargateTaskDefinition, 
         port: number, 
         desiredCount: number, ![Image description](file:////Users/erikl/Documents/Publii/sites/tidy-cloud-aws/input/media/posts/27/Deployed_service_1.png =1982x858)
         serviceName?: string): FargateService {
    const sg = new SecurityGroup(scope, `${id}-security-group`, {
        description: `Security group for service ${serviceName ?? ''}`,
        vpc: cluster.vpc,
    });
    sg.addIngressRule(Peer.anyIpv4(), Port.tcp(port));

    const service = new FargateService(scope, id, {
        cluster,
        taskDefinition: taskDef,
        desiredCount,
        serviceName,
        securityGroups: [sg],
        circuitBreaker: {
            rollback: true,
        },
    });

    return service;
};

Let us now try to deploy this again, and see how this works out! As before, you can run the cdk deploy command to perform the deployment (do not forget to provide your AWS credentials). If you already have the solution from part 4 deployed, you can run the cdk diff command to see what the updates will be. You can also run cdk diff if nothing is deployed to see a list of the CloudFormation resources that will be deployed.

Note: At the time of this writing, the AWS Console provides an old and a new ECS Console experience. I have found that the new experience currently does not have all the features of the old experience, and hence I have used the old one here for screenshots. This may have changed by the time you read this. If you are using the new experience and find that you cannot do a particular task, try to switch to the old experience.

When the deployment is complete, we should see the service resource in our ECS cluster.

We can see that the desired count is 0, as well as the number of running tasks. This is what we expect with the deployment configuration we specified. So far, so good!

If we select the service and click on the Update button, we can change the service configuration to set another value for desired count. There are many entries, but we concern ourselves only with the desired count entry. Scroll down the page until you find the entry number of tasks and change the value to 1.

Click through the rest of the pages with the Next Step buttons with no additional changes, until you get to the Update Service button, and then click on that one. Now it will update the desired count and ECS will try to start the service. At first, the display should show PROVISIONING, then to state PENDING. We want it to reach status RUNNING and then we can test that we can reach the web server, just as we did in part 4.

However, you may notice that we do not actually reach state RUNNING. It seems to be stuck in PENDING state, and eventually ECS will stop trying and remove the task. Something has gone wrong. How do we fix this?

Troubleshooting our deployment

First, let us check if we have any useful logs. We can click around in the ECS Console, and we can look in CloudWatch console under Logs, and we find nothing. This is not so good...

It seems we do not get any useful logs by default, so we need to fix this. Before updating our deployment, let us look further to see if we find anything else that can be useful.

In the Details tab for our service, there is a Network Access section:

We have a default VPC, and all subnets in the default VPC have public internet access, so the subnet list should probably be ok. We can also check the security group listed and check that this is ok. The fourth entry here says Auto-assign public IP DISABLED. When we started a task manually in part 4, we got a public IP address for our task, so this is different. Let us just check the security group also though:

The security group looks as expected, port 80 open for everyone. So now we have two things to update:

Add logging to our deployment
Change so that a public IP address is assigned to the container.

Remember, this is not the ultimate solution, but we take it in small steps and first we want to have it working at all! So let us see what we should change now.

Changing the service deployment

If we look at the FargateService documentation again, there is actually an entry there that says assignPublicIp. It is optional, and the default value is false. So let us add a parameter to addService() function that allows us to set this property.

export const addService = 
function(scope: Construct, 
         id: string, 
         cluster: Cluster, 
         taskDef: FargateTaskDefinition, 
         port: number, 
         desiredCount: number, 
         assignPublicIp?: boolean,
         serviceName?: string): FargateService {
    const sg = new SecurityGroup(scope, `${id}-security-group`, {
        description: `Security group for service ${serviceName ?? ''}`,
        vpc: cluster.vpc,
    });
    sg.addIngressRule(Peer.anyIpv4(), Port.tcp(port));

    const service = new FargateService(scope, id, {
        cluster,
        taskDefinition: taskDef,
        desiredCount,
        serviceName,
        securityGroups: [sg],
        circuitBreaker: {
            rollback: true,
        },
        assignPublicIp,
    });

    return service;
};

To add logging to our container, it not as obvious. There isn't anything obvious on FargateService, and there isn't anything on the FargateTaskDefinition either. It turns out that there is a logging property we can set when we add the container definition. This property requires a LogDriver object, and we can get an object that handles logging to CloudWatch by using the LogDriver.awsLogs() function. There is one mandatory parameter here, and that is streamPrefix. This will set the first part of the name of the CloudWatch log stream.

For now, let us pick the family name in the task configuration. By default, the logs will be around forever, which perhaps a bit too long. So we can also set the desired retention time for the logs. This is a quite temporary lab experiment, so let us keep the retention time short, just a day.

This means that the addTaskDefinitionWithContainer() function gets an update:

export const addTaskDefinitionWithContainer = 
function(scope: Construct, id: string, taskConfig: TaskConfig, containerConfig: ContainerConfig): TaskDefinition {
    const taskdef = new FargateTaskDefinition(scope, id, {
        cpu: taskConfig.cpu,
        memoryLimitMiB: taskConfig.memoryLimitMB,
        family: taskConfig.family,
    });

    const image = ContainerImage.fromRegistry(containerConfig.dockerHubImage);
    const logdriver = LogDriver.awsLogs({ 
        streamPrefix: taskConfig.family,
        logRetention: RetentionDays.ONE_DAY,
    });
    taskdef.addContainer(`container-${containerConfig.dockerHubImage}`, { image, logging: logdriver });

    return taskdef;
};

We also need to update the code in our main program file to enable public IP assignment:

addService(stack, `service-${taskConfig.family}`, cluster, taskdef, 80, 0, true);

Now we should hopefully be ready! When you have done the changes, run cdk diff to see that you get changes that reflect the logging updates and the public IP assignment. If that looks ok, then try cdk deploy and deploy the updates.

Once the deployment is done, let us check how we did. The service is deployed, and we can see that our manual modification of desired count is gone, it is set to 0 again. This is what we want.

In the Details tab of the service, we look at the Network Access section and see that the value of public IP assignment now is set to enabled:

We can then also analyze the task definition for the web server, and look at the specific container configuration, where we will find the log configuration:

So now we are ready to test our update! Perform the same steps as before to update the number of tasks (i.e. desired count) to 1 for the service and check the status for the task started. It will go from PROVISIONING to PENDING, and in this case finally to RUNNING! Success!

Let us also double-check if we get any logs. In the CloudWatch Console, if we select the Logs section, we can actually see a log group in place:

In the log group, we can find a log stream. Opening up the stream itself, we can see a couple of log records:

This is good! The service is running, and we have logs from it!

We have successfully deployed and started the service. You can confirm this by using the public IP address. We got to see that we get the default response from the Apache web server.

Next, you can set the desired count in the addService() function call, set a value 1, and verify that the service starts up and is accessible, with no manual steps needed.

You can also try to stop the task that is running then. After clicking the Stop button, it will stop within seconds. ECS will start a new task to replace the old one. It will not get the same IP address as the old task though, so even though it is accessible, it is not accessible from the same address. This is something we need to improve on.

Final words

In this part, we added the ability to provision a service and get that started automatically. We have some shortcomings in this, in that we do not maintain a fixed address to reach our web server. This is something we need to address later. We also learned that we do not get any logs by default and need to add a configuration entry for this as well.

In addition, we learned that ECS performs some health checks on the tasks in the service. In this case, we had to make it reachable from the internet for the health check to work.

So from our goal list we have a kind of ticked off another entry:

Expose an endpoint for a web server for HTTP traffic from internet.
Web server shall run in a container.
The container itself shall not be directly reachable from internet.
We should be able to have a service set up so that containers will automatically be started if needed.
We should be able to build our custom solution for this web server.
We should be able to get container images from DockerHub.
We do not care about managing the underlying server infrastructure that runs the containers. I.e., we will use Fargate.

In the upcoming parts, we will address the remaining points in the goal list. We will also take a step back and think more about monitoring and structure, and how we can test that we get what we expect before we deploy. This will become more important once our solutions grow larger and more complex.

As always, you can also check out Tidy Cloud AWS for more content that may be of interest if you work with infrastructure-as-code with AWS.

Don't forget to use cdk destroy to remove the infrastructure you deployed! AWS does not bill you for what you use, as much as what you forget to shut down.

How to become an infrastructure-as-code ninja, using AWS CDK - part 4

Erik Lundevall Zara — Mon, 06 Dec 2021 10:30:24 +0000

It is now time to create an infrastructure in AWS to run container-based solutions!

We will use the Amazon Web Services (AWS) service Elastic Container Service (ECS) to run an application packaged as a container. We will also put a load balancer in front of it, so that we can run multiple copies of that containerised application and distribute the load among those copies.

This is part 4 in a series of articles about building cloud infrastructure solutions with AWS Cloud Development Kit (AWS CDK). If you have not already read them, I recommend checking out part 1, 2, and 3 as well, which are all here at dev.to, or on the Tidy Cloud AWS website.

Update:

This article series uses Typescript as an example language. However, there are repositories with example code for multiple languages.

The repositories will contain all the code examples from the articles series, implemented in different languages with the AWS CDK. You can view the code from specific parts and stages in the series, by checking out the code tagged with a specific part and step in that part, see the README file in each repository for more details.

What are containerised applications?

Containers in computer infrastructure and software terms are a way to package applications or software solutions with its dependencies in a way that is portable and (relatively) easy to manage. It has become a popular way to deploy and manage various types of applications.

We will not dive into details about creating, building, and using containers. There are many other resources that cover that extensively. Here are a few links to check out for information about containers (and Docker, which popularised the concept):

Containers in AWS

There are actually many ways to run containers in AWS. Some may argue that there are way too many, which adds to confusion about what to choose.

The Elastic Container Service (ECS) is one of the core services in AWS to run containers, and it is not too hard to get started with, so that is what we will pick here.

It also helps that AWS CDK has substantial support for ECS.

We will also use the flavour of ECS, that is called ECS Fargate. When we use Fargate, we will not need to concern ourselves with the underlying servers that the containers run on. These details are something that Fargate handles for us.

Note: ECS is not a free tier service in AWS. If you spin up containers in ECS, you will pay for the privilege. It will not cost much (probably < $1) unless you forget to shut it down afterwards you are done with this article.

A few pieces of terminology that may be of use here:

A cluster. This is a collection of computational resources (also known as servers) that the containers run on, and the resources needed to manage these computational resources. With ECS Fargate, we do not need to care about the servers in the cluster, but we will still need a cluster to let AWS manage it for us.
A container image. A reference to a specific solution/application packaging that we will use. This is where we have packaged the software components we want to run.
An ECS Task. A collection of one or more container images with associated configuration we want to run as a unit in ECS. The specification of what those images are and their configuration is a TaskDefinition.
An ECS Service. When we want to run an ECS task continuously in one or more copies and make sure it stays up, then we configure an ECS Service. ECS will do the heavy lifting for us to make sure it keeps running. If the task(s) itself dies or fails, ECS will auto-start up new ones.

Unfortunately, the terminology that different platforms to run containers use can differ, so this can become a bit confusing if you look at Kubernetes, or Docker Swarm. They are like ECS, but use partially different terminology.

Goals

Before we begin, let us define what we should accomplish. We will keep it relatively simple, but still not too simple. After that, we will refine our solution to make it better in various ways. Along the way, we should pick up some good practices and feature of the AWS CDK!

Expose an endpoint for a web server for HTTP traffic from internet.
Web server shall run in a container.
The container itself shall not be directly reachable from internet.
We should be able to have a service set up so that containers will automatically be started if needed.
We should be able to build our custom solution for this web server.
We should be able to get container images from DockerHub.
We do not care about managing the underlying server infrastructure that runs the containers. I.e., we will use Fargate.

We will not fix all these points in one go, but iterate on these points. By listing a few points for our goals, we have something to focus on. Once we are done with these, we can refine even further.

The application solution is initially going to be an Apache web server. This is a straightforward solution for us to test that we have something working, as there are pre-built container images to use.

Initialize our project

Similar to what we did in part 2 of this series of articles, we create a new project and use the AWS CDK command-line tool to initialise it.

mkdir my-container-infrastructure
cd my-container-infrastructure 
cdk init --language=typescript

Contrary to the previous project we did, we will not keep everything in a single source code file. We will though remove all sample content that cdk init generated and write our own code from scratch.

So let us start from the top, in our main program file, bin/my-container-infrastructure.ts. Our starting point will be quite similar to our previous project, with an App and a Stack. We will also fetch the default VPC.


import { App, Stack } from 'aws-cdk-lib';
import { Vpc } from 'aws-cdk-lib/aws-ec2';

const app = new App();
const stack = new Stack(app, 'my-container-infrastructure', {
    env: {
        account: process.env.CDK_DEFAULT_ACCOUNT,
        region: process.env.CDK_DEFAULT_REGION,
    },
});

const vpc = Vpc.fromLookup(stack, 'vpc', {
    isDefault: true,
});

If you run this code via cdk synth, get an empty CloudFormation template, which is fine, since we have added no actual resources yet. You will get an error if you have specified no AWS credentials, because it does not know which account and region to look up the VPC from. So make sure you use valid and relevant credentials.

Now we will start with the actual container infrastructure!

Setting up an ECS cluster

The first piece of infrastructure we need to set up to run is an ECS cluster. We could just add a few lines of code in our main files to create the cluster, but we will prepare a bit for future more complex set-up. So instead we will package this in its own module.

In the lib directory, let us create a subdirectory with name containers and in that directory create a file with name container-management.ts. This means the path to the file from the root of the project is lib/containers/container-management.ts.

In this file, we make a function to add a cluster, since we want to add this to our stack to deploy it later. In AWS CDK there is an aws-cdk-lib/aws-ecs sub-module for ECS, in which there is a construct with name Cluster. This is the one we are going to use when we create the cluster. In the documentation for Cluster, we can see that all properties are optional. However, if we want to use our existing VPC, we need to provide that. Otherwise, AWS CDK will create a new VPC for us, which may not be what we want. So with that in mind, our first naïve function signature may look like this:


const addCluster = function(scope: Stack, vpc: IVpc): Cluster {
}

The function will add the cluster we create to the stack we provide and then return a reference to the cluster itself that was created. We pass in the VPC as a parameter, where we use the type IVpc. This is returned by Vpc.fromLookup(). Any type that begins with an “I" is an interface type, which provides a subset of the functionality of a full-blown construct - which is typically the case when the resource is a reference to something that already exists and you not have created yourself.

The implementation is short, like this:


const addCluster = function(scope: Stack, vpc: IVpc): Cluster {
    return new Cluster(scope, 'cluster', {
        vpc,
    });
}

We create a Cluster object, passing in the VPC as a property. The Cluster object we create also gets a name (not the same as a cluster name!), which we have set to cluster.

However, in case we want to reuse the function, this might be troublesome with a hard-coded name for our Cluster object. In AWS CDK, every resource or construct gets an identifying name. It does not have to be globally unique, but at the same level in the resource hierarchy, it has to be unique. So we will need to change that, if we want to be sure to reuse the function without a problem. Thus, we can add a parameter to give the Cluster construct a name as well.


const addCluster = function(scope: Stack, id: string, vpc: IVpc): Cluster {
    return new Cluster(scope, id, {
        vpc,
    });
}

If we look closer to the parameters that Cluster expects in the AWS CDK documentation when we create an object, we can see in its signature that the first parameter is a Construct, not a Stack. A Stack is a kind of Construct, so it is perfectly fine to pass in a Stack. The reason for the first parameter to be a Construct is that we can compose multiple resources to build our own constructs, which is exactly how the AWS CDK itself has built the higher-level building blocks from the CloudFormation-based primitives.

So allowing ourselves to reuse this not only directly in a Stack, but in constructs we create ourself, let us change the type of the first parameter to Construct instead:


const addCluster = function(scope: Construct, id: string, vpc: IVpc): Cluster {
    return new Cluster(scope, id, {
        vpc,
    });
}

Except for the type change, the code is identical. We also need to export the code from our module, so we can use it in our main program. The complete code, including export and import statements, looks like this:


import { IVpc } from 'aws-cdk-lib/aws-ec2';
import { Cluster } from 'aws-cdk-lib/aws-ecs';
import { Construct } from 'constructs';

export const addCluster = function(scope: Construct, id: string, vpc: IVpc): Cluster {
    return new Cluster(scope, id, {
        vpc,
    });
}

In our main program in bin/my-container-infrastructure.ts, we import the function and add a cluster to the stack we created previously. The code would look like this:


import { App, Stack } from 'aws-cdk-lib';
import { Vpc } from 'aws-cdk-lib/aws-ec2';
import { addCluster } from '../lib/containers/container-management';

const app = new App();
const stack = new Stack(app, 'my-container-infrastructure', {
    env: {
        account: process.env.CDK_DEFAULT_ACCOUNT,
        region: process.env.CDK_DEFAULT_REGION,
    },
});

const vpc = Vpc.fromLookup(stack, 'vpc', {
    isDefault: true,
});

const id = 'my-test-cluster';
addCluster(stack, id, vpc);

You can run cdk synth (or cdk synth ---quiet) to check that AWS CDK thinks your code is ok. You can also run a cdk deploy to deploy the ECS cluster. AWS does not charge you for empty ECS clusters, it is only when you actually add containers to run in it you get charged. If you deploy it and look in the view of ECS in AWS Console, it should look like this:

Great work! Now it is time to add an actual task to run into the cluster.

Create an initial task definition

As we mentioned earlier in this article, we set up something to run by using the Apache httpd web server, for which there are container images already which we can download. To create a Task, we need a TaskDefinition. It needs to contain at least one container image definition. Let us assume we need to configure things at the task level and at the container level.

Let us just add some placeholders for task and container configuration, and for now only handle the case with a single container in a task. So a first stab at a function for this could be:


export interface TaskConfig {

}

export interface ContainerConfig {

}

export const addTaskDefinitionWithContainer = 
function(scope: Construct, id: string, taskConfig: TaskConfig, containerConfig: ContainerConfig): TaskDefinition {

};

In our goals, we stated we do not care about managing the underlying server infrastructure that the containers run in; we let AWS handle that for us. This means that we will use a FargateTaskDefinition to define our task. Such a definition requires at least to specify the amount of CPU to allocate and the memory limit for the container. So we should add that to our TaskConfig. Another nice-to-have parameter is the family, which is a name to group multiple versions of a task definition. Our updated TaskConfig then looks like this:


export interface TaskConfig {
    readonly cpu: 256 | 512 | 1024 | 2048 | 4096;
    readonly memoryLimitMB: number;
    readonly family: string;
}

There are only a few fixed values that we can set for CPU, where 1024 represents 1 vCPU (virtual CPU). So we define the cpu property to only include these values. There are also some restrictions on the memory based on the CPU setting, but it is a bit too complex for a simple type definition. So we just expect a number for the memory limit.

Now we can add a task definition in our function:


export const addTaskDefinitionWithContainer = 
function(scope: Construct, id: string, taskConfig: TaskConfig, containerConfig: ContainerConfig): TaskDefinition {
    const taskdef = new FargateTaskDefinition(scope, id, {
        cpu: taskConfig.cpu,
        memoryLimitMiB: taskConfig.memoryLimitMB,
        family: taskConfig.family,
    });

    return taskdef;
};

Since a task definition can have one or more containers, adding container(s) is a separate operation. Thus we will add a method call for this on the task definition, addContainer(). The minimum we have to add is a reference to a container image. If we for now assume we will just use a container image from DockerHub, then we can define ContainerConfig as this for now:


export interface ContainerConfig {
    readonly dockerHubImage: string;
}

Using this input to our addTaskDefinitionWithContainer() function, we can now add a minimal container configuration. The ContainerImage.fromRegistry() function retrieves image information from DockerHub or other public image registries. Will this be enough for us to get it running? Let us find out!


export const addTaskDefinitionWithContainer = 
function(scope: Construct, id: string, taskConfig: TaskConfig, containerConfig: ContainerConfig): TaskDefinition {
const taskdef = new FargateTaskDefinition(scope, id, {
        cpu: taskConfig.cpu,
        memoryLimitMiB: taskConfig.memoryLimitMB,
        family: taskConfig.family,
    });

    const image = ContainerImage.fromRegistry(containerConfig.dockerHubImage);
    taskdef.addContainer(`container-${containerConfig.dockerHubImage}`, { image, });

    return taskdef;
};

We need to add a call to our new function in our main program, so that we can add the task definition to our stack.

const taskConfig: TaskConfig = { cpu: 512, memoryLimitMB: 1024, family: 'webserver' };
const containerConfig: ContainerConfig = { dockerHubImage: 'httpd' };

addTaskDefinitionWithContainer(stack, `taskdef-${taskConfig.family}`, taskConfig, containerConfig);

We add this to the end of our main program. Now we add an ECS cluster to our stack, as well as an ECS Task Definition - at least from the point of view of our AWS CDK code. What about IAM permissions, security groups? AWS CDK will create a few things under the hood for us. We can get an idea of what will actually be created by running the command cdk diff with our updated stack. But first, this is how our main program file looks like now:

import { App, Stack } from 'aws-cdk-lib';
import { Vpc } from 'aws-cdk-lib/aws-ec2';
import { addCluster, addTaskDefinitionWithContainer, ContainerConfig, TaskConfig } from '../lib/containers/container-management';

const app = new App();
const stack = new Stack(app, 'my-container-infrastructure', {
    env: {
        account: process.env.CDK_DEFAULT_ACCOUNT,
        region: process.env.CDK_DEFAULT_REGION,
    },
});

const vpc = Vpc.fromLookup(stack, 'vpc', {
    isDefault: true,
});

const id = 'my-test-cluster';
addCluster(stack, id, vpc);

const taskConfig: TaskConfig = { cpu: 512, memoryLimitMB: 1024, family: 'webserver' };
const containerConfig: ContainerConfig = { dockerHubImage: 'httpd' };

addTaskDefinitionWithContainer(stack, `taskdef-${taskConfig.family}`, taskConfig, containerConfig);

With this program, if we run cdk diff, and we already deployed the empty ECS cluster before, the changes may look like this:


❯ cdk diff
Stack my-container-infrastructure
IAM Statement Changes

┌───┬───────────────────────────────────┬────────┬────────────────┬─────────────────────────────────┬───────────┐
│   │ Resource                          │ Effect │ Action         │ Principal                       │ Condition │
├───┼───────────────────────────────────┼────────┼────────────────┼─────────────────────────────────┼───────────┤
│ + │ ${taskdef-webserver/TaskRole.Arn} │ Allow  │ sts:AssumeRole │ Service:ecs-tasks.amazonaws.com │           │
└───┴───────────────────────────────────┴────────┴────────────────┴─────────────────────────────────┴───────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Resources

[+] AWS::IAM::Role taskdef-webserver/TaskRole taskdefwebserverTaskRole18D47E42 
[+] AWS::ECS::TaskDefinition taskdef-webserver taskdefwebserver2E748BDD

We can see there is an IAM Role with permissions that seems to be related to ECS Task execution. We also see that there is the ECS TaskDefinition. When we use the cdk diff command, we can see what changes a deployment of our AWS CDK code would make to our stack. However, this information is presented in terms of CloudFormation resources. While you do not need to be an expert in CloudFormation, it is certainly beneficial to read and understand CloudFormation resource descriptions. This is also true when we write unit tests for our infrastructure, which we will see in a later article.

If you run cdk deploy to deploy this added infrastructure, we can log in to the AWS Console and see the result. In the ECS Service view in AWS Console, see an ECS task appearing now:

You can drill further into the task definition to see the settings in place there. What we should focus on now, though, is to get it running. At this stage, we will run the task manually from AWS Console. Later, we automate that as well. Select the Cluster view under ECS and click on the cluster name there. Select the Task tab in the new view. Click on the Run new Task button.

In the Run Task view, select launch type Fargate, and operating system family Linux. There is only one task definition in place, and only one version.

Next under VPC and security groups you select the VPC id of the default VPC, and pick one or more of the subnets presented in the list. In the security group setting, you can notice a security group name. This is a security group that the AWS Console has generated for you - not something we have created. It is set up by default to allow incoming traffic on port 80 from anywhere.

Scroll down to the end of the page and click on Run Task. The task should not be set up, first in stage PROVISIONING, then in state RUNNING. It should only take a couple of seconds, but you may need to refresh the view to see the status changes.

If you click on the link under the headline Task in the task status view above, see a detailed view about the task itself. This view includes the private and public IP addresses of the task. Since we used the default VPC and it has only public subnets, the default here is to allocate a public IP address to the task. This is not what we want in the long run, but it is useful for us right now to see that the task with Apache web server is running properly.

Note the public IP address you got in your view, and enter that in a web browser. It should try to access that IP address via HTTP by default and we should see the default response from Apache web server if we try that (the text It works!).

Great work! We have now been able to create a cluster and run a container-based task in that cluster!

Rounding up and final notes

You should probably destroy the provisioned infrastructure before you forget it, so remember to run cdk destroy before you forget what you did here!

Looking at our goal list, we got some points we covered here, and a few that remain:

Expose an endpoint for a web server for HTTP traffic from internet.
Web server shall run in a container.
The container itself shall not be directly reachable from internet.
We should be able to have a service set up so that containers will automatically be started if needed.
We should be able to build our custom solution for this web server.
We should be able to get container images from DockerHub.
We do not care about managing the underlying server infrastructure that runs the containers. I.e., we will use Fargate.

So out of the 7 goals, we have covered 4 of them. We will continue with the remaining points in the next article (part 5), so check out that one! If you are interested in similar materials, you can also visit the Tidy Cloud AWS website to look for more content.

How to become an infrastructure-as-code ninja, using AWS CDK - part 3

Erik Lundevall Zara — Thu, 25 Nov 2021 11:41:09 +0000

Refine the EC2 deployment

In the previous article, we performed a basic set pf an EC2 virtual machine, using AWS CDK. Our goals were these:

An EC2 instance
The instance should run Linux, we will pick Amazon Linux for simplicity
The instance should not be reachable from the internet
We should be able to login and access the machine from a command-line prompt in the AWS Console.
We should use an existing VPC and its subnets.
We do not care about which availability zone the machine ends up in.

If you have not read the previous article, I recommend you to start with that article first. When you are done there, jump back in here!

The goals in bold were the ones we managed to do in the previous article, so now we have one items on the list left to do - be able to login to the machine via the AWS Console, and still keep the machine private and not reachable from the internet.

Update:

This article series uses Typescript as an example language. However, there are repositories with example code for multiple languages.

The repositories will contain all the code examples from the articles series, implemented in different languages with the AWS CDK. You can view the code from specific parts and stages in the series, by checking out the code tagged with a specific part and step in that part, see the README file in each repository for more details.

The road to login - a history lesson

In the old days of EC2, you would generally always create or re-use a key pair in order to login to an EC2 virtual machine. The key data associated with that key pair was something you downloaded, and sometimes may have shared with other colleagues that also needed access to the same machine.

This was a troublesome and cumbersome approach for many reasons, but for a long time pretty much the only approach available. Other public cloud vendors provided better solutions, and in the end so did also AWS.

The approach we are going to use with our virtual machine is to take advantage of a feature in the AWS Systems Manager service, with the name Session Manager. This service allows us to tunnel traffic to an EC2 instance without opening up the security groups for any traffic.

Instead, AWS System Manager has an agent that connect to the AWS Systems Manager service from the EC2 virtual machine. The user who wants to login communicates with the AWS System Manager service, which then can forward traffic via the connection that the Systems Manager agent established to Systems Manager itself.

Luckily for us, we are using Amazon Linux latest version, which already has the System Manager agent installed on the EC2 virtual machine. What we will need to do is to add Identity & Access Management (IAM) permissions to the EC2 virtual machine, so that it (actually the Systems Manger agent) can call Systems Manager and perform the required steps.

EC2 instance profiles

AWS has something that is called instance profiles. It is essentially a configuration that associates a set of IAM permissions in the form of an IAM Role to an EC2 instance. This allows any software that is running on that EC2 instance to use any services that the IAM Role has permissions for.. No need for any access keys or secret information to be stored on the instance itself.

So by using this feature, we can allow the Systems Manager agent on our virtual to perform the necessary operations to allow us to login to the machine, without any port openings or secret keys exposed. We just need the necessary permissions to allow the connection via the AWS Console. If you have full administrative permissions, then you have those permissions.

More detailed permissions to allow such access via AWS Console is out of the scope of this article, that is for a more in-depth article on this particular feature.

The anatomy of the IAM Role

There are two things we need to include in the IAM Role:

Who can assume the role
What permissions are needed to perform the tasks for using Systems Manager Session Manager

Since we are using an EC2 instance to communicate with Systems Manager, it is the EC2 service itself that must be able to assume the role. There are a number of permissions needed for Session Manager to work, but luckily we can simply use a predefined policy that AWS provides for this. This policy has the name AmazonSSMManagedInstanceCore. With this information, we can create the required IAM Role in our AWS CDK code:

const role = new Role(stack, 'ec2-role', {
    assumedBy: new ServicePrincipal('ec2.amazonaws.com'),
    managedPolicies: [ ManagedPolicy.fromAwsManagedPolicyName('AmazonSSMManagedInstanceCore') ],
});

We then need to attach this role to the EC2 instance we create.

const instance = new Instance(stack, 'my-ec2', {
    instanceType: InstanceType.of(InstanceClass.T2, InstanceSize.MICRO),
    machineImage: MachineImage.latestAmazonLinux(),
    role,
    vpc,
});

This means that our complete code now looks like this:

import {App, Stack} from 'aws-cdk-lib';
import {Instance, InstanceClass, InstanceSize, InstanceType, MachineImage, Vpc} from 'aws-cdk-lib/aws-ec2';
import {ManagedPolicy, Role, ServicePrincipal} from 'aws-cdk-lib/aws-iam';

const app = new App();
const stack = new Stack(app, 'my-stack', {
    env: {
        account: process.env.CDK\_DEFAULT\_ACCOUNT,
        region: process.env.CDK\_DEFAULT\_REGION,
    },
});

const role = new Role(stack, 'ec2-role', {
assumedBy: new ServicePrincipal('ec2.amazonaws.com'),

managedPolicies: [ ManagedPolicy.fromAwsManagedPolicyName('AmazonSSMManagedInstanceCore') ],

});

const vpc = Vpc.fromLookup(stack, 'my-vpc', {
    isDefault: true,
});

const instance = new Instance(stack, 'my-ec2', {
    instanceType: InstanceType.of(InstanceClass.T2, InstanceSize.MICRO),
    machineImage: MachineImage.latestAmazonLinux(),
    role,
    vpc,
});

Now with this refinement of our code we can run cdk deploy to deploy our EC2 instance again. After a few minutes, the EC2 instance has been deployed and is running. Now, if you login to the AWS Console and go to the EC2 dashboard with the running instance, you will see a view similar to this below. Select the instance in the checkbox on the left side and click on the Connect button.

You will get a different page with a few options to connect to the virtual machine instance. In this case, you should select the Session Manager tab. If the setup worked as expected, the Connect button at the bottom right should have clear colours and not be greyed out.

Click on this Connect button and a new page/tab should open up, and you should get a command line prompt that you can use. You have now connected to the virtual machine instance!

Feel free to play around and look around on the machine itself, then just terminate the session, run exit command door anything else to end the command-line session.

Final remarks

The last addition we did here allowed us to complete all the goals we had for this initial infrastructure-as-code task using the AWS CDK. Well done!

An EC2 instance
The instance should run Linux, we will pick Amazon Linux for simplicity
The instance should not be reachable from the internet
We should be able to login and access the machine from a command-line prompt in the AWS Console.
We should use an existing VPC and its subnets.
We do not care about which availability zone the machine ends up in.

Feel free to use cdk destroy command to remove the virtual machine instance. There are a few valuable lessons that we have covered so far in this series:

A few key commands with the cdk command-line tool, including init, bootstrap, synth, deploy, destroy.
How to import existing VPC information.
How to create a new EC2 virtual machine
How to create and attach IAM Role with permissions so that software on the machine can use AWS services
How to set AWS account and region for a stack.

It is quite a few things to start with, and you should be proud of yourself to have done all of this!

In the next article, we are going to jump into the world of containers.

DEV Community: Erik Lundevall Zara

Quick blog set-up with Quarto and Fly.io

Installation

Set-up of website/blog content

Set-up of webserver

Version control tweaks

Optional - automate deployment on change through GitHub actions

Custom domain and certificates

Final remarks

An opinionated AWS CDK project template for Python

Template summary

Usage

Reasoning behind the template

Examples

Design considerations

Things to do

Summary

How to become an infrastructure-as-code ninja, using AWS CDK - part 8

Finding and using 3rd party libraries for AWS CDK

Set up our solution monitoring

Recap of solution setup

bin/my-container-infrastructure.ts

lib/containers/container-management.ts

test/containers/container-management.test.ts

Where to start with the monitoring?

Add monitoring foundation

Add monitoring of actual resources

Setting up alarms - what do we need?

Enabling container insights

Adding an alarm

Add an alarm notification topic

Add an alarm notification

Alarm severity and category

Summary and final words

Appendix: Code re-cap

bin/my-container-infrastructure.ts

lib/containers/container-management.ts

lib/monitoring/index.ts

test/containers/container-management.test.ts

test/monitoring.test.ts

How to become an infrastructure-as-code ninja, using AWS CDK - part 7

A recap

Note about code availability

What we will do now

The trouble with default VPCs

Starting from the tests - TDD a load-balanced service

A bit of auto-scaling

Infrastructure-as-code - where is the customer obsession, AWS?

Meeting customers where they are

Migration issues

Other solutions to the problem

How to set up ad hoc (developer) environments

Tweak and persist the package information

Final words

Take a cue to supercharge your configurations

Configuration complexity

Install CUE

Read and convert YAML configuration

Define data validation

Reusable definitions and lists

Adding constraints

Enforcing validation

Final words

How to become an infrastructure-as-code ninja, using AWS CDK - part 6

Different testing

Who writes the tests?

Get started with writing tests

Infrastructure recap

Let us write the first test

Task definition testing

Test for Fargate Task Definition

Test for container definition

Test the service

Does our design suck? Wrapping up

How to migrate from CloudFormation to AWS CDK - part 1

Goals

Requirements

AWS CloudFormation vs AWS Cloud Development Kit

AWS CDK Installation

First sample migration