DEV Community

AWS Heroes profile image Kenta Goto
Kenta Goto for AWS Heroes

Posted on

AWS CDK Unit Testing Guide: When and How to Use Different Test Types

#aws #awscdk #cdk

Types of Unit Tests in AWS CDK

There are three main types of unit tests in AWS CDK:

  • Snapshot tests
  • Fine-grained assertions tests
  • Validation tests

Snapshot Tests

What are Snapshot Tests?

Snapshot tests in AWS CDK are tests that output the AWS CloudFormation template synthesized from CDK code and compare it with the template generated from a previous test run to detect template differences.

import { App } from 'aws-cdk-lib';
import { Match, Template } from 'aws-cdk-lib/assertions';
import { MyStack } from '../lib/my-stack';

describe('MyStack Tests', () => {
  test('Snapshot Tests', () => {
    const app = new App();
    const stack = new MyStack(app, 'MyStack');

    const template = Template.fromStack(stack);
    expect(template.toJSON()).toMatchSnapshot();
  });
});
Enter fullscreen mode Exit fullscreen mode

When you run this test file, a __snapshots__ directory is created directly under the test directory, and a snapshot file named my-stack.test.ts.snap is saved inside. The snapshot file is saved in JSON format of the CloudFormation template. Then it compares the snapshot saved from the previous test run with the template generated by the current CDK code, and if there are differences, the test fails.

If the difference is as expected, you can update the snapshot file to the one generated this time by running the test with the command npx jest --updateSnapshot.

Use Cases for Snapshot Tests

I believe snapshot tests are essential in almost all cases in AWS CDK development, except for some early development phases.

They are particularly effective in the following scenarios:

  1. AWS CDK version updates
  2. CDK code refactoring
  3. Difference management in version control systems

1. AWS CDK Version Updates

One major reason to use snapshot tests is that as long as the snapshot test passes when you update the AWS CDK library version, there is no difference in the CloudFormation template, meaning you can guarantee there is no impact on deployed resources.

The AWS CDK library has various changes added with each version update, and new features are added and bugs are fixed, which can sometimes change the output of CloudFormation templates.

However, in projects developing with AWS CDK, if the content of the generated CloudFormation template changes when upgrading the AWS CDK version, unexpected changes may occur to already deployed resources. Snapshot tests can detect and prevent this in advance.

Basically, on the AWS CDK library development side, which is OSS (Open Source Software), development is carried out to minimize changes to the generated CloudFormation templates, but it may inevitably change sometimes, so it becomes important for users to perform snapshot tests on their side. (However, there are mechanisms in place to prevent breaking changes in AWS CDK OSS development, so you can rest assured.)

2. CDK Code Refactoring

Snapshot tests are also very useful when refactoring CDK code.

Basically, in refactoring, it is required that the behavior (CloudFormation template in CDK) does not change before and after the code change.

By utilizing snapshot tests, you can confirm that the CloudFormation template generated by refactoring is the same as before refactoring.

3. Difference Management in Version Control Systems

When managing code including snapshot files with version control systems like Git, they demonstrate even more powerful effects when making resource definition changes during development and operation.

This is because update differences are visualized and recorded not only at the CDK code granularity but also at the CloudFormation template granularity.

You can also check unexpected changes that are difficult to understand at the CDK code level, minimizing risks related to resource definition changes.

Fine-grained Assertions Tests

What are Fine-grained Assertions Tests?

Fine-grained assertions tests in AWS CDK are tests that extract part of the generated CloudFormation template and check that part. This allows you to test detailed components such as what kind of resources are generated.

For example, you can perform fine-grained tests such as "Are two AWS::SNS::Subscription resources generated?" or "Is nodejs20.x set for the Runtime of AWS::Lambda::Function?"

import { App, assertions } from 'aws-cdk-lib';
import { Match, Template } from 'aws-cdk-lib/assertions';
import { MyStack } from '../lib/my-stack';

const getTemplate = (): assertions.Template => {
  const app = new App();
  const stack = new MyStack(app, 'MyStack');
  return Template.fromStack(stack);
};

describe('Fine-grained assertions tests', () => {
  test('Two SNS subscriptions are created', () => {
    const template = getTemplate();
    template.resourceCountIs('AWS::SNS::Subscription', 2);
  });

  test('Lambda has nodejs20.x', () => {
    const template = getTemplate();
    template.hasResourceProperties('AWS::Lambda::Function', {
      Handler: 'handler',
      Runtime: 'nodejs20.x',
    });
  });
});
Enter fullscreen mode Exit fullscreen mode

Use Cases for Fine-grained Assertions Tests

For these fine-grained assertions tests, the theory of specifically when and what tests to write hasn't been well established yet, I feel.

This is because you can perform various checks at fine granularities such as per resource and per property, and a wide range of test cases can be considered. Also, if you write tests for all resources, it becomes an even more enormous amount, and there are cases where the redundancy and maintenance difficulty outweigh the benefits of testing.

Therefore, while this is my personal judgment criteria, I think it's good to use them in the following scenarios:

  1. Loop processing
  2. Conditional branching
  3. Property override
  4. Definitions you especially want to guarantee
  5. Value specification using props

As a premise, AWS CDK can be written in programming languages, so you can perform resource definition code writing "procedurally", but it's also possible to write resource definitions "declaratively".

*Here, "declarative" refers to declaring the existence of resources, such as "create resource XX." On the other hand, "procedural" refers to writing resource creation procedures procedurally, such as "to create resource XX, first perform this process, then perform this process."

As infrastructure definitions, I think "declarative" is often preferable because you can easily understand what resources are generated just by looking at the definition. Since AWS CDK is primarily a tool for infrastructure definition, it will often be written "declaratively."

And in declarative descriptions like "create resource A," it is self-evident that "resource A is created." For the benefit of confirming something self-evident, fine-grained assertions tests can create code that is almost the same as the resource definition side code, leading to the troublesome nature of dual definition, which can increase test maintenance costs.

For such reasons, as my personal judgment criteria, I think it's good to use them in the above scenarios rather than writing detailed fine-grained assertions tests for all resources.

1. Loop Processing

While I mentioned writing resource definitions "declaratively" above, when using loop processing to generate resources, it becomes "procedural" code writing.

In other words, what kind of resources are generated by this is no longer self-evident.

Therefore, when using loop processing to generate resources, it becomes important to write fine-grained assertions tests to confirm that the loop processing is working correctly.

Specifically, suppose you have the following CDK code:

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Topic } from 'aws-cdk-lib/aws-sns';

export interface MyStackProps extends cdk.StackProps {
  appNames: string[];
}

export class MyStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: MyStackProps) {
    super(scope, id, props);

    // Make unique combinations considering cases with duplicate elements
    const appNames = new Set(props.appNames);

    for (const appName of appNames) {
      new Topic(this, `${appName}Topic`, {
        displayName: `${appName}Topic`,
      });
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

For resource definitions using loop processing like this, you can write fine-grained assertions tests as follows:

import { App } from 'aws-cdk-lib';
import { Match, Template } from 'aws-cdk-lib/assertions';
import { MyStack } from '../lib/my-stack';

describe('Fine-grained assertions tests', () => {
  test('SNS Topics are created', () => {
    const appNames = ['App1', 'App1', 'App2'];
    const expectedNumberOfTopics = 2;

    const app = new App();
    const stack = new MyStack(app, 'MyStack', {
      appNames: appNames,
    });
    const template = Template.fromStack(stack);

    template.resourcePropertiesCountIs(
      'AWS::SNS::Topic',
      {
        DisplayName: Match.stringLikeRegexp('Topic'),
      },
      expectedNumberOfTopics,
    );
  });
});
Enter fullscreen mode Exit fullscreen mode

2. Conditional Branching

When using conditional branching like if statements to change whether resources are generated per environment, it's also important to confirm that the conditional branching is working correctly.

Suppose you have the following CDK code:

import * as cdk from 'aws-cdk-lib';
import { CfnWebACL } from 'aws-cdk-lib/aws-waf';
import { Construct } from 'constructs';

export interface MyStackProps extends cdk.StackProps {
  isProd: boolean;
}

export class MyStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: MyStackProps) {
    super(scope, id, props);

    if (props.isProd) {
      new CfnWebACL(this, 'WebAcl', {
        // ...
      });
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

To confirm that CfnWebACL is created when isProd is true, you can write a test like this:

import { App } from 'aws-cdk-lib';
import { Template } from 'aws-cdk-lib/assertions';
import { MyStack } from '../lib/my-stack';

describe('Fine-grained assertions tests', () => {
  test('Web ACL is created in prod', () => {
    const app = new App();
    const stack = new MyStack(app, 'MyStack', {
      isProd: true,
    });
    const template = Template.fromStack(stack);

    template.resourceCountIs('AWS::WAFv2::WebACL', 1);
  });
});
Enter fullscreen mode Exit fullscreen mode

Now, for cases where you change whether to specify properties per environment:

import * as cdk from 'aws-cdk-lib';
import { Distribution } from 'aws-cdk-lib/aws-cloudfront';
import { Construct } from 'constructs';

export interface MyStackProps extends cdk.StackProps {
  isProd: boolean;
}

export class MyStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: MyStackProps) {
    super(scope, id, props);

    // ...

    new Distribution(this, 'Distribution', {
      // ...
      webAclId: props.isProd ? webAclId : undefined,
    });
  }
}
Enter fullscreen mode Exit fullscreen mode

For example, you can also write a test to confirm that webAclId is "not associated" when isProd is false.

The characteristic is that you can use the Match.absent method to confirm that the corresponding property has "no value specified".

import { App } from 'aws-cdk-lib';
import { Match, Template } from 'aws-cdk-lib/assertions';
import { MyStack } from '../lib/my-stack';

describe('Fine-grained assertions tests', () => {
  test('Web ACL is not associated in dev', () => {
    const app = new App();
    const stack = new MyStack(app, 'MyStack', {
      isProd: false,
    });
    const template = Template.fromStack(stack);

    template.hasResourceProperties('AWS::CloudFront::Distribution', {
      DistributionConfig: {
        // Confirm that WebACLId property is not specified
        WebACLId: Match.absent(),
      },
    });
  });
});
Enter fullscreen mode Exit fullscreen mode

3. Property Override

In AWS CDK, it's common to define resources using L2 Constructs provided by CDK.

However, in cases where you want to set properties not supported by L2 Constructs, you may use escape hatches to cast to L1 Constructs and then override properties using methods like addPropertyOverride.

In this case, you can't use Construct types for property specification and need to write properties according to the CloudFormation template structure yourself, making description mistakes more likely to occur, especially for hierarchical properties.

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Bucket, CfnBucket } from 'aws-cdk-lib/aws-s3';

export class MyStack extends cdk.Stack {
  constructor(scope: Construct, id: string) {
    super(scope, id);

    const bucket = new Bucket(this, 'Bucket');

    // Use escape hatch on Bucket to override properties
    const cfnSrcBucket = bucket.node.defaultChild as CfnBucket;
    cfnSrcBucket.addPropertyOverride('NotificationConfiguration.EventBridgeConfiguration.EventBridgeEnabled', true);
  }
}
Enter fullscreen mode Exit fullscreen mode

When using override to overwrite resource definitions like this, you can write tests to confirm that it's reflected as intended.

import { App, assertions } from 'aws-cdk-lib';
import { Match, Template } from 'aws-cdk-lib/assertions';
import { MyStack } from '../lib/my-stack';

const getTemplate = (): assertions.Template => {
  const app = new App();
  const stack = new MyStack(app, 'MyStack');
  return Template.fromStack(stack);
};

describe('Fine-grained assertions tests', () => {
  // Whether properties are correctly overridden using escape hatch
  test('EventBridge is enabled', () => {
    const template = getTemplate();
    template.hasResourceProperties('AWS::S3::Bucket', {
      NotificationConfiguration: {
        EventBridgeConfiguration: { EventBridgeEnabled: true },
      },
    });
  });
});
Enter fullscreen mode Exit fullscreen mode

4. Definitions You Especially Want to Guarantee

Next is the case of writing tests for definitions you especially want to guarantee.

This is a test for "declarative" code writing as I explained at the beginning.

Earlier, I explained as if we wouldn't write fine-grained assertions tests for "declarative," i.e., resource definitions that are self-evident, but it's also important to write tests for definitions you especially want to guarantee.

For example, you can write tests as a form of "intention declaration" for important definitions compared to other properties, such as "we want to set this property to realize certain requirements." If another developer changes that setting in the future, the corresponding test will fail, and by referring to the failed test, they can understand the original designer's intention, leading to the propagation of intention.

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Bucket } from 'aws-cdk-lib/aws-s3';

export class MyStack extends cdk.Stack {
  constructor(scope: Construct, id: string) {
    super(scope, id);

    new Bucket(this, 'Bucket', {
      lifecycleRules: [{ expiration: cdk.Duration.days(100) }],
    });
  }
}
Enter fullscreen mode Exit fullscreen mode

If you want to guarantee the expiration setting in this lifecycleRules, you can write a test like this:

import { App, assertions } from 'aws-cdk-lib';
import { Template } from 'aws-cdk-lib/assertions';
import { MyStack } from '../lib/my-stack';

const getTemplate = (): assertions.Template => {
  const app = new App();
  const stack = new MyStack(app, 'MyStack');
  return Template.fromStack(stack);
};

describe('Fine-grained assertions tests', () => {
  test('Expiration for lifecycle must be specified', () => {
    const template = getTemplate();

    template.hasResourceProperties('AWS::S3::Bucket', {
      LifecycleConfiguration: {
        Rules: [
          {
            ExpirationInDays: 100,
            Status: 'Enabled',
          },
        ],
      },
    });
  });
});
Enter fullscreen mode Exit fullscreen mode

In this case, when you change property values due to requirement changes during development, you need to change the test side values accordingly. If you just want to confirm whether a property can be specified, you can use the Match.anyValue method to confirm without specifying specific values, which can reduce test maintenance costs.

import { Match } from 'aws-cdk-lib/assertions';

// ...

template.hasResourceProperties('AWS::S3::Bucket', {
  LifecycleConfiguration: {
    Rules: [
      {
        ExpirationInDays: Match.anyValue(),
        Status: 'Enabled',
      },
    ],
  },
});
Enter fullscreen mode Exit fullscreen mode

Also, when adding definitions using methods provided by CDK like addDependency, there may be cases where you want to guarantee that it's reflected as intended.

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { LogGroup, ResourcePolicy } from 'aws-cdk-lib/aws-logs';
import { PolicyStatement, ServicePrincipal } from 'aws-cdk-lib/aws-iam';
import { HostedZone } from 'aws-cdk-lib/aws-route53';
import { Bucket, CfnBucket } from 'aws-cdk-lib/aws-s3';

export interface MyStackProps extends cdk.StackProps {
  domainName: string;
}

export class MyStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: MyStackProps) {
    super(scope, id, props);

    const logGroup = new LogGroup(this, 'QueryLogGroup');
    const hostedZone = new HostedZone(this, 'HostedZone', {
      zoneName: props.domainName,
      queryLogsLogGroupArn: logGroup.logGroupArn,
    });
    const resourcePolicy = new ResourcePolicy(this, 'QueryLogResourcePolicy', {
      policyStatements: [
        new PolicyStatement({
          principals: [new ServicePrincipal('route53.amazonaws.com')],
          actions: ['logs:CreateLogStream', 'logs:PutLogEvents'],
          resources: [logGroup.logGroupArn],
        }),
      ],
    });

    // Make HostedZone depend on QueryLogResourcePolicy
    hostedZone.node.addDependency(resourcePolicy);
  }
}
Enter fullscreen mode Exit fullscreen mode

In this example, you can confirm whether the expected dependencies are added between resources by writing a test like this:

import { App, assertions } from 'aws-cdk-lib';
import { Match, Template } from 'aws-cdk-lib/assertions';
import { MyStack } from '../lib/my-stack';

const getTemplate = (): assertions.Template => {
  const app = new App();
  const stack = new MyStack(app, 'MyStack', {
    domainName: 'example.com',
  });
  return Template.fromStack(stack);
};

describe('Fine-grained assertions tests', () => {
  // Whether intended dependencies are defined by addDependency
  test('HostedZone depends on QueryLogResourcePolicy', () => {
    const template = getTemplate();
    template.hasResource('AWS::Route53::HostedZone', {
      DependsOn: [Match.stringLikeRegexp('QueryLogResourcePolicy')],
    });
  });
});
Enter fullscreen mode Exit fullscreen mode

5. Value Specification Using Props

Fine-grained assertions tests are also useful in scenarios where you specify resource properties using props values passed to Stacks or Constructs. Specifically, you write tests to confirm that props values are correctly reflected in resources.

This helps prevent forgetting to pass values that can occur when resource properties are not "hard-coded" with specific values directly.

import { App, assertions } from 'aws-cdk-lib';
import { Template } from 'aws-cdk-lib/assertions';
import { MyStack } from '../lib/my-stack';

const getTemplate = (): assertions.Template => {
  const app = new App();
  const stack = new MyStack(app, 'MyStack', {
    messageRetentionPeriodInDays: 10,
  });
  return Template.fromStack(stack);
};

describe('Fine-grained assertions tests', () => {
  test('messageRetentionPeriodInDays from props', () => {
    const template = getTemplate();

    template.hasResourceProperties('AWS::SNS::Topic', {
      // Confirm that props values are correctly passed
      ArchivePolicy: { MessageRetentionPeriod: 10 },
    });
  });
});
Enter fullscreen mode Exit fullscreen mode

Also, to test resource definitions with actually deployed CDK code, I think there are many cases where you want to use the props defined for passing to the actual Stack as-is rather than test-specific props.

In this case, you can guarantee that values passed from props are specified without specifying specific values by using the properties that the props have for confirmation, reducing test maintenance costs.

import { App, assertions } from 'aws-cdk-lib';
import { Template } from 'aws-cdk-lib/assertions';
import { myStackProps } from '../lib/config';
import { MyStack } from '../lib/my-stack';

const getTemplate = (): assertions.Template => {
  const app = new App();

  // Use the props defined for passing to the actual Stack as-is
  const stack = new MyStack(app, 'MyStack', myStackProps);

  return Template.fromStack(stack);
};

describe('Fine-grained assertions tests', () => {
  test('messageRetentionPeriodInDays from props', () => {
    const template = getTemplate();

    template.hasResourceProperties('AWS::SNS::Topic', {
      ArchivePolicy: { MessageRetentionPeriod: myStackProps.messageRetentionPeriod },
    });
  });
});
Enter fullscreen mode Exit fullscreen mode

Validation Tests

What are Validation Tests?

The third type I'll explain, validation tests, are tests related to validation as the name suggests.

Validation refers to processing that verifies the validity of values through conditional branching and other means. In AWS CDK, validation processing is sometimes implemented for properties of props that are inputs to Stacks or Constructs.

For example, in validation tests, you can write code to verify that input values for certain properties fall within a specific range.

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Bucket } from 'aws-cdk-lib/aws-s3';

export interface MyStackProps extends cdk.StackProps {
  lifecycleDays: number;
}

export class MyStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: MyStackProps) {
    super(scope, id, props);

    if (!cdk.Token.isUnresolved(props.lifecycleDays) && props.lifecycleDays > 400) {
      throw new Error('Lifecycle days must be 400 days or less');
    }

    new Bucket(this, 'Bucket', {
      lifecycleRules: [
        {
          expiration: cdk.Duration.days(props.lifecycleDays),
        },
      ],
    });
  }
}
Enter fullscreen mode Exit fullscreen mode

*The cdk.Token.isUnresolved method is a method to check whether a value is not a Token.

For such CDK code, you can write validation tests like this. It's a test to confirm that an error occurs when unacceptable input values are passed.

import { App } from 'aws-cdk-lib';
import { MyStack } from '../lib/my-stack';

describe('Validation tests', () => {
  test('lifecycle days must be lower than or equal to 400 days', () => {
    const app = new App();

    expect(() => {
      new MyStack(app, 'MyStack', { lifecycleDays: 500 });
    }).toThrowError('Lifecycle days must be 400 days or less');
  });
});
Enter fullscreen mode Exit fullscreen mode

Use Cases for Validation Tests

When you implement any validation processing for properties received by Stacks or Constructs, whether that validation processing is working correctly is a very important confirmation item, so validation tests are definitely tests you want to write. It would be good to write test cases for each validation.

Conversely, if you don't implement any particular validation processing, they become unnecessary.

Good Things to Know About CDK Unit Tests

Count Checks and Auto-generated Resources

In fine-grained assertions tests, you can write tests to confirm the count of specific resource types using methods like resourceCountIs of the Template class in the assertions module.

const template = Template.fromStack(stack);
template.resourceCountIs('AWS::Logs::LogGroup', 5);
Enter fullscreen mode Exit fullscreen mode

On the other hand, L2 Constructs commonly used in CDK sometimes automatically generate several resources internally to follow best practices or provide a better developer experience. So, for example, like the test above, even if you think you defined 5 resources of that type, there might actually be 6 resources generated.

Also, even if you set the count to 6 in the test considering such cases, when you look at that value later, the breakdown may be unclear, leading to confusion and cognitive load, so be careful. Unless you particularly want to confirm the count including auto-generated resources, auto-generated resources can also be confirmed from snapshot test update differences, so it might be good to consider simply omitting such count checks. If you still want to keep the test, it's also good to write comments clearly so the intention is understood.

Alternatively, please consider using the resourcePropertiesCountIs method to test counts limited to resources with specific properties or values.

const template = Template.fromStack(stack);
template.resourcePropertiesCountIs(
  'AWS::Logs::LogGroup',
  {
    // Limited to log groups with the naming convention '/aws/lambda/my-app/'
    LogGroupName: Match.stringLikeRegexp('/aws/lambda/my-app/'),
  },
  5,
);
Enter fullscreen mode Exit fullscreen mode

Tests per Construct

In this article, I basically introduced examples of unit tests for actually defined Stack classes.

However, when writing CDK code, you often create custom Constructs and combine them to define Stacks.

In such cases, by writing unit tests for each custom Construct, you can perform tests within the scope of that Construct and confirm behavior without being affected by other Constructs. This allows you to guarantee the reliability and reusability of individual Constructs.

Also, by separating test files for each Construct, each test file becomes cohesive by responsibility and simpler, which might increase understandability.

Specifically, you can define an empty stack and add only the custom Construct you want to test to perform tests.

import { Stack } from 'aws-cdk-lib';
import { Template } from 'aws-cdk-lib/assertions';
import { MyConstruct } from '../lib/constructs/my-construct';

test('Construct Tests', () => {
  // Define an empty stack
  const stack = new Stack();

  // Add the Construct you want to test to the above Stack
  new MyConstruct(stack, 'MyConstruct', {
    messageRetentionPeriodInDays: 10,
  });

  const template = Template.fromStack(stack);

  template.hasResourceProperties('AWS::SNS::Topic', {
    ArchivePolicy: { MessageRetentionPeriod: 10 },
  });
});
Enter fullscreen mode Exit fullscreen mode

However, as the number of Constructs increases, trying to write tests for every Construct can result in a very large number of tests. Also, even if you write tests for each Construct, you definitely want to write tests for the Stack that becomes the actual deployed environment configuration. In that case, you need to be careful because if you don't properly separate the test scope and responsibilities to avoid duplication between Stack tests and Construct tests, test maintenance can become difficult.

It might be good to use the distinction of writing Construct-level tests only for Constructs where you especially want to guarantee reusability. Alternatively, if there are no particular cases of reusing Constructs, the option of not writing unit tests for each Construct is also fine.

Recommended Minimal Configuration

Introducing all the tests introduced in this article together can be quite challenging.

Therefore, if you want to introduce CDK unit tests easily with minimal configuration, it's good to start with snapshot tests first. They are easy to introduce and the ability to detect unexpected changes in deployed CloudFormation templates is a very significant benefit.

Summary

I introduced the following three types of unit tests in AWS CDK and their use cases:

  • Snapshot tests
  • Fine-grained assertions tests
  • Validation tests

Top comments (0)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.