CDK CodeChecker v2: Breaking Free from Third-Party Dependencies

Background

Have you ever relied on a tool only to have it unexpectedly fail? I recently experienced this with CodeChecker, a tool that automatically checks Pull Requests. Despite no code changes, it suddenly stopped working. Upon investigation, it became clear that the underlying software had become deprecated.

As you might have seen in my previous blogs, I like to use the Cloud Development Kit (CDK) for my daily Infrastructure as Code (IaC) work with AWS. During my current assignment, creating a data analytics platform for an enterprise in the financial sector, we are using the CDK CodeChecker a lot. It helps us streamline the development process, keep code quality high, make peer reviews a standard, and test changes in an automated manner.

Real World Scenario

In my current assignment, we encountered a problem with the CodeChecker. The problem is in the 3rd party cloud components construct we are using. It seems that the construct is not maintained by the creator/community. So now it raises the following warning:

[WARNING] aws-cdk-lib.CustomResourceProviderRuntime#NODEJS_14_X is deprecated.

The warning basically means that AWS has NodeJS 14 deprecated and you should use higher versions of node. When you fix this with for example an aspect that bumbs your Lambda functions versions used by the custom resource, you will enter in new errors like:

10:22:02 AM | CREATE_FAILED        | Custom::ApprovalRuleTemplate                      | xxx-xxxx-xxx...omResource/Default
Received response status [FAILED] from custom resource. Message returned: Error: Cannot find module 'aws-sdk'
Require stack:
- /var/task/index.js
- /var/task/__entrypoint__.js
- /var/runtime/index.mjs
at Module._resolveFilename (node:internal/modules/cjs/loader:1134:15)
at Module._load (node:internal/modules/cjs/loader:975:27)
at Module.require (node:internal/modules/cjs/loader:1225:19)
at require (node:internal/modules/helpers:177:18)
at Object.<anonymous> (/var/task/index.js:92:18)
at __webpack_require__ (/var/task/index.js:21:30)
at Object.<anonymous> (/var/task/index.js:102:19)
at __webpack_require__ (/var/task/index.js:21:30)
at /var/task/index.js:85:18
at Object.<anonymous> (/var/task/index.js:88:10) (RequestId: df6317ec-19e3-4aec-ab93-e7da70596c82)

Long story short, it is broken. And, since this is a high-level L3 CDK Construct, a lot has been taken care of and abstracted away. Another problem we had with this construct was that it was not compliant according to the security framework used by the enterprise. So it felt a bit like a houtje-touwtje (Band-Aid) solution. So as the current problems made the CodeChecker failing completely it was time to make an own version.

Solution

There are three solutions to the problem.

1. Contributing to the Cloud Components github repository. Unfortunately, this option was doomed for failure as the project isn't maintained and a lot of open pull requests from other developers are still pending. Also solutions to the current problem were already submitted but not merged at all.

2. Creating a fork of the Cloud Components repository and start maintaining ourselves. The problem here is that the Cloud Components repository consisted of more software than only the CodeChecker, which is basically a combination of two projects already. So this would require maintaining a lot more code than anticipated.

3. Simplify the CodeChecker using Standard CDK libraries. Eventually, this felt best and required the least amount of maintainability!

In the meantime AWS also released a blog on this. We can use this as a reference. Let's optimize the CDK CodeChecker to version 2.0.

Go Build

We want to get rid of the cloudcomponents construct. As a starting point I am using the CodeChecker code from my previous blog. The main responsibility for the cloudcomponents construct is to create an approval template, assign it to a repository and create a codebuild project. For the last there is CDK/CloudFormation intergration, but for the first two there isn't, it's basically 1 API call for creation of an approval template, and 1 for the assign to the repository. So how can we tackle this. Well CDK has a custom resource framework especially for single API calls, called the AwsCustomResource.

Looking at the documentation of how to create an approval template, you can use a json template. The template needs to have a 'DestinationReferences', which describes to which branch the template looks; a 'Type', the amount of approvals needed called; 'NumberOfApprovalsNeeded', and whom is allowed to approve; the 'ApprovalPoolMembers'. So let's create the json template:

import json

template = {
    'Version': '2018-11-08',
    'DestinationReferences': [f'refs/heads/{branch}'],
    'Statements': [
        {
            'Type': 'Approvers',
            'NumberOfApprovalsNeeded': required_approvals,
            'ApprovalPoolMembers': [
                f'arn:aws:sts::{Stack.of(self).account}:assumed-role/developer/*',
                f'arn:aws:sts::{Stack.of(self).account}:assumed-role/{self.codechecker_role.role_name}/*',
            ]
        }
    ]
}
json_string = json.dumps(template)

Above you see the template which we can use in our CDK python code. As you can see it allows the developer role and the CodeChecker itself as approvers. The branch and required_approvals are variables so we can create a loop for multiple branches. On the last line we create a json_string, as the API call is expecting a string instead of an object.

Approval template

Now create the approval template with the AwsCustomResource:

from aws_cdk import (
    custom_resources,
    aws_ec2
)

create_approval_template = custom_resources.AwsCustomResource(
    self, 
    f'CreateApprovalTemplateFor{branch}On{repository_name}',
    on_create=custom_resources.AwsSdkCall(
        service='CodeCommit',
        action='createApprovalRuleTemplate',
        physical_resource_id=custom_resources.PhysicalResourceId.of(
            f'{str(required_approvals)}-approval-for-{self._repository.repository_name}-{branch}'
        ),
        parameters={
            'approvalRuleTemplateName': f'{str(required_approvals)}-approval-for-{self._repository.repository_name}-{branch}',
            'approvalRuleTemplateDescription': f'Requires {required_approvals} approvals from the team to approve the pull request',
            'approvalRuleTemplateContent': json_string,
        }
    ),
    on_update=custom_resources.AwsSdkCall(
        service='CodeCommit',
        action='updateApprovalRuleTemplateContent',
        parameters={
            'approvalRuleTemplateName': f'{str(required_approvals)}-approval-for-{self._repository.repository_name}-{branch}',
            'newRuleContent': json_string,
        }),
    on_delete=custom_resources.AwsSdkCall(
        service='CodeCommit',
        action='deleteApprovalRuleTemplate',
        parameters={
            'approvalRuleTemplateName': f'{str(required_approvals)}-approval-for-{self._repository.repository_name}-{branch}'
        }
    ),
    policy=custom_resources.AwsCustomResourcePolicy.from_sdk_calls(
        resources=custom_resources.AwsCustomResourcePolicy.ANY_RESOURCE
    ),
    vpc=vpc,
    vpc_subnets=aws_ec2.SubnetSelection(
        subnet_type=aws_ec2.SubnetType.PRIVATE_ISOLATED
    ),
)

So what happens here, well we are using the AwsCustomResource provider to address single API calls. The provider always uses a on_create, on_update and on_delete action. This is needed to handle the same CloudFormation actions when a stack/resource is created, updated or deleted. You can do without the on_update or on_delete, but that will result in an orphan resource when you clean up your stack, as it will never be deleted.

on_create: The API call we need is createApprovalRuleTemplate for the service CodeCommit. The physical_resource_id is needed to keep track of updates and deletes. At the parameter section we provide input for the API call. Here you can see that we use our template for the parameter: approvalRuleTemplateContent.

on_update: The on_update part is using a different API call, updateApprovalRuleTemplateContent, to update the template itself. So for example when we want more people or roles to allow the approval, we need to update the json template and do the API call to update the template in CodeCommit.

on_delete: The on_delete is basically reverting back the on_create API call. So using the deleteApprovalRuleTemplate call we can remove the template.

I've added VPC configuration here, to make sure that the lambda's created by the provider framework will run in a VPC, this is mandatory at my current client.

Template association

Now we have code for our template creation, we also need to associate the template with a repository. Again like the creation, there is no CloudFormation support for this, but there are API calls. The resource looks the same in essence, but using different API calls.

associate_approval_template = custom_resources.AwsCustomResource(
    self, 
    f'AssociateApprovalTemplateFor{branch}On{repository_name}',
    on_create=custom_resources.AwsSdkCall(
        service='CodeCommit',
        action='associateApprovalRuleTemplateWithRepository',
        physical_resource_id=custom_resources.PhysicalResourceId.of(f'{str(required_approvals)}-approval-for-{self._repository.repository_name}-{branch}-association'),
        parameters={
            'approvalRuleTemplateName': f'{str(required_approvals)}-approval-for-{self._repository.repository_name}-{branch}',
            'repositoryName': self._repository.repository_name
        }
    ),
    on_delete=custom_resources.AwsSdkCall(
        service='CodeCommit',
        action='disassociateApprovalRuleTemplateFromRepository',
        parameters={
            'approvalRuleTemplateName': f'{str(required_approvals)}-approval-for-{self._repository.repository_name}-{branch}',
            'repositoryName': self._repository.repository_name
        }
    ),
    policy=custom_resources.AwsCustomResourcePolicy.from_sdk_calls(
        resources=custom_resources.AwsCustomResourcePolicy.ANY_RESOURCE
    ),
    vpc=vpc,
    vpc_subnets=aws_ec2.SubnetSelection(
        subnet_type=aws_ec2.SubnetType.PRIVATE_ISOLATED
    ),
)
associate_approval_template.node.add_dependency(create_approval_template)

For the linking of the template with a repository there are only two API calls unfortunately. The associateApprovalRuleTemplateWithRepository and disassociateApprovalRuleTemplateFromRepository API call. Therefore we skip the "on_update" call. As we can not do two API calls in 1 on_update, like disassociate and then associate again, we lack the update function here. But that is fine for this use case.

Last we add a dependency between the associate and creation, as the associate can only happen when the template is created.

CodeChecker CodeBuild project

With the approval templates in place we also need the CodeChecker project itself. In my previous blog we used the cloudcomponents PullRequestCheck resource for that. Let's recreate that with native AWS resources. Starting with the CodeBuild project:

from aws_cdk import aws_codebuild

pullrequest_project = aws_codebuild.Project(
    self,
    f"PullRequestCheckFor{branch}",
    project_name=f"{repository_name.lower()}-codechecker-{branch}",
    source=aws_codebuild.Source.code_commit(repository=self._repository),
    role=self.codechecker_role,
    environment=aws_codebuild.BuildEnvironment(build_image=aws_codebuild.LinuxBuildImage.STANDARD_6_0),
    build_spec=aws_codebuild.BuildSpec.from_object_to_yaml(
        {
            "version": "0.2",
            "env": {"git-credential-helper": "yes"},
            "phases": {
                "install": {
                    "commands": [
                        "npm install -g aws-cdk",
                        "pip install -r requirements.txt",
                    ]
                },
                "build": {
                    "commands": [
                        "cdk synth",
                    ]
                },
                "post_build": {
                    "commands": [
                        "pytest --junitxml=reports/codechecker-pytest.xml > pytest-output.txt",
                        'if grep -i "passed" pytest-output.txt; then PYTEST_RESULT="PASSED"; else PYTEST_RESULT="FAILED"; fi',
                        'if [ $PYTEST_RESULT != "PASSED" ]; then PR_STATUS="REVOKE"; else PR_STATUS="APPROVE"; fi',
                        "echo $PR_STATUS",
                        "REVISION_ID=$(aws codecommit get-pull-request --pull-request-id $PULL_REQUEST_ID | jq -r '.pullRequest.revisionId')",
                        "aws codecommit update-pull-request-approval-state --pull-request-id $PULL_REQUEST_ID --revision-id $REVISION_ID --approval-state $PR_STATUS --region $AWS_REGION"
                    ]
                },
            },
            "reports": {
                "pytest_reports": {
                    "files": ["codechecker-pytest.xml"],
                    "base-directory": "reports",
                    "file-format": "JUNITXML"
                }
            }
        }
    ),
    vpc=vpc,
    subnet_selection=aws_ec2.SubnetSelection(
        subnet_type=aws_ec2.SubnetType.PRIVATE_ISOLATED
    ),
)

This is a CodeBuild project which uses the repository as a source. The project installs CDK plus the requirement.txt file with pip in the install phase. As a pre-test in the build stage, CodeBuild tries to do a CDK synth action, to check if it can generate CloudFormation templates. In the post_build phase CodeBuild runs the pytest. It also creates a junitxml file as a report, which you can see in CodeBuild in the console as evidence, see also the reports part. The output is saved in a text file. This is because the if statement checks if the pytest passes or not. If the pytest was successful, it stores "APPROVE" value in the PR_STATUS variable. This variable then is used to add an approval to the Pull Request. In all other cases it will revoke the Pull Request. We are using some variables needed to get the revision id and update the approval state.

Catching the event

With the CodeChecker project created, we need something to trigger the project when a PR is created. Before the cloudcomponents construct was responsible for that, but we will use EventBridge instead:

from aws_cdk import aws_events

pull_request_rule = aws_events.Rule(
    self,
    f"OnPullRequest{branch}EventRule",
    event_pattern=aws_events.EventPattern(
        source=["aws.codecommit"],
        resources=[self.repository.repository_arn],
        detail={
            "event": [
                "pullRequestCreated",
                "pullRequestSourceBranchUpdated",
            ]
        },
    ),
)

pull_request_rule.add_target(
    target=aws_events_targets.CodeBuildProject(
        pullrequest_project, 
        event=aws_events.RuleTargetInput.from_object({
            "sourceVersion": aws_events.EventField.from_path("$.detail.sourceCommit"),
            "environmentVariablesOverride": [
                {
                    "name": "DESTINATION_COMMIT_ID",
                    "type": "PLAINTEXT",
                    "value":  aws_events.EventField.from_path("$.detail.destinationCommit"),
                },
                {
                    "name": "PULL_REQUEST_ID",
                    "type": "PLAINTEXT",
                    "value": aws_events.EventField.from_path("$.detail.pullRequestId"),
                },
                {
                    "name":"SOURCE_COMMIT_ID",
                    "type": "PLAINTEXT",
                    "value": aws_events.EventField.from_path("$.detail.sourceCommit")
                },
                {
                    "name": "REPOSITORY_NAME",
                    "type": "PLAINTEXT",
                    "value": aws_events.EventField.from_path("$.detail.repositoryNames[0]"),
                }
            ]
        }),
    )
)

First we create a rule to check on two type of events, the pullRequestCreated and pullRequestSourceBranchUpdated event. Basically this catches when a Pull Request has been created or the existing Pull Request has been updated.

The CodeBuild project of CodeChecer will be configured as a target. Important here is the environmentVariablesOverride, this makes it possible to use the DESTINATION_COMMIT_ID, PULL_REQUEST_ID, SOURCE_COMMIT_ID and REPOSITORY_NAME inside the CodeBuild project as environment variables. We needed these for getting the REVISION_ID and updating the Pull Request approval state.

Summary

By replacing the cloudcomponents constructs with AWS native resources, we make ourselves less dependent on third party constructs. We only need four blocks of code plus the approval template definition to completely replace the outdated cloudcomponents constructs. The resources are "maintained" by the CDK project and thus can follow your normal update cycle of CDK.

By replacing the cloudcomponents constructs with AWS native resources, we make ourselves less dependant on third-party constructs. Only four blocks of code plus the approval template definition are needed to completely replace the outdated cloudcomponents constructs. These resources are maintained by the CDK project and can follow your normal update cycle of CDK.

Try yourself

Ready to simplify your CodeChecker setup and reduce dependencies on third-party constructs? Dive into the world of AWS native resources and optimize your CDK CodeChecker with the provided guidance. Check out the code and additional resources on GitHub to get started today!