CDK - Using Central Register Pattern for Resource Sharing

Props drilling, massive functions/classes in one file, and deployment deadlock are just some of the common issues you can find in AWS CDK code written by yourself or others (me included). Amazon Web Services Cloud Development Kit is the framework I work with most. AWS CDK is very powerful. Built around CloudFormation it makes architecting, building, and deploying cloud resources very easy...until it's not. This blog post explores using a central register to share resources anywhere in the CDK code base, including between stacks and pitfalls to avoid when doing so.

A year ago I ago I wrote an article "A Novel Pattern for Documenting DynamoDB Access Patterns" (update coming soon) and I mentioned in there the pattern for using a central register for my resources. So here I am over a year later finally writing about it.

The Central Register pattern (check out this page about it on geeksforgeeks, I like how NehalNavlani explains it) has one part where you have a central place to register resources, and you can lookup those resources from anywhere in your code base using an ID. See below for an example of what I wrote for lambda's central register in CDK so I can register resources and have some error checking verifying that I'm not using the same IDs twice. I also have error checking to tell the code that a resources doesn't exist from a provided ID, which can help prevent bad references.

Here's the code that includes 3 main functions: addLambda, getLambda, and validateAndReturnLambdaArn:

To add a Lambda to the central register, all you have to do is import the addLambda and pass it an ID and the instantiated Lambda. I usually use the lambda name, as it's unique per region and makes debugging much easier as the name stays the same through all the steps. Here we create a Lamba and add it to the Lambda Central Register:

To use a Lambda in, say, a step function within the same stack, we can simply do getLambda with the same ID (This example of a different lambda from the above, but created the same way). The key is that we created the lambda somewhere else, but can easily reference it by just importing the getLambda function, and using the correct key:

This works great, reduces the need to do props drilling or adding a lot of properties to look up and keep track of in the stack class or environments that get passed in from one stack to another, but it can come with issues when deploying resources in multiple stacks: Deployment Deadlock. This issue I first saw explained by Lee Gilmore, an AWS Serverless Hero, in one of his articles:

One of the key gotchas to watch out for when using the cross-stack reference approach is breaking your app’s deployment with deployment deadlock, which is usually shown with the following error “Export cannot be deleted as it is in use by another Stack”.

I used to run into this often when using getLambda in an API Gateway stack that depends on a lambda in another stack. Then, either the lambda is renamed, deleted, or some other things can happen that cause a loop where the API Gateway stack cannot deploy before the Lambda stack does, but the Lambda stack won't deploy because the API Gateway stack depends on it. A great solution to this I have found is to use instantiated references using the "from arn" method (different resources name this type of function slightly differently, but the concept is the same). Lots of CDK resources you can create have similar methods, so it doesn't actually create a hard dependency between stacks. The resource does need to exist the first time you deploy, but after that you can delete or update either stack's resources and the other won't care about it. For Lambda, I created the validateAndReturnLambdaArn which acts just like getLambda, with error checking included, but returns the expected arn the lambda will have (not the one generated by CDK, which I believe would link them as dependencies...) so that the "Function.fromFunctionArn" can use it and be passed to API Gateway method creation functions:

Note: on resources like DynamoDB where only the table name is needed, I have a similar function: validateAndReturnDynamoDBTableName that will validate the table is in our code, but not link to it and return just the table name for use. The table name can then be passed in as an environment variable, and can also be used in creating policies for roles for the resources in the other stacks to be given access to the DynamoDB Table.

Using this has been successful on my projects, and makes it very quick to create and use resources in the CDK, while also reducing headaches when deploying updates in pipelines. In the past with Deployment Deadlock, it was almost always a painful manual process to unlink the stacks, redeploy each, then link back together. Something I didn't want to do in the CICD pipelines. Now the pipelines easily keep things up to date as they are updated in our code!

Please feel free to connect with me on LinkedIn, or to join the Believe in Serverless discord where there are over 1000 serverless enthusiasts ranging from just a few days in to people who literally wrote the book on the subject.

Images of code above are nice, but if you want to copy/paste here is the code in text:

import { Function } from "aws-cdk-lib/aws-lambda";
import { getPartitionIdentifier } from "../utilities";

const { region = "", account = "" } = process.env;

export const lambdaLookup: Map<string, Function> = new Map();

export const addLambda = (lambdaName: string, lambda: Function) => {
    if (lambdaLookup.has(lambdaName)) {
        throw Error(`Lambda ID has already been used for this deployment: ${lambdaName}`);
    }
    lambdaLookup.set(lambdaName, lambda);
};

export const getLambda = (lambdaName: string) => {
    const lambda = lambdaLookup.get(lambdaName);
    if (!lambda) {
        throw Error(`Could not find lambda in lookup from ID: ${lambdaName}`);
    }
    return lambda;
};

export const validateAndReturnLambdaArn = (lambdaName: string) => {
    const awsPartition = getPartitionIdentifier(region);
    const lambdaExists = lambdaLookup.has(lambdaName);
    if (!lambdaExists) {
        throw Error(`Could not find lambda in lookup from ID: ${lambdaName}`);
    }
    return `arn:${awsPartition}:lambda:${region}:${account}:function:${lambdaName}`;
};
    const functionName = `${deploymentType}-getQueryData`;
    const lambdaInstance = new DockerImageFunction(construct, functionName, {
        code: DockerImageCode.fromImageAsset(dockerfile),
        functionName,
        timeout: cdk.Duration.seconds(30),
        memorySize: 256,
        role,
        loggingFormat:LoggingFormat.JSON,
    });

    addLambda(functionName, lambdaInstance);

    return lambdaInstance;

export const createLambdaGetIntegration = ({
    construct,
    lambdaName,
    methodName,   
    parentResource,
}: CreateLambdaPostIntegrationProps) => {
    const resource = parentResource.addResource(methodName);
    const lambdaIntegrateArn = validateAndReturnLambdaArn(lambdaName);
    const lambdaToIntegrate = Function.fromFunctionArn(
        construct,
        `${parentResource.path}-GET-${lambdaName}`,
        lambdaIntegrateArn
    );

    resource.addMethod("GET",new LambdaIntegration(lambdaToIntegrate));
};

new LambdaInvoke(construct, `${uniquePrefix}-update-processing-status`, {
    stateName: `${uniquePrefix} - Update Processing Status`,
    lambdaFunction: getLambda(`${deploymentType}-updateProcessingStatus`),
    retryOnServiceExceptions: false,
    resultPath: "$.updateProcessingStatus",
    resultSelector: {
        "payload.$": "$.Payload",
    },
});