This is a step by step guide to running AWS CDK inside an AWS Lambda Function using Lambda layers.
We will create an example CDK code, which will create an S3 bucket (can be any AWS resource, this is just an example) and then create another CDK code to deploy a Lambda function which will run the first one. Do you remember the Russian toy called "matryoshka"?
Why do we need this? There are two main reasons:
- For fun, to play with Lambda layers and AWS CDK.
- It can be sometimes useful and we actually use it in DevOpsBox.
All the source code is available here: https://github.com/devopsbox-io/example-cdk-from-lambda
Prerequisites
You will need several tools:
- AWS CDK (tested with 1.71.0)
- Java JDK (tested with OpenJDK 11.0.9)
- NodeJS (required by AWS CDK, tested with v12.18.3)
- Docker (tested with 18.09.5)
- Maven (tested with 3.6.3)
- AWS CLI (only for testing, tested with 1.18.57)
AWS account with proper credentials is also required. The code will probably work with other versions too.
The solution
First of all, we will create two CDK projects, one inside another (matryoshka):
mkdir example-cdk-from-lambda
cd example-cdk-from-lambda
cdk init app --language=java
mkdir run-cdk-lambda
cd run-cdk-lambda
cdk init app --language=java
Yes, we are using Java here... You can achieve the same results in any other language supported by AWS CDK.
I will not change any package name just to allow you to easily do git diff HEAD~1
and to see all my changes, however, I have removed all the tests and test dependencies - they are not important here.
The smallest matryoshka
Here we just create an S3 bucket but it could be any CDK code.
We need the CDK S3 dependency run-cdk-lambda/pom.xml
<dependency>
<groupId>software.amazon.awscdk</groupId>
<artifactId>s3</artifactId>
<version>${cdk.version}</version>
</dependency>
Next, we have to write some CDK code run-cdk-lambda/src/main/java/com/myorg/RunCdkLambdaStack.java:
new Bucket(this, "created-by-cdk-from-lambda", BucketProps.builder()
.removalPolicy(RemovalPolicy.DESTROY)
.build());
The medium matryoshka
If you want to run the AWS CDK, you will have to execute the cdk binary because this is how it works. That is why we need to create a Lambda handler, which will be just a wrapper and will run the cdk.
We have to add the aws-lambda-java-core dependency to our run-cdk-lambda/pom.xml:
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-core</artifactId>
<version>1.2.1</version>
</dependency>
Then, create the CdkWrapper class run-cdk-lambda/src/main/java/com/myorg/CdkWrapper.java:
public class CdkWrapper implements RequestHandler<Map<String, String>, String> {
public static final String CDK_OUT_DIR = "/tmp/cdk.out";
private static final String CDK_COMMAND = "java -cp . ";
@Override
public String handleRequest(Map<String, String> input, Context context) {
runCdk(RunCdkLambdaApp.class);
return "";
}
public static void runCdk(Class<?> cdkClass) {
ProcessBuilder processBuilder = new ProcessBuilder(
"cdk",
"deploy",
"--verbose",
"--output", CDK_OUT_DIR,
"--app", CDK_COMMAND + cdkClass.getName(),
"--require-approval", "\"never\""
);
processBuilder.redirectOutput(ProcessBuilder.Redirect.INHERIT);
processBuilder.redirectError(ProcessBuilder.Redirect.INHERIT);
Process process;
try {
process = processBuilder.start();
} catch (IOException e) {
throw new RuntimeException("Cannot start cdk process!", e);
}
try {
process.waitFor();
} catch (InterruptedException e) {
throw new RuntimeException("Cdk process interrupted!", e);
}
int exitValue = process.exitValue();
if (exitValue != 0) {
throw new RuntimeException("Exception while executing CDK!");
}
}
}
We are doing this in Java using the ProcessBuilder class because our CDK code is also written in Java and we will use the Java 11 runtime in Lambda.
We also have to pack our Java code into an "uber" jar (jar with all the dependencies). We use maven-shade-plugin to do that run-cdk-lambda/pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
The largest matryoshka
The Java 11 Lambda runtime does not have NodeJS and AWS CDK binaries. We will use Lambda layers to put them inside. We will copy binaries from a docker image created from docker/cdk/Dockerfile:
FROM node:lts
RUN apt-get update && \
apt-get install -y zip && \
rm -rf /var/lib/apt/lists/*
ENV AWS_CDK_VERSION=1.71.0
RUN mkdir -p /nodejs && \
npm config set prefix /nodejs/bin && \
npm install -g aws-cdk@${AWS_CDK_VERSION}
RUN cd /nodejs/bin && \
zip -r --symlinks /opt/aws-cdk.zip *
RUN cd /usr/local && \
zip -r /opt/node.zip bin/node
We need also the code to create the "uber" jar, the docker image with NodeJS and CDK binaries, and copy files from it (actually creating a temporary docker container). This will be a bash script scripts/prepare-bin:
#!/bin/bash
set -e
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
PROJECT_DIR=$(realpath "${SCRIPT_DIR}/..")
DOCKER_IMAGE_TAG=aws-cdk-bin:latest
TMP_BIN_DIR="${PROJECT_DIR}/tmp"
mkdir -p ${TMP_BIN_DIR}
mvn package -f "${PROJECT_DIR}/run-cdk-lambda/pom.xml"
cp ${PROJECT_DIR}/run-cdk-lambda/target/run-cdk-lambda-*.jar ${TMP_BIN_DIR}/run-cdk-lambda.jar
docker build -t ${DOCKER_IMAGE_TAG} ${PROJECT_DIR}/docker/cdk
rm -f ${TMP_BIN_DIR}/docker-cid
docker create --cidfile ${TMP_BIN_DIR}/docker-cid ${DOCKER_IMAGE_TAG}
CID=$(cat ${TMP_BIN_DIR}/docker-cid)
trap "docker rm ${CID}" EXIT
docker cp ${CID}:/opt/node.zip ${TMP_BIN_DIR}/node.zip
docker cp ${CID}:/opt/aws-cdk.zip ${TMP_BIN_DIR}/aws-cdk.zip
Now we have to create the outer CDK code to create the lambda with layers. We need the CDK Lambda dependency pom.xml:
<dependency>
<groupId>software.amazon.awscdk</groupId>
<artifactId>lambda</artifactId>
<version>${cdk.version}</version>
</dependency>
and the CDK code src/main/java/com/myorg/ExampleCdkFromLambdaStack.java:
public class ExampleCdkFromLambdaStack extends Stack {
public ExampleCdkFromLambdaStack(final Construct scope, final String id) {
this(scope, id, null);
}
public ExampleCdkFromLambdaStack(final Construct scope, final String id, final StackProps props) {
super(scope, id, props);
createRunCdkLambda();
}
private void createRunCdkLambda() {
String tmpBinDir = getTmpBinDir();
PolicyStatement cloudformationPolicy = new PolicyStatement(PolicyStatementProps.builder()
.resources(Arrays.asList(
"*"
))
.actions(Arrays.asList(
"cloudformation:DescribeStacks",
"cloudformation:CreateChangeSet",
"cloudformation:DescribeChangeSet",
"cloudformation:GetTemplate",
"cloudformation:GetTemplateSummary",
"cloudformation:DescribeStackEvents",
"cloudformation:ExecuteChangeSet",
"cloudformation:DeleteChangeSet"
))
.build());
// we can restrict this policy to certain buckets only
PolicyStatement s3Policy = new PolicyStatement(PolicyStatementProps.builder()
.resources(Arrays.asList(
"*"
))
.actions(Arrays.asList(
"s3:*"
))
.build());
LayerVersion nodeLayer = LayerVersion.Builder.create(this, "node-layer")
.description("Layer containing node binary")
.code(
Code.fromAsset(tmpBinDir + "/node.zip")
)
.build();
LayerVersion cdkLayer = LayerVersion.Builder.create(this, "aws-cdk-layer")
.description("Layer containing AWS CDK")
.code(
Code.fromAsset(tmpBinDir + "/aws-cdk.zip")
)
.build();
Function lambda = new Function(this, "RunCdk", FunctionProps.builder()
.runtime(Runtime.JAVA_11)
.handler("com.myorg.CdkWrapper")
.code(Code.fromAsset(tmpBinDir + "/run-cdk-lambda.jar"))
.layers(Arrays.asList(
nodeLayer,
cdkLayer
))
.timeout(Duration.seconds(300))
.memorySize(512)
.initialPolicy(Arrays.asList(
cloudformationPolicy,
s3Policy
))
.functionName("run-cdk")
.build());
}
private String getTmpBinDir() {
Path tmpPath = Paths.get("tmp");
return tmpPath.toAbsolutePath().toString();
}
}
This is our third, outermost "matryoshka" layer. Here we are creating the Lambda policies (CloudFormation execution and S3 full access), layers (NodeJS, AWS CDK binaries), and the actual Lambda resource with the com.myorg.CdkWrapper handler and the code from run-cdk-lambda.jar.
We will also change the cdk.json file to run the prepare-bin script before the CDK execution cdk.json:
{
"app": "./scripts/prepare-bin && mvn -e -q compile exec:java",
"context": {
"@aws-cdk/core:enableStackNameDuplicates": "true",
"aws-cdk:enableDiffNoFail": "true",
"@aws-cdk/core:stackRelativeExports": "true"
}
}
Deploying
You have to set your AWS credentials first. You can do this in the ~/.aws/credentials
file or using environment variables. Then, you have to bootstrap the CDK:
cdk bootstrap
Now you can create the Lambda function:
cdk deploy --require-approval never
Testing
Check your S3 buckets using:
aws s3 ls
This should not return any bucket with the "runcdklambdastack-createdbycdkfromlambda" prefix.
Run the Lambda function:
aws lambda invoke --function-name run-cdk tmp/lambda-out
It will run the AWS CDK and create the bucket. The first run can take about 1 minute to complete. Check again your S3 buckets then:
aws s3 ls
It should return a bucket created by CDK run inside a Lambda function (with the "runcdklambdastack-createdbycdkfromlambda" prefix).
Conclusion
Running AWS CDK inside a Lambda function is not an easy task. However, sometimes it is useful and I hope that this article will help you to do it, or maybe will only show you how to use lambda Layers to execute almost any executable binary inside an AWS Lamda. We do use it in DevOpsBox to create appropriate IAM roles and policies based on the roles read from Keycloak.
For more details about the DevOpsBox platform please visit https://www.devopsbox.io/
Top comments (4)
This is a AWESOME post. I am running into an issue when upgrading to CDKv2, the size is over the limit, working through a way to see if anything can help. Now is a DONT KNOW
Thanks @desawsume!
I've updated the repository some time ago (thanks to github.com/YotillaJonas for his PR). I've tested that and it works but didn't try anything bigger.
I would try Lambda container images now. It didn't exist when I was writing the article.
yes, another way to be worry free is using ECR with LAMBDA, 10GB, more than enough for our use case, i might also create a post to share. currently working through the docker stuff
I'm on the same ship - did you end up writing up ideas?