Beat Java cold starts in AWS Lambda's with GraalVM

Joery Vreijsen — Mon, 19 Oct 2020 00:00:00 +0000

Let's face it, against all advice from your co-workers, friends, or even family, you still want to build a blazingly fast lambda with Java. Either to prove a point, to win a bet, or because you are just as much a Java geek as me.

With this blogpost I'm going to walk you through pre-compiling your Java lambda with GraalVM [1] to eliminate slow cold-start times.

Cold starts, what, where, and when?

On the first invocation of your lambda, AWS boots up a container with the right runtime before actually calling your code. When using runtimes like NodeJs or Python, the cold start times are between 200-250 ms [2] while with Java it takes atleast 650 ms if not more depending on the specifics of your function code.

Java's cold start is mainly caused by the JVM, which is started by default when using AWS' pre-configured Java runtime. Luckily AWS also provides us the option to create our own custom runtime which allows us to use GraalVM to pre-compile our Java code into a binary that can be ran without the need of a JVM.

Let's get practical

Let's take a simple Java program like our Lambda Authorizer from the previous blogpost [3], which can be found on Github [4], and start enabling it for GraalVM processing.

AWS Lambda Runtime API

As we are going to build our own custom runtime, we have to interact with the AWS Lambda Runtime API [5] ourselves.
On this API there are three endpoints that we want to use:

/runtime/invocation/next to get our next invocation,
/runtime/invocation/{AwsRequestId}/response to post the response to a specific invocation (request)
/runtime/invocation/{AwsRequestId}/error to post errors during the invocation (request).

A simple bootstrap class to handle this polling of the AWS Runtime API can be found on Github [6].

Reflection

One of the downsides of pre-compiling with GraalVM is that it can't handle reflection very well, which means that we have to create a reflect.json configuration file, where we can point GraalVM to the classes it should prepare for reflective use. Since we use Jackson as our serialization library we should configure all classes that need (de)serialization in the reflect.json file.

Example entry in the reflect.json configuration file:

[
  ...
  {
    "name": "nl.theguild.lambda.model.DefaultResponse",
    "allPublicMethods" : true
  },
  ...
]

For the complete reflect.json check Github [7]

Compiling the image!

Yes! The interesting part! Now that we prepared our little Java project for native-image compilation let's see how it is done.

First we need to make sure our jar comes with a proper manifest stating our mainClass, which can be done by adding the following config to our pom.xml.

<plugin>
    <artifactId>maven-jar-plugin</artifactId>
    <version>3.0.2</version>
    <configuration>
        <archive>
            <manifest>
                <mainClass>nl.theguild.lambda.Main</mainClass>
            </manifest>
        </archive>
    </configuration>
</plugin>

We then can start creating a simple bash script that will preform all the necessary steps to end up with our custom-runtime zip.

Step 1: Create our jar file.

mvn clean install;

Step 2: Run GraalVM inside a docker container, and mount our project.

docker run --rm --name graal -v $(pwd):/${PATH_TO_PROJECT} oracle/graalvm-ce:19.2.0

Step 3: Install and run GraalVM's native-image command while including our reflect.json configuration, and enabling http.

gu install native-image;
native-image \
    -H:EnableURLProtocols=http \
    -H:ReflectionConfigurationFiles=${PATH_TO_PROJECT}/reflect.json \
    -jar ${PATH_TO_JAR};

Step 4: Move our native image to a folder which we can later on zip for usage in the AWS Lambda.

mkdir /${PATH_TO_PROJECT}/target/custom-runtime;
cp ${BINARY_RESULT} ${PATH_TO_PROJECT}/target/custom-runtime;

Step 5: Create a bootstrap file that instructs AWS on how to run the native image.

AWS defines in their documentation [8] that every custom-runtime should be a zip including a bootstrap shell file that can jump start the function.

In our case that bootstrap file can be a simple one looking something like this:

#!/bin/sh
set -euo pipefail
./${PROJECT_NAME}

Adding all those steps into one single bash script results in the following file:

PROJECT_NAME=aws-enriching-lambda-authorizer
PROJECT_VERSION=1.0.0

# Generate Jar file
mvn clean install;

# Generate Native Image
docker run --rm --name graal -v $(pwd):/${PROJECT_NAME} oracle/graalvm-ce:19.2.0 \
    /bin/bash -c "gu install native-image; \
                  native-image \
                        -H:ReflectionConfigurationFiles=/${PROJECT_NAME}/reflect.json \
                    -jar /${PROJECT_NAME}/target/${PROJECT_NAME}-${PROJECT_VERSION}.jar \
                    ; \
                    mkdir /${PROJECT_NAME}/target/custom-runtime \
                    ; \
                    cp ${PROJECT_NAME}-${PROJECT_VERSION} /${PROJECT_NAME}/target/custom-runtime/${PROJECT_NAME}";

echo -e "#!/bin/sh \n \
set -euo pipefail \n \
./${PROJECT_NAME}" > target/custom-runtime/bootstrap;

# Make bootstrap executable
chmod +x target/custom-runtime/bootstrap;

# Zip
rm $PROJECT_NAME-custom-runtime.zip
cd target/custom-runtime || exit
zip -X -r ../../$PROJECT_NAME-custom-runtime.zip .

HOORAY!

We now have our aws-enriching-lambda-authorizer-custom-runtime.zip that we can use in our AWS Lambda and enjoy our rapid fast cold start timings... WHILE USING JAVA!

Navigate to the AWS Console -> Lambda -> Create function, choose custom-runtime -> use default bootstrap and click create.

Now that we have a lambda, we can go to Actions and upload our zip file, after which we can configure a test-event.

As our authorizer lambda is triggered by the api-gateway we can use the default Amazon API Gateway AWS Proxy event, and simply add an "Authorization": "Bearer 12345" header to the request.

When we now hit test we see our lambda responding in just 283ms, matching the cold-start times of other runtimes, show that to your co-workers, friends, and family!

Source

Github: https://github.com/VR4J/aws-enriching-lambda-authorizer/tree/feature/graal-vm

References

[1] https://www.graalvm.org/docs/introduction/
[2] https://levelup.gitconnected.com

[3] https://www.kabisa.nl/tech/enriching-requests-with-an-aws-lambda-authorizer

[4] https://github.com/VR4J/aws-enriching-lambda-authorizer

[5] https://docs.aws.amazon.com/lambda/latest/dg/runtimes-api.html

[6] https://github.com/VR4J/aws-enriching-lambda-authorizer/blob/feature/graal-vm/src/main/java/nl/theguild/lambda/Main.java

[7] https://github.com/VR4J/aws-enriching-lambda-authorizer/blob/feature/graal-vm/reflect.json

[8] https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html

DEV Community: Joery Vreijsen