AWS Lambda SnapStart - Part 6 Priming the request invocation for Java 11 and Micronaut, Quarkus and Spring Boot frameworks

Introduction

In part 5 of the series, we discussed the huge impact of priming for our scenario. During priming, we invoke the DynamoDB Client getItem method, which forces Jackson Marshallers to initialize, which is quite an expensive one-time operation for the life cycle of the Lambda function. Only because of this optimization, we observed a huge decrease (up to 900 milliseconds) in the cold start times for all scenarios. But this made me think: can we optimize it any further? In case of using frameworks like Micronaut, Quarkus, and especially Spring Boot, we still observed bigger cold start times (especially at p90s) compared to using the pure Lambda solution. This is because of the translation layer (aka proxy) between the programming model of the used framework and Lambda itself. In the case of Spring Boot, reflection adds on cold start. So I wanted to figure out if I can use priming to make the faked request invocation and preload and prewarm things. So let's explore it.

Priming the request invocation

1) Mirconaut

Mirconaut uses io.micronaut.function.aws.proxy.MicronautLambdaHandler to proxy the incoming request and uses com.amazonaws.serverless.proxy.model.AwsProxyRequest (from the artefact aws-serverless-java-container-core ) as input.
So let's construct a mocked AwsProxyRequest so that the request to "/products/0" will be processed by the method

@Get("/products/{id}")
public Optional<Product> getProductById(@PathVariable String id) 

of the GetProductByIdController. It took me a while to figure out the minimal information to be passed, as it's a (faked) internal invocation without authorization, header, and other metadata required. I came up with the following solution :

final AwsProxyRequest awsProxyRequest = new AwsProxyRequest ();
awsProxyRequest.setHttpMethod("GET");
awsProxyRequest.setPath("/products/0");
awsProxyRequest.setResource("/products/{id}");
awsProxyRequest.setPathParameters(Map.of("id","0"));
final AwsProxyRequestContext awsProxyRequestContext = new AwsProxyRequestContext();
final ApiGatewayRequestIdentity apiGatewayRequestIdentity= new ApiGatewayRequestIdentity();
apiGatewayRequestIdentity.setApiKey("blabla");
awsProxyRequestContext.setIdentity(apiGatewayRequestIdentity);
awsProxyRequest.setRequestContext(awsProxyRequestContext);

We'll also use com.amazonaws.serverless.proxy.internal.testutils.MockLambdaContext from the same artefact aws-serverless-java-container-core to mock the com.amazonaws.services.lambda.runtime.Context.

So let's use this in priming

 @Override
public void beforeCheckpoint(Context<? extends Resource> context) throws Exception {
    try (MicronautLambdaHandler micronautLambdaHandler = new MicronautLambdaHandler()) {        
    micronautLambdaHandler.handleRequest(getAwsProxyRequest(), new MockLambdaContext());
}

The getProductById method of the GetProductByIdController class will subsequently invoke the getItem on the DynamoDB Client with product id 0 itself, so we'll get the accumulated effect of priming. Before presenting the results, let's explore how to do the same with Quarkus.

2) Quarkus

Quarkus uses io.quarkus.amazon.lambda.runtime.QuarkusStreamHandler to proxy the incoming request and uses java.io.InputStream. I decided to go the same way as with Micronaut and used com.amazonaws.serverless.proxy.model.AwsProxyRequest (from the artefact aws-serverless-java-container-core ) to construct the same input. I then converted the AwsProxyRequest object to a byte array using Jackson

ObjectWriter ow = new ObjectMapper().writer().withDefaultPrettyPrinter();
return ow.writeValueAsBytes(getAwsProxyRequest());

and then proxied the request during priming like this

@Override
public void beforeCheckpoint(org.crac.Context<? extends Resource> context) throws Exception {
    new QuarkusStreamHandler().handleRequest
    (new ByteArrayInputStream(convertAwsProxRequestToJsonBytes()), new ByteArrayOutputStream(), new MockLambdaContext());
}

This will proxy the request "/products/0" to the handleRequest method of the GetProductByIdHandler. It will be nice if Quarkus directly supports AwsProxyRequest instead of InputStream in QuarkusStreamHandler in the future, which will make our code a bit cleaner. Before presenting the results, let's explore how to do the same with Spring Boot.

3) Spring Boot

Conceptually, this works the same as with Micronaut. We construct the same AwsProxyRequest input and use the already created SpringBootLambdaContainerHandler handler to proxy the stream. SpringBootLambdaContainerHandler already supports proxying the AwsProxyRequest directly as one of the offered options. So priming looks like this:

@Override
public void beforeCheckpoint(org.crac.Context<? extends Resource> context) throws Exception {
     handler.proxy(getAwsProxyRequest(), new MockLambdaContext());
}

This will proxy the request "/products/0" to ProductController's method

@RequestMapping(path = "/products/{id}", method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
   public Optional<Product> getProductById(@PathVariable("id") String id)

Now let's compare the cold start times due to the effect of priming of DynamoDB getItem invocation (prefix d in the column name) described in the part 5 and the whole (faked) request invocation through the proxy, including subsequent DynamoDB getItem invocation (prefix a) described in this article. Please note that this optimization doesn't have any effect on the pure Lambda example without using any framework.

Framework	d p50	a p50	d p90	a p90	d p99	a p99
Pure Lambda	352.45	352.45	401.43	401.43	433.76	433.76
Micronaut	597.91	431.64	732.01	515.78	755.53	526.11
Quarkus	459.24	413.48	493.33	458.42	510.32	500.21
Spring Boot	600.66	419.47	1065.37	582.64	1173.93	622.23

We see a very big effect of this optimization, especially for Micronaut and Spring Boot frameworks. The cold start times of all 3 frameworks are now really much closer to the pure Lambda ones.

Measuring end-to-end AWS API Gateway latency

Now it's time to re-measure APIGateway end-to-end request latencies in case of cold starts from the previous article. We'll use the same prefixes as in the previous table. Please note that this optimization doesn't have any effect on the pure Lambda example without using any framework.

Framework	d p50	a p50	d p90	a p90	d p99	a p99
Pure Lambda	877	877	1090	1090	1098	1098
Micronaut	1083	900	1221	1247	1570	1325
Quarkus	946	920	1094	1049	1243	1111
Spring Boot	1068	950	2021	1341	2222	1689

We also observe a very big improvement here, especially for Spring Boot, but also for Micronaut and Quarkus frameworks at various percentiles.

Conclusions

With priming of the invocation of the entire request, we could achieve further significant reduction of the cold start times using all 3 frameworks, Micronaut, Quarkus, and Spring Boot for our scenario. The measure of cold start times became much closer to those of the pure Lambda function. Of course, end to end APIGateway request latency is also reduced. We were required to write additional code for that, but the already existing AwsProxyRequest class made our life a bit easier, as we had to set only a small number of properties to make it work. Maybe adding some additional utilities provided out of the box for this purpose can reduce our amount of work further. Anyway, we have to understand the internals of the frameworks used and whether there is an optimization potential through the whole invocation chain. This optimization works the same way for all downstream services from AWS or not, that you invoke in your Lambda implementation through abstractions provided by Micronaut, Quarkus, and Spring Boot frameworks.

Is this the end state of what we can optimize for the Serverless architectures like API Gateway -> SnapStart enabled Lambda written in Java (optionally using frameworks) (-> DynamoDB)?

Maybe not. I'll have to think about other optimization ideas and try them out. Stay tuned!