Micronaut 4 application on AWS Lambda- Part 4 Reducing Lambda cold starts with SnapStart and API Gateway request event priming

Introduction

In the part 1 of our series about how to develop, run and optimize Micronaut web application on AWS Lambda, we demonstrated how to write a sample application which uses the Micronaut framework, AWS Lambda, Amazon API Gateway and Amazon DynamoDB. We also made the first Lambda performance (cold and warm start time) measurements and observed quite a big cold start time.

In the part 2 of the series, we introduced Lambda SnapStart and measured how its enabling reduces the Lambda cold start time by far more than 50% depending on the percentile.

In the part 3 of the series, we introduced how to apply Lambda SnapStart priming technique by starting with DynamoDB request priming with the goal to even further improve the performance of our Lambda functions. We saw that by doing this kind of priming by writing some additional code we could significantly further reduce (depending on the percentiles by more than 50%) the Lambda cold start times compared to simply activating the SnapStart.

We also clearly observed the impact of the AWS SnapStart Snapshot tiered cache in all our measurements so far.

In this part of our article series, we'll introduce another Lambda SnapStart priming technique which is API Gateway request event priming. We'll then measure the Lambda performance by applying it and compare the results with other already introduced approaches.

Sample application with the activated AWS Lambda SnapStart using API Gateway request event priming

We'll re-use the same sample application introduced in the [https://dev.to/aws-heroes/micronaut-4-application-on-aws-lambda-part-1-introduction-to-the-sample-application-and-first-1g62) of our series.

Activating Lambda SnapStart is also a prerequisite for this method.

Globals:
  Function:
    CodeUri: target/aws-lambda-micronaut-4.9-1.0.0-SNAPSHOT.jar
    Runtime: java21
    SnapStart:
     ApplyOn: PublishedVersions    
....

This can be done in the globals section of the Lambda functions, in which case SnapStart applies to all Lambda functions defined in the AWS SAM template, or you can add the 2 lines

SnapStart:
 ApplyOn: PublishedVersions 

to activate SnapStart only for the individual Lambda function.

You can read more about the concepts behind Lambda SnapStart in the part 2.

SnapStart and runtime hooks offer you new possibilities to create your Lambda functions for low startup latency. With the pre-snapshot hook, we can prepare our Java application as much as possible for the first call. We load and initialize as much as possible which our Lambda function needs before the snapshot is created. This technique is known as priming.

Here I'll present another experimental priming technique that pre-initializes the entire web request (API gateway request event). This pre-initializes more than DynamoDB request described in part 3 but also requires significantly more code to be written. The idea is nevertheless comparable. Activating Lambda SnapStart is also a prerequisite for this method. Let's take a look at the implementation in the AmazonAPIGatewayPrimingResource class:

@Singleton
public class AmazonAPIGatewayProxyRequestPrimingResource implements OrderedResource 
{

   @Inject
   private JsonMapper objectMapper;

   @Override
   public void beforeCheckpoint(Context<? extends Resource> context) throws Exception 
   {

   APIGatewayProxyRequestEvent requestEvent = LambdaEventSerializers
    .serializerFor(APIGatewayProxyRequestEvent.class,      
    AmazonAPIGatewayProxyRequestPrimingResource.class.getClassLoader())
    .fromJson(getAPIGatewayProxyRequestEventAsJson());

   new GetProductByIdHandler().execute(requestEvent);
   }


   @Override
   public void afterRestore(Context<? extends Resource> context) throws Exception {}

   private String getAPIGatewayProxyRequestEventAsJson() throws Exception {
    return objectMapper.writeValueAsString(this.getAPIGatewayProxyRequestEvent());
   }

   private APIGatewayProxyRequestEvent getAPIGatewayProxyRequestEvent() throws Exception {
     final APIGatewayProxyRequestEvent aPIGatewayProxyRequestEvent = new APIGatewayProxyRequestEvent ();
     aPIGatewayProxyRequestEvent.setHttpMethod("GET");
     aPIGatewayProxyRequestEvent.setPath("/products/0");
     aPIGatewayProxyRequestEvent.setPathParameters(Map.of("id","0"));
     return aPIGatewayProxyRequestEvent;
   }
}

We use Lambda SnapStart CRaC runtime hooks here. To do this, we need to declare the following Micronaut CRaC dependency in pom.xml:

<dependency>
   <groupId>io.micronaut.crac</groupId>
   <artifactId>micronaut-crac</artifactId>
</dependency>

As we can see, in the method getAPIGatewayProxyRequestEvent we create an object of type APIGatewayProxyRequestEvent and set some of its properties like HTTP Method to "GET" and path parameter ID to 0. This basically mocks APIGatewayProxyRequestEvent which is the input parameter of the handleRequest method. Only these properties of the APIGatewayProxyRequestEvent are required to be set to invoke GetProductByIdHandler Lambda function which then accesses the product id by invoking requestEvent.getPathParameters().get("id"). Similar we can create APIGatewayProxyRequestEvent for the HTTP POST method by setting it as a parameter of the setHttpMethod method and set HTTP body by invoking setBody method. getAPIGatewayProxyRequestEventAsJson method converts APIGatewayProxyRequestEvent object to JSON.

The following piece of code in the beforeCheckpoint method, which is the main priming entry point

APIGatewayProxyRequestEvent requestEvent = LambdaEventSerializers
  .serializerFor(APIGatewayProxyRequestEvent.class,  
 AmazonAPIGatewayProxyRequestPrimingResource.class.getClassLoader())
.fromJson(getAPIGatewayProxyRequestEventAsJson());

we do additional priming, which we in depth described in the Using insights from AWS Lambda Profiler Extension for Java to reduce Lambda cold starts article.

The last piece of code in the beforeCheckpoint is new GetProductByIdHandler().execute(requestEvent);

Through the preinitialized call of the handleRequest method of the GetProductByIdHandler, the DynamoDB request priming presented in the part 3 is also carried out automatically by the DynamoDB call.

To ensure that only this priming takes effect, please temporary delete following AmazonDynamoDBPrimingResource class, otherwise DynamoDB request priming takes places which is completely unnecessary, as it happens anyway during the priming introduced in this article.

However, I consider this priming technique to be experimental, as it naturally leads to a lot of extra code, which can be significantly simplified using a few utility methods. Therefore, the decision to use this priming method is left to the reader.

Measurements of cold and warm start times of our application with Lambda SnapStart and API Gateway request event priming

In the following, we will measure the performance of our GetProductByIdFunction Lambda function, which we will trigger by invoking curl -H "X-API-Key: a6ZbcDefQW12BN56WEM49" https://{$API_GATEWAY_URL}/prod/products/1.

The results of the experiment are based on reproducing more than 100 cold starts and about 100,000 warm starts with the Lambda function GetProductByIdFunction (we ask for the already existing product with ID=1) for a duration of about 1 hour. We give Lambda function 1024 MB memory, which is a good trade-off between performance and cost. We also use (default) x86 Lambda architecture. For the load tests I used the load test tool hey, but you can use whatever tool you want, like Serverless-artillery or Postman.

We will measure with tiered compilation (which is default in Java 21, we don't need to set anything separately) and compilation option XX:+TieredCompilation -XX:TieredStopAtLevel=1. To use the last option, you have to set it in template.yaml in JAVA_OPTIONS environment variable as follows:

Globals:
  Function:
    ...
    Environment:
      Variables:
        JAVA_TOOL_OPTIONS: "-XX:+TieredCompilation -XX:TieredStopAtLevel=1"

Please also note the effect of the AWS SnapStart Snapshot tiered cache. This means that in the case of SnapStart activation, we get the largest cold starts during the first measurements. Due to the tiered cache, the subsequent cold starts will have lower values. For more details about the technical implementation of AWS SnapStart and its tiered cache, I refer you to the presentation by Mike Danilov: "AWS Lambda Under the Hood". Therefore, I will present the Lambda performance measurements with SnapStart being activated for all approx. 100 cold start times (labelled as all in the table), but also for the last approx. 70 (labelled as last 70 in the table), so that the effect of Snapshot Tiered Cache becomes visible to you. Depending on how often the respective Lambda function is updated and thus some layers of the cache are invalidated, a Lambda function can experience thousands or tens of thousands of cold starts during its life cycle, so that the first longer lasting cold starts no longer carry much weight.

To show the impact of the SnapStart with API Gateway request event priming, we'll also present the Lambda performance measurements without SnapStart being activated from the part 1, with SnapStart being activated but without applying the priming technique as measured in the part 2 and with SnapStart being activated and DynamoDB request priming neing applied as measured in the part 3.

Cold (c) and warm (w) start time with tiered compilation in ms:

Scenario Number	c p50	c p75	c p90	c p99	c p99.9	c max	w p50	w p75	w p90	w p99	w p99.9	w max
No SnapStart enabled	4948	5038	5155	5387	5403	5404	5.37	6.01	7.10	16.01	52.05	1535
SnapStart enabled but no priming applied, all	1926	1981	3213	3232	3242	3245	5.33	5.96	6.93	14.43	38.76	2617
SnapStart enabled but no priming applied, last 70	1900	1959	2001	2063	2063	2063	5.29	5.91	6.93	14.66	37.84	1588
SnapStart enabled and DynamoDB request priming applied, all	743	787	879	1300	1798	1798	5.42	6.10	7.27	14.90	36.08	1095
SnapStart enabled and DynamoDB request priming applied, last 70	730	787	878	1301	1301	1301	5.37	6.10	7.33	15.02	33.85	433
SnapStart enabled and API Gateway request event priming applied, all	696	739	819	918	936	936	5.33	6.01	7.27	15.38	40.34	285
SnapStart enabled and API Gateway request event priming applied, last 70	673	714	777	845	845	845	5.25	5.96	7.16	15.14	38.15	261

Cold (c) and warm (w) start time with -XX:+TieredCompilation -XX:TieredStopAtLevel=1 compilation in ms:

Scenario Number	c p50	c p75	c p90	c p99	c p99.9	c max	w p50	w p75	w p90	w p99	w p99.9	w max
No SnapStart enabled	4993	5145	5392	5697	5852	5856	5.33	5.91	6.88	15.50	52.47	1616
SnapStart enabled but no priming applied, all	1895	1947	2025	2154	3368	3369	5.55	5.82	6.72	14.86	104.68	2609
SnapStart enabled but no priming applied, last 70	1891	1923	1989	2066	2066	2066	5.13	5.73	6.61	14.17	35.01	1637
SnapStart enabled and DynamoDB request priming applied, all	696	755	1611	1632	1632	1632	5.21	5.92	7.16	15.09	43.72	952
SnapStart enabled and DynamoDB request priming applied, last 70	663	693	759	826	826	826	5.13	5.82	7.05	14.39	35.57	370
SnapStart enabled and API Gateway request event priming applied, all	679	723	1375	1400	1460	1460	5.33	6.05	7.27	15.63	41.64	709
SnapStart enabled and API Gateway request event priming applied, last 70	657	685	733	801	801	801	5.29	5.96	7.21	15.38	36.66	280

Conclusion

In this part of the series, we introduced how to apply Lambda SnapStart priming technique we called API Gateway event request priming with the goal to further improve the performance of our Lambda functions compared to the DynamoDB request priming. We saw that by doing this kind of priming by writing some additional code we could further reduce the Lambda cold start times, but not so significantly compared to the DynamoDB request priming introduced in part 3.

We also clearly observed the impact of the AWS SnapStart Snapshot tiered cache in our measurements.

We also saw that the cold start time measurements for both Java compilation options produce different results depending on the percentile, but the -XX:+TieredCompilation -XX:TieredStopAtLevel=1 compilation option produced slightly lower cold start times for last 70 measurements.

However, as I've already pointed out, I consider this priming technique to be experimental, as it naturally leads to a lot of extra code, which can be significantly simplified using a few utility methods. Therefore, the decision to use this priming method is left to the reader. You can stick to applying DynamoDB request priming having a bit higher Lambda cold start time.

In the next part of our article series, we'll introduce how to adjust our sample application to one from which we can build the GraalVM Native Image and deploy it as a Lambda Custom Runtime. We'll then measure the Lambda performance with it and compare the results with other already introduced approaches.