AWS SnapStart - Part 12 Measuring cold starts and deployment time with Java 21 using different Lambda memory settings

#java #aws #serverless #coldstart

Introduction

In part 11 of our series, we measured the cold starts of the Lambda function with Corretto Java 21 runtime without SnapStart enabled, with SnapStart enabled, and also applied DynamoDB invocation priming optimization with different deployment packages/jar sizes between 137KB and 50MB. In this article, we will measure the cold starts but will use different memory settings.

Measuring cold starts with Java 21 with and without SnapStart enabled using different Lambda memory settings.

In our experiment, we'll reuse the application introduced in part 9 for this. Here is the code for the sample application. There are basically 2 Lambda functions that both respond to the API Gateway requests and retrieve the product by id received from the Api Gateway from DynamoDB. One Lambda function, GetProductByIdWithPureJava21Lambda, can be used with and without SnapStart, and the second one, GetProductByIdWithPureJava21LambdaAndPriming, uses SnapStart and DynamoDB request invocation priming. We'll measure cold starts using the following memory settings in MB: 256, 512, 768, 1024, 1536, and 2048.

The results of the experiment below were based on reproducing approximately 100 cold starts for the duration of our experiment, which ran for approximately 1 hour. For it (and all experiments from my previous articles), I used the load test tool hey, but you can use whatever tool you want, like Serverless-artillery or Postman

Cold start time without SnapStart in ms:

Experiment description	p50	p75	p90	p99	p99.9	max
256 MB cold start time w/o SnapStart	8056.84	8298.23	8494.11	9123.72	9256.72	9417.65
512 MB cold start time w/o SnapStart	4684.92	4781.67	4944.95	5014.81	5156.32	5321.34
768 MB cold start time w/o SnapStart	3638.06	3720.91	3815.31	4636.18	4787.51	4895.48
1024 MB cold start time w/o SnapStart	3157.6	3213.85	3270.8	3428.2	3601.12	3725.02
1536 MB cold start time w/o SnapStart	2665.49	2700.56	2808.68	3306.94	3476.05	3613.84
2048 MB cold start time w/o SnapStart	2414.66	2463.17	2521.7	2839.59	3012.07	3156.33

Let's put this data into a graph:

Cold start time with SnapStart without Priming in ms:

Experiment description	p50	p75	p90	p99	p99.9	max
256 MB cold start time with SnapStart w/o Priming	5623.35	5736.89	6526.39	7020.38	7076.74	7080.47
512 MB cold start time with SnapStart w/o Priming	2861.27	3035.06	3504.91	3881.08	3927.91	3931.82
768 MB cold start time with SnapStart w/o Priming	1976.74	2053.27	2492.62	2787.87	2799.04	2801.67
1024 MB cold start time with SnapStart w/o Priming	1626.69	1741.10	2040.99	2219.75	2319.54	2321.64
1536 MB cold start time with SnapStart w/o Priming	1312.14	1369.76	1703.23	1753.32	2078.04	2078.96
2048 MB cold start time with SnapStart w/o Priming	1172.00	1282.32	1612.12	1845.01	2312.60	2313.33

Let's put this data into a graph:

Cold start time with SnapStart enabled and with DynamoDB invocation Priming in ms:

Experiment description	p50	p75	p90	p99	p99.9	max
256 MB cold start time with SnapStart w/o Priming	1135.11	1227.14	1457.33	1713.47	1715.19	1716.63
512 MB cold start time with SnapStart with Priming	862.32	935.03	1206.47	1394.62	1398.81	1399.09
768 MB cold start time with SnapStart with Priming	839.36	918.36	1107.10	1248.17	1387.67	1389.05
1024 MB cold start time with SnapStart with Priming	702.55	759.52	1038.50	1169.66	1179.05	1179.36
1536 MB cold start time with SnapStart with Priming	659.02	723.22	981.97	1254.43	1376.62	1377.26
2048 MB cold start time with SnapStart with Priming	687.96	763.32	1054.18	1174.35	1184.96	1186.08

Let's put this data into a graph as well:

I also measured the deployment time of such a sample project with all memory settings mentioned above, with SnapStart enabled for all functions, and the GetProductByIdWithPureJava21LambdaAndPriming function additionally using priming. Here are the results:

Experiment description	average deployment time in minutes/seconds
256 MB deployment time with SnapStart	2m 44 s
512 MB deployment time with SnapStart	2m 18 s
768 MB deployment time with SnapStart	2m 17 s
1024 MB deployment time with SnapStart	2m 08 s
1536 MB deployment time with SnapStart	2m 06 s
2048 MB deployment time with SnapStart	2m 06 s

Conclusions

In this article, we measured the cold start time of the Lambda function without SnapStart and with SnapStart, and for the latter with additional priming of DynamoDB invocation.

In the case of not enabling SnapStart, we observed that increasing the memory from 256 MB to 2048 MB constantly brought a significant reduction in the cold start time.

In case of enabling SnapStart but not using priming, we observed that increasing the memory from 256 MB to 1024 MB brought a significant reduction of the cold start time, but after tha,t the impact of increasing the memory to 1536 MB or even to 2048 MB varied depending on percentile and being significant until p90 and much lower for the percentiles higher than 90.

In case of enabling SnapStart and with additional priming of DynamoDB invocation, we observed that increasing the memory from 256 MB to 1024 MB brought a significant reduction of the cold start time, but after that, the impact of increasing the memory to 1536 MB or 2048 MB was low for all percentiles.

So, for the SnapStart-enabled Lambda function and considering our use case with the Lambda function reading from the DynamoDB table, the Lambda memory setting of 1024 MB is a good choice.

Deployment time for the SnapStart enabled on all 3 Lambda functions and additionally applying priming on one of them, also became lower by giving the Lambda function more memory.

In the next article, I will also provide measurements for the warm execution times for this use case, as using different memory settings not only affects the cold start times but also the warm execution times and, therefore, Lambda costs. Stay tuned!