Introduction
In part 12 of our series, we measured the cold starts of the Lambda function with Corretto Java 21 runtime without SnapStart enabled, with SnapStart enabled, and also applied DynamoDB invocation priming optimization with different Lambda function memory settings. In this article, we’d like to do the same measurement but for Corretto Java 17 runtime and then do a comparison between both.
Measuring cold starts with Java 17 with and without SnapStart enabled using different Lambda memory settings
In our experiment, we'll reuse the application introduced in part 8 for this. Here is the code for the sample application. There are basically 2 Lambda functions that both respond to the API Gateway requests and retrieve the product by id received from the Api Gateway from DynamoDB. One Lambda function, GetProductByIdWithPureJava17Lambda, can be used with and without SnapStart, and the second one, GetProductByIdWithPureJava17LambdaAndPriming, uses SnapStart and DynamoDB request invocation priming. We'll measure cold starts using the following memory settings in MB: 256, 512, 768, 1024, 1536, and 2048.
The results of the experiment below were based on reproducing approximately 100 cold starts for the duration of our experiment, which ran for approximately 1 hour. For it (and all experiments from my previous articles), I used the load test tool hey, but you can use whatever tool you want, like Serverless-artillery or Postman
Cold start time without SnapStart in ms:
| Experiment description | p50 | p75 | p90 | p99 | p99.9 | max |
|---|---|---|---|---|---|---|
| 256 MB | 7309.66 | 7432.6 | 7575.51 | 7662.51 | 7782.25 | 7962.01 |
| 512 MB cold start time w/o SnapStart | 4213.37 | 4256.07 | 4325.17 | 4496.15 | 4661.23 | 4786.71 |
| 768 MB | 3310.08 | 3414.9 | 3551.82 | 4271.48 | 4421.09 | 4594.42 |
| 1024 MB | 2880.53 | 2918.79 | 2974.45 | 3337.29 | 3515.86 | 3651.65 |
| 1536 MB | 2390.15 | 2434.52 | 2464.46 | 2668.95 | 2812.15 | 2987.04 |
| 2048 MB | 2198.02 | 2272.5 | 2397.97 | 2757.06 | 2892.65 | 3005.31 |
Cold start time with SnapStart without Priming in ms:
| Experiment description | p50 | p75 | p90 | p99 | p99.9 | max |
|---|---|---|---|---|---|---|
| 256 MB | 4972.83 | 5227.66 | 5754.12 | 7551.76 | 7559.31 | 7562.5 |
| 512 MB | 2550.59 | 2604.69 | 2765.67 | 2942.48 | 3108.76 | 3110.69 |
| 768 MB | 1801.28 | 1887.92 | 2251.03 | 2604.69 | 2681.29 | 2681.34 |
| 1024 MB | 1521.33 | 1578.64 | 1918.35 | 2113.65 | 2115.77 | 2117.42 |
| 1536 MB | 1204.06 | 1325.32 | 1507.70 | 1817.56 | 1821.19 | 1821.6 |
| 2048 MB | 1129.45 | 1286.17 | 1583.38 | 1819.37 | 1998.60 | 2000.17 |
Cold start time with SnapStart enabled and with DynamoDB invocation Priming in ms:
| Experiment description | p50 | p75 | p90 | p99 | p99.9 | max |
|---|---|---|---|---|---|---|
| 256 MB | 1126.07 | 1183.77 | 1397.42 | 1608.90 | 1621.82 | 1622.23 |
| 512 MB | 831.84 | 881.49 | 1136.24 | 1348.03 | 1386.29 | 1387.52 |
| 768 MB | 819.46 | 891.23 | 1141.93 | 1243.19 | 1808.50 | 1809.34 |
| 1024 MB | 692.79 | 758.00 | 1003.80 | 1204.06 | 1216.15 | 1216.88 |
| 1536 MB | 713.17 | 773.31 | 995.80 | 1124.94 | 1372.50 | 1372.73 |
| 2048 MB | 797.64 | 858.87 | 1080.86 | 1296.49 | 1376.62 | 1377.05 |
I also measured the deployment time of such a sample project with all memory settings mentioned above, with SnapStart enabled for all functions, and the GetProductByIdWithPureJava17LambdaAndPriming function, additionally using priming. Here are the results:
| Experiment description | average deployment time in minutes/seconds |
|---|---|
| 256 MB deployment time with SnapStart | 2m 57 s |
| 512 MB deployment time with SnapStart | 2m 19 s |
| 768 MB deployment time with SnapStart | 2m 04 s |
| 1024 MB deployment time with SnapStart | 2m 10 s |
| 1536 MB deployment time with SnapStart | 2m 12 s |
| 2048 MB deployment time with SnapStart | 2m 13 s |
Conclusions
In this article, we measured the cold start time of the Lambda function without SnapStart and with SnapStart, and for the latter with additional priming of DynamoDB invocation for Java 17 runtime.
In the case of not enabling SnapStart, we observed that increasing the memory from 256 MB to 2048 MB constantly brought a significant reduction in the cold start time. We observed the same with Java 21 runtime, but cold start times with Java 17 runtime were a bit lower than with Java 21 runtime (for the relatively early version of Java 21 used for the measurements)
In case of enabling SnapStart but not using priming, we observed that increasing the memory from 256 MB to 1536 MB brought a significant reduction in the cold start time with very low impact of increasing the memory from 1536 MB or even to 2048 MB. We observed similar results with Java 21 runtime, but the impact of increasing memory even beyond 1024 MB already become lower. The cold start times with Java 17 runtime were also a bit lower than with Java 21 runtime (for the relatively early version of Java 21 used for the measurements)
In case of enabling SnapStart and with additional priming of DynamoDB invocation, we observed that increasing the memory from 256 MB to 1024 MB brought a significant reduction of the cold start time, but after that, the impact of increasing the memory to 1536 MB or 2048 MB was low or even negative for all percentiles. We observed the same with Java 21 runtime. Also, the results of using Java 17 and Java 21 runtimes are very close.
So, for the SnapStart-enabled Lambda function and considering our use case with the Lambda function reading from the DynamoDB table, the Lambda memory setting of 1024 MB is a good choice.
Deployment time for the SnapStart enabled on all 3 Lambda functions and additionally applying priming on one of them also became lower by giving the Lambda function more memory, but only until 786 MB. After that, we even observed a small increase in the deployment time. With the Java 21 runtime, we observed a decrease in the deployment time by giving the Lambda function more memory until 1024 MB, and then a relatively stable deployment time by giving the Lambda function even more memory.
Top comments (0)