AWS SnapStart - Part 22 Measuring cold and warm starts with Java 17 using synchronous HTTP clients

Introduction

In the previous parts of this series, we've done many measurements with AWS Lambda using Java 17 runtime with and without using AWS SnapStart, and additionally using SnapStart and priming DynamoDB invocation :

• cold starts using different deployment artifact sizes
• cold starts and deployment time using different Lambda memory settings
• warm starts using different Lambda memory settings
• cold and warm starts using different compilation options

In this article, we'll now add another dimension to our Java 17 measurements: the choice of HTTP Client implementation. Starting from AWS SDK for Java version 2.22, AWS added support for their own implementation of the synchronous CRT HTTP Client. The asynchronous CRT HTTP client has been generally available since February 2023. In this article, we'll explore synchronous HTTP clients first and leave asynchronous ones for the next article.

I will also compare it with the same measurements for Java 21 already performed in the article Measuring cold and warm starts with Java 21 using different synchronous HTTP clients

Measuring cold and warm starts with Java 17 using synchronous HTTP clients

In our experiment, we'll reuse the application introduced in part 8 for this. Here is the code for the sample application. There are basically 2 Lambda functions that both respond to the API Gateway requests and retrieve the product by id received from the API Gateway from DynamoDB. One Lambda function, GetProductByIdWithPureJava17Lambda, can be used with and without SnapStart, and the second one, GetProductByIdWithPureJava17LambdaAndPriming, uses SnapStart and DynamoDB request invocation priming.

As we did our measurements for Java 17 in the previous articles of the series, we have always used the default HTTP Client implementation, which is Apache HTTP Client (we'll use the measurements for the comparison in this article), we'll explore 2 other options as well.

There are now 3 synchronous HTTP Client implementations available in the AWS SDK for Java.

1. Url Connection
2. Apache (Default)
3. AWS CRT

This is the order for the lookup of the synchronous HTTP Client in the classpath.

Let's figure out how to configure the HTTP Client. There are 2 places to do it : pom.xml and DynamoProductDao

Let's consider 3 scenarios:

Scenario 1) UrlConnection HTTP Client. Its configuration looks like this:
In the pom.xml, the only enabled HTTP Client dependency has to be:

<dependency>
  <groupId>software.amazon.awssdk</groupId>
  <artifactId>url-connection-client</artifactId>
</dependency>

In DynamoProductDao, the DynamoDBClient should be created like this:

DynamoDbClient.builder()
    .region(Region.EU_CENTRAL_1)
     .httpClient(UrlConnectionHttpClient.create())
    .overrideConfiguration(ClientOverrideConfiguration.builder()
      .build())
    .build();

Scenario 2) Apache HTTP Client. Its configuration looks like this:
In the pom.xml, the only enabled HTTP Client dependency has to be:

<dependency>
   <groupId>software.amazon.awssdk</groupId>
   <artifactId>apache-client</artifactId>
</dependency>

In DynamoProductDao, the DynamoDBClient should be created like this:

DynamoDbClient.builder()
    .region(Region.EU_CENTRAL_1)
     .httpClient(ApacheHttpClient.create())
    .overrideConfiguration(ClientOverrideConfiguration.builder()
      .build())
    .build();

Scenario 3) AWS CRT synchronous HTTP Client. Its configuration looks like this:
In the pom.xml, the only enabled HTTP Client dependency has to be:

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>aws-crt-client</artifactId>
</dependency>

In DynamoProductDao, the DynamoDBClient should be created like this:

DynamoDbClient.builder()
    .region(Region.EU_CENTRAL_1)
     .httpClient(AwsCrtHttpClient.create())
    .overrideConfiguration(ClientOverrideConfiguration.builder()
      .build())
    .build();

For the sake of simplicity, we create all HTTP Clients with their default settings. Of course, there is the optimization potential there to figure out the right HTTP Client settings.

The results of the experiment below were based on reproducing more than 100 cold and approximately 100.000 warm starts with the experiment, which ran for approximately 1 hour. For it (and experiments from my previous article), I used the load test tool hey, but you can use whatever tool you want, like Serverless-artillery or Postman. I ran all these experiments for all 3 scenarios using 2 different compilation options in the AWS SAM template.yaml each:

1. No options (tiered compilation will take place)
2. JAVA_TOOL_OPTIONS: "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling)

We found out in the article Measuring cold and warm starts with Java 17 using different compilation options that with both we've got the lowest cold and warm start times. We’ve also got good results with "-XX:+TieredCompilation -XX:TieredStopAtLevel=2” compilation option, but I haven’t done any measurement with this option yet.

Let's look into the results of our measurements.

Cold (c) and warm (m) start time with compilation option "tiered compilation" without SnapStart enabled in ms:

Scenario Number	c p50	c p75	c p90	c p99	c p99.9	c max	w p50	w p75	w p90	w p99	w p99.9	w max
Url Connection	2615.48	2672.78	2726.22	3660.9	3817.73	3993.28	6.82	7.57	8.80	22.01	50.82	2172.5
Apache	2831.33	2924.85	2950.12	3120.34	3257.03	3386.67	5.73	6.50	7.88	20.49	49.62	1355.08
AWS CRT	2340.71	2406.5	2482.01	2578.71	2721.06	2890.88	5.73	6.61	8.00	21.07	70.39	980.93

Cold (c) and warm (m) start time with compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) without SnapStart enabled in ms:

Scenario Number	c p50	c p75	c p90	c p99	c p99.9	c max	w p50	w p75	w p90	w p99	w p99.9	w max
Url Connection	2610.59	2700.55	2800.53	3028.36	3184.08	3326.09	7.04	7.88	9.31	22.55	55.04	1286.36
Apache	2880.53	2918.79	2974.45	3337.29	3515.86	3651.65	6.11	7.05	8.94	23.54	62.99	1272.96
AWS CRT	2268.78	2314.49	2341.29	2461.23	2613.98	2754.08	5.55	6.30	7.57	20.49	75.70	1010.95

Cold (c) and warm (m) start time with compilation option "tiered compilation" with SnapStart enabled without Priming in ms:

Scenario Number	c p50	c p75	c p90	c p99	c p99.9	c max	w p50	w p75	w p90	w p99	w p99.9	w max
Url Connection	1510.72	1566.07	1797.68	2006.60	2012.63	2014.23	6.93	7.87	9.38	23.92	935.81	1343.25
Apache	1506.20	1577.06	1845.01	2010.62	2280.46	2281	5.82	6.72	8.39	22.81	798.46	1377.54
AWS CRT	1196.86	1313.44	1584.96	1781.58	1872.88	1873.52	5.55	6.41	7.87	21.40	681.26	1164.44

Cold (c) and warm (m) start time with compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) with SnapStart enabled without Priming in ms:

Scenario Number	c p50	c p75	c p90	c p99	c p99.9	c max	w p50	w p75	w p90	w p99	w p99.9	w max
Url Connection	1567.63	1647.96	1889.80	2026.76	2075.97	2076.57	7.10	8.00	9.69	25.41	953.93	1190.54
Apache	1521.33	1578.64	1918.35	2113.65	2115.77	2117.42	6.01	7.05	8.94	23.92	101.41	1077.45
AWS CRT	11176.70	1259.45	1621.82	1854.25	1856.11	1857.59	5.55	6.30	7.63	21.40	670.53	990.96

Cold (c) and warm (m) start time with compilation option "tiered compilation" with SnapStart enabled and with DynamoDB invocation Priming in ms:

Scenario Number	c p50	c p75	c p90	c p99	c p99.9	c max	w p50	w p75	w p90	w p99	w p99.9	w max
Url Connection	666.97	745.23	965.42	1084.10	1108.20	1108.66	7.21	8.07	9.61	24.22	145.49	377.43
Apache	708.90	790.50	960.61	1041.61	1148.80	1149.91	5.64	6.61	8.38	21.07	141.53	373.37
AWS CRT	679.76	851.18	1026.11	1102.68	1111.53	1111.64	5.92	6.72	8.26	22.09	171.22	1065.32

Cold (c) and warm start (m) time with compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) with SnapStart enabled and with DynamoDB invocation Priming in ms:

Scenario Number	c p50	c p75	c p90	c p99	c p99.9	c max	w p50	w p75	w p90	w p99	w p99.9	w max
Url Connection	673.67	748.22	946.31	1184.96	1213.73	1214.34	7.16	8.13	9.83	25.89	141.53	275.35
Apache	692.79	758.00	1003.80	1204.06	1216.15	1216.88	6.21	7.27	9.38	25.09	103.03	256.65
AWS CRT	640.19	693.49	1022.02	1229.60	1306.90	1307.14	5.64	6.51	8.13	22.81	171.22	877.24

Conclusion

In terms of the HTTP Client choice for Java 17, AWS CRT HTTP Client is the preferred choice in case SnapStart isn't enabled, or SnapStart is enabled, but no priming is applied. In case of priming of the DynamoDB invocation, the results in terms of the cold starts for all 3 HTTP Clients are close to each other, as the initialization of the DynamoDB Client with the HTTP Client and the most expensive first invocation (priming) happens already during the deployment phase of the Lambda function and doesn't impact the further invocations that much. The Apache HTTP Client is probably the most powerful choice, but it shows the worst results for the cold starts when SnapStart is not enabled.

We observed the same also for Java 21, see the measurements in the article Measuring cold and warm starts with Java 21 using different synchronous HTTP clients.

The warm execution times are more or less close to each other for all 3 HTTP clients and compilation options, and vary a bit in favor of one or another HTTP Client, depending on the percentile and compilation option. We observed the same also for Java 21

Can we reduce the cold start a bit further? From our article Measuring cold starts with Java 17 using different deployment artifact sizes, we know that smaller deployment artifact sizes lead to lower cold start times. The usage of AWS CRT HTTP Client adds 18 MB to the deployment artifact size for our sample application (total size 32MB versus 14 MB for URL Connection and Apache HTTP Clients). If we look into the deployment artifact with AWS CRT HTTP Client, we'll discover the following additional packages for each operating system: Linux, OS X, and Windows.

If we take a look into those folders, we'll see, for example, the following content for the linux folder (the same applies for windows and osx folders) :

As we see, the content of such folders is natives file for each operating system and processor architecture: for osx it's libaws-crt-jni.dylib file, for windows - aws-crt-jni.dll and for linux - libaws-crt-jni.so. If we already know that we'll run our Lambda only on Linux x86 architecture, we can delete the osx and windows folders completely and subfolders for arm architecture in the linux folder. This will reduce the deployment artifact size from 32 to 19 MB for AWS CRT HTTP Client and further reduce the cold start time a bit.

The choice of HTTP Client is not only about minimizing cold and warm starts. The decision is much more complex and also depends on the functionality of the HTTP Client implementation and its settings, like whether it supports HTTP/2. AWS published the decision tree, which HTTP client to choose depending on the criteria.

In the next article of the series, we'll make the same measurements for Java 17 but using the asynchronous HTTP Clients.

Update on 06.06.2024. For the CRT client, we can set the classifier (i.e., linux-x86_64) in our POM file to only pick the relevant binary for our platform. See platform-specific-jars and How can I use Powertools for AWS Lambda (Java) with the AWS CRT HTTP Client?. Big thanks to Maximilian Schellhorn for the hint!