AWS SDK for Java 2.x asynchronous HTTP clients and their impact on cold start times and memory consumption of Lambda and Java 11

Introduction

Historically, AWS SDK for Java 2.x offered the possibility to create both synchronous and asynchronous clients for AWS services, i.e., S3Client and S3AsyncClient, DynamoDbClient and DynamoDbAsyncClient, and so on. The difference is, of course, the programming model, i.e., the invocation of the getItem method on the DynamoDB asynchronous client, like

dynamoDbAsyncClient.getItem(GetItemRequest.builder()
      .key(Map.of("PK", AttributeValue.builder().s(id).build()))
      .tableName(PRODUCT_TABLE_NAME)
      .build());

will return the java.util.concurrent.CompletableFuture instead of GetItemResponse itself. CompletableFuture is used in Java as an abstraction to writing non-blocking code by running a task on a separate thread from the main application thread and notifying the main thread about its progress, completion, or failure. We can then invoke methods like get, join, or whenComplete to process the result of the asynchronous invocation. Here are other examples of asynchronous programming with the AWS SDK for Java 2.x.

(Asynchronous) HTTP Client Options for Java 11 Runtime

When building the client for the AWS service, you can pass the appropriate HTTP client implementation to it.

private static final DynamoDbAsyncClient dynamoDbAsyncClient = DynamoDbAsyncClient.builder()
    .region(Region.EU_CENTRAL_1)
    .httpClient(NettyNioAsyncHttpClient.create())
    .build();

For a synchronous AWS client, the default implementation is ApacheHTTPClient. For information about configuring the ApacheHttpClient, see Configuring the Apache-based HTTP client.
As a lightweight option to the ApacheHttpClient, you can use the UrlConnectionHttpClient. For information about configuring the UrlConnectionHttpClient, see Configuring the URLConnection-based HTTP client.

Both synchronous HTTP client implementations are abstracted by the
SdkHttpClient.

For an asynchronous AWS client, the default implementation is NettyNioAsyncHttpClient.

This asynchronous HTTP client implementation is abstracted by the
SdkAsyncHttpClient

All 3 until now mentioned HTTP client implementations (synchronous and asynchronous) are designed for the long living applications (i.e., deployed in a web application server like Tomcat or Netty). The creation of such an HTTP client takes seconds and actively uses caching (and therefore memory), which perfectly makes sense if the execution environment (i.e., web application server) stays for hours, days, or even weeks. In case of using Serverless on AWS and therefore Lambda with the exection environment which can be desposed after minutes or an hour, the creation of such a HTTP client (best practice is to create it in the static initializer block of the class) adds a precious time to the cold start time (which increases response time) and consumes more memory (which is a cost factor for AWS Lambda). Moreover, the creation of an HTTP client is one of the major factors that contribute to the increase in the cold start time of the Lambda function. For the AWS Lambda more lightweight approach is required.

AWS Common Runtime (CRT) HTTP Client

In the beginning of the Februar 2023 the general availability (GA) of the AWS Common Runtime (CRT) HTTP Client in the AWS SDK for Java 2.x. has been announced. With release 2.20.0 of the SDK, the AWS CRT HTTP Client can now be used in production environments (it was in preview since more than 3 years).

The AWS CRT HTTP Client is an asynchronous, non-blocking HTTP client that can be used by AWS services to invoke other AWS APIs. You can use it as an alternative to the default Netty implementation of the SdkAsyncHttpClient interface. The AWS CRT HTTP Client is built on top of the Java bindings of the AWS CRT, which is written in C. It has a faster startup time and consumes less memory than other HTTP clients supported in the SDK.

Measuring the cold start times and memory consumption with asynchronous HTTP client implementations

Now it's time to measure the cold start times and memory consumption of both asynchronous HTTP client implementations, Netty and AWS CRT. We will also make the comparison with and without SnapStart enabled on the Lambda function. I wrote a lot about the SnapStart for Java 11 in general in the first part of my series and also measured the cold start applying the priming optimization. In all cases, I used the default synchronous implementation of the HTTP client, which is ApacheHTTPClient.

Now we'll modify our implementation to use the DynamoDbAsyncClient. Let's start with NettyNioAsyncHttpClient. As it is the default asynchronous HTTP client implementation, it's sufficient modify the source code of the DynamoProductDao to build DynamoDbAsyncClient like this

DynamoDbAsyncClient dynamoDbAsyncClient = DynamoDbAsyncClient.builder()
    .region(Region.EU_CENTRAL_1)
    .build();

To explicitly configure NettyNioAsyncHttpClient, we first need to define its dependency in pom.xml like this

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>netty-nio-client</artifactId>
</dependency>

and then pass it like this

DynamoDbAsyncClient dynamoDbAsyncClient = DynamoDbAsyncClient.builder()
    .region(Region.EU_CENTRAL_1)
    .httpClient(NettyNioAsyncHttpClient.builder().build())
    .build();

We also need to slightly modify the function getProduct to adapt to the asynchronous programming style. They mainly added the join() invocation on CompletableFuture to get GetItemResponse itself.

public Optional<Product> getProduct(String id) {
    GetItemResponse getItemResponse = dynamoDbClient.getItem(GetItemRequest.builder()
      .key(Map.of("PK", AttributeValue.builder().s(id).build()))
      .tableName(PRODUCT_TABLE_NAME)
      .build()).join();
    if (getItemResponse.hasItem()) {
      return Optional.of(ProductMapper.productFromDynamoDB(getItemResponse.item()));
    } else {
      return Optional.empty();
    }
  }

The default properties of the NettyNioAsyncHttpClient will be good enough for most use cases. For more information about its configuration options, see Configuring the Netty-based HTTP client.

To use AwsCrtAsyncHttpClient instead of NettyNioAsyncHttpClient, we have to replace the dependency in pom.xml to

<dependency>
   <groupId>software.amazon.awssdk</groupId>
   <artifactId>aws-crt-client</artifactId>
</dependency>

and then replace it when creating DynamoDbAsyncClient like this

DynamoDbAsyncClient dynamoDbAsyncClient = DynamoDbAsyncClient.builder()
    .region(Region.EU_CENTRAL_1)
    .httpClient(AwsCrtAsyncHttpClient.builder().build())
    .build();

The default properties of the AwsCrtAsyncHttpClient will be good enough as well for most use cases. For more information about its configuration options, see Configuring the AWS CRT-based HTTP client.

As for other examples, we'll use Java 11 Corretto and 1024 MB of memory. Now let's measure the cold start (CS) times in milliseconds (I produced approx 100 of them for each use case) and average memory consumption (in MBs) of the Lambda function with both asynchronous HTTP Clients with and without SnapStart enabled on the Lambda function (which retrieves the product by id). In case of SnapStart enabled, I also used priming as described in my article.

Measurement	CS p50	CS p90	CS p99	Memory
Netty w/o SnapStart	3577	3675	3680	148
Netty with SnapStart	364	385	407	125
AWS CRT w/o SnapStart	2080	2158	2209	130
AWS CRT with SnapStart	324	359	392	118

Of course, the result will vary depending on the memory settings of the Lambda function.

Conclusion

In this article, we explored the ways to use asynchronous HTTP clients in the AWS SDK for Java 2.x for the AWS services that require HTTP communication. We also introduced AWS Common Runtime (CRT) HTTP Client, which shortly became generally available, and explained how to use and configure it.

Then we measured the cold start times and average memory consumption of the Lambda function with both asynchronous HTTP clients with and without SnapStart enabled, and proved that usage of AWS CRT HTTP Client has significantly reduced the cold start times compared to the Netty HTTP client without SnapStart enabled. It also has slightly better cold start times (8-10%) in case we enabled Snap Start on the Lambda function and used priming. Also, AWS CRT HTTP client consumes slightly less memory compared to the Netty HTTP client for all use cases.

I also measured the cold start times using the synchronous Apache HTTP client. Without SnapStart enabled, the cold start times were comparable to the measurements with the Netty Nio Async HTTP client.

With Apache Http client, SnapStart enabled, and priming the cold start times for the same application as described in my article, were like this

Measurement	p50	p90	p99
Apache with SnapStart	352	401	434

It was clear that with SnapStart enabled, and priming the impact of the choice of the HTTP client (synchronous or asynchronous) will be much less noticeable, as the instantiation of the client and the priming invocation happen in the snapshot phase, and the restore times will be comparable. But anyway, especially the usage of the asynchronous AWS CRT HTTP client additionally reduced the cold start times by 5-10% comparing to the usage of the synchronous Apache HTTP client. It's worth noticing that the usage of the AWS CRT HTTP client added more than 10 MB to the deployment package size of the Lambda function.

Some final thoughts: it's a valid question whether the usage of an asynchronous programming model will be beneficial for your concrete use case, especially if you use AWS Lambda, as most of them will require less than 1792 MB of memory to achieve the optimal price and performance, and therefore the Lambda function will have only one processor core available to you. But even having one (not full) core can help you to speed up things. For example, if you make the call to the database (like DynamoDB) and wait for the result (IOWait) to be delivered. Even in this case, you can do some computation (or another call to the database) in parallel. CompletableFuture abstraction gives you a lot of functionality to do it.

Word of caution: I experienced some challenges when using both asynchronous HTTP clients (AWS CRT and Netty) with SnapStart enabled and priming. It worked well when I enabled SnapStart and instantiated the DynamoDbAsyncClient client in the static initializer block and then invoked getItem on it within the handleRequest method (so no priming was used). But when I did priming with CRaC and additionally invoked getItem within beforeCheckpoint method, this invocation succeeded, but then the invocation of getItem during within the handleRequest method (after the restore phase) got the API Gateway timeout after 29 seconds. I haven't figured out the reason for it yet (it seems currently to be a general issue with SnapStart and priming and has nothing to do with (a)synchronous AWS (DynamoDB) client). But for the experiment to be executed, I used 2 different instances of DynamoDbAsyncClient in the modified DynamoProductDao: one for the getItem invocation in the priming (beforeCheckout) method and another for the getItem invocation from the handleRequest method. Of course, it's not a clean solution (that's why I only explained it, but have not commited it to my GitHub yet), but for my experiment it was a sufficient hack to pre-load all the classes involved in the invocation and to force Jackson Marshallers to initialize which is quite expensive one time operation for the life cycle of the Lambda function. I'll update this part of the article for sure when I find a better solution for this problem.

Update on April 10. AWS reported the rollout of the fix for the described issue with using the asynchronous client, AWS SnapStart, and priming.

Comments

[Dmitriy Musatkin]

You can reduce size of artifacts that CRT Http client brings in by selecting a specific platform using classifiers as described here - github.com/awslabs/aws-crt-java#re...