AWS SDK for Java 2.x asynchronous HTTP clients and their impact on cold start times and memory consumption of Lambda and Java 11

Introduction

Historically AWS SDK for Java 2.x offered the possibility to create both synchronous and asynchronous clients for AWS services, i.e. S3Client and S3AsyncClient, DynamoDbClient and DynamoDbAsyncClient and so on. The difference is of course the programming model, i.e. the invocation of the getItem method on the DynamoDB asynchronous client like

dynamoDbAsyncClient.getItem(GetItemRequest.builder()
      .key(Map.of("PK", AttributeValue.builder().s(id).build()))
      .tableName(PRODUCT_TABLE_NAME)
      .build());

will return the java.util.concurrent.CompletableFuture instead of GetItemResponse itself. CompletableFuture is used in Java as an abstraction to writing non-blocking code by running a task on a separate thread than the main application thread and notifying the main thread about its progress, completion or failure. We can then invoke methods like get, join or whenComplete to process the result of the asynchronous invocation. Here are other examples of the asynchronous programming with the AWS SDK for Java 2.x.

(Asynchronous) HTTP Client Options for Java 11 Runtime

When building the client for AWS service you can pass the appropriate HTTP client implementation to it

private static final DynamoDbAsyncClient dynamoDbAsyncClient = DynamoDbAsyncClient.builder()
    .region(Region.EU_CENTRAL_1)
    .httpClient(NettyNioAsyncHttpClient.create())
    .build();

For synchronous AWS client the default implementation is ApacheHTTPClient. For information about configuring the ApacheHttpClient, see Configuring the Apache-based HTTP client.
As a lightweight option to the ApacheHttpClient, you can use the UrlConnectionHttpClient. For information about configuring the UrlConnectionHttpClient, see Configuring the URLConnection-based HTTP client.

Both synchronous HTTP client implementations are abstracted by the
SdkHttpClient.

For asynchronous AWS client the default implementation is NettyNioAsyncHttpClient.

This asynchronous HTTP client implementation is abstracted by the
SdkAsyncHttpClient

All 3 until now mentioned HTTP client implementations (synchronous and asynchronous) are designed for the long living applications (i.e. deployed in the web application server like Tomcat or Netty). The creation of such a HTTP client takes seconds and actively uses caching (and therefore memory) which perfectly makes sense if the execution environment (i.e. web application server) stays for hours, days or even weeks. In case of using Serverless on AWS and therefore Lambda with the exection environment which can be desposed after minutes or an hour, the creation of such a HTTP client (best practice is to create it in the static initializer block of the class) adds a precious time to the cold start time (which increases response time) and consumes more memory (which is a cost factor for AWS Lambda). Moreover, the creation of HTTP client is one of the major factors which contributes to increase of the cold start time of the Lambda function. For the AWS Lambda more lightweight approach is required.

AWS Common Runtime (CRT) HTTP Client

In the beginning of the Februar 2023 the general availability (GA) of the AWS Common Runtime (CRT) HTTP Client in the AWS SDK for Java 2.x. has been announced. With release 2.20.0 of the SDK, the AWS CRT HTTP Client can now be used in production environments (it was in preview since more than 3 years).

The AWS CRT HTTP Client is an asynchronous, non-blocking HTTP client that can be used by AWS servicec to invoke other AWS APIs. You can use it as an alternative to the default Netty implementation of the SdkAsyncHttpClient interface. The AWS CRT HTTP Client is built on top of the Java bindings of the AWS CRT, which is written in C. It has faster startup time and consumes less memory than other HTTP clients supported in the SDK.

Measuring the cold start times and memory consumption with asynchronous HTTP client implementations

Now it's time to measure the cold start times and memory consumption of the both asynchronous HTTP client implementations Netty and AWS CRT. We will also make the comparison with and without SnapStart enabled on Lambda function. I wrote a lot about the SnapStart for Java 11 in general in the first part of my series and also measured the cold start applying the priming optimization. In all cases I used the default synchronous implemention of the HTTP client which is ApacheHTTPClient.

Now we'll modify our implementation to use the DynamoDbAsyncClient. Let's start with NettyNioAsyncHttpClient. As it is te default asynchronous HTTP client implementation, it's sufficient modify the source code of the DynamoProductDao to build DynamoDbAsyncClient like this

DynamoDbAsyncClient dynamoDbAsyncClient = DynamoDbAsyncClient.builder()
    .region(Region.EU_CENTRAL_1)
    .build();

In order to explicitly configure NettyNioAsyncHttpClient, we first need to define its dependency in pom.xml like this

    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>netty-nio-client</artifactId>
    </dependency>

and then pass it like this

DynamoDbAsyncClient dynamoDbAsyncClient = DynamoDbAsyncClient.builder()
    .region(Region.EU_CENTRAL_1)
    .httpClient(NettyNioAsyncHttpClient.builder().build())
    .build();

We also need to slightly modify the function getProduct to adopt to the asynchronous programming style. The mainly added the join() invocation on CompletableFuture to get GetItemResponse itself.

  public Optional<Product> getProduct(String id) {
    GetItemResponse getItemResponse = dynamoDbClient.getItem(GetItemRequest.builder()
      .key(Map.of("PK", AttributeValue.builder().s(id).build()))
      .tableName(PRODUCT_TABLE_NAME)
      .build()).join();
    if (getItemResponse.hasItem()) {
      return Optional.of(ProductMapper.productFromDynamoDB(getItemResponse.item()));
    } else {
      return Optional.empty();
    }
  }

The default properties of the NettyNioAsyncHttpClient will be good enough for the most use cases. For more information about its configuration options, see Configuring the Netty-based HTTP client.

In order to use AwsCrtAsyncHttpClient instead of NettyNioAsyncHttpClient we have to replace the dependency in pom.xml to

    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>aws-crt-client</artifactId>
    </dependency>

and then replace it when creating DynamoDbAsyncClient like this

DynamoDbAsyncClient dynamoDbAsyncClient = DynamoDbAsyncClient.builder()
    .region(Region.EU_CENTRAL_1)
    .httpClient(AwsCrtAsyncHttpClient.builder().build())
    .build();

The default properties of the AwsCrtAsyncHttpClient will be good enough as well for the most use cases. For more information about its configuration options, see Configuring the AWS CRT-based HTTP client.

As for other examples we'll use Java 11 Corretto and 1024 MB of memory. Now let's measure the cold start (CS) times in milliseconds (I produced approx 100 of them for each use case) and average memory consumption (in MBs) of the Lambda function with both asynchronous HTTP Clients with and without SnapStart enabled on the Lambda function (which retrieves the product by id). In case of SnapStart enabled, I also used priming as described in my article.

Measurement	CS p50	CS p90	CS p99	Memory
Netty w/o SnapStart	3577	3675	3680	148
Netty with SnapStart	364	385	407	125
AWS CRT w/o SnapStart	2080	2158	2209	130
AWS CRT with SnapStart	324	359	392	118

Of course the result will vary depending on the memory settings of the Lambda function.

Conclusion

In this article we explored the ways to use asynchronous HTTP clients in the AWS SDK for Java 2.x for the AWS services which require HTTP communication. We also introduced AWS Common Runtime (CRT) HTTP Client which shortly became general available and explained how to use and configure it.

Then we measured the cold start times and average memory consumption of the Lambda function with both asynchronous HTTP clients with and without SnapStart enabled and prooved that usage of AWS CRT HTTP Client has significantly reduced the cold start times comparing to the Netty HTTP client without SnapStart enabled. It also has a slightly better cold start times (8-10%) in case we enabled Snap Start on the Lambda function and used priming. Also AWS CRT HTTP client consumes slightly less memory comparing to the Netty HTTP client for all use cases.

I also measured the cold start times using the synchronous Apache HTTP client. Without SnapStart enabled the cold start times were comparable to the measurements with Netty Nio Async HTTP client.

With Apache Http client, SnapStart enabled and primin the cold start times for the same application as described in my article were like this

Measurement	p50	p90	p99
Apache with SnapStart	352	401	434

It was clear, that with SnapStart enabled and priming the impact of the choice of the HTTP client (synchronous or asynchronous) will be much less noticable, as the instantiation of the client and the priming invocation happens in the snapshot phase and the restore times will be comparable. But anyway, especially the usage of the asynchronous AWS CRT HTTP client additionally reduced the cold start times by 5-10% comparing to the usage of synchronous Apache Http client. It's worth noticing that the usage of AWS CRT HTTP client added more than 10 MB to the deployment package size of Lambda function.

Some final thoughts : it's a valid question whether the usage of asynchronous programming model will be beneficial for your concrete use case, especially if you use AWS Lambda, as most of them will require less than 1792 MB of memory to achieve the optimal price and performance and therefore Lambda function will have only one processor core available to you. But even having one (not full) core can help you to speed up things. For example, if you make the call to the database (like DynamoDB) and wait for the result (IOWait) to be delivered. Even in this case you can do some computation (or another call to the database) in parallel. CompletableFuture abstraction gives you a lot of functionally to do it.

Word of caution: I experienced some challenges when using both asynchronous HTTP clients (AWS CRT and Netty) with SnapStart enabled and priming. It worked well when I enabled SnapStart and instantiated the DynamoDbAsyncClient client in the static initializer block and then invoked getItem on it during within the handleRequest method (so no priming used). But when I did priming with CRaC and additionally invoked getItem within beforeCheckpoint method, this invocation succeded, but then the invocation of getItem during within the handleRequest method (after the restore phase) got the API Gateway time out after 29 seconds. I haven't figured our the reason for it yet (it seems currently to be a general issue with SnapStart and priming and has nothing to do with (a)synchronous AWS (DynamoDB) client). But in order the experiment to be executed I used 2 different instances of DynamoDbAsyncClient in the modified DynamoProductDao : one for the getItem invocation in priming (beforeCheckout) method and another for the getItem invocation from the handleRequest method. Of course, it's not a clean solution (that's why I only explained it, but have not commited it to my GitHub yet), but for my experiment it was a sufficient hack to pre-load all the classes involved in the invocation and to force Jackson Marshallers to initialize which is quite expensive one time operation for the life cycle of the Lambda function. I'll update this part of article for sure when I find a better solution for this problem.

Update on April 10. AWS reported the rollout of the fix of the described issue with using the asynchronous client, AWS SnapStart and priming.

Comments

[Dmitriy Musatkin]

You can reduce size of artifacts that CRT Http client brings in by selecting a specific platform using classifiers as described here - github.com/awslabs/aws-crt-java#re...