Avoid Spring RestTemplate Default Implementation to Prevent Performance Impact

Introduction

In the world of software engineering, with so many frameworks offering outstanding functionality and out-of-the-box features, we tend to overlook the underlying implementation and configuration. One such case is Spring's RestTemplate: any APIs that use the default RestTemplate constructor or the RestTemplateBuilder's build method (no parameters) to create a RestTemplate instance will also suffer from its shortcomings.
In this post, I’ll talk about the performance issues that it possesses and its solutions.

In case you also want to know about RestClient performance improvements, kindly visit this post Never Use Spring RestClient Default Implementation in Production

📬 I also write about production engineering and real-world system behavior in my newsletter.

If topics like resilience, distributed systems, and performance tuning interest you, you can subscribe here:

📬 Subscribe to Newsletter🙏🏻

Bottleneck

Let's take a simplified example to understand the problem better.

Below is the illustration of the problem.

As depicted in above diagram, assume that we have a weather API that provides an HTTP GET endpoint to fetch data, which in-turn uses a downstream API (forecast API), let's say the weather API's response time is 200 ms (milliseconds), so with simple calculation this API could serve 5 requests per second, however, if it uses RestTemplate default implementation then maximum no. of HTTP connections per host (in this case for forecast API) is defaulted to 2 connections. So, even though the API is capable of serving 5 requests per second due to a max no. of 2 HTTP connections, the remaining 3 requests have to wait until a response is received from the downstream API (assuming here forecast API response time is 200 ms as well). This creates a bottleneck and a performance issue.

Deepdive

In Spring Framework, the RestTemplate class utilizes an underlying HttpClient implementation for handling HTTP requests. By default, RestTemplate uses the SimpleClientHttpRequestFactory, which creates a new HttpURLConnection (based on the JDK's own HTTP libraries) for each request. This factory does not have built-in connection pooling.
However, starting from Spring 4.3, the RestTemplate class can be configured to use HttpComponentsClientHttpRequestFactory, which is based on Apache HttpClient and provides connection pooling capabilities. The number of connections in the pool can be customized through the configuration.
But again, the HttpComponentsClientHttpRequestFactory does not specify the maximum number of connections in the pool. Instead, it uses the Apache HttpClient's default value, which is 2 connections per route (per target host).

Solutions

There are multiple ways to fix this problem, depending on the level of granular customization needed for an API. We need to start by adding Apache HttpClient to the project's dependencies.

Add Apache HttpClient dependency to the project

Here is an example for maven project.

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.13</version> <!-- or the latest version -->
</dependency>

1. First Method

High-level setting with minimum configuration. Setting max connections and connections per route

    @Bean
    public CloseableHttpClient httpClient() {
        return HttpClientBuilder.create()
                .setMaxConnTotal(100)      // Set required maximum total connections
                .setMaxConnPerRoute(20)    // Set required maximum connections per route
                .build();
    }

    @Bean
    public RestTemplate restTemplate() {
        return new RestTemplate(new HttpComponentsClientHttpRequestFactory(httpClient()));
    }

GitHub link to the complete file
BasicRestClientCustomConnectionConfig.java

2. Second Method

More granular settings, creating a finer level configuration. Setting connection timeout, max connection and connections per route

    @Bean
    public PoolingHttpClientConnectionManager customizedPoolingHttpClientConnectionManager(){
      /*
        Setting the connection caching time to 5 minutes, so that the connection is in an open state for 5 mins. 
Every connection will add some extra time, as every 5 mins SSL handshake will happen. However, software is all about trade-offs, and here we are making sure we don't keep calling the cached API DNS
      */
      PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager(5, TimeUnit.MINUTES);
      connManager.setMaxTotal(100);    // Set required maximum total connections
      connManager.setDefaultMaxPerRoute(20);     // Set required maximum connections per route
      return connManager;
    }

    @Bean
    public RestTemplate restTemplate() {
        RequestConfig reqConfig = RequestConfig.custom()
                                  .setConnectionRequestTimeout(4000) //in milliseconds
                                  .setConnectionTimeout(4000)
                                  .setSocketTimeout(4000)
                                  .build();

        HttpClientBuilder clientBuilder = HttpClientBuilder.create()
                                  .setConnectionManager(customizedPoolingHttpClientConnectionManager)
                                  .setConnectionManagerShared(true)  //this is important to set as true in case of more than one downstream APIs as we want to set a common HTTP connection pool                   .setDefaultRequestConfig(reqConfig);

        HttpComponentsClientHttpRequestFactory reqFactor = new HttpComponentsClientHttpRequestFactory();
        reqFactor.setHttpClient(clientBuilder.build();

        return new RestTemplate(reqFactor);
    }

GitHub link to complete file AdvanceRestClientCustomConnectionConfig.java

Conclusion

Once you have defined the max total connections and connections per route as stated above. More connections are available at the API level to make more parallel downstream API calls, which will improve the overall performance as the API can serve more requests simultaneously.
Going to our earlier example, now with more connections, it would look like the diagram below.

If you have reached here, then I have made a satisfactory effort to keep you reading. Please be kind to leave any comments or ask for any corrections.

My Other Blogs:

• Shift Left Performance Testing in Spring Boot: Stability Through Control
• Never Use Spring RestClient Default Implementation in Production
• When Resilience Backfires: Retry and Circuit Breaker in Spring Boot
• Cracking Software Engineering Interviews
• Setup GraphQL Mock Server
• ⁠Supercharge Your E2E Tests with Playwright and Cucumber Integration
• Team Agreement in Software Engineering
• My firsthand experience with web component - learnings and limitations
• Streaming logs of all pods at once
• Certified Kubernetes Application Developer (CKAD) Exam Tips

Comments

[Ionut Ciuta]

Hey, this article is quite useful, but your code is messed up in multiple place.
e.g. PoolingHtppClientConnectionManager has a typo, customizedPoolingHtppClientConnectionManager() doesn't have a return statement etc.

[AK DevCraft]

Thanks for catching those, I appreciate it! I've fixed those.