We live in era of microservices. Those have A LOT of communication happening inside their ecosystems. You might have seen so-called death stars of microservices:
image by Cornell University researchers
Every single click in modern internet triggers multitudes of network calls and, as you probably know, network is unreliable. This is one of the reasons you should set request timeouts when fetching something over the wire. It's better to incorporate some other techniques as well, but that's the topic for another article (and even whole book).
Why bother about timeouts?
Setting them is very important, because network might fail, your instance might get slow or crash. You don't want users to wait indefinitely just because 1 of your 1000 servers suddenly shut down. People hate waiting, it affects their happiness and, therefore, your revenue. If network call gets stuck, you should abandon it, retry and, unless your cluster is having problems, it most likely to succeed.
Let's see how we can implement this in Java and Go.
Common Java approach
Most popular http client in java is Apache HttpClient. Here's how configuring request timeouts will look like:
RequestConfig config = RequestConfig.custom()
.setConnectTimeout(100)
.setConnectionRequestTimeout(100)
.setSocketTimeout(300)
.build();
Those 3 are all different and independent. Biggest problem we're facing here is how to represent 500 ms as 3 different timeouts, should it be 100 + 100 + 300 or 50 + 50 + 400? And, more importantly, do we even care if connect timeout will take 200 ms? Imagine that server will complete request in 50 ms, so total response time would be 250 ms, in most situations you don't care, it's completely fine!
On the other side you can't set timeouts much bigger, because it will lead to a longer requests. Also, socket timeout is just a timeout between any consecutive packets being read from the socket, not the whole response being sent back to you.
Nonetheless, it's all we can do using Apache HttpClient. Let's take a look at Go.
Go standard library
Go, however, has support for whole client call timeout:
callTimeoutMs := 2000
httpClient := http.Client{Timeout: time.Duration(callTimeoutMs) * time.Millisecond}
_, err := httpClient.Get("http://httpbin.org/delay/1") // this waits 1 second
if err != nil {
log.Fatal(err)
}
Here we're setting call timeout to 2 seconds and calling httpbin, which will imitate work for 1 second long and return a response after. Launching this will lead to a successful call.
If we set call timeout to 1 second it will fail with a message similar to given , which is exactly what we are looking for!
2019/11/25 19:06:09 Get http://httpbin.org/delay/1: net/http:
request canceled (Client.Timeout exceeded while awaiting headers)
On the contrary, configuring call timeout only is not always the best we can achieve. Imagine having a long and heavy request which takes 10 seconds to complete. It would be nasty to wait 10 seconds for response and figure out that servers was trying to establish connection all this time and no actual work was done.
Solution to this? Connect timeout. Yes, the one we were sort of blaming earlier. Combined with call timeout it gives robust protection to your network interactions:
callTimeoutMs := 10000
connectTimeoutMs := 25
httpClient := http.Client{
Timeout: time.Duration(callTimeoutMs) * time.Millisecond,
Transport: &http.Transport{
DialContext: (&net.Dialer{
Timeout: time.Duration(connectTimeoutMs) * time.Millisecond,
}).DialContext,
},
}
_, err := httpClient.Get("http://httpbin.org/delay/5") // this waits for 5 seconds
if err != nil {
log.Fatal(err)
}
Exceeding connect timeout will give us desirable behavior:
2019/11/25 19:12:25 Get http://httpbin.org/delay/5: dial tcp 54.172.95.6:80: i/o timeout
Go gives you a lot of flexibility with it's standard HTTP client, and library is supported by the vendor - perfect combination, that's why I like it very much! Now let's get back to Java world one more time.
So, is Java doomed?
No. There are less popular alternatives, such as JDK 11 http client and OkHttp, which supports call timeout features. Sadly, those are not the top results when searching on Google, so people are less aware of them or not willing to start using them.
JDK 11
It is a modern http client delivered with standard library, which supports HTTP/1.1, HTTP/2, async calls via CompletableFuture and provides convenient api to work with. Let's combine both timeouts with it:
HttpClient httpClient = HttpClient.newBuilder()
.connectTimeout(Duration.ofMillis(50))
.version(HttpClient.Version.HTTP_1_1)
.build();
HttpRequest request = HttpRequest.newBuilder()
.GET()
.uri(URI.create("http://httpbin.org/delay/1"))
.timeout(Duration.ofMillis(2000))
.build();
httpClient.send(request, HttpResponse.BodyHandlers.ofString());
This client will provide nice and easy to understand error messages(especially compared to Go) for connect and call timeouts respectively:
Exception in thread "main" java.net.http.HttpConnectTimeoutException:
HTTP connect timed out
Exception in thread "main" java.net.http.HttpTimeoutException:
request timed out
OkHttp
OkHttp supports call and connect timeouts too. Both can be configured at client abstraction level, which is slightly more convenient compared to JDK11 implementation:
OkHttpClient httpClient = new OkHttpClient.Builder()
.connectTimeout(Duration.ofMillis(100))
.callTimeout(Duration.ofMillis(1000))
.build();
How to choose timeout values, anyway?
Last thing I'd like to mention is how to derive those 2 numbers. Call timeout is more about SLA/SLO you have with other services and connect timeout is about expectations from underlying network. For example, if you're sending requests to the same datacenter, then 100ms would be fine (although it should establish connection faster than 5ms), but operating on top of mobile networks (which are more error-prone) will require higher connect timeouts.
Wrap-up
In this article we discussed why timeouts are important, why 'classic' timeouts doesn't meet modern requirements and which instruments to use to force them. I hope you found something useful in it.
Top comments (0)