DEV Community: Rey Pham

Power up your REST service with Batch API

Rey Pham — Mon, 13 Jul 2020 09:48:10 +0000

RESTful APIs is pretty common nowadays, as a clear and well-structured resource-oriented standard. But we often have to face the use case that requires to call multi APIs to collect all the necessary data. This quickly becomes a problem if your client is a mobile device with limited network, and only interested in a subset data of all your API responses.

A popular solution to this problem is GraphQL , a query language from Facebook. Although GraphQL is a powerful tool, it still requires you to define all the schemas & functions before the client can use it. In this post, I will show you a different approach, by creating a Batch Request API, that can quickly integrate into your REST service without the need to define any schema.

The code

For a Batch Request API, we'll need an Engine that able to build & execute requests sequentially, then collect all responses. For all of the hard part, I will use this JsonBatch library, that provides many out-of-box features like: build & extract JSON object, conditional & looping requests, aggregate functions, ... You can test out it features here

Below is the code to setup a Batch Engine :

@Bean
public BatchEngine getBatchEngine(ObjectMapper objectMapper, RequestDispatcherService dispatcherService) {
    Configuration conf = Configuration.builder()
            .options(Option.SUPPRESS_EXCEPTIONS)
            .jsonProvider(new JacksonJsonProvider(objectMapper))
            .mappingProvider(new JacksonMappingProvider(objectMapper))
            .build();
    JsonBuilder jsonBuilder = new JsonBuilder(Functions.basic());
    return new BatchEngine(conf, jsonBuilder, dispatcherService);
}

And next is the Controller. The logic is quite simple, we only need to build the original request and pass it to Batch Engine along with batch template. After Batch Engine return response, we just warp it up with ResponseEntity and return.

@RequestMapping(value = "/batch", method = RequestMethod.POST, produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
public ResponseEntity batchRequests(@RequestHeader MultiValueMap<String, String> headers, @RequestBody BatchRequest batchRequest) {
    Request request = buildOriginalRequest(headers, batchRequest.getData());
    Response response = batchEngine.execute(request, batchRequest.getTemplate());
    return new ResponseEntity(response.getBody(), HttpStatus.resolve(response.getStatus()));
    return responseEntity;
}

private Request buildOriginalRequest(MultiValueMap<String, String> headers, Object body) {
    Request request = new Request();
    request.setBody(body);
    HashMap<String, List<String>> headerMap = new HashMap<>();
    headers.forEach(headerMap::put);
    request.setHeaders(headerMap);
    return request;
}

BatchRequest is a simple Json object:

{
  "data": { input data needed to build requests }
  "template": { batch template to instruct how to build & execute requests }
}

We also need to implement a simple RequestDispatcher:

@Service
public class RequestDispatcherService implements RequestDispatcher {

    @Autowired
    private RestTemplate restTemplate;

    @Override
    public Response dispatch(Request request, JsonProvider jsonProvider, DispatchOptions options) {
        HttpHeaders requestHeaders = new HttpHeaders();
        requestHeaders.putAll(request.getHeaders());
        HttpEntity<Object> httpEntity = request.getBody() != null ?
                new HttpEntity<>(request.getBody(), requestHeaders) 
                : new HttpEntity<>(requestHeaders);
        ResponseEntity responseEntity = restTemplate.exchange(
                "Your service host" + request.getUrl(),
                HttpMethod.resolve(request.getHttpMethod()),
                httpEntity,
                Object.class);
        return buildResponse(responseEntity);
    }

    private Response buildResponse(ResponseEntity responseEntity) {
        Response response = new Response();
        Map<String, List<String>> headers = new HashMap<>();
        responseEntity.getHeaders().forEach(headers::put);
        response.setHeaders(headers);
        response.setStatus(responseEntity.getStatusCodeValue());
        response.setBody(responseEntity.getBody());
        return response;
    }

}

And it's done. Now let's test it.

A real scenario

Currently, in our system (a Fintech product), we have this scenario: After user login to our system, we will retrieve all the user's information, that includes:

User profile
User’s company profile
User’s wallet information

Normally we have to call 4 APIs to collect all responses, but only need a subset of data. With Batch API, we only have to make a single call with request body like that:

{
  "data": {
    "token": "user's token"
  },
  "template": {
    "requests": [
      {
        "http_method": "GET",
        "url": "/oauth/payload",
        "headers": {
          "Authorization": "Bearer @{$.original.body.token}@"
        },
        "body": null,
        "requests": [
          {
            "http_method": "POST",
            "url": "/report/users",
            "headers": {
              "Authorization": "Bearer @{$.original.body.token}@"
            },
            "body": {
              "id": "$.responses[0].body.data.user_id"
            },
            "requests": [
              {
                "http_method": "POST",
                "url": "/report/wallets",
                "headers": {
                  "Authorization": "Bearer @{$.original.body.token}@"
                },
                "body": {
                  "user_id": "$.responses[0].body.data.user_id"
                },
                "requests": [
                  {
                    "http_method": "POST",
                    "url": "/report/companies",
                    "headers": {
                      "Authorization": "Bearer @{$.original.body.token}@"
                    },
                    "body": {
                      "id": "$.responses[1].body.data.users[0].company_id"
                    }
                  }
                ]
              }
            ]
          }
        ]
      }
    ],
    "responses":[
      {
        "body": {
          "id": "$.responses[1].body.data.users[0].id",
          "full_name": "@{$.responses[1].body.data.users[0].first_name}@ @{$.responses[1].body.data.users[0].last_name}@",
          "email": "$.responses[1].body.data.users[0].email",
          "company": {
            "id": "$.responses[3].body.data.companies[0].id",
            "name": "$.responses[3].body.data.companies[0].name"
          },
          "wallets": [
            {
              "id": "$.id",
              "balance": "$.balance",
              "currency": "$.currency",
              "__array_schema": "$.responses[2].body.data.wallets"
            }
          ]
        }
      }
    ]
  }
}

I have run a performance test with both ways. Below is the test result when run multi requests:

And next is the test result when run a batch request:

As you can see, the batch request way only took on average ~141 ms while the multi requests way took ~369 ms.

Next, let’s look at the size of response. Below is response size when run multi requests:

And here is the response size when run batch request:

By strip out all the unnecessary data from response, we able to reduce the response size 30+ times (9730 > 272)

Conclusion

As you can see, with a Batch API, we can archive almost same result as GraphQL without the limit of pre-defined schema

Originally published at rey5137.com

JsonBatch — Journey from zero to full-fledged Batch Engine — part 1

Rey Pham — Wed, 17 Jun 2020 02:12:26 +0000

Recently, I have published JsonBatch — an Engine to run batch requests with JSON based REST APIs. Although it is a small library, I found it quite satisfying to go through all processes, from designing to developing it. So today, I want to share my journey with you.

I. The idea

It all started with a requirement for our company product. To support one of our users migrate their own system to our system, we need to provide them an Adapter that will bridge the gap between their APIs to our APIs and maintain it for a while until they finish the migrating process.

It’s a common requirement, but we want to reuse it somehow, so we don’t have to repeat the same works with other users. At first, we thinking about providing user a way to config what endpoint that their API will route to, and how to map our response back to their response. Quite straight forward, but the reality is not so simple. Due to the different design of 2 systems, their API will mostly have to call multi-requests and then aggregate all responses.

That is when an idea came to me. What we need is a Batch Engine that can be configured easily & dynamically.

II. The first prototype

So for a Batch Engine, it should take a list of requests, execute sequentially, and collect all responses.

The execution logic is simple, but here comes the first challenge: How to build the request body?
It’s easy if the client can supply all the request bodies, but what if a request body requires data from the response of another request? The Engine needs a way to know how & where to extract data. Because all of the requests & responses will be JSON, so if you want to locate a field, the first answer that came to me is using JsonPath. Fortunately, Java already has a good library Jayway’s JsonPath that supports all my needs.

So now we done with the How question, next is the Where question. The solution I came up with is building a grand JSON contains all the requests & responses. Something like this:

{
 "original": {
   "http_method": "...",
   "url": "...",
   "headers": {
     "header_1": [ "..." ],
     "header_2": [ "..." ],
     ...
   },
   "body": { ... }
 },
 "requests": [
     {
       "http_method": "...",
       "url": "...",
       "headers": {
         "header_1": [ "..." ],
         "header_2": [ "..." ],
         ...
       },
       "body": { ... }
     },
     ...
 ],
 "responses": [
     {
       "status": ...,
       "headers": {
         "header_1": [ "..." ],
         "header_2": [ "..." ],
         ...
       },
       "body": { ... }
     },
     ...
 ]
}

With that, only 1 problem left: we need a schema format to instruct the Engine build the actual JSON body. A quick search and I found out JSON Schema. But after some research, I think JSON Schema is more suited to validating than generating, and the schema format has a structure that quite different from actual JSON. So I’m left with designing my own custom schema.

With some experiments, I came up with a schema that’s simple but good enough. The schema is same as actual JSON but only the string field has a specific format. For example:

{
  "field_1": 1,
  "field_2": "int $.responses[0].body.field_a",
  "field_3": "int[] $.responses[*].body.field_a"
}

With this schema, the Engine will build a JSON with:

field_1 = 1
field_2 is integer, and extract value from JsonPath: $.responses[0].body.field_a
field_3 is integer array, and extract value from JsonPath: $.responses[*].body.field_a

An actual JSON will look like that:

{
  "field_1": 1,
  "field_2": 2,
  "field_3": [2, 3, 4]
}

You can see the structure of schema & actual is quite similar.

Now, after all the pieces came together, it took me about 1 week to code this first prototype. In the next part, I will share how this prototype continues to evolve.

Originally published at rey5137.com

Bad chunk header mystery

Rey Pham — Wed, 18 Mar 2020 09:47:39 +0000

Recently, I encounter an interesting issue in my project at work. I’m working on many Springboot-based microservices, and sometimes, an exception is thrown after using RestTemplate exchange() method:

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.springframework.web.client.ResourceAccessException: I/O error on POST request for “...": Bad chunk header: {...}; nested exception is org.apache.http.MalformedChunkCodingException: Bad chunk header: {...} at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) at javax.servlet.http.HttpServlet.service(HttpServlet.java:660) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

Below is the simplified sequence diagram:

After investigating, here is what I found about this issue:

The exception is thrown after service A received the response from service B.
The issue only happens when I run some performance test cases that flood service A with requests.
The issue is pretty consistent, although not at an exact time, but happens most of the times I run test case.
The interesting thing here is the log from service B & C didn’t show anything wrong when this issue happened, indicated that request is succeeded and response is returned. But is it true???
The exception message is short but still left an important hint: header. Did service A receive an inappropriate header, or more correctly, service B sent an unwanted header? How can we know what header is wrong?

Further investigating, I found that service B just forwards request to service C, then forwards the response from service C to service A. Something like this:

public ResponseEntity<?> forwardRequest(@RequestBody Request request) {
    return restTemplate.exchange(url, HttpMethod.POST, request, ...)
}

If so, did the inappropriate header not come from service B, but from service C instead?

Luckily, we can insert code to log all the headers of service C response. After adding and re-running test case, here is what I get when the exception occurs:

Header: [Connection] > [close]
Header: [Content-Type] > [application/json;charset=UTF-8]
Header: [Date] > [DD/mm/yyyy]
Header: [Transfer-Encoding] > [chunked]

For easier, here is normal response’s headers:

Header: [Content-Type] > [application/json;charset=UTF-8]
Header: [Date] > [DD/mm/yyyy]
Header: [Transfer-Encoding] > [chunked]

Finally, we can see the culprit. It is the header Connection: close that service C returned and service B blindly returned to service A.

But what does Connection: close mean? And how it is included in the service C response?

From RFC 2616, Section 14.10 , Connection: close is included to signal that the connection will be closed after completion of the response. Normally, connection with keepAlive property will not be closed by the server so the client can keep using it for another request. But tomcat has a max-keepalive-timeout setting, that will close connection that reaches a certain keepAlive time. And that is when the server will include Connection: close header.

Conclusion

Now we know that blindly forwarding responses without filtering out headers is a bad practice. In case you don’t care about the header, just extract and return only the response body is enough.

Originally published at rey5137.com