Grouping AJAX requests in a pool

Original article: https://aralroca.com/2018/09/10/grouping-ajax-requests-in-a-pool/

In this article I would like to explain what I did to improve the speed in the communication between client and server. It's important to understand that this is not a global solution for all AJAX request. Instead, it can only be applied in some particular type of request, as we will see soon if you keep reading.

Note that in most projects other solutions could be more efficient.

What's the initial problem?

I'm currently working in a complex React application where the user can mount their own interactive widgets by using React components. Some of these interactive widgets need to do some AJAX request to load / insert some data (or whatever) on componentDidMount, componentWillUnmount or more (as we will see soon).

To implement this first approach, we can make every interactive widget (React container) call the POST /whatever on componentDidMount method.

Image1. In this example is POST /evaluate

In this implementation, each container is the responsible of doing the corresponding POST /evaluate. Or, using Redux, each container is the responsible to dispatch an action that, in turn, will do the request. After resolving each promise, each container decides what to do with the evaluation.

At the beginning, in this example, is going to emit at least 5 requests at the same tick of the clock. And, after resolving these requests, React is going to change the DOM at least 5 times, in different renders.

This implementation can be enough quickly for some cases. However, remember that the user can mount their own page with a big amount of interactive widgets. So this means that 20, 30 or more request can be emitted at the same ticking.

Unfortunately, there is a limitation on how many requests we can emit at the same time, so the rest are added in a queue that increments the total time. Moreover, in this /evaluate we are evaluating the same things through different widgets (for example, the item "a" is evaluated 3 times in the Image1).

Our mission in this article is to improve the request time by grouping all these requests into one, and removing duplicates.

Type of request to group

Before starting the implementation, the first important step is to know which is the request target. We can't group every type of request, at least without modifying the behaviour on back-side.

How should the request be?

• It should accept an array as a parameter.
• The response is an array in the same order.
• If any item can't be resolved, instead of using a 500 Internal Server Error, the status should be 200 OK. The error should be in the response array index.
• Each item should spend approximately the same time to be resolved. If the evaluation of "a" is taking 10 times more than the evaluation of "f", this wouldn't be a good approach because we prefer to load each interactive widget independently.

Grouping AJAX requests in a container

After analysing the initial problem, a common solution we can apply, in order to improve the loading speed of the page, is using a parent container to group all the requests while removing the duplicated items.

This parent container in the componentDidMount method does this AJAX call (or uses a Redux Action to do that). Then, this parent container distributes the results to its children (or, using Redux, each children container gets their results from the store).

In this way, instead of emitting 20 or 30 request at the same time, we group all these request into one. Also, after resolving the promise of the request, React is going to render the new DOM for all the interactive widgets at the same time.

More problems on the way...

In the above example we only took care about componentDidMount method. However, in reality, each interactive widget can have an "interval" property in the configuration. This widgets are able to send different requests on each "interval" tick.

In this case we are having more troubles to group all requests emitted in each tick of the clock in the parent container. However, it's possible. In order to fix the problem we can create a common interval in the parent container with the greatest common divisor of all the children intervals. This global interval checks in every tick which requests need to emit in order to group them. Also, another alternative is to create different intervals on the parent container without time duplicates.

By the way, let me tell you something else: Some interactive widgets can be connected and the "interval" property can be changed depending on the output of another widget.

More troubles... Still not impossible to group requests depending on each ticking by using a parent container, but maybe we need to re-think a painless and more flexible way to implement this.

Grouping AJAX requests in a pool

A different way, instead of implementing all the logic of all the cases in the parent container, is to use an AJAX pool to directly group all the request emitted in the same ticking into only one request.

The pool is adding in a queue all things to evaluate emitted in the same ticking. In the next tick it will do the request by sending all the queue as param.

To use this pool, it's necessary that the interactive widgets use the corresponding service instead of sending directly the request.

Instead of:

axios.post('/evaluate', { data: [a, b] })
.then(res => {
 // ...
})

Use:

EvaluationService.evaluate([a, b])
.then(res => {
 // ...
})

These promises always return the filtered result to each widget.

Each service will use an AJAX pool or not, depending on the type of the request. In this case in the EvaluationService we are going to use this pool.

This EvaluationService is the responsible of initialising the pool, adding the items into the queue, removing duplicates and saving the indexes. Then, when the request is resolved, it will filter the required items from the total response.

import AjaxPool from './services/ajax-pool';

const pool = new AjaxPool();

export default class EvaluateService {
  static evaluate(data) {
    const id = pool.initPool();

    const indexes = data
      .map((item) => {
        let index = pool.findInQueue(id,
          existingItem => _.isEqual(existingItem, item),
        );

        if (index === -1) {
          index = pool.addToQueue(id, exp);
        }

        return index;
      });

    return pool
      .request(id, '/evaluate', queue => ({  data: queue }), 'post')
      .then((allEvaluations) => indexes.map(index => allEvaluations[index]));
  }
}

Every time we call the evaluate method of this service, it first calls the initPool to get the corresponding "id" of the pool. This "id" is unique for each AJAX request. If there are more than one execution in the same tick of the clock, the same "id" should be used in all the group.

The purpose of the AJAX pool is to resolve all the promises of the group with the same response, but using just one AJAX request.

import uuid from 'uuid';
import axios from 'axios';

const DEFAULT_DELAY = 0; // Wait the next ticking

export default class AjaxPool {
  constructor(milliseconds = DEFAULT_DELAY) {
    this.DELAY_MILLISECONDS = milliseconds;
    this.queues = {};
    this.needsInitialization = true;
    this.requests = {};
    this.numRequest = {};
  }

  /**
   * Initialising the queue 
   */
   initPool() {
     if (this.needsInitialization) {
       this.requestID = uuid();
       this.queues[this.requestID] = [];
       this.needsInitialization = false;
       this.numRequest[this.requestID] = 0;
     }

     return this.requestID;
   }

   findInQueue(id, method) {
     if (typeof method !== 'function') {
       return -1;
     }

     return _.findIndex(this.queues[id], method);
   }

   cleanRequest(id) {
     this.numRequest[id] -= 1;

     if (this.numRequest[id] === 0) {
       delete this.requests[id];
       delete this.queues[id];
       delete this.numRequest[id];
     }
   }

   /**
    * Add to queue
    * 
    * @param {any} queueElement 
    * @return {number} index of element on the queue
    */
   addToQueue(id, queueElement) {
     return this.queues[id].push(queueElement) - 1;
   }

   request(id, url, getData, method = 'get') {
     this.numRequest[id] += 1;
     return new Promise((res, rej) => {
       _.delay(() => {
         this.needsInitialization = true;

         if (!this.requests[id]) {
           const data = typeof getData === 'function' ? getData(this.queues[id]) || {} : {};
           this.requests[id] = axios[method](url, data);
         }
         // For each request in the same "ticking" is doing one AJAX 
         // request, but all resolve the same promise with the same result
         this.requests[id]
           .then((result) => {
             if (result.error) {
               rej(result.error);
             } else {
               res(result);
             }

             this.cleanRequest(id);
           })
           .catch((err) => {
             this.cleanRequest(id);
             rej(err);
           });
       }, this.DELAY_MILLISECONDS);
     });
   }
}

In this case we won't use a big delay, it's just going to be 0 milliseconds to wait the next ticking. However, it's possible to use some milliseconds as a param to construct the pool. For example, if we use 100ms, it will group more requests.

const pool = new AjaxPool(100);

📕 Code: https://stackblitz.com/edit/ajax-pool

Conclusion

Grouping requests in a pool:

• Improves the total loading time in Client, avoiding the addition of some requests in a queue.
• The server has less requests, reducing costs.
• It's reusable and every component of the project can use it without extra logic.

Although:

• It's not always the best solution, only for a specific type of requests.

Comments

[rhymes]

Nice idea and solution, although you had to change the server API as well.

I guess before you had a single resource, like POST /evaluate/1 and then turned it into a batch system.

The drawback of your system is that you need a "god object" that keeps track of all the input data but it's a solid idea to avoid duplicate calls. If more than one widget depends on "state A" then you end up asking the server about it only once.

I saw you tested also Promise.all :-)

A possible alternative, to limit the amount of connections to the server, could have been HTTP/2's multiplexing (while keeping the single resource) paired with caching.

Multiplexing opens one single TCP connection per origin, so you shouldn't have the issue of too many widgets connections on the same page.

Caching allows you to avoid evaluating the same value twice on the server (even if introducing caching you get another set of problems).

The only issue of your solution, correct me if I'm wrong, is that if the batch call fails, it fails for every widget on the page. In the HTTP/2 scenario if one the HTTP calls fails, the others go on indipendently.

[Aral Roca]

Thank you for the comment. I just learned something new 😊 I'm going to start learning HTTP/2 to understand how multiplexing works.

My solution was thought for a project that uses AWS lambda on back-side. I'm not sure that AWS lambda supports HTTP/2, but I'm going to investigate! I like your proposal.

[rhymes]

It's just an idea, your solution might be enough for your use case for a long time. Worst case scenario as you said: you learn something new :-)

AFAIK API Gateway does not support HTTP/2 :-(

[Abhinav Galodha]

Nice article, Could you please share the details of the tool to create the animation?

[Aral Roca]

I used illustrator + imovie to make a video, then Gifski to convert this video into Gif

[Abhinav Galodha]

Thanks, Aral.

[Mihail Malo]

Have you considered GraphQL?
It was originally designed for precisely this usecase:
All components declare their data needs, then this can potentially be batched down into one query, and all of the data comes back in one response.