Making A Bunch of Requests from a Node Server? Try Promise.all!

Sometimes you find yourself needing to make a ton of http requests. For the most part this a bad idea, and you should really be abstracting your requests and not hammering a REST API since that's how you break things on the internet.

Buuuuuut for testing purposes or just trying to get something to friggin' work, we might be feeling a lil' hacky. We are developers ya know, and girls just want to have fun!

Disclaimer:

Please don't use this on any existing REST service that you don't control. This will possibly break any rate restrictions that service has put in place, and worse case could overload their servers.

I found myself in one of these scenarios while testing a bunch of mock spreadsheet data, where I wanted to make hundreds of requests to some server routes our team was building. Thus, the advent of this Node code you see below you.

Before continuing, this hack expects some knowledge around JS Promises, ES6 Syntax, and Node Modules. With that out of the way let's look at this solution and break it down:

Our Solution

This module doesn't do anything super fancy. But it does utilize some super fancy built in javascript Objects.

• It formats an array of promises that we can feed into Promise.all.
• When those asynchronously resolve we get a big happy bundle (an array for those strictly typed folks) of response objects :).

How do we Achieve this?

We leverage a handy Array prototype .map method to turn our array of request options, into new array containing promises:

const promiseArray = reqArray.map(req => new Promise(
  // things our Promise should do
));

Each Promise will resolve result of the request or reject the error if the request fails, asynchronously of course:

async (resolve, reject) => {
  try {
    resolve(await httpPromise(req));
  } catch (err) {
    reject(err);
  }
}

finally we just return the result of Promise.all which we pass our newly created array of Promises, or console an error if we get errors in those requests.

return Promise.all(promiseArray)
  .then((responses) => responses)
  .catch(err => {
    return { "message": "bulk request failed", "error": err }
  });

Hope this helps with whatever crazy asynchronous actions you're trying to accomplish. This can easily be refactored for use in other environments besides Node and can be used with other events (Database queries, Cloud Resource Interactions) that you want Javascript to handle only upon completion.

Happy Hacking :)