Discussion on: Rate limiting API calls - sometimes a Bottleneck is a good thing

[Aroh Sunday]

This is where I tried it with Promise.all but got the same error. Is this code below look like what you suggested above?

const toReadInParalel = async arrayOfIds => {
  let cvUrl = await getUploadedCVUrl(arrayOfIds);
  let parsedResult = [];
  let save;
  let start = +new Date();
  let parserError;
  await Promise.all(
    cvUrl.map(async url => {
      const file = url.fileurl;
      const filename = url.filename;
      const jobpositionIdFromArray = url.jobPositionId;
      console.log({ file, filename, jobpositionIdFromArray });
      if (!file && !filename) return;

      let singleResult = await throttleApiCall(filename, file);
      console.log({ singleResult });
      const { data } = singleResult;
      if (data.Error) {
        parserError = true;
      }
      if (data) {
        let newparsed = new ParsersCollection(data);
        newparsed.jobPositionId = jobpositionIdFromArray;
        save = await newparsed.save();
        parsedResult.push(save);
      }
    }),
  );

[Aroh Sunday]

I need your assistance to get this resolve. Thanks

[Aroh Sunday]

@ross Coundon, I would appreciate your assistance from the wealth of your experience in the field interacting with several third part API on how I can make a concurrent request to the server without experience the "socket hangout issue". In the first interaction of the loop shown above, I will get results from the third-party API but on the second iteration, there will be a delay for a response from the third party and hence the error message below

Trace: { Error: socket hang up
    at createHangUpError (_http_client.js:323:15)
    at TLSSocket.socketOnEnd (_http_client.js:426:23)
    at TLSSocket.emit (events.js:194:15)
    at TLSSocket.EventEmitter.emit (domain.js:441:20)
    at endReadableNT (_stream_readable.js:1125:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  code: 'ECONNRESET',

[Ross Coundon]

Hi - I'm not sure what to suggest, are you able to share what the 3rd party API is? Do they provide any documentation/information on acceptable usage, time between requests, number of concurrent requests etc?

[Aroh Sunday]

They do not have that spell out on their API documentation. I have sent a mail to them to inquire about the acceptable usage, the time between requests, number of concurrent requests