DEV Community

Farmin Farzin
Farmin Farzin

Posted on โ€ข Edited on โ€ข Originally published at farmin.dev

2

Running Promises in Series with a delay between each (Throttling API calls)

I wanted to call an endpoint updating an entity and I wanted to do it for 500 entities.
The API server had "rate limit" and I couldn't just use Promise.all since it will call the endpoint in parallel and the rate-limit would block my calls.

So I decided to write a script to bulk call the endpoint in series and have a 2 seconds wait before each call to make sure API server would not block them.

As I did that and it worked great, I thought it'd be advantageous to share!
Is there a better way for doing this?

How I'm doing this is starting from 0 and then adding up to that based on the array of data that I want to update. In the chain of promises that I have, I just added a delay before the API call, every time that I add 1 to the value that I'm passing to update the data till that number would be equal to my array length.

import fetch from "node-fetch";
import { data } from "./data.js";

const bearerToken = ""

const updateData = (id) => {
  return fetch(
    `https://www.url/${id}`,
    {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${bearerToken}`,
      },
    }
  );
};

const delay = () => {
  console.log(`Waiting: 2 seconds.`);
  return new Promise((resolve) => {
    setTimeout(() => {
      resolve("2");
    }, 2000);
  });
};

const startTime = Date.now();

const doNextPromise = async (id) => {
  delay()
    .then((x) => {
      console.log(
        `Waited: ${x} seconds now calling the endpoint for updating data ${data[id]}`
      );
      return updateData(data[id])
        .then((res) => {
          if (res.status !== 200) {
            throw `Error updating data ${data[id]}: ${res.status}`;
          }
          return res.json();
        })
        .then((res) =>
          console.log(`Response: ${JSON.stringify(res)} for data ${data[id]}`)
        )
        .catch((e) => console.log(`Error: ${e}`));
    })
    .then((res) => {
      id++;
      if (id < data.length) doNextPromise(id);
      else console.log(`Total: ${(Date.now() - startTime) / 1000} seconds.`);
    });
};
doNextPromise(0);
Enter fullscreen mode Exit fullscreen mode

Image of Datadog

Create and maintain end-to-end frontend tests

Learn best practices on creating frontend tests, testing on-premise apps, integrating tests into your CI/CD pipeline, and using Datadogโ€™s testing tunnel.

Download The Guide

Top comments (2)

Collapse
 
jonrandy profile image
Jon Randy ๐ŸŽ–๏ธ โ€ข โ€ข Edited

This doesn't bypass the rate limit at all - quite the opposite in fact. It makes your calls obey the rate limit

Collapse
 
farminf profile image
Farmin Farzin โ€ข

Good point! I guess that was my mindset when I was doing that since it was blocking me. But youโ€™re right it is in fact obeying the rate limit to make those calls

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more