JavaScript Secrets: How to Implement Retry Logic Like a Pro

Introduction

As enthusiastic JavaScript users, we often rely on REST APIs and microservices to fetch data for our applications, or to create and update information. Sometimes, we encounter issues like temporary errors, unstable connections, or service downtimes. These can disrupt our applications because we might not have effective strategies in place to try the operation again, leading to a disappointing user experience. 🌐💡

To improve this situation and ensure our applications run smoothly, it’s essential to implement solid retry strategies. This approach allows our applications to handle disruptions gracefully, maintaining stability and keeping users happy. By preparing our applications to retry failed operations intelligently, we enhance their resilience against common network issues. Let’s make our applications more reliable and user-friendly by mastering retry strategies. 🚀🛡️

Click here for GitHub Repo for code samples

Let’s jump straight into exploring different methods to implement retry logic! 🤿

So, what are retry strategies?

Imagine your application as a determined traveler navigating through a landscape riddled with obstacles. Just as a traveler might encounter closed roads or bad weather, our applications often face similar hurdles in the digital realm — like a sudden service outage or a glitch in the network. Retry strategies are our map and compass in these scenarios, guiding our application on when and how to make another attempt to reach its destination successfully. 🗺️⏳

They help our applications understand whether a problem is just a temporary glitch or something more serious. Based on this understanding, our applications can decide to wait a bit and try again, perhaps taking a slightly different path by adjusting the timing or method of the request.

---

Below are some strategies (in short):

Let’s explore different retry strategies through code examples having simulation of request failure as well, each offering a distinct method for overcoming digital challenges and keeping our applications on track. 🛤️

📈 Exponential Backoff
Think of this as gradually increasing your steps as you try to leap over a puddle. Initially, you start with small jumps, but with each attempt, you increase your leap exponentially. This strategy helps lessen the burden on our digital pathways, making it less likely for our efforts to splash down into the water again.

Doubles the wait time after each retry to reduce system load and failure likelihood.

const axios = require('axios');

let attemptCounter = 0; // Keep track of attempts

/**
 * Attempts to fetch data from a given URL using axios with simulated failures.
 * The function simulates network failures for the first 2 attempts by throwing an error,
 * demonstrating how retry mechanisms can handle transient errors.
 * After the simulated failures, actual axios requests are made.
 * 
 * Parameters:
 * - url: The URL to fetch data from.
 * - retries: The number of retries allowed.
 * - delay: The initial delay before retrying, which doubles with each retry.
 * 
 * The function uses exponential backoff for the delay between retries,
 * effectively handling temporary network issues by giving time for recovery.
 */
const fetchData = async (url, retries, delay) => {
  try {
    attemptCounter++;
    if (attemptCounter <= 2) {
      throw new Error('Simulated network failure');
    }

    const response = await axios.get(url);
    console.log(`Success: ${response.status}`);
    return response.data;
  } catch (error) {
    console.log(`Attempt ${attemptCounter} failed with error: ${error.message}. Waiting ${delay} ms before retrying.`);
    if (retries > 0) {
      await new Promise(resolve => setTimeout(resolve, delay));
      return fetchData(url, retries - 1, delay * 2);
    } else {
      throw new Error('All retries failed');
    }
  }
};

const url = 'https://jsonplaceholder.typicode.com/posts/1';
fetchData(url, 3, 1000).catch(console.error);

/**
 * Output:
 * Attempt 1 failed with error: Simulated network failure. Waiting 1000 ms before retrying.
 * Attempt 2 failed with error: Simulated network failure. Waiting 2000 ms before retrying.
 * Success: 200
 */

➡️ ️Linear Backoff
This is like walking on a straight path, where you take a step forward at regular intervals. After each retry, you wait a bit longer, but the wait time increases by the same amount each time. It’s steady and predictable, making sure we don’t rush and stumble.

Increases wait time by a constant amount after each retry for predictable delays.

const axios = require('axios');

let attemptCounter = 0; // Tracks the number of attempts for simulating failures

/**
 * This function demonstrates a linear backoff retry strategy using axios for HTTP requests.
 * It simulates network failures for the initial attempts and then succeeds, showcasing
 * how applications can recover from transient issues with appropriate retry logic.
 *
 * The linear backoff strategy increases the delay between retries by a fixed increment,
 * providing a balanced approach to managing retry intervals and allowing the system
 * some time to recover before the next attempt.
 *
 * Parameters:
 * - url: The URL to fetch data from.
 * - retries: The total number of retries allowed.
 * - delay: The initial delay before the first retry.
 * - increment: The amount by which the delay increases after each retry.
 *
 * On failure, the function waits for the specified delay, then retries the request
 * with an increased delay, based on the linear backoff calculation.
 */
const fetchDataWithLinearBackoff = async (url, retries, delay, increment) => {
  try {
    attemptCounter++;
    if (attemptCounter <= 3) {
      throw new Error('Simulated network failure');
    }

    const response = await axios.get(url);
    console.log(`Success: ${response.status}`);
    return response.data;
  } catch (error) {
    console.log(`Attempt ${attemptCounter} failed with error: ${error.message}. Waiting ${delay} ms before retrying.`);
    if (retries > 0) {
      await new Promise(resolve => setTimeout(resolve, delay));
      return fetchDataWithLinearBackoff(url, retries - 1, delay + increment, increment);
    } else {
      throw new Error('All retries failed');
    }
  }
};

const url = 'https://jsonplaceholder.typicode.com/posts/1';
fetchDataWithLinearBackoff(url, 5, 1000, 2000).catch(console.error);

/**
 * Output:
 * Attempt 1 failed with error: Simulated network failure. Waiting 1000 ms before retrying.
 * Attempt 2 failed with error: Simulated network failure. Waiting 3000 ms before retrying.
 * Attempt 3 failed with error: Simulated network failure. Waiting 5000 ms before retrying.
 * Success: 200
 */

🕛 Fixed Delay
Imagine pausing to breathe at regular intervals, no matter how far you’ve run. This strategy keeps the waiting time the same between each retry, providing our applications with a consistent rhythm to follow, ensuring they don’t wear themselves out too quickly.

Maintains a constant wait time between retries, regardless of attempt count.

const axios = require('axios');

let attemptCounter = 0; // To track the attempt number for simulating a scenario

/**
 * Demonstrates implementing a fixed delay retry strategy for HTTP requests using axios.
 * It simulates failures for the initial attempts to illustrate how fixed delay retries
 * can effectively manage transient errors by waiting a predetermined amount of time
 * before each retry attempt, regardless of the number of attempts made.
 *
 * The fixed delay approach ensures that retries are spaced out by a consistent interval,
 * offering a straightforward and predictable method to allow for system recovery or error
 * resolution before the next attempt. This strategy is particularly useful in scenarios
 * where the expected recovery time is consistent.
 *
 * Parameters:
 * - url: The endpoint URL to make the HTTP GET request to.
 * - retries: The number of retry attempts before giving up.
 * - delay: The fixed time in milliseconds to wait before each retry attempt.
 */
const fetchData = async (url, retries, delay) => {
  try {
    attemptCounter++;

    if (attemptCounter <= 3) {
      throw new Error('Simulated network failure');
    }

    const response = await axios.get(url);
    console.log(`Success: ${response.status}`);
    return response.data;
  } catch (error) {
    console.log(`Attempt ${attemptCounter} failed with error: ${error.message}. Waiting ${delay} ms before retrying.`);
    if (retries > 0) {
      await new Promise(resolve => setTimeout(resolve, delay));
      return fetchData(url, retries - 1, delay);
    } else {
      throw new Error('All retries failed');
    }
  }
};

const url = 'https://jsonplaceholder.typicode.com/posts/1';
fetchData(url, 4, 1000).catch(console.error);

/**
 * Output:
 * Attempt 1 failed with error: Simulated network failure. Waiting 1000 ms before retrying.
 * Attempt 2 failed with error: Simulated network failure. Waiting 1000 ms before retrying.
 * Attempt 3 failed with error: Simulated network failure. Waiting 1000 ms before retrying.
 * Success: 200
 */

🌀 Fibonacci Backoff
Inspired by nature’s spiral, this approach increases the wait time following the elegant Fibonacci sequence. Each step forward combines the wisdom of the last two, finding a harmonious balance between rushing and waiting too long, guiding us through challenges with natural grace.

Uses the Fibonacci sequence to determine the wait time, balancing between aggressive and moderate delays.

const axios = require('axios');

let attemptCounter = 0; // Tracks the current attempt for simulation

/**
 * Calculates the Fibonacci number for a given index.
 * The Fibonacci sequence is a series of numbers where each number is the sum
 * of the two preceding ones, usually starting with 0 and 1.
 *
 * - index: The position in the Fibonacci sequence.
 */
const calculateFibonacciNumber = (index) => {
    if (index <= 1) return index;
    let previous = 0, current = 1, temp;
    for (let i = 2; i <= index; i++) {
        temp = previous + current;
        previous = current;
        current = temp;
    }
    return current;
};

/**
 * Performs an HTTP GET request using axios with retries based on the Fibonacci backoff strategy.
 * Initially simulates network failures for the first few attempts to illustrate how the application
 * recovers using retry strategies with increasing delays based on the Fibonacci sequence.
 *
 * - url: The URL to send the request to.
 * - retries: The number of retries allowed before failing.
 * - baseDelay: The base delay in milliseconds for the Fibonacci backoff calculation.
 */
const fetchData = async (url, retries, baseDelay) => {
  try {
    attemptCounter++;

    if (attemptCounter <= 2) {
      throw new Error('Simulated network failure');
    }

    const response = await axios.get(url);
    console.log(`Success: ${response.status}`);
    return response.data;
  } catch (error) {
    console.log(`Attempt ${attemptCounter} failed with error: ${error.message}. Waiting for the next attempt.`);
    if (retries > 0) {
      const delay = calculateFibonacciNumber(5 - retries + 1) * baseDelay;
      console.log(`Waiting ${delay} ms before retrying.`);
      await new Promise(resolve => setTimeout(resolve, delay));
      return fetchData(url, retries - 1, baseDelay);
    } else {
      throw new Error('All retries failed after ' + attemptCounter + ' attempts');
    }
  }
};

const url = 'https://jsonplaceholder.typicode.com/posts/1';
fetchData(url, 5, 100).catch(console.error);

/** 
 * Output:
 * Attempt 1 failed with error: Simulated network failure. Waiting for the next attempt.
 * Waiting 100 ms before retrying.
 * Attempt 2 failed with error: Simulated network failure. Waiting for the next attempt.
 * Waiting 100 ms before retrying.
 * Success: 200
 */

🎲 Randomised Retry
As if rolling dice to decide how long to wait, this strategy introduces an element of chance in our retry timing. This randomness helps distribute our attempts more evenly, ensuring that not everyone rushes through the door at once, reducing the pressure on our systems.

Selects a random wait time before retrying to distribute attempts and reduce system load.

const axios = require('axios');

let attemptCounter = 0; // To track the number of attempts and simulate network failures

/**
 * Performs an HTTP GET request using axios with retries that incorporate randomized delays.
 * This strategy simulates network failures for the initial attempts to demonstrate how the application
 * can recover by retrying with random delays between a specified minimum and maximum range.
 * The randomization of retry intervals helps distribute the load and reduce peak pressure on the system.
 *
 * - url: The endpoint URL for the HTTP GET request.
 * - retries: The number of retries before giving up.
 * - minDelay: The minimum delay in milliseconds before retrying.
 * - maxDelay: The maximum delay in milliseconds before retrying.
 */
const fetchData = async (url, retries, minDelay, maxDelay) => {
  try {
    attemptCounter++;

    if (attemptCounter <= 2) {
      throw new Error('Simulated network failure');
    }

    const response = await axios.get(url);
    console.log(`Success: ${response.status}`);
    return response.data;
  } catch (error) {
    console.log(`Attempt ${attemptCounter} failed with error: ${error.message}.`);
    if (retries > 0) {
      const randomDelay = Math.random() * (maxDelay - minDelay) + minDelay; // Calculate a random delay
      console.log(`Waiting ${Math.round(randomDelay)} ms before retrying.`);
      await new Promise(resolve => setTimeout(resolve, Math.round(randomDelay)));
      return fetchData(url, retries - 1, minDelay, maxDelay);
    } else {
      throw new Error(`All retries failed after ${attemptCounter} attempts`);
    }
  }
};

const url = 'https://jsonplaceholder.typicode.com/posts/1';
fetchData(url, 3, 500, 1500).catch(console.error);

/**
 * Output:
 * Attempt 1 failed with error: Simulated network failure.
 * Waiting 1487 ms before retrying.
 * Attempt 2 failed with error: Simulated network failure.
 * Waiting 777 ms before retrying.
 * Success: 200
 */

⚡ Immediate Retry
Sometimes, the best approach is to try again right away, especially when we sense a quick solution might be just around the corner. This strategy is all about seizing the moment, ready to spring back into action at the first sign of a hiccup.

Retries immediately without delay, ideal for quickly resolvable issues.

const axios = require('axios');

let attemptCounter = 0; // Tracks the current attempt number to simulate network failures

/**
 * Executes an HTTP GET request using axios, employing an immediate retry strategy upon failure.
 * This approach simulates network failures for the first few attempts, illustrating how the application
 * adapts by retrying the operation immediately without any delay, making it suitable for quickly resolvable
 * transient issues.
 *
 * Immediate retries are used in scenarios where there is a high probability that errors are temporary
 * and can be resolved without introducing a delay, effectively improving the chances of a successful request
 * in environments with fluctuating network stability.
 *
 * - url: The endpoint URL for the HTTP GET request.
 * - retries: The number of allowed retries before giving up.
 */
const fetchData = async (url, retries) => {
  try {
    attemptCounter++;

    if (attemptCounter <= 3) {
      throw new Error('Simulated network failure');
    }

    const response = await axios.get(url);
    console.log(`Success: ${response.status}`);
    return response.data;
  } catch (error) {
    console.log(`Attempt ${attemptCounter} failed with error: ${error.message}.`);
    if (retries > 0) {
      console.log('Retrying immediately.');
      return fetchData(url, retries - 1);
    } else {
      throw new Error(`All retries failed after ${attemptCounter} attempts`);
    }
  }
};

const url = 'https://jsonplaceholder.typicode.com/posts/1';
fetchData(url, 5).catch(console.error);

/**
 * Output:
 * Attempt 1 failed with error: Simulated network failure.
 * Retrying immediately.
 * Attempt 2 failed with error: Simulated network failure.
 * Retrying immediately.
 * Attempt 3 failed with error: Simulated network failure.
 * Retrying immediately.
 * Success: 200
 */

---

Utilizing npm Packages for Retry Logic**

axios-retry
axios-retry offers a straightforward way to add retry functionality to your axios requests. This library can automatically retry failed requests under certain conditions, such as network errors or receiving specific HTTP response codes, and it supports configurable retry strategies, including exponential backoff.

To use axios-retry in a your application, first, ensure you have axios and axios-retry installed.

npm install axios axios-retry

or

yarn add axios axios-retry

Then, you can configure axios-retry in your application like so:

import axios from 'axios';
import axiosRetry from 'axios-retry';

// Configure axios-retry to automatically retry requests
axiosRetry(axios, {
  retries: 3, // Number of retry attempts
  retryDelay: axiosRetry.exponentialDelay, // Use exponential backoff delay between retry attempts
});

// Example of making a request with axios that will be retried upon failure
axios.get('https://jsonplaceholder.typicode.com/posts/1')
  .then(response => console.log(response.data))
  .catch(error => console.error(error));

Using axios-retry in frontend can significantly simplify handling retries for HTTP requests in your applications, allowing you to make your web apps more robust and reliable with minimal additional code.
npm package -> axios-retry

retry
The retry package provides a flexible way to add retry functionality to your code, suitable for any asynchronous operation or logic you want to attempt multiple times upon failure. Here's a basic guide on how to use it:

First, you need to install the package using npm or yarn:

npm install retry

or

yarn add retry

Here’s a simple example of how to use retry to perform a task that might fail:

const retry = require('retry');
const axios = require('axios'); // Assuming axios is used for HTTP requests

async function fetchData(url) {
  const operation = retry.operation({
    retries: 3, // Maximum amount of retries
    factor: 2, // The exponential factor for delay
    minTimeout: 1000, // The number of milliseconds before starting the first retry
    maxTimeout: 2000, // The maximum number of milliseconds between two retries
  });

  operation.attempt(async currentAttempt => {
    try {
      const response = await axios.get(url);
      console.log('Data:', response.data);
    } catch (error) {
      console.log(`Attempt ${currentAttempt} failed: ${error.message}`);
      if (operation.retry(error)) {
        console.log(`Retrying...`);
        return;
      }
      console.error('Request failed after retries:', error.message);
    }
  });
}

fetchData('https://jsonplaceholder.typicode.com/posts/1');

The retry package is powerful and flexible, making it suitable for a wide range of retry scenarios beyond just HTTP requests. You can adapt the retry logic to fit various asynchronous tasks, such as database operations, filesystem access, or any other operations that might require retries upon failure.

npm package -> retry

---

Best Practices for Implementing Retry Logic

• Determine When to Retry: Not all errors should trigger a retry. Evaluate whether the error is transient and likely to be resolved with subsequent attempts.

• Limit Retries: Set a maximum number of retries to prevent infinite loops.

• Consider the User Experience: Ensure that retry logic does not degrade the user experience, especially in client-facing applications.

---

Conclusion 🚀
Incorporating retry logic into your JavaScript applications is a critical step toward ensuring reliability and resilience. Whether you choose to implement these strategies directly or leverage existing npm packages, the ability to gracefully handle transient errors will significantly enhance your application’s robustness and user satisfaction.

Connect with me on LinkedIn: https://www.linkedin.com/in/anu95/

Follow me on Instagram:
https://www.instagram.com/code_with_onu/

Happy coding! 💻 ❤️

Anurag Gupta
SDE-II, Microsoft

Comments

[Filip Danić]

Good, detailed post! Some additional thoughts:

1. There’s a great post on the AWS Blog* that explains more strategies and why jitter (randomness) is so important.
2. The npm package exponential-backoff is also worth looking at, in particular, if you want better out-of-the-box support for multiple jitter approaches

*link: aws.amazon.com/blogs/architecture/...

[Anurag Gupta]

Thanks for putting this here. Really helpful!

[Ski]

I prefer loop style implementation of retry over recursive one

let retryCount = 0
while(true) {
   try {
       return await somethingThatCanFail()
   } catch(e) {
       retryCount++
       if(retryCount > maxRetryCount) {
           throw e
       } else {
           console.log(...)
           await new Promise(r => setTimeout(1000 * 2**retryCount))
       }
   }
}

[Vitaly Tomilov]

It is a bit an overkill. Here's generic retry solution - gist.github.com/vitaly-t/6e3d28585...

Because it fully supports callbacks, you can easily add any resilience strategy - linear/exponential, whatever, into the delay's callback.