forEach vs. for: The Asynchronous Showdown!

Understanding How Looping Choices Affect Asynchronous Performance

In my recent project, I needed to update a large number of records in a NoSQL database — over a thousand. Since updating them all at once wasn’t feasible and the process was asynchronous too, I decided to process them in smaller batches of 20 records each. I implemented this in a loop using forEach, where each set of 20 would be updated before moving on to the next set. However, after putting this approach into practice, I noticed that not all records were being updated as expected, even within these smaller batches. For instance, when I attempted to update a batch of 25 records, only 10 were successfully updated. There were no logs for the other 10 records, and the number of records that failed to update varied randomly — sometimes 12, sometimes 5, or sometimes 7.

Upon further debugging, I observed that these records were likely to be skipped during the update process. Now, why did this happen? Let’s explore this issue to understand the behaviour and identify potential reasons behind it.

Problem
After some research, I discovered that using an asynchronous function inside a forEach loop without a proper mechanism to await all iterations can lead to problems. In this case, we used async/await within the forEach loop without ensuring that all iterations were awaited (see code below):

async function patchRecords(records) {
    // Here, the length of records is 25
    let successfulUpdates = 0; // Initialize a counter for successful updates

    records.forEach(async (item) => {
        await databaseName.patch(item);
        successfulUpdates++; // Increment counter on successful update
    });
    // Return the number of records updated successfully
    return successfulUpdates;
}

const response = await patchRecords(records); // Make sure to await this call

In the code above, there is no mechanism to ensure that all asynchronous iterations of the loop are completed. The forEach method fires multiple asynchronous calls to patch the records, but the patchRecords function quickly reaches the return statement without waiting for those patches to finish. As a result, while the updates are processed in the background, the function moves on without waiting for them to complete. This can lead to some pending promises remain unfulfilled, which may cause skipped or discarded updates.

Solution
So how can we solve this? Before we jump into the solution, let’s first understand the different ways to iterate over data. Fundamentally, there are two approaches: Iterating over data sequentially or in parallel.

If you want to iterate over data asynchronously in a sequential manner, using a forEach loop can lead to the issues we discussed. Instead, it’s better to use a modern for…of loop or a simple for loop, as these allow await to work properly and ensure that all updates are processed without being skipped (see the code below).

async function patchRecords(records) {
    let successfulUpdates = 0; // Counter for successful updates

    for (const item of records) {
        try {
            // Query to patch each record
            await databaseName.patch(item);
            successfulUpdates++; // Increment counter on success
        } catch {
            // Handle any errors
        }
    }

    return successfulUpdates; // Return the number of records updated successfully
}

const response = await patchRecords(records); // Make sure to await this call

On the other hand, if you want to process records in parallel, again using forEach won’t work. While each asynchronous callback does return a promise, some of those promises may remain unfulfilled because they aren’t awaited. Instead, use map to generate an array of promises and then await them with Promise.all (see the code below).

async function patchRecords(records) {
    // Create an array of promises for patching each record
    const patchPromises = records.map(async (item) => {
        try {
            await databaseName.patch(item);
            return true; // Successful update
        } catch {
            return false; // Failed update
        }
    });

    // Wait for all patch operations to complete
    const results = await Promise.all(patchPromises);

    // Count and return the number of successful updates
    return results.filter(result => result).length;
}

const response = await patchRecords(records); // Ensure this is awaited
// Some other code

This shows how choosing the right looping method can greatly impact the expected results of your asynchronous operations. The type of loop you use — whether it’s forEach, for…of, or a standard for loop — plays a crucial role in how well your code manages asynchronous tasks.

Wrapping up

• Challenge: Updating records in bulk using forEach led to inconsistent updates, with many records being skipped.
• Root Cause: forEach doesn’t wait for asynchronous operations to complete, leading to unfulfilled promises.
• Solution: Use a for…of loop to wait for each record update sequentially for accuracy, or use map with Promise.all for concurrent updates.

Did I explain things clearly? let me know in comments below.

I enjoyed sharing my experience, and I hope you found it valuable too! As I’m still exploring this space, I’d love your feedback and suggestions. ☺️ If you see areas where I can improve, or have topics you’d like me to cover, please don’t hesitate to reach out! 🔉 website. I’m excited to learn and grow with your help. 👋🏻