Iterating through items and dealing with asynchronous logic (i.e. API calls) are probably two of the most common tasks we have to perform as JavaScript devs. In this article, we will discuss the best approaches to combine async/await and iterative logic. There will be a time when you would want to run async operations inside for loops (or any type of other loops). Let’s take a look at how to deal with such situations.
Update: I uploaded a video based on this article as well.
Reading Promises in sequence
Let’s say we have a list of files and we would like to read and log the contents of each file in the sequence. How would we do this? Well, we can use a for … of the loop inside an async function. Here’s the code snippet.
async function printFiles () {
let fileNames = ['picard', 'kirk', 'geordy', 'ryker', 'worf'];
for (const file of fileNames) {
const contents = await fs.readFile(file, 'utf8');
console.log(contents);
}
}
💡 Be advised that if you want to read files in sequence you can not use a forEach loop.
Let’s elaborate on this with a quick example.
async function someFunction(items) {
items.forEach( async(i) => {
const res = await someAPICall(i);
console.log('--->', res);
});
}
function someAPICall(param) {
return new Promise((resolve, reject)=>{
setTimeout(()=>{
resolve("Resolved" + param)
},param);
})
}
someFunction(['3000','8000','1000','4000']);
In the code above we have a simple async function called someFunction, it takes in an array as a parameter, iterates the array and for each item makes an API request (via out fake API function). In this case, we want to resolve the API calls in sequence. We want our output to print the following
// expected
3000
8000
1000
4000
Instead of this output, we see the following result
// actual
1000
3000
4000
8000
Instead of running the API calls in sequence the forEach loop just sets the API calls one after another. It doesn’t wait for the previous call to finish. This is why we get the promises that resolve first. This is the main reason we can not use a forEach loop.
On contrary, we can use a reduce function to iterate over the array and resolve the promises in sequence. Let’s take a quick look at an example of this.
function testPromise(time) {
return new Promise((resolve, reject) => {
setTimeout(() => {
console.log(`Processing ${time}`);
resolve(time);
}, time);
});
}
let result = [3000,2000,1000, 4000].reduce( (accumulatorPromise, nextID) => {
return accumulatorPromise.then(() => {
return testPromise(nextID);
});
}, Promise.resolve());
result.then(e => {
console.log("All Promises Resolved !!✨")
});
Pretty neat isn’t it? Another way of resolving promises in the sequence is with an async generator.
async function* readFiles(files) {
for(const file of files) {
yield await readFile(file);
}
};
Generators and support by most modern browsers and Node 10 and up. You can learn more about generators and iterators in Javascript here.
Resolving Promises in parallel
Next, let’s take a look at how we can resolve promises in parallel. Let’s go back to our first example. Instead of reading the files in the sequence we now want to read them in parallel. In this scenario, we don’t care about what order the contents get printed in our console. Therefore we can simply use a Promise.all() function with a map.
async function printFiles () {
let fileNames = ['picard', 'kirk', 'geordy', 'ryker', 'worf'];
await Promise.all(fileNames.map(async (file) => {
const contents = await fs.readFile(file, 'utf8');
console.log(contents);
}));
}
Each of the async callback function calls do return a promise, we are stashing them and resolving them all at once in parallel with a Prmiss.all().
I hope this quick read gave you an insight into how to use asynchronous code within loops. If you enjoyed this article please follow me on twitter @HaqueShadid. That’s all for today, until next time.
References:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_Generators
https://stackoverflow.com/questions/37576685/using-async-await-with-a-foreach-loop
https://css-tricks.com/why-using-reduce-to-sequentially-resolve-promises-works/
Top comments (11)
I think in practice, I'd be horrified if someone tried to use a reduce over a simple for..of for such a simple case but it was a good article. Thanks!
Hello @dylanwatsonsoftware , I'm new with JS. I'm really curious about why you horrified, may you give me the best solution for this simple case? Thanks!!
Hmm just rereading the article. The example I was referring to is:
But you could write it much more simply, something like:
Some cases you might need the first one... But usually the 2nd one is much easier to read, which means it's less likely to contain a bug.
Thank you, the 2nd is easy for me to understand, actually it took me a while to understand why the first one can run.
don't think reduce is a neat way,I found it difficult to read as well, async generator is neat though
Excellent topic and article! The code examples were very clear.
The reduce example is neat, after I re-read a few times to understand it. I think it's unnecessary syntactic sugar for building the .then() chain imperatively using a for loop as others have mentioned. I could be wrong and maybe this pattern of building a .then() chain with reduce is common in practice. But I still think it's better to be simple than smart, even if it means more lines of code. IMO, a reduce is better suited for more trivial problems like calculating a sum.
As for resolving promises in parallel, there is a simple improvement you can make to your code that will get you the contents in the right order to be printed to your console. All you need is to create a new outputArray and a counter=0 variable outside of your iteration, and as you are iterating through the fileNames, pass in the array index of the fileName, and push each promise result into the outputArray using the same index to build outputArray in the same order as the input array. Also within each promise, increment the counter variable and check for when counter=fileNames.length to know when all your promises have resolved before and when you can call console.log(outputArray).
Thank you @z2lai interesting point. I will make a follow up post on this one
I forgot to mention that using a counter variable in my example to determine when all promises have been resolved is actually a replacement for Promise.all() (it might even be that this is how Promise.all() is implemented under the hood).
This approach is very well explained in YDKJS's section on "Interaction" in the Asynchrony chapter. I recommend people to go through that entire section: github.com/getify/You-Dont-Know-JS...
This is a great blog post. Your suggestions just save my project more bandwidth. We have a particular endpoint that update the exchange rates of multiple currency pairs, previously this endpoint takes about 30s to complete, now 3s.
Thanks, you deserve some Satoshi!
i think best way to achieve this making prototype function and then good to go
Array.prototype.forAsyncSerial = async function (fn = null) {
let result = [];
for (let i = 0; i < this.length; i++) {
if (typeof fn !== 'function') result[i] = await this[i];
else result[i] = await fn(this[i], i);
}
return result;
};
Array.prototype.forAsyncParallel = async function (fn = null) {
if (typeof fn !== 'function') return await Promise.all(this);
else return await Promise.all(this.map(fn));
};
Thank you