Discussion on: Playing with promises and concurrency in TypeScript

[Olivier Guimbal]

In the same spirit, I very often use this helper, which is very handy to limit your concurrency while keeping things parallel.

For instance, if you want to gather 4938935 users based on a list of IDs, its not advisable to launch 4938935 requests in parallel, but you could use parallel(100, userIds, fetchUser) to always run 100 concurrent requests until all are processed.

/**
 * Similar to Promise.all(), 
 * but limits parallelization to a certain number of parallel executions.
 */
export async function parallel<T>(concurrent: number
         , collection: Iterable<T>
         , processor: (item: T) => Promise<any>) {
    // queue up simultaneous calls
    const queue = [];
    const ret = [];
    for (const fn of collection) {
        // fire the async function, add its promise to the queue, and remove
        // it from queue when complete
        const p = processor(fn).then(res => {
            queue.splice(queue.indexOf(p), 1);
            return res;
        });
        queue.push(p);
        ret.push(p);
        // if max concurrent, wait for one to finish
        if (queue.length >= concurrent) {
            await Promise.race(queue);
        }
    }
    // wait for the rest of the calls to finish
    await Promise.all(queue);
}

`

[Javier Toledo]

Interesting, that's a very cool way to use Promise.race!