DEV Community

Alex Aslam
Alex Aslam

Posted on

Optimizing Async Operations: When Promise.all Destroys Your Performance (And When to Use Loops Instead)

#webdev #programming #javascript #beginners

The Async Performance Trap We Fell Into

Our database migration script was taking 8 hours to process 50,000 records. The code looked elegant:

await Promise.all(records.map(processRecord)); // "This should be fast!"
Enter fullscreen mode Exit fullscreen mode

But it crashed our database with 500 concurrent connections.

Here’s how we fixed it—and when loops actually beat Promise.all.

1. The Dark Side of Promise.all

🚨 Problem #1: Uncontrolled Concurrency 

// Spawns 50,000 promises at once!
await Promise.all(hugeArray.map(async (item) => {
  await db.insert(item); // 💥 Database melts down
}));
Enter fullscreen mode Exit fullscreen mode

✅ Fix: Batch with p-limit

import pLimit from 'p-limit';
const limit = pLimit(10); // Max 10 concurrent ops

await Promise.all(
  hugeArray.map(item => limit(() => db.insert(item)))
);
Enter fullscreen mode Exit fullscreen mode

🚨 Problem #2: No Error Handling 

// One failure rejects ALL promises
await Promise.all([
  fetchUser(1),
  fetchUser(2), // If this fails, entire batch dies
]);
Enter fullscreen mode Exit fullscreen mode

✅ Fix: Use Promise.allSettled + filtering

const results = await Promise.allSettled(promises);
const successes = results.filter(r => r.status === 'fulfilled');
Enter fullscreen mode Exit fullscreen mode

🚨 Problem #3: Memory Overload 

// Loads ALL records into memory before processing
const records = await getAllRecords(); // 500MB array
await Promise.all(records.map(processRecord));
Enter fullscreen mode Exit fullscreen mode

✅ Fix: Stream with for-await-of

for await (const record of getRecordsStream()) {
  await processRecord(record); // Processes one at a time
}
Enter fullscreen mode Exit fullscreen mode

2. When Old-School Loops Win

Scenario Promise.all Loop
Order matters ❌ No ✅ Yes
Memory-constrained ❌ No ✅ Yes
Backpressure needed ❌ No ✅ Yes

Example: Sequential Processing 

// Slower but safer
for (const item of items) {
  await process(item); // One at a time
}
Enter fullscreen mode Exit fullscreen mode

Example: Batched Loops 

// Best of both worlds
const batchSize = 100;
for (let i = 0; i < items.length; i += batchSize) {
  const batch = items.slice(i, i + batchSize);
  await Promise.all(batch.map(process)); // Controlled concurrency
}
Enter fullscreen mode Exit fullscreen mode

3. Real-World Benchmarks

Method Time (10K ops) Memory Usage
Promise.all (unlimited) 2 min 💥 1.2GB 🚨
p-limit (concurrency 10) 8 min 200MB
Batched loops (100/chunk) 6 min 50MB

Lesson learned: "Faster" isn’t always better.

Key Takeaways

⚡ Promise.all is great for → Small, independent, non-IO-heavy tasks
🐢 Loops are better for → Order-sensitive, memory-bound, or backpressured workflows
🛠 Hybrid approach → Batched Promise.all inside loops

Which async pattern burned you? Share your war story!

Further Reading

Top comments (0)