DEV Community: Bouhadjila Hamza

How Sets, Maps, and One NPM Package Saved My Backend's Performance

Bouhadjila Hamza — Sat, 12 Jul 2025 10:39:43 +0000

TL;DR: If you're doing a lot of fetch() in loops, especially to internal services, you're probably wasting time and compute. Here's how using Set, Map, and a custom batching/retry tool cut our response time from 4s to 900ms — and what you can do to replicate it.

The Problem

Imagine you're building a simple dashboard that lists books from your favorite public API — say Open Library. For each book, you also want to display the author's details, which are hosted on a different endpoint.

You start with this basic idea:

const books = await fetchBooks(); // returns 50 books

const results = await Promise.all(
  books.map((book) => fetchAuthor(book.author_id))
);

It works, but suddenly things slow down.

Every request to your dashboard makes 50+ fetch calls to the author endpoint. You’re:

Re-fetching the same authors multiple times
Sending all requests in parallel (no concurrency control)
Making your backend cry 😢

The Fix

1. Deduplicate with `Set()`

You realize that many books share the same authors. So instead of fetching for each book, you deduplicate them:

const uniqueAuthorIds = new Set(books.map(b => b.author_id));

Now instead of 50 fetches, maybe you're down to 20. ✅

2. Use `Map()` for Fast Lookup

After fetching the author info, store them in a Map for constant-time access:

const authorMap = new Map();
authorResponses.forEach(({ item, result, success }) => {
  if (success) authorMap.set(item, result.data);
});

const booksWithAuthors = books.map(book => ({
  ...book,
  author: authorMap.get(book.author_id) || null
}));

3. Throttle & Retry with `oh-no-again`

I built a utility called oh-no-again to help manage request batching with:

Concurrency limits
Retry logic
Timeouts
Lifecycle hooks

How to Use `oh-no-again.js`: Full Tutorial

Let’s walk through the full flow using Open Library’s Books and Authors API.

Step 1: Install the package

npm install oh-no-again

Step 2: Fetch books from Open Library

You’ll need a mock function to fetch a list of books. Each book contains an author_id:

async function fetchBooks() {
  return [
    { title: 'Book One', author_id: 'OL23919A' },
    { title: 'Book Two', author_id: 'OL23919A' },
    { title: 'Book Three', author_id: 'OL2162282A' },
  ];
}

Step 3: Create a deduplicated list of authors

const books = await fetchBooks();
const uniqueAuthorIds = new Set(books.map(b => b.author_id));

Step 4: Batch fetch author info with retries and timeouts

import { requestBatcher } from 'oh-no-again';

const authorResponses = await requestBatcher(
  Array.from(uniqueAuthorIds),
  5, // Max 5 concurrent fetches
  (authorId) => ({
    method: 'GET',
    url: `https://openlibrary.org/authors/${authorId}.json`,
    headers: {
      accept: 'application/json',
    },
  }),
  {
    delay: 50,           // Delay between retries in ms
    timeout: 1000,       // Timeout per request in ms
    retries: 2,          // Retry twice before giving up
    returnMeta: true,    // Attach the item and result in the output
    failFast: false,     // Don't abort all on first error
    hooks: { //optional
      onRetry: (err, attempt) =>
        console.warn(`Retry #${attempt + 1}:`, err.message),
      onAbort: (err) => console.warn('Aborted:', err.message),
      onFailure: (err) => console.error('Final failure:', err.message),
      onSuccess: (res) => console.log('Success:', res),
    }
  }
);

Step 5: Map responses to a lookup Map

const authorMap = new Map();
authorResponses.forEach(({ item, result, success }) => {
  if (success) authorMap.set(item, result.data);
});

Step 6: Attach authors to books

const booksWithAuthors = books.map(book => ({
  ...book,
  author: authorMap.get(book.author_id) || null
}));

console.log(booksWithAuthors);

You’re done! 🚀

The Result

✅ Response time dropped from 4s → 900ms on average\
✅ No redundant requests\
✅ Fine-tuned control over retries and batching\
✅ Cleaner, more reliable backend

Takeaways

Avoid fetching inside loops without deduplication
Use Set to ensure you don’t repeat work
Use Map for blazing-fast ID-based access
Use batching + concurrency control (like oh-no-again.js)
Profile before optimizing randomly

Want to Try It?

Check out oh-no-again on npm:\
👉 https://www.npmjs.com/package/oh-no-again

Planned features:

Dynamic batch sizing
Circuit breaker logic
Built-in metrics/logging support

Happy batching! 🚀

Build a Rock-Solid API with Retry & Timeout in 10 Minutes ⏱️

Bouhadjila Hamza — Sat, 05 Jul 2025 16:42:49 +0000

🚀 Retry, Abort, and Batch like a Pro — Build a Resilient Express API with `oh-no-again.js` 😬

Are you tired of flaky APIs ruining your vibe? Let’s fix that with a fun Express.js project using the free and awesome Rick and Morty API and a utility package built from pain and frustration: oh-no-again.

✅ Retry failed requests\
✅ Batch concurrent fetches\
✅ Timeout and abort support\
✅ All powered by native fetch

🌟 What Are We Building?

An Express.js API endpoint:

GET /characters?ids=1,2,3

It will:

Fetch multiple Rick and Morty characters in parallel
Retry failed requests automatically with exponential backoff
Abort slow requests with a timeout
Report success and failure clearly

📆 Step 1: Setup the Project

mkdir rickmorty-batch-api
cd rickmorty-batch-api
npm init -y
npm install express oh-no-again

✍️ Step 2: Create the API

Create a file called server.js:

const express = require('express');
const { requestBatcher } = require('oh-no-again');

const app = express();
const PORT = process.env.PORT || 3000;

app.use(express.json());

app.get('/characters', async (req, res) => {
  try {
    const ids = (req.query.ids || '')
      .split(',')
      .map((id) => parseInt(id.trim()))
      .filter(Boolean);

    if (!ids.length) {
      return res
        .status(400)
        .json({ error: 'Please provide character IDs: ?ids=1,2,3' });
    }

    const result = await requestBatcher(
      ids,
      3, // concurrency
      (id) => ({
        url: `https://rickandmortyapi.com/api/character/${id}`,
        method: 'GET',
        headers: { 'Content-Type': 'application/json' },
      }),
      {
        retries: 3,
        delay: 200,
        timeout: 100,
        returnMeta: true,
        hooks: {
          onRetry: (err, attempt) =>
            console.warn(`[RETRY] Attempt ${attempt + 1}: ${err.message}`),
          onFailure: (err) => console.error('[FAILURE]', err.message),
          onSuccess: (res) =>
            console.log('[SUCCESS] Character fetched.', res.name),
        },
      },
    );

    return res.json({ count: result.length, result });
  } catch (err) {
    res.status(500).json({
      error: 'Failed to fetch users',
      message: err.message,
    });
  }
});

app.listen(PORT, () => {
  console.log(`Server is running on port ${PORT}`);
});

📊 Step 3: Test It

Start the server:

node server.js

Then visit:

http://localhost:3000/characters?ids=1,2,3,9999

You’ll get a result like:

{
  "count": 4,
  "result": [
    {
      "item": 1,
      "result": { "id": 1, "name": "Rick Sanchez", ... },
      "success": true
    },
    {
      "item": 2,
      "result": { "id": 2, "name": "Morty Smith", ... },
      "success": true
    },
    ...
    {
      "item": 9999,
      "error": "Fetch failed with 404 Not Found. Body: {}",
      "success": false
    }
  ]
}

🔍 Under the Hood — What is `oh-no-again`?

oh-no-again is a lightweight utility that gives you:

Feature	Description
`retryHelper`	Retries any async function with `AbortSignal` support
`requestBatcher`	Handles concurrency, batching, retries and metadata
Built-in fetch	Uses native fetch, no axios dependency
Hook system	`onRetry`, `onFailure`, `onSuccess`, `onBatchComplete`

🔗 API Recap

`requestBatcher(data, concurrency, taskFn, options)`

taskFn: (item) => {
  url: string,
  method?: string,
  headers?: Record<string, string>,
  body?: any
}

Options include:

retries, delay, timeout
returnMeta: true returns { success, result, item }
failFast: true throws error immediately on any failure
hooks: lifecycle tracking

🤔 Final Thoughts

This mini project demonstrates how retry logic and batching can be elegant, safe, and fun.\
No more callback hell. No more flaky APIs ruining your night.

And thanks to the open Rick and Morty API — we had a blast doing it.

📂 Resources

🌌 Rick and Morty API: https://rickandmortyapi.com
📆 oh-no-again: https://www.npmjs.com/package/oh-no-again
💻 GitHub Repo: github.com/yupher/oh-no-again

🙌 Built by Bouhadjila Hamza

Because we've all said: "oh no… not again."