DEV Community: Raja Jaganathan

Boosting Node.js Throughput with setImmediate

Raja Jaganathan — Sat, 01 Mar 2025 18:45:49 +0000

Node.js is awesome for non-blocking tasks, but heavy work can still jam the event loop, slowing things down. That’s where setImmediate helps—it moves tasks to the next step, keeping your app running smoothly. In this short guide, we’ll tackle a real problem, check a use case, and see how setImmediate increases throughput. Let’s begin!

Problem: Analysing Large Text Data (Customer Reviews)
Imagine your app gets 1,00,000 customer reviews from an API and needs to analyse their sentiment (positive or negative). Doing it all at once blocks the event loop.

Problem Code:

const reviews = Array(100000).fill().map((_, i) => ({
  id: i,
  text: `This is review number ${i}. Product is good but service can improve.`
}));

function analyzeSentiment(reviews) {
  return reviews.map(review => {
    const words = review.text.split(' '); // Tokenize
    let score = 0;
    // Simple scoring: count positive/negative words
    words.forEach(word => {
      if (['good', 'awesome'].includes(word)) score += 1;
      if (['bad', 'improve'].includes(word)) score -= 1;
    });
    return { id: review.id, score };
  });
}

console.log('Starting...');
const results = analyzeSentiment(reviews); // Blocks for ~100-200ms
console.log('Analyzed reviews:', results.length);
console.log('Done!');

What’s the Issue? Analysing 1,00,000 reviews—splitting text and scoring—takes enough CPU time to freeze the event loop. On a server, this delays API responses or other tasks, hurting performance.

Solution: Use setImmediate to split it:


function analyzeSentimentChunk(reviews, callback) {
  const CHUNK_SIZE = 10_000;
  let index = 0;
  const result = [];

  function processChunk() {
    const end = Math.min(index + CHUNK_SIZE, reviews.length);
    for (; index < end; index++) {
      const words = reviews[index].text.split(' ');
      let score = 0;
      words.forEach(word => {
        if (['good', 'awesome'].includes(word)) score += 1;
        if (['bad', 'improve'].includes(word)) score -= 1;
      });
      result.push({ id: reviews[index].id, score });
    }

    if (index < reviews.length) {
      setImmediate(processChunk);
    } else {
      callback(result);
    }
  }

  setImmediate(processChunk);
}

const reviews = Array(100000).fill().map((_, i) => ({
  id: i,
  text: `This is review number ${i}. Product is good but service can improve.`
}));

console.log('Starting...');
analyzeSentimentChunk(reviews, (results) => {
  console.log('Analyzed reviews:', results.length);
});
console.log('This runs instantly!');

Why It Works: We process 10_000 reviews per cycle, and setImmediate schedules the next batch. The event loop stays free for other tasks.

Benefits:

Non-blocking: App keeps handling requests or I/O.
Better Throughput: More tasks run at once.

Drawbacks:

Takes a bit longer due to scheduling.
Chunk size needs tuning for best results.

Real Use: Ideal for text analysis (sentiment, keywords), log processing, or batch data transformations in APIs.

How Throughput Improves

Throughput is how many tasks your app can finish per second. Without setImmediate, a 100-200ms blocking task stops everything for that time. Split into 10-20ms chunks, Node.js can handle other work in between—like serving API requests. This keeps your app responsive and lets it process more tasks under load, though exact gains depend on your setup and traffic.

Conclusion

setImmediate boosts throughput in Node.js by splitting big tasks (like analysing customer reviews). It adds a small delay but keeps the app responsive.
Why not process.nextTick?
--It runs too fast, delaying I/O by overloading the current tick.
Why not setTimeout(() => {}, 0)?
It’s slower with extra timer delay.
For heavy CPU tasks (like image processing), worker threads are better—they run separately and other use case consider stream API. But for a simple fix, setImmediate rocks—try it to keep your app smooth and responsive!

An Alternative to filter().map(): Using flatMap()

Raja Jaganathan — Sat, 22 Feb 2025 04:36:39 +0000

When working with arrays, developers often use Array.filter().map() to transform data. But did you know Array.flatMap() can achieve the same result in a single step? Let’s explore when to use each.

Old Approach with 2 iteration: filter().map()

const persons = [
  { name: "Raja", age: 36 },
  { name: "Jaganathan", age: 65 },
  { name: "Kumar", age: 50 }
];

const result = persons.filter(p => p.age > 60).map(p => p.name);
console.log(result); // ["Jaganathan"]

Here, .filter() first selects the matching objects, and then .map() extracts names. Though it involves two loops, modern JavaScript engines optimize it well, making it faster in many cases.

New Approach with 1 iteration: flatMap()

const result = persons.flatMap(({ age, name }) => age > 60 ? name : []);
console.log(result); // ["Jaganathan"]

With flatMap(), we combine filtering and mapping in one pass, making the code more concise. However, performance may slightly drop because flatMap() internally flattens arrays, adding overhead.

Performance Comparison

Multiple approaches have been tested on jsperf.app. Thanks @miketalbot and @webjose.

🔹 flatMap() vs filter().map() – There’s no clear winner, but flatMap() shows a slight performance advantage in some cases.

When to Use Each

✅ Use filter().map() when clarity and semantic intent are important.
✅ Use flatMap() when reducing iterations and simplifying code structure matters more than micro-optimizations.

Happy Coding! 😊

Write Maintainable React Component

Raja Jaganathan — Wed, 26 Oct 2022 13:29:07 +0000

I recently came to the conclusion that React container components do not scale well. we usually put the business logic in Container components, which eventually grow into very large containers with too many responsibilities. so it's very difficult to maintain, refactor, and work with.

There are numerous approaches to writing maintainable React components. But for the purposes of this article, I'm leaning toward the useState -> useReducer -> Domain Hook path.

The useReducer hook is an alternative to the useState hook in appropriate situation. Let's see how to component transition from useState to useReducer and then useReducer in the next level with a custom hook also how React can help us solve this problem without the use of any additional libraries.

Domain Component Objectives:

Consider managing a simple inventory system. Here we are more interested in building product list components with minimal functionality such as marking as sold out, adding stock, sorting by title, and rating, and when one of the products is completely sold out, the render stock availability is red badge(Not Available) otherwise it will be green badge (Available).

With useState:

Here is ProductList component created using React.useState

import React from "react";
import { mockProducts, Product } from "../products-mock";

export function ProductList() {
  const [products, setProducts] = React.useState(mockProducts);

  const handleSortBy = (by: "rating" | "title") => {
    let sortedProducts;
    if (by === "rating") {
      sortedProducts = [...products].sort((a, b) => a[by] - b[by]);
    } else {
      sortedProducts = [...products].sort((a, b) => a[by].localeCompare(b[by]));
    }
    setProducts(sortedProducts);
  };

  const isProductsInStock = products.every((p) => p.stock > 0);

  const handleSoldOut = (id: number) => {
    const results = products.map((p) => {
      if (p.id === id) {
        return {
          ...p,
          stock: 0,
        };
      }
      return p;
    });
    setProducts(results);
  };

  const handleAddStock = (id: number, stock: number) => {
    const results = products.map((p) => {
      if (p.id === id) {
        return {
          ...p,
          stock: stock,
        };
      }
      return p;
    });
    setProducts(results);
  };

  return (
    <div className="container">

      <pre>
        Product Availability :
        {isProductsInStock ? " Available " : " Not Available "}
      </pre>


      <div className="action-buttons">
        <button onClick={() => handleSortBy("rating")}>sort by rating</button>
        <button onClick={() => handleSortBy("title")}>sort by name</button>
      </div>


      <ul className="product-list">
        {products.map((product) => {
          return (
            <li className="product-list__item">
              <span>
                {product.title} - ( {product.stock} )
              </span>
              <button onClick={() => handleSoldOut(product.id)}>Sold out</button>
              <button
                onClick={() =>
                  handleAddStock(product.id, Math.ceil(Math.random() * 50 + 50))
                }
              >
                Add Stock
              </button>
            </li>
          );
        })}
      </ul>
    </div>
  );
}

Keep in mind that all logic is close to the component and reveals how we manipulate the model within the component itself.

Pros:

Basic features are simple to implement.

Cons:

The component became fat as a result of state management within the component.
Separate of concerns
Difficult to manage dependent states such as isLoading, success, failure while fetching Product List, and so on.
With independent state variables, a complex operation like filtering based on other state variables is overly complex.
Undo/Redo functions

With useReducer:

The useReducer hook accepts a reducer type (state, action) => newState and returns a state object paired with a dispatch method much like Redux.

import React from "react";
import { mockProducts, Product } from "../products-mock";

type ProductListState = {
  products: Product[];
};

enum ProductListActionKind {
  SORT_BY_RATING = "SORT_BY_RATING",
  SORT_BY_NAME = "SORT_BY_NAME",
  PRODUCTS_IN_STOCKS = "PRODUCTS_IN_STOCKS",
  PRODUCT_SOLD_OUT = "PRODUCT_SOLD_OUT",
  ADD_STOCK_PRODUCT = "ADD_STOCK_PRODUCT",
}

type ProductListAction = {
  type: ProductListActionKind;
};

function productListReducer(
  state: ProductListState,
  action: ProductListAction
) {
  switch (action.type) {
    case ProductListActionKind.SORT_BY_RATING: {
      return {
        ...state,
        products: [...state.products].sort((a, b) => a.rating - b.rating),
      };
    }
    case ProductListActionKind.SORT_BY_NAME: {
      return {
        ...state,
        products: [...state.products].sort((a, b) =>
          a.title.localeCompare(b.title)
        ),
      };
    }
    case ProductListActionKind.PRODUCT_SOLD_OUT: {
      return {
        ...state,
        products: state.products.map((p) => {
          if (p.id === action.payload.id) {
            return {
              ...p,
              stock: 0,
            };
          }
          return p;
        }),
      };
    }
    case ProductListActionKind.ADD_STOCK_PRODUCT: {
      return {
        ...state,
        products: state.products.map((p) => {
          if (p.id === action.payload.id) {
            return {
              ...p,
              stock: action.payload.stock,
            };
          }
          return p;
        }),
      };
    }
    default:
      return state;
  }
}

export function ProductList() {
  const [state, dispatch] = React.useReducer(productListReducer, {
    products: mockProducts,
  });

  const isProductsInStock = state.products.every((p) => p.stock > 0);

  return (
    <div className="container">
      <pre>
        Product Availability :
        {isProductsInStock ? " Available " : " Not Available "}
      </pre>

      <div className="action-buttons">
        <button
          onClick={() =>
            dispatch({ type: ProductListActionKind.SORT_BY_RATING })
          }
        >
          sort by rating
        </button>
        <button
          onClick={() => dispatch({ type: ProductListActionKind.SORT_BY_NAME })}
        >
          sort by name
        </button>
      </div>
      <ul className="product-list">
        {state.products.map((product) => {
          return (
            <li className="product-list__item">
              <span>
                {product.title} - ( {product.stock} )
              </span>
              <button
                onClick={() =>
                  dispatch({
                    type: ProductListActionKind.PRODUCT_SOLD_OUT,
                    payload: {
                      id: product.id,
                    },
                  })
                }
              >
                Sold out
              </button>
              <button
                onClick={() =>
                  dispatch({
                    type: ProductListActionKind.ADD_STOCK_PRODUCT,
                    payload: {
                      id: product.id,
                      stock: Math.ceil(Math.random() * 50 + 50),
                    },
                  })
                }
              >
                Add Stock
              </button>
            </li>
          );
        })}
      </ul>
    </div>
  );
}

pros:

Isolate state management with reducer functions
The component appears to be cleaner than before.
Undo/Redo capability

cons:

There is no way to write a faster component because there is a lot of boilerplate code, such as constants/enum for action type.

Introduce Custom Hook with abstraction:

Abstraction: we extracted all business logic into custom hooks called useProducts along with useReducer hooks, and now the client doesn't have to worry about constant types, importing all the action types inside the component. Also it's not necessary to understand the underlying implementation. It reduces the possibility of errors during dispatch type construction and we can refactor reducer function without worrying too much about the consumer of useProducts hook.

We abstracted dispatch method with a nice set of API methods exposed, and we can test the custom hooks separately.

src/shared/hooks/useProducts.js


export function useProduct() {
  const [{ products }, dispatch] = React.useReducer(productListReducer, {
    products: mockProducts,
  });

  const sortBy = (by: "rating" | "title") => {
    const type =
      by === "rating"
        ? ProductListActionKind.SORT_BY_RATING
        : ProductListActionKind.SORT_BY_NAME;
    dispatch({ type });
  };

  const isProductsInStock = () => products.every((p) => p.stock > 0);
  const setSoldOut = (id: number) =>
    dispatch({ type: ProductListActionKind.PRODUCT_SOLD_OUT, payload: { id } });

  const addStock = (id: number, stock: number) =>
    dispatch({
      type: ProductListActionKind.ADD_STOCK_PRODUCT,
      payload: { id, stock },
    });

  return {
    products,
    sortBy,
    isProductsInStock,
    setSoldOut,
    addStock,
  };
}

Encapsulation: Much cleaner than before, capable of managing complex state through useReducer and abstracting useReducer's functions like dispatch. In addition, I encapsulated all of the behaviour under a custom hook with useful methods like sortBy, setSoldOut, and addStock.

import React from "react";
import { useProduct } from "../hooks/useProduct";

export function ProductList() {
  const { products, sortBy, isProductsInStock, setSoldOut, addStock } =
    useProduct();

  return (
    <div className="container">
      <pre>
        Product Availability :
        {isProductsInStock() ? " Available " : " Not Available "}
      </pre>
      <div className="action-buttons">
        <button onClick={() => sortBy("rating")}>sort by rating</button>
        <button onClick={() => sortBy("title")}>sort by name</button>
      </div>
      <ul className="product-list">
        {products.map((product) => {
          return (
            <li className="product-list__item">
              <span>
                {product.title} - ( {product.stock} )
              </span>
              <button onClick={() => setSoldOut(product.id)}>Sold out</button>
              <button
                onClick={() =>
                  addStock(product.id, Math.ceil(Math.random() * 50 + 50))
                }
              >
                Add Stock
              </button>
            </li>
          );
        })}
      </ul>
    </div>
  );
}

Dealing with the custom hook's performance issue:

So far, so good. But what if we need to pass these methods to the down in the component? Yes, it will re-render an unnecessary. How do you handle unnecessary re-rendering? Here React.useCallback and React.useMemo come in handy.

 // omitted unwanted snippet

export function useProduct() {
  const [{ products }, dispatch] = React.useReducer(productListReducer, {
    products: mockProducts,
  });

  const sortBy = useCallback((by: "rating" | "title") => {
    const type =
      by === "rating"
        ? ProductListActionKind.SORT_BY_RATING
        : ProductListActionKind.SORT_BY_NAME;
    dispatch({ type, payload:{}});
  }, []);

  const isProductsInStock = useMemo(() => {
    return products.every((p) => p.stock > 0);
  }, [products]);

  const setSoldOut = useCallback((id: number) => {
    dispatch({ type: ProductListActionKind.PRODUCT_SOLD_OUT, payload: { id } });
  }, []);

  const addStock = useCallback((id: number, stock: number) => {
    dispatch({
      type: ProductListActionKind.ADD_STOCK_PRODUCT,
      payload: { id, stock },
    });
  }, []);

 return [
  {
    products,
    isProductsInStock,
  },
  {
    sortBy,
    setSoldOut,
    addStock,
  },
  ];

}

Custom Hook Expert Level:

The pattern below is an excellent way to avoid using useCallback multiple times with useMemo; it is more concise and cleaner than using useCallback multiple times.

const handlers = React.useMemo(
  () => ({
    sortBy(by: "rating" | "title") {...},
    setSoldOut(id: number) {...},
    addStock(id: number, quantity: number) {...},
  }),
  []
);

 // omitted unwanted code

export function useProduct() {
  const [{ products }, dispatch] = React.useReducer(productListReducer, {
    products: mockProducts,
  });

  const isProductsInStock = useMemo(() => {
    return products.every((p) => p.stock > 0);
  }, [products]);

  const handlers = useMemo(
    () => ({
      sortBy(by: "rating" | "title") {
        const type =
          by === "rating"
            ? ProductListActionKind.SORT_BY_RATING
            : ProductListActionKind.SORT_BY_NAME;
        dispatch({ type, payload: {} });
      },
      setSoldOut(id: number) {
        dispatch({
          type: ProductListActionKind.PRODUCT_SOLD_OUT,
          payload: { id },
        });
      },
      addStock(id: number, quantity: number) {
        dispatch({
          type: ProductListActionKind.ADD_STOCK_PRODUCT,
          payload: { id, stock: quantity },
        });
      },
    }),
    []
  );

  return [
    { products, isProductsInStock },
    handlers
  ] as const;
}

Bonus:

How do we take our useProduct custom hook to the next level?

What if we need to create a CategoryList with useCategory custom hook with operations such as addCategory, removeCategory, and updateCategory, sortByCategory and so on ? that is very similar to a ProductList and useProducts

You may encounter code duplication because the same data structures are used in underlying. so it's time to create a new custom hook!, let's call it useArray with these features add, remove, update and sortBy and so on.

const [data, {add, update, remove, sortBy}] = useArray(initialState)

We can now remove the reducer function and its implementation from the useProduct and useCategory custom hook, making both domain custom hooks and components more concise.

Wrap up:

Custom hooks are a natural way to encapsulate and abstract business logic, and they allow us to create very powerful abstractions for React applications.

Use your imagination to take it to the next level.

Good luck!

Why "Skip Nav" Link is important!

Raja Jaganathan — Sun, 28 Jun 2020 04:05:09 +0000

A little while back, I'd the opportunity to work for accessibility implementation. one of my colleague Nawaz Khan a11y expert asked me to implement SkipNav for the page. The term "Skip Navigation" links are completely new for me and I've come up with a decent implementation that might be useful to everyone who cares about accessibility and usability. In this article, I'd like to discuss what is the problem without Skip Navigation links and how to build a Skip Navigation links for our website or application.

Problem:

Take a glance at https://www.nytimes.com/ for a minute, see how many links present in the header, sidebar, footer etc..,?

The main content is not usually the first thing on a web page. Keyboard and screen reader users generally must navigate a long list of navigation links, sub-lists of links, corporate icons, site searches, and other elements before ever arriving at the main content

How to navigate to the main content without visiting all header navigation links?
How to avoid visiting all the links for every page when navigating to another page.

The solution is "Skip Navigation" Links

Solution:

Generally, skip nav links are placed first or second links of the website so that users can easily navigate to the main content or desired section easily in a few tabbing. we might have seen many different approaches to solve the problem but I see below two-approach are used widely

Providing visible links at the top of the page
Making the link invisible until it receives keyboard focus

We'll talk about the 2nd approach and its implementation.

Let's see how https://www.nytimes.com/ solved this problem visually.

As you can see in the clip, showing the skip nav link when the link receives the keyboard focus.

That's cool. Now see how user navigates to the site to index section of the site which declared way down in the site.

Let's implement the same functionality with the help of CSS and React.

Here we're controlling link toggle logic only in CSS through :focus pseudo selector like below, we are hiding the element normally then we'll show link in the screen only when the link element receives the focus. Note that we are not using display: none or visibility: hidden these make it inaccessible, Technically those types of elements (applied display: none and visibility: hidden) are not part of Accessibility tree. Hence we are using clip CSS properties.

.skip-nav-link {
    border: 0;
    clip: rect(0 0 0 0);
    height: 1px;
    margin: -1px;
    overflow: hidden;
    padding: 0;
    position: absolute;
    width: 1px;
}

.skip-nav-link:focus {
    background: white;
    clip: auto;
    height: auto;
    left: 10px;
    padding: 1rem;
    position: fixed;
    top: 0px;
    width: auto;
    z-index: 1001;
}

React implementation is dead simple and self-explanatory too. These 2 component does the job better SkipNavLink and SkipNavContent


export const SkipNavLink = ({ id, children = 'Skip to content', ...props }) => {
  return (
    <a {...props} href={`#${id}`} className="skip-nav-link">
      {children}
    </a>
  );
};

export const SkipNavContent = ({ id, ...props }) => {
  return <div id={id} {...props} />;
};

Here is the usage of components

...
<SkipNavContent id="main-content">
    <main>
        Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
        tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
        quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
        consequat. Duis aute irure dolor in reprehenderit in voluptate velit ease
    </main>
</SkipNavContent>
...

ReactDOM.createPortal(
    <React.Fragment>
        <SkipNavLink id="main-content">
            Skip to main
        </SkipNavLink>
        <SkipNavLink id="sitemap-links">
            Skip to sitemap
        </SkipNavLink>
    </React.Fragment>,
    document.getElementById('skip-links-root')!
)

To get some rough ideas on how laid out our HTML page.

<body>
    <div class="wrapper">
        <div id="skip-links-root">
            <!-- Skip Nav links -->
        </div>
        <header>
            <nav>
                <!-- Nav links -->
            </nav>
        </header>
        <div id="root">
            <!-- React content goes here -->
        </div>
    </div>
</body>

Just take a moment to think about how tiny changes improved the user experience a lot better.

That's it.

Happy Tabbing ⌨️! ❤️

Reference:

https://webaim.org/techniques/skipnav/

Express: scalable way to handle errors

Raja Jaganathan — Sat, 06 Jun 2020 16:43:17 +0000

Error handling is an important concept to understand for writing any better application. However often, I have seen error handling is misused or handled wrong way in many different application, especially in express.js application.

In this article, we'll talk about a way to handle the error in a better and scalable way.

Let’s build an user registration API with all functionality like validating user input, handling business validation, save the user's registration etc..,


app.post(
  "api/user/registration",
  ...
);

We will demonstrate 3 approaches such as novice, advanced and pro version of user's registration api.

Novice approach

In general we will come up with below code snippet without any proper design in place or this may be our first rough cut to build a registration endpoint.


app.post("api/user/registration", postRegistration);

function postRegistration(req, res, next) {
  const {
    first_name,
    last_name,
    email,
    password,
    re_password,
    terms_condition,
  } = req.body;

  const errors = [];

  // Required field validation
  if (!first_name) {
    errors.push("first_name is required");
  }
  if (!last_name) {
    errors.push("last_name is required");
  }
  if (!email) {
    errors.push("email is required");
  }
  if (!email) {
    errors.push("email is required");
  }
  if (!password) {
    errors.push("password is required");
  }
  if (!re_password) {
    errors.push("re_password is required");
  }
  if (!terms_condition) {
    errors.push("terms_condition is required");
  }

  // Length validation
  if (password.length > 8) {
    errors.push("Password has to be longer than 8 characters");
  }

  // cross field match validation
  if (password === re_password) {
    errors.push("Password and re_password has to match");
  }

  // Bad request error
  if (errors.length > 0) {
    return res.status(400).json({ errors: errors });
  }

  // Server business logic validation
  const businessErrors = [];

  if (email.includes("dummy@gmail.com")) {
    businessErrors.push("EMAIL_ALREADY_TAKEN");
  }

  if (password.includes("qwerty")) {
    businessErrors.push("AUTH_WEAK_PASSWORD");
  }

  if (businessErrors.length > 0) {
    return res.status(400).json({ businessErrors: businessErrors });
  }

  // await UserRegistrationRepo.register(req.body)
  res.json({ error: false, msg: "Registration is successful" });
}

Let's try to find the problem with above approach. It's easy to find many problems at a very first glance like code duplication, too many responsibilities for postRegistration function because it does multiple thing such as validating the input, handle business validation and handling database operation etc..,

Moving on to advanced version.

Advanced

In this version we will try to rectify the problem that we encountered in novice approach by eliminating the code duplication, split into responsibility, clean logical separations etc..,

app.post(
  "api/user/registration",
  validateRegistrationInput,
  validateBusinessRegistration,
  postRegistration
);

To avoid code duplication we have created our own util function to validate the different rules such as required, minLength etc..,

lib/util/validation.js

export function validate(input, validationRule) {
  return Object.keys(validationRule).reduce((errors, key) => {
    const currentRule = validationRule[key];
    if (currentRule.required) {
      if (!input[key]) {
        errors.push(`${key} is required field`);
      }
    }

    if (currentRule.minLength) {
      console.log({ errors, key, currentRule, input });
      if (input[key] && input[key].length < currentRule.minLength) {
        errors.push(
          `${key} has to more than ${currentRule.minLength} characters`
        );
      }
    }
    //TODO:cross field match validation
    return errors;
  }, []);
}

controller/registration.js

Let's see how our registration's controller code looks like

import { validate } from './validation'

const validationRule = {
  first_name: {
    required: true,
  },
  last_name: {
    required: true,
  },
  email: {
    required: true,
  },
  password: {
    required: true,
    minLength: 8,
  },
  re_password: {
    required: true,
    ref: "password",
    exactMatch: true,
  },
  terms_condition: {
    required: true,
  },
};

export function validateRegistrationInput(req, res, next) {
  const {
    first_name,
    last_name,
    email,
    password,
    re_password,
    terms_condition,
  } = req.body;

  const errors = validate(req.body, validationRule);

  // Bad request error
  if (errors.length > 0) {
    return res.status(400).json({ errors: errors });
  }
  next();
}

export function validateBusinessRegistration(req, res, next) {
  // Server business logic validation
  const { email, password } = req.body;
  const businessErrors = [];

  if (email.includes("dummy@gmail.com")) {
    businessErrors.push("EMAIL_ALREADY_TAKEN");
  }

  if (password.includes("qwerty")) {
    businessErrors.push("AUTH_WEAK_PASSWORD");
  }

  if (businessErrors.length > 0) {
    return res.status(400).json({ errorMessages: businessErrors });
  }

  next();
}

export function postRegistration(req, res, next) {
  // await UserRegistrationRepo.register(req.body)
  res.json({ success: true, data: { message: "Registration is successful" }});
}

Let's discuss some of the pros & cons of this advanced version of api/user/registration

Pros:

Reduced code duplication
clean separation
adhered single responsibility

Cons:

Not leveraging centralize error handling
Own implementation of validation (time-consuming to implement and test case to cover the all use-case)
Inconsistent error schema structure (How to provide consistent error schema to client ?)
- res.status(400).json({ errorMessages: businessErrors });
- res.status(400).json({ errors: errors });
Inconsistent response schema structure (How to provide consistent response schema to client ?)
- res.json({ success: true, data: { message: "Registration is successful" } });
- res.json({ error: false, msg: "Registration is successful" });

Just want to more emphasis on consistency, because consistency will lead a better, clean and understandable code. Also, which help us to refactor the code though code is messy.

Pro: Scalable Error handling

In pro version, we will take care of following things

Validation with Schema library (Yup/Joi)
Unified AppilcationError interface
Custom Error creation
Central error handling

Validation with Schema library

I'd like to introduce the schema based validation library such as Yup/Joi . Let's define the validation schema for our registration endpoint like below.

Take a look at our userRegistrationSchema function. See how elegantly we are validating against our javascript object without much code to write and also think about readablity perceptive, it has improved a lot and schema based validation is help us to reduce bug too!.

validation.js

import * as Yup from "yup";

export function userRegistrationSchema() {
  return Yup.object().shape({
    first_name: Yup.string().required(),
    last_name: Yup.string().required(),
    email: Yup.string().email().required(),
    password: Yup.string()
      .min(8, "Password has to be longer than 8 characters!")
      .required(),
    re_password: Yup.string()
      .oneOf([Yup.ref("password"), null], "Passwords must match")
      .required("Re-enter password is a required field"),
    terms_condition: Yup.boolean().oneOf(
      [true],
      "Please accept terms and conditions"
    ),
  });
}

Unified Appilcation Error interface

Let's create our own error class for whole application so that it will provide consistent error interface, error schema structure to the application and clients.

In other way, we may prefer to create our own custom error class which extends javascript's Error class, like below

class ResourceNotFound extend Error { }

It's up to us to decide but I felt too many error classes which bring some maintenance, enforce the consistency for error classes seems like it is unnecessary for javascript application. For example, In node.js internally errors are categorized into few type of errors.

Let's define ApplicationError class

lib/api/applicationError.js

export class ApplicationError extends Error {
  static type = {
    APP_NAME: "APP_NAME",
    INTERNAL: "INTERNAL",
    NETWORK: "NETWORK",
    UNKNOWN: "UNKNOWN",
  };

  constructor(options, overrides) {
    super();
    Object.assign(options, overrides);

    if (!ApplicationError.type.hasOwnProperty(options.type)) {
      throw new Error(`ApplicationError: ${options.type} is not a valid type.`);
    }

    if (!options.message) {
      throw new Error("ApplicationError: error message required.");
    }

    if (!options.code) {
      throw new Error("ApplicationError: error code required.");
    }

    this.name = "ApplicationError";
    this.type = options.type;
    this.code = options.code;
    this.message = options.message;
    this.errors = options.errors;
    this.meta = options.meta;
    // {
    //   analytics:  {},
    //   context: {}
    // }
    this.statusCode = options.statusCode;
  }
}

Great, now we have defined ApplicationError but think a moment about Yup's ValidationError and ApplicationError both are completely different interface.

How do we provide a consistent error interface ?

Since we are handling the third party exceptions like Yup validation or MongoException, which has the different error schema that will create a problem. We could solve this problem elegantly with the help of factory function. so that we can swap the Yup with Joi or something in later point even without knowing or altering too much in existing code.

Our Factory function name called createError that will take care of converting third party exception or any error to ApplicationError exception. Here the errorFactory.js appears below

lib/api/errorFactory.js

import * as Yup from 'yup'
import { ApplicationError } from './applicationError'

export function createError(error, overrides) {
  const isYupError = error instanceof Yup.ValidationError
  if (isYupError) {
    const yupError = mapYupValidationError(error)
    return new ApplicationError(yupError, overrides)
  }
  return new ApplicationError(error, overrides)
}

function mapYupValidationError(error) {

  return {
    type: ApplicationError.type.APP_NAME,
    code: 'VALIDATION_ERROR',
    message: error.message,
    errors: error.inner,
    statusCode: 400,
    meta: {
      context: error.value
    }
  }
}

Custom Error Creation

Return to our registration API, We might encounter the few business exceptions while developing the registration endpoint. Those few exceptions are

if email is already taken (EMAIL_ALREADY_TAKEN)
if user enters a weak password (AUTH_WEAK_PASSWORD)
...

As said, we don't want to create new custom error class for each type of error. then how do we create a custom error with the help of ApplicationError ?

controller/registration/error.js


import { ApplicationError } from '../../lib/api'

export const Errors = {
  EMAIL_ALREADY_TAKEN: {
    type: ApplicationError.type.APP_NAME,
    code: 'EMAIL_ALREADY_TAKEN',
    message: 'The given email address is already taken :(',
    statusCode: 400
  },
  AUTH_WEAK_PASSWORD: {
    type: ApplicationError.type.APP_NAME,
    code: 'AUTH_WEAK_PASSWORD',
    message: 'The given password is easy to guess, provide strong password',
    statusCode: 400
  }
}

In later we could use like below

new ApplicationError(RegistrationError.EMAIL_ALREADY_TAKEN);

one important thing to note, these business validation error.js are co-locating with our registration's controller is good thing.

Bonus: Common Errors

I'd like to show some of common errors which helps for REST API development.

lib/api/commonError.js

import { ApplicationError } from "./applicationError";

const HTTPError = {
  // Predefined 4xx http errors
  BAD_REQUEST: {
    type: ApplicationError.type.NETWORK,
    code: "BAD_REQUEST",
    message: "Bad request",
    statusCode: 400,
  },
  UNAUTHORIZED: {
    type: ApplicationError.type.NETWORK,
    code: "UNAUTHORIZED",
    message: "Unauthorized",
    statusCode: 401,
  },
  FORBIDDEN: {
    type: ApplicationError.type.NETWORK,
    code: "FORBIDDEN",
    message: "Forbidden",
    statusCode: 403,
  },
  RESOURCE_NOT_FOUND: {
    type: ApplicationError.type.NETWORK,
    code: "RESOURCE_NOT_FOUND",
    message: "Resource not found",
    statusCode: 404,
    meta: {
      translationKey: "app.common.error.RESOURCE_NOT_FOUND",
    },
  },

  // Predefined 5xx http errors
  INTERNAL_SERVER_ERROR: {
    type: ApplicationError.type.NETWORK,
    code: "INTERNAL_SERVER_ERROR",
    message: "Something went wrong, Please try again later.",
    statusCode: 500,
    meta: {
      shouldRedirect: true,
    },
  },
  BAD_GATEWAY: {
    type: ApplicationError.type.NETWORK,
    code: "BAD_GATEWAY",
    message: "Bad gateway",
    statusCode: 502,
  },
  SERVICE_UNAVAILABLE: {
    type: ApplicationError.type.NETWORK,
    code: "SERVICE_UNAVAILABLE",
    message: "Service unavailable",
    statusCode: 503,
  },
  GATEWAY_TIMEOUT: {
    type: ApplicationError.type.NETWORK,
    code: "GATEWAY_TIMEOUT",
    message: "Gateway timeout",
    statusCode: 504,
  },
};

export { HTTPError };

Bonus: Response Schema

In order to send the consistent response schema to the client we may need to define a function, called sendResponse, so that we can enforce to use sendResponse instead of res.json()

import { ApplicationError, createError } from '../error'

export function formatError(error, overrides = {}) {
  // `Error.stack`'s `enumerable` property descriptor is `false`
  // Thus, `JSON.stringify(...)` doesn't enumerate over it.
  const stackTrace = JSON.stringify(error, ['stack'], 4) || {}
  const newError = JSON.parse(JSON.stringify(error))

  // No need to send to client
  newError.statusCode = undefined
  delete newError.meta

  return {
    error: {
      ...newError,
      stack: stackTrace.stack
    },
    success: false,
    ...overrides
  }
}

export function formatResponse(result, override = {}) {
  return {
    data: result,
    success: true,
    ...override
  }
}

export function sendResponse(res, payload, statusCode = 200, context = {}) {
  return res.status(statusCode).json(formatResponse(payload))
}

As you can see here, validation.js and error.js are co-locating to registration endpoint would be great again.

Central error handling

It's time to reveal the core technique of this article that is centralized error handling in express.js application.

Define error-handling middleware functions in the same way as other middleware functions, except error-handling functions have four arguments instead of three: (err, req, res, next)

we should define an error-handling middleware last, after other app.use() and routes calls.

app.use("/api", userRegistrationRouter);

app.use(errorHandler);

how it works

In general, if error occur in synchronous code inside route handlers and middleware require no extra work. If synchronous code throws an error, then Express will catch and process it.

The errors returned from asynchronous functions invoked by route handlers and middleware, you must pass them to the next(error) function, where Express will catch and process them.

Like below we need to throw the error or pass the error to express middleware

controller/registration.js

import { userRegistrationSchema } from "./validation";
import { createError, sendJson, ApplicationError } from "../../lib/api";
import { Errors } from "./error";

export async function validateUserRegistration(req, res, next) {
  try {
    await userRegistrationSchema().validate(req.body, { abortEarly: false });
  } catch (e) {
    return next(createError(e));
  }
  next();
}

export function validationBusinessRule(req, res, next) {
  const { email, password } = req.body;

  if (email.includes('dummy@gmail.com')) {
    throw new ApplicationError(Errors.EMAIL_ALREADY_TAKEN);
  }

  if (password.includes('qwerty')) {
    throw new ApplicationError(Errors.AUTH_WEAK_PASSWORD);
  }
  next()
}
export function postRegistration(req, res, next) {
  // await UserRegistrationRepo.register(req.body)
  sendJson(res, { message: "Registration is successful" });
}

validationBusinessRule we synchronously throwing error hence we need not to call like next(error) and validateUserRegistration asynchronously we are catching the error so pass like next(error) to catch the express middleware.

Here is our centralized error middleware looks like

lib/errorHandler.js

import { sendResponse, formatError, CommonError } from "../lib/api";

export function errorHandler(err, req, res, next) {
    const { analytics = {} } = err.meta || {};
  // logging for analytics
  console.log({ analytics });

  if (err instanceof ApplicationError) {
    const code = err.statusCode || 500
    return res.status(code).json(formatError(err))
  }

  if (err instanceof Error) {
    const newError = createError(err)
    const code = newError.statusCode || 500
    return res.status(code).json(formatError(newError))
  }

  const unknownError = new ApplicationError(CommonError.UNKNOWN_ERROR)

  return sendResponse(res, unknownError, statusCode);
}

Most importantly, we did not handling errors in every middleware, All error handling moves to centralized error middleware with aim that we have great opportunities to cover other scenario easily such as

Logging error details
Send details for analytics
Formatting the error for consist error schema

Finally to test our registration endpoint using cURL command like below

curl --location --request POST 'http://localhost:3000/api/user/registration' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'first_name=raja' \
--data-urlencode 'last_name=jaganathan' \
--data-urlencode 'password=qwerty1234' \
--data-urlencode 're_password=qwerty1234' \
--data-urlencode 'email=dummy@gmail.com' | python -mjson.tool

{
    "error": {
        "name": "ApplicationError",
        "type": "APP_NAME",
        "code": "AUTH_WEAK_PASSWORD",
        "message": "The given password is easy to guess, provide strong password"
    },
    "success": false
}

That's it. Pretty neat right!!!

You can find the repo here 💌https://github.com/RajaJaganathan/express-error-handling

Thanks for reading!

Why nested functions in JavaScript

Raja Jaganathan — Mon, 25 May 2020 15:48:26 +0000

Often many people confused about nested function in JavaScript, why the nested function is useful, and what is the use case for nested function? There are so many places we might encounter such nested functions in our code or blog post or articles.

In this article, I would like to talk about the basics of why nested functions are useful in many ways.

I'll start asking some basic questions along with some constraints that way it may help to think differently to solve the problem. Let's assume that we have employee array like below

const employee = [
  { id: 1, name: "Raja", age: 28, status: "active" },
  { id: 2, name: "Andy", age: 32, status: "Inactive" },
  { id: 3, name: "Kumar", age: 45, status: "active" },
  { id: 4, name: "Charles", age: 35, status: "Inactive" },
];

Use case 1 #

How do you sort an array by property (age, status, etc)?

Constraint: Do not pass an array as a reference to any other functions and function should be more reusable.

Typically, with our prior programming knowledge, we do sorting using Array.prototype.sort function like below

function sortBy(array, propName) {
  return array.sort(function (a, b) {
    if (a[propName] > b[propName]) {
      return 1;
    } else if (a[propName] < b[propName]) {
      return -1;
    } else {
      return 0;
    }
  });
}

console.log(sortBy(employee, "age"));
console.log(sortBy(employee, "status"));

The above code does the job nicely but just take a look into our first constraint for the first use case. so it's clearly saying that's not the way to do. Note that, we are passing array reference to the function :(

In general, this is how we do sorting in JavaScript

// How do to pass another arguments to tell sort based on which property of an object ???????
Array.prototype.sort(function(a, b))

As we know Array.prototype.sort interface only accepts 2 arguments, So How do we pass propName as another argument? now we clearly understand that sort method accepts only 2 parameters we can't pass any extra argument to the function anymore.

Okay then how do we solve the problem?.

Hint: can we make our custom argument accessible inside sort callback function without adding an extra argument to Array.prototype.sort?

This is a perfect moment to use function inside a function (often called higher-order function) so that we could return a function which accepts only 2 arguments as per sort interface requirement, the high order function can accept n of arguments so that we can access our custom argument inside the sort callback function.

Here is the second version of sortBy function


function sortBy(propName) {
  return function (a, b) { // obeying sort interface function signature
    if (a[propName] > b[propName]) { // propName accessible from highorder function
      return 1;
    } else if (a[propName] < b[propName]) {
      return -1;
    } else {
      return 0;
    }
  };
}

console.log(employee.sort(sortBy("age")));
console.log(employee.sort(sortBy("status")));

I would like to discuss one more use case for higher-order functions with filter use case

Use case 2:

How do you filter an array with an inactive employee?

Constraint: Assume that we have function for filter out all active employee(isActiveEmployee) function already exists, Use the same function to get all inactive employee without code duplication.

function isActiveEmployee(employee) {
    return employee.status == 'active';
}

const activeEmployee = employee.filter(isActiveEmployee);
// const inactiveEmployee = employee.filter(??????);

Okay we might come up with below solution easy without thinking anything

function isInactiveEmployee(employee) {
     return employee.status !== 'active';
}

//(or)

 function isInactiveEmployee(employee) {
     return employee.status === 'inactive';
}

Just gentle reminder of use case 2's constraint, it said, we should n't duplicate the code, so can we do it a better way?

So think of we have already code which finds all active employees, so it just to invert the result of isActiveEmployee function we will get the all inactive employee even without write a brand new function for filter out inactive employee.

Method looks like

 function negate(item){
     return !isActiveEmployee(item)
 }

But we can do even better, we may think of negate as utility functions so it can be written like below

  function negate(predicate){
      return function() {
          return !predicate.apply(this, arguments)
      }
  }

Here is solution for filtered out all inactive employee

const inactiveEmployee = employee.filter(negate(isActiveEmployee));

console.log({inactiveEmployee});

Hope now you can see some of the benefits of high order function such as

Cleaner abstraction
working with composable functions and many more

Prove that JavaScript runs in a single thread

Raja Jaganathan — Sun, 05 Apr 2020 07:15:27 +0000

JavaScript runs on a single thread

we might hear about this sentence many times in different situations like interviews, blogs, conferences, workplaces, etc.

Is that really true?

Official docs say:

despite the fact that JavaScript is single-threaded — by offloading operations to the system kernel whenever possible.

This is again completely true for web browser too,

Of course, Yes it is, But can you prove that by writing some code?

Take a moment to think about this and demonstrate it with a code example.

Solution

let loop = true;

setTimeout(() => { 
    loop = false;
});

while (loop) {
    console.log('loop', loop);    
}

console.log('after loop');

Explanation:

We have just declared a variable called loop and it's defaulted to true.

setTimeout(() => {
    loop = false;
});

Immediately after the above line encountered by JavaScript interpreter, JavaScript scheduled a task in the event loop since these are web API. JavaScript doesn't want to block the main thread so it moves the task to event loop. Here are some of the web APIs which browser provides like XMLHttpRequest, setInternal, setTimeout, etc.

while (loop) {
    console.log('loop', loop);    
}

Here is an interesting moment in our example. As many developers know the above code will execute infinitely. But take a moment to think about what was happened to our scheduled task in the event loop. Will modify loop value as false ?.

Actually, It will never be loop false, because until the main thread execution is completed fully, the main thread won’t get the opportunity to pick a task from task queue in the event loop even though it’s completed and added callback in the task queue.

In other words, In general, the main thread will able to pick a completed task from task queue in the event loop, only after the main thread cleared all of its calls in the stack. Even though it was scheduled before reaching infinity code, no other threads can modify the loop's variable value.

So note that

console.log('after loop');

line is unreachable code.

Solution with Comments

let loop = true;

setTimeout(() => {  // callback execution is unreachable because main thread is still busy with below infinite loop code. 
    console.log('It will never reach here :ohh');
    loop = false;
});

while (loop) {
    console.log('loop', loop);  // Infinite loop
}

console.log('after loop');  // Unreachable code

Hence it's running on a single thread.