How to Implement pagination on API Endpoint Using Nodejs and TypeScript

Introduction

Pagination is a technique used in web development to split large sets of data or content right into smaller, more manageable chunks. Instead of displaying all the data at once, pagination organizes it into separate pages, typically with a limited number of items per page. Hence, users can then navigate through the pages to view different portions of the dataset.

Problem Statement

Imagine a scenario as a developer where you want to display let's say 100k records at once on a web page, you'll then notice that this would likely crash the browser or make it unresponsive. This is why implementing pagination can address this type of challenge in your application.

Pagination comes with good benefits, let's highlight some few

🕝 Benefits of pagination

✅ Improved Performance: By paginating data, only a portion of it is displayed at a time, and this leads to faster page load times especially for users on slow internet or slower devices.

✅ Enhanced User Experience: Makes it easier for users to navigate smaller, more focussed portion of the data by quickly navigating to the actual record they want.

✅ Scalability: When large datasets are splitted into chunks, this can help developers scale their applications effectively.

✅ Reduced Server Load: Pagination reduces strain on the server, prevents performance issues during high traffic period.

✅ SEO Benefits: Search Engine Optimization gets improved, by making it easier for SEO crawlers to index and understand the content of the website, since each paginated data can have its own unique URL.

Having x-rayed some of the benefits of pagination, let's now look into the implementation of pagination in our choice tools; Nodejs, TypeScript & MongoDB.

🛠️ API Use Case

For the purpose of this article, we'll consider an e-com application where we want to fetch products from the server, based on several criteria, and have the returned data displayed on the web page, most likely the landing page.

💡 2 Important Concepts (2IC)

In setting up the Endpoint we'll need the following:

⏩️ API Request method => GET
⏩️ URL Query parameters be defined based on:

🔄 Featured products
🔄 Name of products
🔄 Category of products
🔄 SortBy value
🔄 PriceRange value
🔄 Page value
🔄 Limit value

Please note that you can change these query parameters to meet the needs of your application. Howerver, for the purpose of this article, we'll go with the aforementioned parameters.

So, let's dive in.

🪜 API Setup & Implementation

To begin with, ensure you have the following import statements:

import express, { Request, Response } from 'express';
import { Document } from 'mongoose';
import Product from './models/productsModel';
import Category from './models/categoriesModel';

The first import statment is used to get specific types Request and Response from the 'express' module. These types are used to define the types of request and response objects in Express applications.

The second import statement of Document, allows you to work with MongoDB documents in a type-safe manner using TypeScript, while the third and fourth import statements, represent the Product and Category models in your application, which will be used for querying data.

Next, we'll setup a product middleware to handle incoming request in the application, and also create a new interface to define the shape of the paginated products that will be sent out as response data:

  const productRouter = express.Router();

  interface PaginatedProducts {
    products: Document[];
    total: number;
    limit: number;
    page: number;
    pages: number;
    totalPublished: number;
}

productRouter.get("/products", async (req:Request, res:Response) => {});

🏛️ The Query Building Phase

The productRouter is all set and inside the middleware, We want to first receive the query parameters, by destructuring the req.query object

 const { featured, name, category, sortBy, priceRange, page = 1, limit = 10 } = req.query;

We'll define a new query object and conditionally populate this object based on the received query parameter requested by the client.

 const query: Record<string, any> = {};
   query.quantity = { $gte: 1 }
    if (featured) { query.featured = true }

We'll also define the options object and set values for the following properties; skip, limit and sort:

 const options = { skip: (parseInt(page as string, 10) - 1) * 
       parseInt(limit as string, 10),
          limit: parseInt(limit as string, 10),
          sort: sortBy === "asc" ? { name: 1 } : { name: -1 },
        };

And finally, we want to use the query and options objects to query the products model to retrieve the specified data. Also, note that we want to fetch products owned by different vendors, and have used the populate method in mongoose to get data from the users model taking the userName field. Hence:

 const products = await Product.find(query, null, 
  options).populate("vendor", "userName");

Holistically, putting the query building all together will yield:

import express, { Request, Response } from 'express';
import { Document } from 'mongoose';
import Product from './models/productsModel';
import Category from './models/categoriesModel';

const productRouter = express.Router();

  interface PaginatedProducts {
    products: Document[];
    total: number;
    limit: number;
    page: number;
    pages: number;
    totalPublished: number;
}

productRouter.get("/products", async(req:Request, 
 res:Response) => {
  try {
      const { featured, name, category, sortBy, priceRange, page = 1, limit = 10 } = req.query;
        const query: Record<string, any> = {};
        query.quantity = { $gte: 1 }
        if (featured) {
            query.featured = true;
        }
        if (category) {
            const categoryData = await Category.findOne({ 
             name: category });
          if (categoryData) {
             query.category = categoryData._id;
           }
        }
        if (name) {
            query.name = name;
        }
        if (priceRange) {
            const [minPrice, maxPrice] = String(priceRange).split('-').map(Number);
            query.price = { $gte: minPrice, $lte: maxPrice };
        }
        const options = { 
            skip: (parseInt(page as string, 10) 
            - 1) * parseInt(limit as string, 10),
            limit: parseInt(limit as string, 10),
            sort: sortBy === "asc" ? { name: 1 } : { name: -1 
         },
        };
        const products = await Product.find(query, null, options).populate("vendor", "userName");
        const total = await Product.countDocuments(query);
        const totalPublished = await Product.countDocuments({ isDisplayed: true, ...query });
        const totalPages = Math.ceil(total / parseInt(limit as string, 10));
        const paginatedProducts: PaginatedProducts = {
            products,
            total,
            limit: parseInt(limit as string, 10),
            page: parseInt(page as string, 10),
            pages: totalPages,
            totalPublished
        };
        return res.status(200).send(paginatedProducts);
    } catch (error) {
        console.error('Error in getPaginatedProducts middleware:', error);
        return res.status(500).json({ error: 'Internal Server Error' });
    }
});

🔎 API Testing

To test the result of the code we have written in the query building phase, we'll head over to postman and make a GET request using the URL structure below;

http://localhost:8000/api/v1/users/products?page=1&limit=10&sortBy=name&category=65a52ecace1687debeaa7e7e

And if everything works out well, we should see a screen much similar to:

If you got to this point, congratulations 👏🏻 🎉 your pagination functionality is all set for use.

📨 Conclusion

In conclusion, pagination stands as a pivotal technique in modern web development, offering a solution to the challenge of efficiently managing and presenting large datasets. By breaking down extensive data collections into smaller chunks. Pagination enhances user experience & efficiently treats performance issues,

By implementing pagination, developers can ensure that users can seamlessly access and navigate through datasets without encountering browser crashes or unresponsiveness.

If you found this article helpful, kindly like, share and leave your comments or queries in the comment section, and i'll treat them asap. Until next time, see you soon. 🏄‍♀️