How to paginate your NodeJS APIs

#api #node #mongoose #sequelize

Why paginate?

A lot of times when we make calls to a large REST API a lot of results is been returned, hence the need to paginate or break down the results into chunks to make the responses easier to handle.

What to expect

I'll be sharing code-snippets on how I handle pagination when building APIs with Node when querying a Javascript Array when making calls to a MongoDB database with mongoose or using sequelize for SQL databases (PostgreSQL/MySQL)

First you'll need the NPM package paginate-info you can install it by running:

$ npm install -S paginate-info

Javascript Array

You can copy this example JSON file and create your own data.json file.

Now let's see how we can paginate the Javascript Array

import { calculateLimitAndOffset, paginate } from 'paginate-info';
import data from './data.js'

const getAllData = (req, res) => {
  const { query: { currentPage, pageSize } } = req;
  const { limit, offset } = calculateLimitAndOffset(currentPage, pageSize);
  const count = data.length;
  const paginatedData = data.slice(offset, offset + limit);
  const paginationInfo = paginate(currentPage, count, paginatedData);

  return res.status(200).json({
    success: true,
    data: { result: paginatedData, meta: paginationInfo }
  });
}

From the above code we are getting our page values currentPage and pageSize from the request query and passing it to the paginate-info calculateLimitAndOffset function, which in turn returns to us our limit and offset. You can read more of how this works by visiting the docs for paginate-info

If for example we have 100 documents in our array and we decided to get the second page with a page size of 20 elements.
req.query.currentPage = 2, req.query.pageSize= 10.

Mongoose implementation(MongoDB)

/--- Start of MongoDB example --- /
import { calculateLimitAndOffset, paginate } from 'paginate-info';

/**
   * @function getAll
   * @param {Object} req request object
   * @param {Object} res response object
   * @returns {Object} response object
   * @description gets all available results
   */
  const getAll = async (req, res) => {
    const {
      query: { currentPage, pageSize }
    } = req;
    try {
      const count = await model.estimatedDocumentCount();
      const { limit, offset } = calculateLimitAndOffset(page, pageSize);
      const rows = await model.find({})
        .limit(limit)
        .skip(offset);
      const meta = paginate(currentPage, count, rows, pageSize);
      return handleServerResponse(res, 200, { rows, meta });
    } catch (error) {
      return handleServerError(res, error);
    }
  }
  /----End of MongoDB implementation---/

Like with our array example we are getting our currentPage and pageSize from our request query. We get our get the total count of our documents using the estimatedDocumentCount provided by Mongoose. We pass the limit and offset generated from our calculateLimitAndOffset function to the mongoose find() limit and skip function respectively. Our meta data is generated the same way as the array.

Sequelize (PostgreSQL/MYSQL) implementation


 * @function getAll
 * @param {Object} req - server request
 * @param {Object} res - server response
 * @returns {Object} - custom response
*/
const getAll = async (req, res) => {
  try {
    const {
      query: {
        currentPage, pageSize
      }
    } = req;
    const { limit, offset } = calculateLimitAndOffset(currentPage, pageSize);
    const { rows, count } = await model.findAndCountAll({ limit, offet});
    const meta = paginate(currentPage, count, rows, pageSize);
    return response(res, 200, 'success', { rows, meta });
  } catch (error) {
    return response(res, 500, 'error', serverError);
  }
};

/** End of PostgreSQL/SQL(sequelize) implementation */

So for the SQL implementation its the same routine, only that this time we use sequelize's findAndCountAll method to get both the documents and the count, destructured as rows and count respectively.

So that's all for pagination from me. For more info as to the paginate-info package, check it out here on NPM.

I'll appreciate questions and your feedback in the response section.