DEV Community: Anthony Hagidimitriou

How I made pagination easy

Anthony Hagidimitriou — Sat, 06 Jan 2024 08:45:03 +0000

We've all been there. We have some data to show in a list or a grid on a page, but it becomes too long or too much to show all at once. What can we do about it? Well, the simplest solution to this would be to split it up into separate pages. Let's break down exactly what we need to do.

Based on the image above, we have three distinct parts that we need to figure out. The "previous" page link, the "next" page link and the page numbers in the center that should be shown to the user.

The data structure

Let's first start by defining our data structure:

type Page = {
  type: "page";
  href: string;
  label: string;
  current?: boolean | undefined;
};

type Gap = { type: "gap" };

type PaginationLinks = {
  previous: string | undefined;
  list: (Page | Gap)[];
  next: string | undefined;
};

The PaginationLinks type will be our representation of the final object returned. The previous and next attributes will be relative URLs (e.g /store/popular?page=3). The list attribute will contain either pages that we can render or gaps that can be expressed as an ellipse (e.g. ...) to show a break between the adjacent pages and the start/ending pages (e.g. 1, 2, ..., 5, 6, 7).

Now that we have a clearly defined data structure, let's set up the skeleton of our function that will generate the final object.

The generating function

type PaginationLinksParams = { url: URL; totalPages: number };

export function generatePaginationLinks({
  url,
  totalPages,
}: PaginationLinksParams): PaginationLinks {
  // There's no reason to handle an invalid number of pages OR if the total
  // number of pages is 0.
  if (totalPages <= 0) {
    throw new Error("Total number of pages must be greater than 0");
  }

  // Our actual logic will go here soon

  return { previous: undefined, next: undefined, list: [] };
}

To give a quick summary of the function, we bring in the current URL and the total number of pages. The current URL gives us the base of the relative URLs we need to make, as well as getting the current page number (if it is present).

Now that we have a function to generate our final links, let's find out the current page number so that we can generate the previous and next links.

function getCurrentPage(url: URL, totalPages: number): number {
  const unvalidatedPage = url.searchParams.get("page");
  const page = parseInt(unvalidatedPage ?? "1");

  if (isNaN(page)) {
    throw new Error(
      `The 'page' passed in is not a valid number: ${unvalidatedPage}`
    );
  } else if (page > totalPages) {
    throw new Error(
      "The current page should not be larger than the total number of pages"
    );
  } else if (page <= 0) {
    throw new Error("The current page should not be smaller or equal to 0");
  }

  return page;
}

The getCurrentPage function does some nice things for us. It first gets the current page as an integer OR defaults to page number 1. It then makes sure that it is a valid number and lies within the bounds that we expect (e.g. page 1-10).

Using a default page is just a small helper for when we are on the first page (e.g. /store/popular) and want to show it implicitly.

Since this function is now defined, we can finally use it within our generatePaginationLinks function as shown below:

export function generatePaginationLinks({
  url,
  totalPages,
}: PaginationLinksParams): PaginationLinks {
  // ...

  const currentPage = getCurrentPage(url, totalPages);

  return { previous: undefined, next: undefined, list: [] };
}

So, what will we do with the current page? We'll create a function to generate our next and previous links.

type PreviousPageParams = { type: "previous"; url: URL; currentPage: number };

type NextPageParams = {
  type: "next";
  url: URL;
  currentPage: number;
  totalPages: number;
};

type AdjacentPageParams = PreviousPageParams | NextPageParams;

function getAdjacentPage(params: AdjacentPageParams): string | undefined {
  // We need to make a copy of the URL so that any changes made to the
  // query string, do not affect any other instances (due to shared
  // memory pointers).
  const url = new URL(params.url);
  const currentPage = params.currentPage;

  // Make sure that we do not generate any invalid URLs based on the current
  // page number (e.g. we lie within the bounds of 0 and the total number
  // of pages).
  if (params.type === "next" && currentPage >= params.totalPages) {
    return undefined;
  } else if (params.type === "previous" && currentPage <= 1) {
    return undefined;
  }

  if (params.type === "next") {
    url.searchParams.set("page", (currentPage + 1).toString());
  } else if (params.type === "previous") {
    url.searchParams.set("page", (currentPage - 1).toString());
  }

  return `${url.pathname}?${url.searchParams.toString()}`;
}

Since there's a little bit more involved here, let's break this down into individual parts.

The AdjacentPageParams type helps us determine which parameters are required to generate the previous page OR the next page. By using a common type key, it helps us narrow down which one we want to make as well as accessing the correct parameters (through the use of type hints).

Within the actual function, we use the type key to check which link we are generating so that we can do the appropriate checks. An example of this is the check for whether the current page is less than 1 for the previous link. Once we've finally verified that the page we want is going to be valid, we can return the relative URL with the new page number set.

Let's now add this to our generatePaginationLinks function:

export function generatePaginationLinks({
  url,
  totalPages,
}: PaginationLinksParams): PaginationLinks {
  // ...

  const currentPage = getCurrentPage(url, totalPages);

  const previous = getAdjacentPage({ type: "previous", url, currentPage });
  const next = getAdjacentPage({ type: "next", url, currentPage, totalPages });

  return { previous, next, list: [] };
}

We can now remove the explicit undefined from the returned object since we have the previous and next link generating!

Before we can start generating our list of links, we will be needing a helper function to minimise boilerplate code. This function will help with generating a page object to show within the list. This function is as follows:

type PageParams = { url: URL; page: number; current?: boolean | undefined };

function getPage(params: PageParams): Page {
  const url = new URL(params.url);
  url.searchParams.set("page", params.page.toString());

  return {
    type: "page",
    href: `${url.pathname}?${url.searchParams.toString()}`,
    label: `${params.page.toString()}`,
    current: params.current,
  };
}

Now that we have our helper function, we can now make the list of links. Let's break this down to keep it simple.

type PageListParams = { url: URL; currentPage: number; totalPages: number };

function getPageList(params: PageListParams): (Page | Gap)[] {
  return [];
}

Within this function, we want to take in the URL (so we know how to generate the relative pages), the current page and the total number of pages to stay within our bounds.

The first page we will add is the current page we are on. This page can be added simply as follows:

function getPageList(params: PageListParams): (Page | Gap)[] {
  const { url, currentPage: page } = params;

  const adjacentLinks: (Page | Gap)[] = [
    getPage({ url, page, current: true })
  ];

  return adjacentLinks;
}

Pretty simple, hey?

Let's add in the previous two and next two links (so currentPage - 1, currentPage - 2, currentPage + 1 and currentPage + 2).

function getPageList(params: PageListParams): (Page | Gap)[] {
  const { url, currentPage: page, totalPages: total } = params;

  const adjacentLinks: (Page | Gap)[] = [
    ...(page - 2 > 0 ? [getPage({ url, page: page - 2 })] : []),
    ...(page - 1 > 0 ? [getPage({ url, page: page - 1 })] : []),

    getPage({ url, page, current: true }),

    ...(page + 1 <= total ? [getPage({ url, page: page + 1 })] : []),
    ...(page + 2 <= total ? [getPage({ url, page: page + 2 })] : []),
  ];

  return adjacentLinks;
}

Now, this looks really complicated but it really isn't. What we're doing is first checking "should we add the page?" (yes if we are within the bounds of 1-total) and then, we just spread the sub array into the top level array. This saves us from running a filter over all the elements to filter out undefined or null.

I have some more good news for you. It doesn't get more complicated than what I've just shown you.

We can now add in the initial two pages and the final two pages, as well as the 'gap' or ellipsis between the initial/final pages and the relative center pages:

function getPageList(params: PageListParams): (Page | Gap)[] {
  const { url, currentPage: page, totalPages: total } = params;

  const adjacentLinks: (Page | Gap)[] = [
    // Show first 2 initial pages if we have room and a gap after these pages
    // to seperate against the middle or "adjacent" pages.
    ...(page > 3 ? [getPage({ url, page: 1 })] : []),
    ...(page > 4 ? [getPage({ url, page: 2 })] : []),
    ...(page > 5 ? [{ type: "gap" as const }] : []),

    ...(page - 2 > 0 ? [getPage({ url, page: page - 2 })] : []),
    ...(page - 1 > 0 ? [getPage({ url, page: page - 1 })] : []),

    getPage({ url, page, current: true }),

    ...(page + 1 <= total ? [getPage({ url, page: page + 1 })] : []),
    ...(page + 2 <= total ? [getPage({ url, page: page + 2 })] : []),

    // Show the gap between the adjacent pages and the final pages
    ...(page < total - 4 ? [{ type: "gap" as const }] : []),
    ...(page < total - 3 ? [getPage({ url, page: total - 1 })] : []),
    ...(page < total - 2 ? [getPage({ url, page: total })] : []),
  ];

  return adjacentLinks;
}

This is all that needs to be done to generate the adjacent links for the list, while also not getting any overlaps (e.g. duplicates) or potential boundary issues.

So to finally finish this pagination generation function, we can add a call in the exported function as follows:

export function generatePaginationLinks({
  url,
  totalPages,
}: PaginationLinksParams): PaginationLinks {
  // ...

  const currentPage = getCurrentPage(url, totalPages);

  const previous = getAdjacentPage({ type: "previous", url, currentPage });
  const next = getAdjacentPage({ type: "next", url, currentPage, totalPages });

  const list = getPageList({ url, currentPage, totalPages });

  return { previous, next, list };
}

That's it! You now have a pagination function that you can use within your templates or front-end frameworks.

The full code is available on GitHub below:

You Don’t Need Microservices: A Real-World Perspective

Anthony Hagidimitriou — Sat, 30 Dec 2023 07:36:05 +0000

In the world of software architecture, microservices are often heralded as the go-to solution for scalability and flexibility.

However, my experience with a project at my first job utilising microservices, has revealed a different story.

This particularly challenging project highlighted the pitfalls of this architectural pattern, especially when implemented without proper consideration.

It became clear that microservices, while powerful, can introduce significant complexity, overhead, and potential for disarray, particularly in handling inter-service communication and data consistency.

These real-world challenges have led me to question the widespread push towards microservices, especially for projects where simpler, more unified approaches could be more effective.

The project

I worked on a new mobile application that was built from scratch.

It started as a proof of concept which demonstrated the basic functionality of the project to the client, and was ultimately used as the base of the first deployed version.

Microservices were used within the backend of the application.

As new features were added or changed, new services were added.

There was no common base they worked from; it was a matter of what was thought to be added and that was it.

By the end of the project, there was over 25 individual services that could have been shrunk into 4 or 5 (such as ‘authentication’, ‘users’ and ‘discoverability’).

A summary of my experience

When I first joined the project, there was a lot to learn.

Since I was new to microservices, I had a very neutral stance towards it.

Yet, as I kept working on this project over the next 6 months up until deployment, many issues arose.

One of these issues was dealing with authentication.

Given that we were roughly 3 months away from our expected launch date, we did not have any kind of authentication within the app (to validate requests once logged in).

I had spoken to the tech lead and he believed having a users’ id (a generated UUID) was enough to authenticate a request (which by the way, was never validated on the server apart from being a valid UUID).

I argued against this given that security is a very important thing to get right, especially when user data can easily be obtained from requests.

I was then told that other work was more important and that relying purely on a UUID was “enough”.

Later down the track, a security report was made from a 3rd party which rated this decision at a severity level of around 9.9/10 (meaning, resolve this immediately).

The tech lead then had to sacrifice many nights of sleep to implement a JWT authentication package that we could add in to our existing services.

Now, given that there was no common base with all of these services, the authentication package needed to be installed and set up in all of the services.. one by one.

This package then made things more complex to develop with because we needed to pass the JWT to the service on each request.

Given that we had services that called other services, this token then needed to be passed along the line.

At this point it was also next to impossible to run services locally to develop with given that they relied on other services to run.

Another pain point of microservices is dealing with a common database. Where should the source of truth be stored?

Well, in this project, we just relied on having a database running based on manual deployments.

There were no database migrations, let alone any foreign keys used.

Any changes that needed to be made to the database were made by hand.

In my experience, microservices often lead to a fragmented codebase.

Each service, ideally isolated in functionality, ends up duplicating common code, making maintenance and updates a nightmare.

The overhead of managing these numerous, interdependent services isn't just technical; it requires a significant investment in both infrastructure and specialised skills.

Within the project I worked on, I highly emphasise this point given that we had 3 separate deployment environments for all the services (development, staging and production).

An alternative solution

Throughout development of the project I had questioned the tech lead as to why he chose to go with a microservices backend instead of a ‘monolithic’ approach.

I never got a proper answer and was told to “do my own research” into microservices (and hence why thi post has been written).

It's crucial to recognise that 'monolithic' isn't synonymous with 'outdated' or 'inefficient'.

Monolithic architectures, especially for smaller projects, offer a level of simplicity and coherence that microservices struggle to match.

Development and testing are streamlined, as everything resides within a single, unified codebase.

This ultimately leads to a simpler development environment and deployment infrastructure.

Unfortunately, the tech lead did not agree with me on my reasoning.

microservices are often touted for their scalability.

However, this benefit is irrelevant for many applications, particularly those in their infancy.

Small teams can find the distributed nature of microservices overwhelming, detracting from the core functionality of the product.

When Monoliths Triumph

There are numerous examples of successful applications that have thrived with a monolithic architecture.

These examples showcase that starting simple, with a focus on core functionality, can often be the key to a product's success.

Moreover, transitioning to microservices is always an option when the time is right.

I believe that a monolithic architecture would have alleviated a lot of issues within development of this application, as well as speed up the rate of development.

Things such as authentication would be easily managed; database migrations would have been a trivial integration and adding new “services” or “modules” would have been as simple as adding a new route with some functions related to what needed to be done.

Ultimately, implementing microservices from the get-go can be akin to premature optimisation.

It's like building a highway when a simple road would suffice.

The key is to assess your project's current needs and scale your architecture accordingly.

This project was not available to the public, so why did it need to have individual services?

Concluding Thoughts

Microservices, while beneficial in certain contexts, are not a one-size-fits-all solution.

They come with their own set of complexities and challenges that can hinder rather than help, especially in the early stages of a project.

A monolithic approach, on the other hand, offers simplicity, ease of development, and a unified codebase, making it an ideal choice for many projects.

As my experience suggests, sometimes, the best approach is to keep it simple.

To summarise, I learnt a lot from this project.

What to do, and definitely what NOT to do regarding building a backend.

I highly recommend building a backend as a monolith to begin with given that it is the easiest to get going and the quickest to bring to market.

Once you reach scale and run into scaling issues, only then should you consider moving to microservices.