How To Implement Caching in Express Using Redis and Docker-Compose

Introduction

As the number of users of an application increases, the load on that application increases. This often leads to an increase in latency, which is the amount of time the application takes to return a response to a user. This is particularly prevalent in applications that persist data, as every request made by a client has to be processed by the servers, followed by appropriate queries to a database. The higher the number of requests, the more work the servers and databases have to do, leading to increased response times.

Caching helps reduce the latency of applications by storing frequently accessed data in memory. This ensures that not every request ends up as a network call to the database, as servers can simply access data from the cache if it was stored there prior to the request. In this article, we will explore how to implement caching in an Express application using Redis.

Redis

Redis stands for REmote Dictionary Server is an open-source, in-memory key-value data store which is known for its speed, versatility, and robust caching capabilities. It supports various data types like strings, hashes, lists, sets, and more.

Redis: Other Use Cases

Aside it's use as a database cache, redis also has other uses cases such as:

• Session Storage: Sessions can be used in web applications to maintain state and store user-specific data such as recent actions, personal information, and more across multiple requests. Redis's in-memory nature makes it an excellent choice for storing session data, and providing fast read and write operations.

• Pub/Sub Messaging System: A messaging system allows for real-time communication between different parts of an application or even between different applications. Redis can act as a message broker and allows for the creation of a channel where publishers can send messages which are later consumed by consumers for further processing.

• Rate Limiting: It is often important to control the rate of requests made by clients to an API service in order to protect that API from abuse and attacks. Redis can be used to log the number of times a user makes a request to an API and when the number of requests exceeds a certain threshold, an error can be sent to the user.

Setting up Redis as a Cache in Express

To follow along with this tutorial, you are going to have to node, docker and docker-compose installed on your computer as well as have a basic understanding of each of them. You'll also need to have a basic understanding of the express framework.

• Open your terminal and create a folder for the application using
  mkdir express-redis-cache or whatever you want to name your application. Navigate to the folder in the terminal by typing cd express-redis-cache. Initialize an npm project using npm init -y.

• We are going to be running redis in a docker container so we are going to create a docker-compose file to hold the configurations. Create a docker-compose file at the roof of the folder using the command touch compose.yml and then copy the code below and paste.
  

version: "3.8"

services:
  redis:
    image: redis
    ports:
      - 6379:6379

All we are simply doing here is specifying the docker-compose version in the first line and under the services section, we are listing the services (containers) that will be part of the application. Each service represents a container that will be managed by Docker Compose. In this file, we are listing redis as a service and specifying that the redis container should run on port 6379 on our machine as well as port 6379 on the redis container. To start the redis container, we simply run the command docker compose up -d. We use the -d flag to run the container in a detached mode i.e. the container does not stop after we close the terminal.

• In the next step, we are going to install our dependencies. We simply do that by running npm install axios express redis nodemon. Ideally, we would install nodemon as a dev dependency but we'll ignore it in this case.

• After installation, open the package.json file and add the scripts to run the application. Your script section should look like the image below.

We use nodemon so that whenever we make a change, the express server is restarted with our changes automatically and as such, we do not have to manually restart the server every time for our changes to take effect.

• Next, create the app.js file by running the command touch app.js in the terminal. Open the file and follow along.

• First, we import the necessary packages as below.
  

const express = require("express");
const { createClient } = require("redis");
const axios = require("axios");

• We'll be making use of axios to make a request to https://swapi.dev/api which is an API that has information on various elements in start wars. The goal of the tutorial is to build an application to make a request to the API above and cache the data. Subsequent requests for that particular data would then be made to the cache instead of the API.

• Copy and paste the code below after importing the appropriate packages.
  

const app = express();
const port = 3000;
const startWarsUrl = "https://swapi.dev/api";

• Here we are simply creating an instance of the express application, defining the port in which the express application would listen and saving the URL to the API in a constant.

• After doing the above, we are going to be instantiating the redis client and then as per best practices by the redis documentation we are going to be listening for the error and connection events.
  

// connect to redis db on localhost
const redisClient = createClient();

// listen for errors
redisClient.on("err", (err) => {
  console.log("redis client error", err);
});

// listen for successful connection
redisClient.connect().then(() => {
  console.log("redis connected successfully");
});

• The function below would be responsible for fetching data from the API. it takes a parameter titled dataToFetch and in the first line in the function, you can see the data available to you in the form of a comment. You can navigate to the URL for more information.

const fetchStarWarsData = async (dataToFetch) => {
  // data to fetch can be people, planets, films, species, vehicles and starships
  const response = await axios.get(`${startWarsUrl}/${dataToFetch}`);

  return response.data;
};

• We are then going to create an endpoint that allows us to pass a parameter and then use the fetchStarWarsData function to fetch the data from the API if it is not cached in redis.

app.get("/star-wars/:dataToFetch", async (req, res) => {
  try {
    const dataToFetch = req.params.dataToFetch;

    // check if data is cached and return cached data
    const cacheResult = await redisClient.get(dataToFetch);
    if (cacheResult) {
      const parsedResult = JSON.parse(cacheResult);

      return res.status(200).json({ isCached: true, data: parsedResult });
    }

    // fetch data from api and cache if data is not already cached
    result = await fetchStarWarsData(dataToFetch);
    await redisClient.set(dataToFetch, JSON.stringify(result), {
      EX: 300,
      NX: true,
    });

    return res.status(200).json({ isCached: false, data: result });
  } catch (error) {
    return res.status(500).json({ message: error.message });
  }
});

The EX parameter specified in the line to cache the result simply sets the time in seconds that data should be saved in the cache. While the NX parameter when set to true, implies that redis should only cache data that does not already exist.

• Finally all we have to do is listen for connections using the code below

app.listen(port, () => {
  console.log(`server is listening on port ${port}`);
});

• We then run the command npm run dev to run the app in using the nodemon package or run the command npm run start to run the app using node.

• To test the application, open Postman or any API testing software of your choice and make a GET request to the endpoint localhost:3000/star-wars/planets or localhost:3000/star-wars/films or whatever data you want to fetch. You can visit the API to be sure.

• Your first response should look like the one below.

• Make the same request again and you should get a response like the one below.

• Notice the isCached field in both of the returned responses. In the first response, it was false as we were making a request to the external API and in the second response, it was true as we were getting the result from the database cache.

Overall, your code should look like the one below:

const express = require("express");
const { createClient } = require("redis");
const axios = require("axios");

const app = express();
const port = 3000;
const startWarsUrl = "https://swapi.dev/api";

// connect to redis db on localhost
const redisClient = createClient();

// listen for errors
redisClient.on("err", (err) => {
  console.log("redis client error", err);
});

// listen for successful connection
redisClient.connect().then(() => {
  console.log("redis connected successfully");
});

const fetchStarWarsData = async (dataToFetch) => {
  // data to fetch can be people, planet, films, species, vehicles and startships
  const response = await axios.get(`${startWarsUrl}/${dataToFetch}`);

  return response.data;
};

app.get("/star-wars/:dataToFetch", async (req, res) => {
  try {
    const dataToFetch = req.params.dataToFetch;

    // check if data is cached and return cached data
    const cacheResult = await redisClient.get(dataToFetch);
    if (cacheResult) {
      const parsedResult = JSON.parse(cacheResult);

      return res.status(200).json({ isCached: true, data: parsedResult });
    }

    // fetch data from api and cache if data is not already cached
    result = await fetchStarWarsData(dataToFetch);
    await redisClient.set(dataToFetch, JSON.stringify(result), {
      EX: 300,
      NX: true,
    });

    return res.status(200).json({ isCached: false, data: result });
  } catch (error) {
    return res.status(500).json({ message: error.message });
  }
});

app.listen(port, () => {
  console.log(`server is listening on port ${port}`);
});

Conclusion

In this article, we built an application using Node.js, Express, Docker Compose, and Redis to demonstrate how to fetch data from an API, cache it, and return the cached data as a response to the client. By implementing caching, subsequent requests for the same data are served from the cache instead of making repeated API calls, significantly improving response times and reducing server load. While this tutorial covers the basics, the principles discussed here provide a foundation for more advanced caching strategies and optimizations in your applications.

If you enjoyed this article, kindly leave a like. It would be much appreciated. Thanks...