This time we are going to implement caching in an existing node.js application using Redis, but first let's understand what it is and how it can help you.
What is caching?
Cache is a high-speed data storage layer, so that future requests for that data are served up faster than is possible by accessing the primary data storage location, such as a database.
How does Caching work?
The data in a cache is usually stored on fast-access hardware, such as RAM and its primary function is to increase data recovery performance.
Unlike databases, in which data is generally more durable, a cache system prefers to invest in the speed at which the data is returned and the persistence of the data is temporary.
Basically, all caching data resides in-memory (RAM), unlike databases that store data on hard drive or on SSDs.
Why do we cache?
Caching is important because you can get performance improvements, solving many problems without much effort.
Its use can be applied in different contexts. If you are consuming a third party api that has a limited number of requests per day, by using cache this is no longer a problem. Or if you make a request to the database that takes a very long time to complete, you can solve this quickly by caching it.
But perhaps the most common problem is if you have a certain resource in your api that is constantly being consumed but its data rarely changes, in this case, its wise to cache it to relieve the database. And basically many problems that come with the scalability of the application can be solved with caching.
Why Redis?
Redis is a fast, open-source, in-memory key-value data structure store.
Long story short, Redis allows you to store key-value pairs on your RAM. Since accessing RAM is faster than accessing a hard drive or an SSD. We are talking about speed.
Let's code
My approach in this example is very simple. Usually when we are going to implement caching in our application it is because we supposedly already have a functional api and we already have an idea of its problems/limitations.
Let's pretend that this is our api:
const express = require("express");
const Posts = require("./models/Posts");
const app = express();
app.get("/post/:id", async (req, res) => {
const { id } = req.params;
const data = await Posts.findById(id);
return res.json(data);
});
app.listen(3000);
Basically we are making a simple request to the database to have the data related to a single post. However let's assume that this post is quite popular and we decided to cache it.
First, we will install and import ioredis, so that we can communicate with our redis instance. And then we will create our client using the default host and port.
const express = require("express");
const Redis = require("ioredis");
const Posts = require("./models/Posts");
const app = express();
const redis = new Redis();
First, let's go to our route and add a middleware called cache (which we have yet to create):
app.get("/post/:id", cache, async (req, res) => {
// Hidden for simplicity.
});
Then we have to assign the key and the value so that we can save the post in the cache. The key will be the post id, but first I want to point out that our data variable is an object, so in order for us to save it as the value of our key we will have to convert it to a string.
And we will cache the post before we return it, like this:
app.get("/post/:id", cache, async (req, res) => {
// Hidden for simplicity.
redis.set(id, JSON.stringify(data));
return res.json(data);
});
Another point I want to address is the durability of our key in the cache. As we know, the idea is to persist the data only for a certain amount of time. In this example I decided to persist the data for only 15 seconds. It is done as follows:
app.get("/post/:id", cache, async (req, res) => {
// Hidden for simplicity.
redis.set(id, JSON.stringify(data), "ex", 15); // expires in 15s
return res.json(data);
});
Now we will create the cache middleware:
const cache = (req, res, next) => {
// Logic goes here
};
The first step is to acquire the post id through the parameters. Then we will try to access the appropriate data from the post by checking the key (id) in the Redis store. If an error occurs, we will return the error.
If the value (result) is found (if it is not null), then we will return the data from the Redis store without having to make a request to the database again. But remember that the value is a string so we will have to convert it back to an object.
const cache = (req, res, next) => {
const { id } = req.params;
redis.get(id, (error, result) => {
if (error) throw error;
if (result !== null) {
return res.json(JSON.parse(result));
} else {
return next();
}
});
};
But if the key is not found in our Redis store, we will make a request to the database and then cache the data.
The final code should be as follows:
const express = require("express");
const Redis = require("ioredis");
const Posts = require("./models/Posts");
const app = express();
const redis = new Redis();
const cache = (req, res, next) => {
const { id } = req.params;
redis.get(id, (error, result) => {
if (error) throw error;
if (result !== null) {
return res.json(JSON.parse(result));
} else {
return next();
}
});
};
app.get("/post/:id", cache, async (req, res) => {
const { id } = req.params;
const data = await Posts.findById(id);
redis.set(id, JSON.stringify(data), "ex", 15);
return res.json(data);
});
app.listen(3000);
What about you?
Have you used Redis yet?
Top comments (9)
How do you make the id unique for each key in the redis key pair structure, i.e lets say you have a collection called "Post", and another "Comment" and they both have a document with same id, is there a way to map this id to their Model like so:
post-5
comment-5
...to avoid conflict with the ids within the redis in-memory database
There are several ways around this. But I think the simplest would be to write the key like this:
post-${id}and then there would be a middleware to fetch the cached post or the comment, this according to the controller.I hope I understood your question 🙏
Okay thanks, you did get my question right.
This actually means that there will be a different middleware to handle this, right?
Thank god 🙏 Yes, that would imply having a middleware for the posts and another one for the comments. I think it would be a quick and simple solution. 😊
Great! Thank Francisco, i really appreciate.
Once again, awesome piece🤝🏼🙂
Thank you very much 🙏
Thanks for sharing. You made it pretty clear.
Thanks for the feedback! My intention is to make things as easy as possible to understand! 😊
redis is not the great solution if your system is in realtime that have 10000 req per second . they are missing a local cache per pod .