DEV Community: Vignesh M

A simple "Cache Invalidation" strategy, Part 2

Vignesh M — Sun, 17 Mar 2019 12:15:37 +0000

"There are only two hard problems in Computer Science: cache invalidation, and naming things."
— Phil Karlton

This is in continuation to my last post, A simple caching strategy for Node REST APIs, Part 1 in which we talked about implementing a simple cache middleware, it can be summarised in this flow chart above. If you have not read that post, please read it before continuing. That post ended with a question, "How can we do cache Invalidation?", we will explore this question now.

Alright, let's do this one more time.

Ques. Why did we need caching?
Ans. So that users could get data quicker.
Ques. Why would we need cache invalidation?
Ans. So that users get recent real-time data.

And what is Cache Invalidation?

Cache invalidation is a process in a computer system whereby entries in a cache are replaced or removed.

"Replaced": Cache is replaced by recently updated data
"Removed": Entire cache is removed.

Out these two ways, "Remove" is the easiest to implement, the cache is cleared and we let it be rebuilt with new data.

Cache Invalidation Strategy

This has only two steps, but the implementation can vary drastically from architecture to architecture.

Find all sources from where data change can be triggered: Data could be changed through an API endpoint, a periodic task or from a trigger somewhere hidden deep inside your codebase. Your job is to find them all and
Add a method to clear/update cache after the data is changed. Simple as that. 😅

Again, the exact implementation for this can vary, you may

Add a cache clearing method at a low level like, $afterUpdate of your database model.
Add it to every method that changes data. It all depends on the complexity of the application.

For this post, we will extend our cache middleware to handle invalidations too, let's see how that works. This example assumes that only way to change any data of the resource is by doing a POST request. We will add a new method called clear to our middleware.



// middlewares/cache.js

const NodeCache = require('node-cache')
const cache = new NodeCache({ stdTTL: 5 * 60 })

function getUrlFromRequest(req) {
    ...
}

function set(req, res, next) {
    ...  
}

function get(req, res, next) {
    ...
}

+ function clear(req, res, next) {
+   cache.keys(function(err, keys) {
+       if (!err) {
+           let resourceUrl = req.baseUrl;
+           const resourceKeys = keys.filter(k => k.includes(resourceUrl));
+           cache.del(resourceKeys);
+       }
+   });
+   return next();
+ }

module.exports = { get, set, clear }

And use it in our routes



// products/routes.js

router.get(
    ...
)

router.post(
    '/',
    productsController.create,
    cache.clear, // 👈
    responseHandler
)

And Done!

Whenever a POST request is made, the data would have been changed, thus we trigger a cache clear, so the cache can be rebuilt when the next GET request comes in.

What exactly is happening in the cache.clear?



// middlewares/cache.js

...
...

function clear(req, res, next) {
    cache.keys(function(err, keys) {
        if (!err) {
            // again, it depends on your application architecture,
            // how you would retrive and clear the cache that needs to be cleared.
            // You may use query path, query params or anything. 
            let resourceUrl = req.baseUrl;
            const resourceKeys = keys.filter(k => k.includes(resourceUrl));

            cache.del(resourceKeys);
        }
    });
    return next();
}

cache.keys returns all stored cache keys,
req.baseUrl has the basepath of the request, eg. 'products'
resourceKeys gets all the keys which have the base path as a substring. (Remember?, the keys were simply the GET URLs pointing to the resource)
cache.del clears the cache pointed by the keys.

For Example, if our cache had values with keys like

A POST request to /products/ will clear all these 👆 caches and they will be rebuilt when a new GET request comes in.

For my example, simply clearing all cache which had keys within the scope of the POST request's base path worked.

With this setup, theoretically, we could set our cache TTL to be infinite, because every change to the data will clear it and cache will always have the recent data. But for sake of sanity, we kept our TTL to 15 mins. Now our users always had the recent data, faster.

That's all for today. Happy Coding!

Follow me on Twitter | Github, I build and post cool stuff. 👨‍💻

A simple caching strategy for Node REST APIs, Part 1

Vignesh M — Sun, 03 Mar 2019 21:26:19 +0000

Hello World, this is the beginning of 2 Part Series on "How to make your REST APIs blazing fast 🚀". These are from my personal experiences and from projects that I built.

Some time ago, I was working on a marketplace platform, where the users can list their products to sell. On the Home Page, it would load a bunch of products, and with the products self-data, it will also load some stats, previous sale history, recent lists data etc. We also let the user sort, filter and do more actions right on the page without reloading or re-fetching for a quick experience. But this came at a cost. For the API to send all this data, it had to do a bunch of calculations which ended up taking a few hundred milliseconds, in range 200-400ms, and worse during high traffic. So we started looking into ways to improve this. This series talks about those methods.

Part 1: A simple caching strategy for Node REST APIs
Part 2: Cache invalidation 😭

So let's jump right into Part 1

Here is the endpoint that we will work on. It simply takes in some query, fetches data from the database, processes it and returns back a JSON response.

// products/routes.js

router.get(
  '/',
  processQuery,
  productsController.index,
  responseHandler
)

okay now, let's add some Cache 💸!

For this example, We will use node-cache, we will put it in a single file, then it can be easily replaced by any cache storage by changing just few lines.

First of all, install the node-cache package.

$ npm install node-cache --save

We will create a cache middleware, it can be used with any endpoint we want easily. This is how the middleware looks like.

// middlewares/cache.js

const NodeCache = require('node-cache')

// stdTTL: time to live in seconds for every generated cache element.
const cache = new NodeCache({ stdTTL: 5 * 60 })

function getUrlFromRequest(req) {
  const url = req.protocol + '://' + req.headers.host + req.originalUrl
  return url
}

function set(req, res, next) {
  const url = getUrlFromRequest(req)
  cache.set(url, res.locals.data)
  return next()
}

function get(req, res, next) {
  const url = getUrlFromRequest(req)
  const content = cache.get(url)
  if (content) {
    return res.status(200).send(content)
  }
  return next()
}

module.exports = { get, set }

Let's go over the functions one by one.

getUrlFromRequest takes the request and returns the complete request URL.
We use this URL as the unique KEY for our cache.
set saves our processed response (res.locals.data) to the cache with the complete URL as the KEY.
get uses the URL as the KEY to retrieve the previously stored cached response, if it finds the data, it sends it back as the response, else the request is forwarded to the next middleware.

Our cache middleware is ready! Let's plug it in with our product route.

// products/routes.js

const cache = require('./cache-middleware') // 👈 import our cache middleware

router.get( 
  '/',
  cache.get,  // 👈
  processQuery,
  productsController.index,
  cache.set, // 👈
  responseHandler
)

That's all, Our Endpoint is already faster! But how 😯??

We have added our two middlewares get and set to the route. When a new request comes in, it will first go through cache.get, since we don't have anything in the cache yet, the request passes down to the next middlewares and arrives at cache.set, which will save the response in the cache for next 5 minutes.

Any request that comes in the next 5 minutes, will be able to retrieve this cache form cache.get and will immediately return it to the users. No calculations are done. The database isn't touched.

By doing this we were able to bring down our response time to just a few milliseconds 🎉.

But, yes, this is not the final solution, there are minor issues with this approach. Users on the site won't get real-time data, the data shown can be max 5 mins old. While this approach may work for some use cases, this was not an acceptable solution for us, our users needed real-time data. So we had to look into this more. We had to look into Cache Invalidation 😈, which we will talk about in our next Part. 👋

Follow me on Twitter | Github, I build and post cool stuff. 👨‍💻

React.lazy, What and how to use in your app

Vignesh M — Wed, 24 Oct 2018 14:09:42 +0000

React 16.6 is out and with it comes the new Lazy API. The Rreact.lazy function lets you render a dynamic import as a regular component.

React.lazy takes a function that must call a dynamic import(). This must return a Promise which resolves to a module with a default export containing a React component.

Usage:

const LazyComponent = React.lazy(() => import('./Component'));

Show a fallback with Suspense

Wrap the component where it's used with Suspense and pass a fallback. The fallback prop accepts any React elements that you want to render while waiting for the component to load.

<Suspense fallback={<div>Loading Component...</div>}>
  <LazyComponent />
</Suspense>

I made a simple example to show

The app shows a Press Me!! button which when pressed fetches a random user data from randomuser then loads and renders the User component.

const User = React.lazy(() => import('./User'));

....

<React.Suspense fallback={<div>Loading Component...</div>}>
  {user && <User user={user}/>}
  {loading ? (
    <div>Loading User...</div>
  ) : (
    !user && <button onClick={this.loadUser}>Press Me!!</button>
  )}
</React.Suspense>

Live: react-lazy-example
Code: github

Run the app, open your Network tab in Dev console, Press the button and see the lazily loaded javascript chunk. 🎉🎉

Hi, I'm M Vignesh

Vignesh M — Wed, 22 Feb 2017 18:39:24 +0000

I have been coding for 4 years.

You can find me on GitHub as vigzmv

I live in New,Delhi.

I work as Intern/part time at small startups.

I mostly program in these languages: Python â¤ Javascipt

I am currently learning more about Machine Learning and Javascipt's cool parts.

Nice to meet you.