Solving cache invalidation in 15 lines of code

#javascript #performance #programming

Caching lets you manage the trade-off between freshness and speed when dealing with slow asynchronous calls.

If you only care about freshness, it's easy. Just set a really short cache time (or forego caching entirely). The data will always be up-to-date, but you'll constantly be waiting on the slow calls to finish.

If you only care about speed, it's also easy. Just set a super long cache time. You'll rarely have to wait for the slow calls, but the data may be really out-of-date.

The problem is, developers often want both - fresh data as fast as possible. Cache invalidation is one way to achieve this, but it's incredibly hard to do well. In fact, there's a common saying:

The two hardest problems in Computer Science are naming, cache invalidation, and off-by-one errors.

Stale-while-revalidate (or SWR) is another attempt to solve this problem that is much simpler.

When a stale item is requested from the cache, you immediately return it (maximum speed). You then kick off a background task to update the value (maximum freshness).

In fact, it's so simple, we can implement it in only 15 lines of javascript code:

const cache = new Map();
async function swr(key, refresh, staleAfter = 5000) {
  let data = cache.get(key) || {ts: 0, val: null, promise: null}
  cache.set(key, data);
  // Item is stale, start refreshing in the background
  if (!data.promise && Date.now() - data.ts > staleAfter) {
    data.promise = refresh()
      .then((val) => { data.ts = Date.now(); data.val = val; })
      .catch((e) => console.error(e))
      .finally(() => (data.promise = null));
  }
  // No data yet, wait for the refresh to finish
  if (data.promise && !data.ts) await data.promise;
  return data.val;
}

And this is how you would use it:

const data = await swr("cache-key", expensiveFuncion, 5000)

The first time you run that code, it will wait for the expensive function to finish. Subsequent calls will always return immediately. If the returned value is stale (more than 5 seconds old), it will refresh it in the background.

You can get a lot fancier with the implementation, but the basics of SWR really are that simple!

If you are building a React App and want to use this method to fetch data for your front-end, I highly recommend the useSWR React hook by Vercel.