Persisting DataLoader between requests and using it as a cache

Mohammed Rabeeh — Sat, 14 Nov 2020 13:46:27 +0000

I'm working on a project where I use the DataLoader to batch requests (N + 1 Problem). I accidentally created DataLoader beforehand and started to pass it in the context, instead of initializing the DataLoader in the context. I'll explain.

DataLoader is meant to be used for "per request batching and caching". Therefore, you would do something like this to implement DataLoader normally.

// app.js

app.register(..., {
  schema: ...
  context: () => {
    return {
      userLoader: new DataLoader(batchFunction);
    };
  },
});

This would make it so that the DataLoader exist only in the context of a particular request. However, what I accidentally did was this:

// app.js

userLoader = new DataLoader(batchFunction);

app.register(..., {
  schema: ...
  context: () => {
    return {
      userLoader
    };
  },
});

This started to cache the data between requests. It was not hitting my database for the data it already had. So I started wondering if there was anything wrong in caching the data in this way. I get batching. I get caching without having to manually implement it. I can clear the data->id from the cache when the database is updated. So far so good.

But there has to be issues with this implementation. One limitation I found is that as the cache grows, eventually the entire database will be cached. I don't know if that's an issue or not. This can be avoided by using { cache:false } option in the loader to cache only the required data.

It would be great if someone more experienced could give me some feedback and shed some more light on this. Thank you!

DEV Community: Mohammed Rabeeh

Persisting DataLoader between requests and using it as a cache