How to make your app indefinitely lazy – Part 3: Vendors and Cache

Well, hello there! And welcome to part 3 of my ultimate guide! Brace yourself, because you are about to become a real pro.

⏱️ Reading time: ~18-20 minutes
🎓 Level: Intermediate+

Series Contents:

1. How to make your app indefinitely lazy – Part 1: Why lazy loading is important
2. How to make your app indefinitely lazy – Part 2: Dependency Graphs
3. How to make your app indefinitely lazy – Part 3: Vendors and Cache
4. How to make your app indefinitely lazy – Part 4: Preload in Advance

Earlier we covered how to make our project's dependency tree as clean as possible and why it is important for lazy loading. And in this article, we will cover the following:

• How we should download vendor files to ensure the best lazy loading.
• What do "Lazy Loading" and "Cache" optimization strategies have in common, and how does using one affect another.
• What is cacheability and how to make our application as cacheable as possible.
• As well as how to correctly set up Webpack's cache groups and not mess up performance.

---

How to split your vendors

Lazy loading isn't just about splitting our own source files. It also applies to everything our website delivers, including external NPM packages. And it also applies to using cache properly.

Terms "vendors" and "cache" will be related in this article. So, let's briefly talk about what caching is first. Cache is yet another strategy to boost the loading time. In simplified terms, it works this way:

• When a user opens our website for the very first time, no cache is applied; thus, all the files must be downloaded. At the same time, all the files the browser has downloaded are saved locally in cache.
• Later, when a user reloads the page or opens a website a second time and onward, the files are retrieved from the local cache, which significantly reduces the loading time.

Now, let's go back to vendors. Imagine we use a few NPM packages in our project and we import different members from them in different lazy chunks.

// Home
import { chunk, difference, intersection, sortedUniq, takeWhile } from 'lodash-es';
import { addDays, addHours, addYears, addMonths } from 'date-fns';

// Page 1
import { countBy, partition, sample, sampleSize, orderBy } from 'lodash-es';
import { differenceInBusinessDays, differenceInCalendarDays, differenceInCalendarQuarters, differenceInHours } from 'date-fns';

// Page 2
import { debounce, memoize, throttle, once, curry } from 'lodash-es';
import { isToday, isTomorrow, isAfter, isDate } from 'date-fns';

// Page 3
import { cloneDeep, isArrayLike, isNative, isObject, isEqual } from 'lodash-es';
import { format, formatDate, formatISO, formatDuration } from 'date-fns';

How should the browser download the members?

Common configuration mistakes

Some developers believe that creating a single vendor file for the assembly is a good approach for delivering vendors. And the reason why they do that is that:

It is easier to save and retrieve from cache a single file rather than several.

But that's actually a bad choice. Because:

1. Retrieving multiple files from cache is not actually slower than retrieving a single file.
2. If a user doesn't have cache, they must download all the vendors initially. If some of them could be downloaded lazily, unnecessary delays happen. And since the initial loading time is the most crucial in user's perspective, UX of the application decreases.
3. Plus, since all the vendors are included in a single file, any time we import a new member from any NPM package, the hash code of this file changes. Which causes cache to be lost. Which causes problem #2.

Let's discuss the 3rd point in more detail. Usually, in real production applications, we need to control the cache of our files. Real websites change over time when introducing new features and fixing bugs. And if our cache strategy is too aggressive, users may end up retrieving files only from cache, thus not being able to see any changes we deploy.

To avoid such problems, usually developers include some unique identifiers into generated filenames. So, once we deploy a new update:

1. filenames of our application change;
2. users lose their cache;
3. and the browser is able to download the required changes.

And there are various ways to achieve it, but the ideal way is to include file content hash into its name:

module.exports = {
  // ...
  output: {
    // ...
    filename: '[name].[contenthash].js',
  },
};

Besides [contenthash], you can also use Webpack's [hash], [chunkhash], or [fullhash]. You can also provide some unique strings yourself like a date timestamp of when the build was happening. Or you can also use hash configuration from the HTMLWebpackPlugin. And none of these options, besides using [contenthash], are ideal to maintain the best cache strategy. But we will cover it a little later.

But for now, you should understand that using [contenthash] is considered the best practice to build an application. Therefore, once the content of any generated file changes, its name changes too, which results in cache being lost. And if we store all vendors inside a single file, any additional import from any NPM library will lead this file to be changed.

So, remember: if you see something like this in your assembly, chunk groups are probably set up incorrectly.

Another similar mistake some developers make, is generating vendor chunks based on the name of the NPM package. And such an approach is actually much better in general but may have similar problems of violating the lazy loading principle, because chances are you will be downloading these files initially. Plus, again, this strategy is still not the best in terms of caching.

Correct configuration

So, how can we make our vendors load truly lazily? We need to split our vendors into lazy-loaded parts:

• If any exported entities are required to be downloaded initially, only they should be done so.
• But the rest should be downloaded only when they are actually used, e.g., methods countBy and differenceInBusinessDays should only be downloaded when Page 1 is opened.

For the example above, the ideal solution should look like this:

We have an initial chunk for vendors, plus we have asynchronous vendor chunks. Chapter1 would need to download only its own vendor chunk, while Chapter2 would need to download its own. As in lazy loading it should be.

In Webpack, such vendor splitting is really easy to achieve:

module.exports = {
  // ...
  optimization: {
    splitChunks: {
      chunks: 'all',
    },
  },
}

📌 Set up your Webpack to split vendor packages and load them lazily.

---

Make it cacheable

But what do I mean by saying "cacheability"? We do need to reset cache of our files every time we deploy an update. However, we don't need to lose cache for all the files. In fact, we can make only required files to lose their cache. And by "cacheability", I mean how many files don't lose their cache after we make a change.

Use Webpack correctly

As I said, most of the existing strategies other than using [contenthash] are ideal to maintain a decent cache strategy.

• Using [fullhash] or HTMLWebpackPlugin's hash leads to removing cache from all the files each time we make the slightest change.
• Using [hash] and [chunkhash] will result in slightly more stable files. Although, they still will not be as stable as when using [contenthash].
• As creating your own strategy, it's either very complicated or you will end up losing cache for all the files by the smallest change.
  • For example, in my previous company for some period of time, we used Date.now() instead of the built-in hash functions. Which made all files update their names in every single build. Even if we didn't have any changes.

module.exports = {
  // ...
  output: {
    // Bad examples
-    filename: `[name].${Date.now()}.js`,
-    filename: '[name].[fullhash].js',
-    filename: '[name].[hash].js',
-    filename: '[name].[chunkhash].js',
    // Good example
+    filename: '[name].[contenthash].js',
  },
  plugins: [
    new HtmlWebpackPlugin({
      // Bad example
-     hash: true,
    })
  ],
};

📌 Use only Webpack's [contenthash] to make your application as cacheable as possible.

Clean your dependency tree

From the first glance, "lazy loading" and "caching" seem to be unrelated optimization strategies. However, they do share some commonalities. For example, when we are trying to optimize the dependency tree, we are also optimizing the cacheability of our project.

Let's regard the following example. Due to a terrible source file dependency graph, Webpack created an entangled execution dependency graph. Like on the picture below.

Let's say we made a one-line change that would affect the content of [id].[hash].chunk6.js. We'd expect only this file to lose its cache. However, in reality, chunks 1-5, as well as pages 1-3, as well as main.js, - they all also update their content hash. And therefore, with the execution graph above, a single one-line change may lead most of the files to lose their cache.

To understand what files will be affected when we make a certain change, we must take a look at the execution dependency graph and analyze it "backwards". Chunks 1-5 are all dependent on chunk 6. Pages 1-3 are all dependent on chunks 1-5. And main is always dependent on pages 1-3. Thus, they all lose their cache.

Now, let's imagine we fixed the graph.

In reality, the number of generated files should've been changed in such a case. But to make it simpler let's imagine that the files are generated completely the same, while the number of connections became much lower.

What happens when we make exactly the same one-line change?

If we analyze the dependency graph backwards, we'll see that now only chunk1, chunk5, page1, and main, will have their hash updated. Therefore, only 5 files lose their cache.

Now, a small task for you: what files will be changed, if chunk8 is updated?

Answer

Files main, page 2, page 3, chunk 2, chunk 3, and chunks 8, will lose their cache.

📌 Keep your dependency graph clean to improve cacheability of your project. The cleaner the graph is, the fewer files lose their cache each time we make a change.

Plus, I wanted to highlight one more important thing regarding cache and cacheability. You might have noticed, that in all the cases I mentioned, main.js loses its cache. Even when we fixed our dependency tree. It happens because the main initial JavaScript will always be dependent on all the other generated files from the assembly. Therefore, this file always loses its cache any time we make any change.

If users open our website after we deploy an update, it is impossible for them to retrieve initial files from cache. Therefore, caching optimizations are not as efficient for them as for other files. This is yet another reason, why we should care more about the initial loading time.

ℹ️ Regardless of the size of our change, the main javascript file will always lose its cache. That's the second most important reason why we should make our initial javascript files as lean as possible.

---

Set up vendors cache groups

And now, since we already know what "cacheability" is and what defines how good it is, we are ready to continue with our vendors discussion.

You might have noticed that in our "correct configuration" example the size of the initial vendor file is quite big. And with the approach I described it may become even bigger, especially when we use multiple NPM libraries: react, react-dom, zustand, zod, axios, etc. Because even if some libraries can be downloaded lazily, quite many of them still must be downloaded initially. However, we can fix this issue by setting up cache groups.

With the cache groups configuration, we can tell Webpack how to generate JavaScript chunks, including what source code files and/or vendors should be included in certain chunks. If we set them up correctly, we will improve cacheability. Likewise, the opposite, so we should be careful.

We should use cache groups only when we are sure that the group we create will be stable over time.

ℹ️ In order to make cache groups stable, we should include only "stable" vendors in the groups we create.

For example, react, react-dom, axios and zod, all must be used entirely in our application, and therefore can be included in a cache group. But date-fns or lodash-es can vary in their content based on what exported entities are used in our project, therefore we should not create cache groups for them.

To keep things easy, I'd say that we should only care about creating groups for the initially loaded vendors. Creating other groups may affect cacheability negatively unless we have a complete grasp on what's going on with our assembly.

And if are working on a Micro-FE application, we can't really operate initial chunks, so we have to manually choose which packages are downloaded initially and which of them should be included in groups.

In order to create such groups, we should use splitChunks.cacheGroups:

  module.exports = {
    optimization: {
     splitChunks: {
       cacheGroups: {
        react: {
          filename: `react.[contenthash:8].js`,
          // it's better to create groups for initial vendors only,
          //  but don't use it in micro-fe apps
          chunks: 'initial',
          // Will include `react`, `react-dom`, and `react-router-dom`
          //  in a single chunk
          test: /react/,
        },
       },
     },
    },
  };

📌 Set up Webpack's cache groups for stable vendor packages to additionally split them when needed.

---

Conclusion

Alright, that was long, but that's it for today. Thank you for joining me once again on our journey to make our web applications indefinitely lazy. If you have any questions feel free to ask them in the comments. And you can also read the following articles from this series.

And to summarize this article, let's list the rules we learned today:

• 📌 Set up your Webpack to split vendor packages and load them lazily.
• 📌 Use only Webpack's [contenthash] to make your application as cacheable as possible.
• 📌 Keep your dependency graph clean to improve cacheability of your project. The cleaner the graph is, the fewer files lose their cache each time we make a change.
• 📌 Set up Webpack's cache groups for stable vendor packages to additionally split them when needed.

You are almost there, just one more push:
How to make your app indefinitely lazy – Part 4: Preload in Advance

Here are my social links: LinkedIn Telegram GitHub. See you ✌️