Preface
I use Gatsby at work and in my personal projects because I believe it is the best tool out there right now in terms of efficiency as a developer and value added to my clients. The thing that keeps me using Gatsby is they really focus on performance and we all know performance matters when it comes to retaining users. As amazing as Gatsby is, it does not fully take performance off of our plate so we never have to worry about it again. As developers we should be testing the speed of our websites after every code and content change because no tool is going to handle every edge case in the world for us. Gatsby and websites in general are fast out of the box, but it is our job not to mess it up. In this post I want to share with you a case where Gatsby itself was not enough to handle our performance requirements and how we tackled the issue by constantly testing and making incremental changes.
The Performance Issue We Were Facing
At my work we primarily use 2 testing tools to measure our website performance.
In Lighthouse our website was scoring in the mid 70s (out of 100) and two of the things that were pointed out to improve were
- Reduce JavaScript execution time
- Minimize main-thread work
In Web Page Test our website had a very high time until the page was considered fully loaded and high load times are bad. I say "high" subjectively compared to the performance we were accustomed to seeing for the same exact website. An interesting thing about this Web Page Test tool is that you can block certain HTTP requests from happening which is a really handy way to test whether or not the presence of a certain request is the cause of performance issues. It turns out after blocking the gatsby generated javascript files on the page our website load time was cut in half!
The conclusion we drew from both of these testing tools was that the downloading, parsing, and execution time for our javascript scripts was too high.
Understanding Why Gatsby was Failing Us
In truth Gatsby did not fail us, but the out of the box solution that Gatsby provides for code splitting did. Gatsby provides a very in depth article to how they handle code splitting here so I'm not going to spend a lot of time going over it.
Dynamic Pages are the Real Issue
We are using Gatsby I believe in a very unique way where we have a custom CMS / design system feeding Gatsby data to create static pages with. Our CMS breaks up pages into different sections that we call modules.
The red lines separate what we call a module on our website and content writers in our CMS can compose a page of any of these modules which means on the Gatsby side we have to have code like this:
export default function Page ({pageFromCMS}) {
return pageFromCMS.modules.map((module) => {
const Module = findModuleComponent(module.id)
return <Module module={module}/>
})
}
This is not the real code but it very much illustrates what we are trying to accomplish. The idea is that we just want to take the modules that the CMS has for any given page and loop over them to dynamically put them on the page.
The issue with this code is that inside the function above called findModuleComponent we have to do something like:
import ModuleOne from './module-one'
import ModuleTwo from './module-two'
const modules = {
'moduleOne': ModuleOne,
'moduleTwo': ModuleTwo
}
export function findModuleComponent (moduleId) {
if (!modules.hasOwnProperty(moduleId)) {
throw new Error(`Module ${moduleId} does not exist`)
}
return modules[moduleId]
}
Do you spot the issue here and how it relates to code splitting from the title of this article?
Basic Understanding Code Splitting
If you have two import statements at the top of a file Gatsby / Webpack is going to bundle those imports into one javascript file during the build, and make something like https://www.dumpsters.com/component---src-templates-page-js-123eb4b151ebecfc1fda.js.
Bringing It All Together
Our requirements for our CMS to have any module on any page forces us to dynamically render the modules on the Gatsby side. In order to dynamically render any module we have to have a map of module names to react components which forces us to import all of our react components in the same file. The act of having all of these imports in the same file makes Gatsby/Webpack think that every module/import is needed on every single page so there is essentially no code splitting at all for our page specific code. This is a real problem because we could easily have 100 total modules and any given page probably only uses 10 of them so we have a lot of unneeded javascript on our pages.
Solving the Issue
We need a way to only import the modules that we need for any given page without sacrificing the dynamic nature of our CMS. Introducing dynamic imports mentioned by react and also Webpack. The issue with the dynamic imports right now is that it relies on React.lazy which does not support server side rendering. We absolutely need server side rendering, it is another big reason we chose to use Gatsby to statically render our HTML pages. React themselves acknowledges this limitation of React.lazy and they recommend using loadable components to address the issue for now.
Implementing Loadable Components in Gatsby
If you follow the documentation for loadable components you will probably quickly get confused when you get to the third step which is about how to set up the server side of your application. This step is confusing because Gatsby already takes care of these things for you! Gatsby itself is in charge of doing the server rendering and you will not need to override it to make loadable components work. Instead if you just follow the first 2 steps in the documentation then it will be enough to get started.
Step 1
You will need to use a custom babel plugin so you need to overwrite the Gatsby default one as described here.
.babelrc
{
"plugins": [
"@loadable/babel-plugin"
],
"presets": [
[
"babel-preset-gatsby",
{
"targets": {
"browsers": [">0.25%", "not dead"]
}
}
]
]
}
make sure to install @loadable/babel-plugin and babel-preset-gatsby
Step 2
You will need to add a custom webpack plugin.
gatsby-node.js
const LoadablePlugin = require('@loadable/webpack-plugin')
exports.onCreateWebpackConfig = ({ stage, getConfig, rules, loaders, plugins, actions }) => {
actions.setWebpackConfig({
plugins: [new LoadablePlugin()]
})
}
again make sure to install @loadable/webpack-plugin and @loadable/component
Changing Our Code
Now that we have loadable components lets use its dynamic import abilities.
import loadable from '@loadable/component'
export default function Page ({pageFromCMS}) {
return pageFromCMS.modules.map((module) => {
const moduleFileName = findModuleFileName(module.id)
const ModuleComponent = loadable(() => import(`../modules/${moduleFileName}`))
return <ModuleComponent module={module}/>
})
}
If we stopped now we would be most of the way there with code splitting happening at the module level and therefore we are not including a bunch of unneeded javascript on our pages. There is an issue with code like this though.
What will happen is:
- The static HTML will render to the user.
- React will hydrate itself onto the static HTML
- Your current DOM will be destroyed by React because it takes time for the dynamic import to resolve
- The modules will be added back to the page once the dynamic import actually loads the javascript file it needs.
This has a nasty effect of having content on the screen, it disappearing, and then reappearing which is a terrible UX. In order to solve this issue we did something clever/hackish (I'll let you decide). Essentially the loadable components library allows you to specify fallback content as a prop until it is able to load the javascript file. We don't want to use a loading spinner because that is still going to flash content, instead we know the HTML is already statically rendered on the page so we grab the HTML for that module with a document.querySelector and then specify it as the fallback content until the module's javascript has loaded.
This post is getting sort of long so I'm going to share will you some psuedo code / real code of the final solution.
import loadable from '@loadable/component'
return page.modules.map((module, index) => {
const { moduleFileName, shouldLoadJavascript } = retrieveModulePath(module.id)
if (isServer()) {
// The server should always render the module so we get the static HTML.
// RENDER YOUR MODULE
}
const wasUserPreviouslyOnSite = window.history.state
const htmlEl = document.querySelector(`[data-module-index="${index.toString()}"]`)
if (htmlEl && !shouldLoadJavascript && !wasUserPreviouslyOnSite) {
// These modules do not require javascript to work, don't even load them
// RENDER THE STATIC HTML ONLY HERE - something like <div dangerouslySetInnerHTML={{ __html: htmlEl.outerHTML }}></div>
}
const fallback = htmlEl && htmlEl.outerHTML ? <div dangerouslySetInnerHTML={{ __html: htmlEl.outerHTML }}></div> : null
// RENDER THE MODULE NORMALLY HERE WITH THE FALLBACK HTML SPECIFIED
})
The above code accomplishes a lot of different things for us:
- Dynamic importing code for better code splitting
- Allows us to opt into not importing code at all for modules that don't need JS to work.
- Prevents any flash of content from happening.
Conclusion
Sometimes you have to go beyond what our tools offer us out of the box and that is okay. Gatsby is an excellent tool I plan on using for a long time but it needed some super powers added to it with loadable components. We saw a total of about 200KB of javascript removed from our site when we implemented something like this code and yes we have seen improvements in our page speed when using lighthouse and web page test.
I know I left some of the code above open ended but I really can't share much more since it is a company project. Feel free to reach out to me though if you have questions and I will guide you as much as I can without handing you the word for word solution.
Any follows on dev.to and twitter are always appreciated!
Cover Photo by José Alejandro Cuffia on Unsplash
Top comments (12)
Hey Tommy,
Just wanted to thank you for sharing this solution with us, this is the only article I was able to find on gatsby addressing this issue. Was just wondering if you have a working example of that pseudo code you outlined at the end of your examples. Would you be willing to share it with us? :-)
Thank you again for writing this article and outlining your solution!
Hey Lilian,
I'm happy to share the real code now as we actually made the decision to remove React altogether for our simple site so this blog post quickly went out of date for us. I actually posted on Reddit a few days back with some open source plugins we created to make removing React easy reddit.com/r/gatsbyjs/comments/c6z...
The real code is below
As you can see for each module we check whether or not the module should load javascript by checking the
shouldLoadJavascriptflag, if it does not load javascript then we just render the module statically. Here is our module map file that thegetFromMapfunction is working onMate, you've made my day! :-)
Thank you so much for being so forthcoming and for your quick reply. There are so many issues or edge cases where I can get multiple sources addressing them, but for this one, your article is the only one that addressed exactly what I was looking for and in such a straightforward way.
I hope you won't stop sharing your knowledge and expertise. You've already saved me hours of headaches :-)
Can you please explain a bit more about he
isServer()and "shouldLoadJS" Stuff?I don't really got this in the Gatsby-Context. If the Page was build and served with gatsby, the Page become statically, right? So why is there an "isServer"-Request needed (and how do you test for "isServe"?)
Or is it to force to reload Component, if you are in
gatsby developmode?And for
shouldLoadJS... what exactly the functionality of it? How do I decide if "javascript" is needed or not by this module. Does it means if you import additional JS inside this module?I am really curious about how you're handling the
lazyLoad: index > 1part.Could you enlighten me on this?
Sorry for the late reply, this
lazyLoad: index > 1is there mostly for an<Image/>component we have. We only wanted images to lazy load if they were later down the page, so the first module we had would never lazy load images.Can you give a hint about this
isServer()function I've asked above this comment?Thanks a lot in advance
I'm curious about this, as well. But I suppose you could just check for the existence of
window.This is exactly, almost word for word, the issue I’ve been trying to solve. Thanks so much for sharing in such great detail.
Hey Tommy,
Thanks for the great article.
Where was your .babelrc file located? My gatsby projects only have this file in the .cache/tests/ directory, which is reset on each build
Just tried viewing your site on mobile. Not sure if it’s working
I don't really have a personal site at the moment, do you remember what site you were visiting?