We knew we had a problem.
In 2019, SitePoint was getting Lighthouse Speed scores under 10 on mobile, and between 20 and 30 on desktop.
Our efforts to control UX bloat were failing in the wake of a publishing business environment that sprang new leaks just as we’d finished temporarily plugging the last one. Our reliance on advertising, controlled by external parties, was a major obstacle to improved site performance. Our traffic growth had turned into decline.
On a site that provided people with a place to come and learn to code with best practices, this was not a good look. And it wasn't a site we could feel proud of, either.
To make matters worse, operational bottlenecks had arisen that made adaptation a tricky logistical business. Our team was struggling to make changes to the site: having focused on our Premium experience for several years, we were down to one developer with WordPress and PHP experience. To test out code changes, the team would have to wait in a queue to access our staging server.
It wasn’t energizing work for anyone, and it certainly wasn't efficient.
It was time to make some changes, and we set out to look for a solution. After a lot of research, we decided that Gatsby would be a great fit for our team. It would play to our talent strengths, help us solve all of the issues we had identified, and allow us to keep using WordPress for the backend so the editorial process wouldn’t need to change.
Why We Moved to Gatsby
Early in the research process, Gatsby started to look like a serious frontrunner. SitePoint isn't a small site, so we knew that the tech we chose had to be able to handle some pretty intense demands. Gatsby checked all of our boxes:
- We could code everything in React, a tech that every member of the front-end team knows and uses daily.
- Gatsby is super fast at its core — performance was at the heart of this project, and we could start from a good footing.
- The entire site is rendered as static, which would be great for SEO.
- We could build it as a new project, which meant no worrying about the existing codebase, which brought a huge amount of legacy code with it.
- We could use Gatsby Cloud, allowing the team to get feedback on the build at any time just by pushing the branch to GitHub.
- DDoS attacks on WordPress wouldn’t cause us issues, as the front-end is completely stand-alone.
More Maintainable CSS with styled-components
Since we were going to rebuild the site from scratch, we planned to make some design changes at the same time. To help with this work we decided to use styled-components.
styled-components keeps the site's styling easy to maintain, and we know where to look when we want to change the style of something — the style is always with the component.
How We Made the Build Happen
We started by following Gatsby's basic docs and pulling in our posts with the gatsby-source-wordpress plugin.
This was a big initial test for us: we had to see if it was even possible to use Gatsby for our site.
After 20 years of blogging, we have over 17,000 posts published. We knew the builds would take a long time, but we had to find out if Gatsby could deal with such a massive amount of content. As you've probably figured, the test delivered good news: Gatsby works.
A quick tip for other teams working with large sites: to make development a better experience, we used environment vars to prevent Gatsby from fetching all of the site's posts in development. There's nothing quite like a 60 minute hot reload to slow progress.
if (hasNextPage && process.env.NODE_ENV != "development") {
return fetchPosts({ first: 100, after: endCursor });
}
From this point, we ran into some limitations with the WordPress source plugin. We couldn't get all the data we needed, so we moved to the WordPress GraphQL plugin.
We use Yoast to set our metadata for SEO, and had to ensure we were pulling in the correct information. We were able to do this with WordPress GraphQL. By doing it this way, the content team could still edit metadata the same way, and the data would still be dynamic and fetched on each build.
During the build, we would have three or four people in the team working on parts of the new blog. In the past, if they wanted to get feedback they'd have to push to our staging server and make sure nobody was already using it.
We found that Gatsby Cloud was a great solution to this issue. Now when someone pushes to a branch in GitHub, it creates a build in Gatsby Cloud along with a preview link. Our developers could share this link and get immediate testing and feedback much more effectively than before.
This faster feedback cycle made it easy to have multiple people on the team working on the build and put an end to a major bottleneck.
Launch Day Fun
On the big day, we launched the new site and ran through our initial tests. The new blog was flying — every page load felt instant.
We ran into some problems on SitePoint Premium, which started running into slows and even crashes. The culprit was a new element on blog pages that pulled in the popular books people were currently reading. It would do this via a client-side API call, and it was too much for Premium to handle due to the amount of traffic we get on the blog side.
We quickly added some page caching to the API to temporarily solve the issues. We realized we were doing this wrong — we should have been sourcing this data at build time, so that the popular books are already loaded when we serve the page to the user.
This is the main mindset shift you need to make when using Gatsby: any data that you can get at build time should be fetched at build time. You should only use client-side API calls when you need live data.
Once we'd re-written the API call to happen during the build, the first load of a blog page was even quicker — and Premium stopped crashing.
What We Still Need to Solve
While it's hard to overstate how much better our on-site experience is today, there are still a few pain points we need to solve.
If a new article is published, or if content is updated — as it is multiple times per day — we need to re-run the Gatsby build before these changes show up.
Our solution for that right now is a simple cron job that runs at pre-scheduled times over the course of a day. The long-term solution to this is to add a webhook to the WordPress publish and update button, so that a new build is triggered once pressed.
We also need to get incremental builds running. Right now, the entire site needs to be rebuilt each time, and given our content archive, this can take a while. Gatsby just introduced incremental builds as we went live, and we're working on implementing this on our site. Once that's set up our builds will be much faster if the only thing that has changed is content.
Our speed score is still not where we want it to be. While the site feels subjectively very fast, we are still not getting consistent scores in Lighthouse. We want to get both mobile and desktop into the green zone (scores of 90+) for optimal user experience and SEO.
Would We Do It Again?
A launch of this type would normally be a pretty nerve-wracking event, and take a lot of work from the team on launch day.
With Gatsby, our launch was really easy. We just had to move WordPress onto a new domain, and point sitepoint.com at the Gatsby version of the site.
Then we sat back and watched the numbers to see what happened to our traffic. Within a few days, the data was starting to come in and we were seeing a 15% increase in traffic. User engagement metrics were up across the board. And we hadn't even removed our ads yet (which, you may have noticed, we've since done).
It's not hard to figure out why the effects were so immediate. We had better SEO running on static HTML and CSS pages, and massive speed improvements made possible by the move to Gatsby.
Since we made the move, we've increased our Lighthouse speed scores from 6-15 on mobile to the 50-60 range, and from the 30s on desktop into the 70s. We wanted to ensure speed remained top of mind with this change, so we're using a great tool called Calibre that runs speed tests over a number of top pages each day and alerts us to the scores. We are using this tool to continue to improve our score, so I hope to have another article for you in three months when we get everything to stay in the 90+ range.
The team loves working in Gatsby. The blog codebase was something that nobody wanted to work on. Now, everyone wants to take those cards thanks to the great developer experience.
If you've been eyeing a move to Gatsby and wondering if it's ready for prime time, take our advice — it's worth the switch.
Top comments (1)
Great to hear how Gatsby has improved the SEO for a big publisher/website like Sitepoint. At the start it sounds overwhelming how to move 17,000 Wordpress posts to a new frontend, but with Gatsby it is possible.
I assume the editors can preview everything in the Wordpress domain. But curious about how editors feel with the building time.