DEV Community

Discussion on: I built a large production static site for a publisher (100K articles) [without GatsbyJS], Ask Me Anything!

Collapse
 
ctrl_alt_aldr profile image
Christian | 🧑🏼‍💻 • Edited

Did you use an alternative static site gen? (Hugo, Next.js?, etc) and if not how did you approach splitting up and managing the code?

I'm a newbie to this field but without a static gen (or using something like PHP) I can't imagine how you'd wrangle implementing a header to each page for example, and changing it manually across each of the 100k articles opposed to once ^

Collapse
 
jjaybrown profile image
Jason Brown

When we started the project it was before Next.js had reached any level of maturity.

We built a pipeline using gulp and metalsmith, which takes partials and markdown, renders HTML and then inlined CSS and JS which had been minified in a previous stage.

Webpack would have been a better alternative had we understood more about the complexity we'd face using gulp for concurrency. The more tasks we needed, the longer the build would run.

We could use partials and mustache to extend templates.

For publishing or re-building based on changes, we would run a jenkins task within our CI and that would re-build all 100K article, updating the header for example.

Collapse
 
bradtaniguchi profile image
Brad

How long do the re-builds take for all those articles?

How difficult would it be to move from a gulp based setup to webpack?

Thread Thread
 
jjaybrown profile image
Jason Brown

A full site build would take 30-40min, which isn't just articles but also RSS feeds, pagination for categories and landing pages.

We didn't see a need to move to webpack, we also couldn't do incremental builds with webpack due to how it chunks JS.

Any changes would be a roll forward type thing, so you could see older content over time look or act differently to newer content. Meant we were also able to identify when breaking changes took place and it wouldn't affect the whole site.