Adding a dynamic sitemap.xml to a next.js site

While building my blog with Next.js I naturally wanted to support sitemaps because it can supposedly help out the search engines. For my small blog it will not make any difference but for larger sites it's more important. Google has this to say.

Using a sitemap doesn't guarantee that all the items in your sitemap will be crawled and indexed, as Google processes rely on complex algorithms to schedule crawling. However, in most cases, your site will benefit from having a sitemap, and you'll never be penalized for having one.

Page for the sitemap

The first thing we need to do is to create a sitemap.xml.ts page in the "page"-folder. This will expose a https://yourdomain.com/sitemap.xml url that you can submit to search engines. if you want to you can omit the .xml part and only use /sitemap, Google search console will accept the url anyway.

We want to make sure that we set the Content-Type header to text/xml and to write our xml output to the response stream.

export default class Sitemap extends React.Component {
  static async getInitialProps({ res }: any) {
    const blogPosts = getBlogPosts();
    res.setHeader("Content-Type", "text/xml");
    res.write(sitemapXml(blogPosts));
    res.end();
  }
}

Generating the base xml

For the site map we want to list all pages on the site, apart from the blog posts we have to add all additional pages that we want the search engines to find.

I have an about page that I add manually together with the index page but if you have many pages I suggest you create an array and add them in a more automated way.

I won't go into the inner workings of all the properties of a sitemap but I want to mention the <priority>-tag that lets you set a value between 0 and 1 indicating how important you think the page is.
<lastmod> is used to indicate when the page was changed.

const sitemapXml = (blogPosts: IBlogPostListItem[]) => {
  const { postsXml, latestPost } = blogPostsXml(blogPosts);
  return `
  <?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
      <loc>https://www.bergqvist.it/</loc>
      <lastmod>${latestPost}</lastmod>
      <priority>1.00</priority>
    </url>
    <url>
      <loc>https://www.bergqvist.it/about</loc>
      <priority>0.5</priority>
    </url>
    ${blogPostsXml}
  </urlset>`;
};

Adding xml for the blog posts

As mentioned above I want to add the dynamic blog post pages to the site map as well. In the blogPostsXml-function I generate xml for all posts and keep track of when the latest post was posted.

const blogPostsXml = (blogPosts: IBlogPostListItem[]) => {
  let latestPost = 0;
  let postsXml = "";
  blogPosts.map(post => {
    const postDate = Date.parse(post.createdAt);
    if (!latestPost || postDate > latestPost) {
      latestPost = postDate;
    }
    postsXml += `
    <url>
      <loc>${post.url}</loc>
      <lastmod>${postDate}</lastmod>
      <priority>0.80</priority>
    </url>`;
  });
  return {
    postsXml,
    latestPost
  };
};

Last words

Make sure to check that you do not add any pages to the sitemap that is blocked in your robots.txt.

If you have a large site with many pages (100 or more) you can split up the sitemap into several smaller ones, for very large sites this is a must!

I hope this could help out someone and please leave a comment. A full gist of the code can be found here

Comments

[Fredrik Bergqvist]

Isn't that what I did? :)

[Hammed Oyedele]

Oh...I actually skipped that part 🤭🤭🤭

[Michael J. Ryan]

Just a minor note... You probably want to stick to the most recent 100 entries by last modified date and let any search engine navigate/remember the rest. Inflating the sitemap too big can work against you.

[Fredrik Bergqvist]

Yes, I did touch on it but maybe I should make it more clear.

[Mfou'ou Medjo stanly]

Thanks great article !!

[Fredrik Bergqvist]

Thank you! Glad it could be of some help :)