DEV Community

Auto-Generate sitemap.xml in Next.js

Martin Beierling-Mutz on December 29, 2018

Hurray! You created all the components and styling for your beautiful and performant Next.js website. What now? There are some key files you want ...

Read full post

Remi Vledder • Jun 23 '20

Another example for a next js site. I've adjusted it slightly to also handle sub-folders:

const fs = require('fs');
const path = require('path');

const removeFileExt = (dirname) => {
  const parsedPath = path.parse(dirname);
  if (parsedPath.dir) {
    return `${parsedPath.dir}/${parsedPath.name}`;
  }
  return parsedPath.name;
};

module.exports = () => {
  const fileObj = {};

  const walkSync = (dir) => {
    const files = fs.readdirSync(dir);
    files.forEach((file) => {

      // specific for Next, skip files starting with '-' or between brackets '[...]'
      const partialNextFile = /^_\w*/;
      const dynamicNextFile = /^\[(.*?)\]/;
      if (partialNextFile.test(file) || dynamicNextFile.test(file)) {
        return;
      }

      const fullPath = `${dir}${file}`;
      const fileStat = fs.statSync(fullPath);

      if (fileStat.isDirectory()) {
        walkSync(`${fullPath}/`);
      } else {
        const parsedPath = path.parse(fullPath);
        if (!parsedPath.ext) {
          // skip system files
          return;
        }

        const filePathAry = path.format(parsedPath).split(path.sep);
        filePathAry.splice(0, 3);

        const fileDir = filePathAry.join('/');
        const cleanPath = removeFileExt(fileDir);

        fileObj[`/${cleanPath}`] = {
          page: `/${cleanPath}`,
          lastModified: fileStat.mtime,
        };
      }
    });
  };

  walkSync('./src/pages/');
  walkSync('./src/posts/');

  return fileObj;
};

Remi Vledder • Jun 23 '20

Very nice! I've made some changes in my project, perhaps this is of any use to someone.

In the walkSync method I used the 'dir' parameter to replace pages/. This makes the method reusable for any other paths such as posts.

// ... not shown for brevity
const cleanFileName = filePath.substr(0, filePath.lastIndexOf('.')).replace(dir, '');
// ... not shown for brevity

Also, by using an early return I filtered out any files that do not compile to an actual file:

// ... not shown for brevity
files.forEach((file) => {
      const partialNextFile = /^_\w*/; // any filenames starting with _*
      const dynamicNextFile = /^\[(.*?)\]/; // any filenames between brackets [*]
      if (partialNextFile.test(file) || dynamicNextFile.test(file)) {
        // skip
        return;
      }
// ... not shown for brevity

Guillermo Gonzalez • Apr 24 '19 • Edited

Thanks! I think there's a bug in the code:

${Object.keys(pathsObj).map(
path => <url> <loc>https://embiem.me${path}</loc> <lastmod>${ formatDate(new Date(pathsObj[path].lastModified)) }</lastmod> </url>
)}

Should be:

${Object.keys(pathsObj).map(
path => <url> <loc>https://embiem.me${path}</loc> <lastmod>${ formatDate(new Date(pathsObj[path].lastModified)) }</lastmod> </url>
).join("")}

The original version is outputting commas between each XML element.

chadarri • May 8 '19

Hello Martin,

I am trying to add the sitemap and robots.txt files to my web NextJs app. I follow your post and I generated the two files without any problems but what is the next step to make accessible via URL the file in production ?

The two files are in the Out directory but they are not accessible in the url online.

Sorry for my bad english, I hope you will understand my problem.

Charlotte

Martin Beierling-Mutz • May 12 '19

Hey. This really depends on your deployment setup. As long as all files in the out folder are pushed to your static file host, you should be good.