Automagically creating a Sitemap for React, Node.js, MongoDB website

Sitemaps play an important role in SEO optimisation. Thus having a current sitemap is important. Automating a sitemap is easy. My sitemap generator runs on my node.js web server, automatically updating my sitemap.xml in my create-react-app website, public folder when I create or delete a page from site.

Step 1. Create a new generate-sitemap.js file

Step 2-A. Import the following

import { writeFileSync } from "fs";
import mongoose from "mongoose";

//Import the model which you want to autogenerate. 
import Product from "../models/productModel.js";

Step 2-B. Connect to your DB

const mongodbUrl = config.MONGODB_URL;
mongoose
    .connect(mongodbUrl, {
        useNewUrlParser: true,
        useUnifiedTopology: true,
        useCreateIndex: true,
    })
    .catch((error) => console.log(error.reason));

Step 2-C. Create the sitemap generator function

export const generateSitemap = async () => {
    const products = await Product.find();
    const sitemap = ``;
}

Step 2-D. Add the static pages

(non-changing urls i.e About, Home, Contact, Search, TOS, Privacy Policy, etc).

const generateSitemap = async () => {
    const products = await Product.find();
    const sitemap = `
    <?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
        <loc>https://www.yourWebsiteHere.com/</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/about</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/contact</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/search</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/privacy</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/terms-of-use</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/shipping-and-returns</loc>
    </url>
</urlset>
`;
}

export default generateSitemap 

Step 2-E. Add the dynamic pages after the last </url>

It can be products, categories, articles, etc.

  ${products
            .map((page) => {
                return `
              <url>
                  <loc>${`https://www.yourWebsiteHere.com/product/${page.slug}`}</loc>
              </url>
            `;
            })
            .join("")}

Step 2-F: Write the file to your desired location.

I'm saving it to my public folder.

writeFileSync("./frontend/public/sitemap.xml", sitemap);
console.log("genarated Sitemap successfully");
    return true;

Step 3. Recreate the sitemap every time an new product is added.

I want the generateSitemap function to run every time I add a new product. So I'm calling it after my product is created.

import generateSitemap from "../scripts/generate-sitemap.js";

productRouter.post(
    "/",
    isAuth,
    expressAsyncHandler(async (req, res) => {
            const createdProduct = await product.save();
            //calling generateSitemap
            generateSitemap();
            res.send({
                message: "Product has been added successfully!",
                product: createdProduct,
            });

    });

You can do something similar or create a cron job to recreate the sitemap at a set interval.

I'm using node-cron for that. I've scheduled it to run every morning at 6am and I've set the timezone closest to mine.

Install node-cron using npm install --save node-cron

import cron from "node-cron";
cron.schedule(
    "0 6 * * *",
    () => {
        generateSitemap();
    },
    {
        scheduled: true,
        timezone: "Asia/Colombo",
    },
);

Finally once your site map is generated you can access it by typing your websitename.com/sitemap.xml or if you're on localhost then localhost:port/sitemap.xml.

Complete Code for generate-sitemap.js

import { writeFileSync } from "fs";
import mongoose from "mongoose";
import config from "../config.js";

import Product from "../models/productModel.js";

const mongodbUrl = config.MONGODB_URL;
mongoose
    .connect(mongodbUrl, {
        useNewUrlParser: true,
        useUnifiedTopology: true,
        useCreateIndex: true,
    })
    .catch((error) => console.log(error.reason));

const generateSitemap = async () => {
    const products = await Product.find();
    const sitemap = `
    <?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
        <loc>https://www.yourWebsiteHere.com/</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/about</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/contact</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/search</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/privacy</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/terms-of-use</loc>
    </url>
    <url>
        <loc>https://www.yourWebsiteHere.com/rules/shipping-and-returns</loc>
    </url>
        ${products
            .map((page) => {
                return `
              <url>
                  <loc>${`https://www.yourWebsiteHere.com/product/${page.slug}`}</loc>
              </url>
            `;
            })
            .join("")}
    </urlset>
    `;

    writeFileSync("./frontend/public/sitemap.xml", sitemap);
    console.log("genarated Sitemap successfully");
    return;
};

export default generateSitemap;

Comments

[TechSynthesis]

If you have issues with the post or if you get stuck at any point do let me know. This is my first time documenting something like this.

[Dragos George]

Thanks, this is amazing!