DEV Community

Cover image for Sitemap couldn't fetch in Google Search Console - fix
1

Sitemap couldn't fetch in Google Search Console - fix

To set up your site to be indexed by Google Search, you need to have a robots.txt file and a sitemap.xml file in your site.

Here’s how I set up my own sitemap and robots file using Next.js. My solution takes TypeScript files and converts them into .txt and .xml files, which are then placed in the public directory of the site.

robot.ts
This is the code for my robots.txt file:

import { MetadataRoute } from "next";

export default function robots(): MetadataRoute.Robots {
  return {
    rules: {
      userAgent: "*",
      allow: "/",
    },
    sitemap: "https://www.thook.xyz/sitemap.xml",
    host: "https://www.thook.xyz",
  };
}
Enter fullscreen mode Exit fullscreen mode

sitemap.ts
Below is the code for generating my sitemap.xml file dynamically:

import { MetadataRoute } from "next";

export default function sitemap(): MetadataRoute.Sitemap {
  const routes: MetadataRoute.Sitemap = [
    {
      url: "",
      changeFrequency: "monthly",
      priority: 1,
    },
    {
      url: "/search",
      changeFrequency: "monthly",
      priority: 0.5,
    },
  ];

  return routes.map(({ url, ...rest }) => ({
    ...rest,
    url: `https://thook.xyz${url}`,
    lastModified: new Date(),
  }));
}
Enter fullscreen mode Exit fullscreen mode

The Problem 🤔

While my robots.txt worked flawlessly, my sitemap.xml kept returning an error:

sitemap unsuccessful

It seemed like Google Search Console had cached an old version of my sitemap, and nothing I did seemed to fix the issue.

The Solution 🚀

To work around this, I created a new sitemap2.xml file and placed it in the public directory of my Next.js application:

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml">
    <url>
        <loc>https://thook.xyz/</loc>
        <lastmod>2025-01-01T11:39:05.531Z</lastmod>
        <changefreq>yearly</changefreq>
        <priority>1</priority>
        <xhtml:link rel="home" href="https://thook.xyz/" />
    </url>
    <url>
        <loc>https://thook.xyz/search</loc>
        <lastmod>2025-01-01T11:39:05.531Z</lastmod>
        <changefreq>monthly</changefreq>
        <priority>0.8</priority>
        <xhtml:link rel="search" href="https://thook.xyz/search" />
    </url>
</urlset>
Enter fullscreen mode Exit fullscreen mode

And just like that, Booom!💥.
Jan 1st problem solved ✅.

sitemap successful

Lessons Learned 📚

Here’s what I learned about using Google Search Console:

  • Be patient: Google Search Console can take time to crawl and update your site. Don’t panic.
  • Try a fresh name: If you’re facing persistent errors, create a new sitemap with a different name that Google hasn’t cached yet.

Thanks for reading.

Speedy emails, satisfied customers

Postmark Image

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.

Sign up

Top comments (0)

Heroku

Build apps, not infrastructure.

Dealing with servers, hardware, and infrastructure can take up your valuable time. Discover the benefits of Heroku, the PaaS of choice for developers since 2007.

Visit Site

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay