How to create Sitemap.xml for ASP.net Core Razor Pages

Recently I wanted to create a sitemap for my Razor Page web application. Adding a sitemap to a website is a relatively straightforward process, but I found out that many examples over the web are a bit outdated. So I decided to document how I added it.

The following information is related to Microsoft.AspNetCore.App version 7.0.7.

In general, creating a sitemap.xml for web applications can significantly enhance their search engine visibility. A sitemap provides a roadmap for search engine bots, enabling them to index your website content more efficiently.

Step 1: Understand the Structure of sitemap.xml

A sitemap XML file lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it changes, and its importance relative to other URLs).

Here is a very basic XML sitemap that includes the location of a single URL:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
        <loc>https://www.example.com/foo.html</loc>
        <lastmod>2022-06-04</lastmod>
    </url>
</urlset>

There are more parameters defined in the protocol specification, but Google claims to ignore them, so I think they could be omitted:

• Google ignores <priority> and <changefreq> values.

• Google uses the <lastmod> value if it's consistently and verifiably (for example by comparing to the last modification of the page) accurate.

Step 2: Create a Model for the Sitemap

First, I created a model for the sitemap node. This model will represent individual URLs, their priority, and other metadata.

    public class SitemapNode
    {
        public SitemapFrequency? Frequency { get; set; }
        public DateTime? LastModified { get; set; }
        public double? Priority { get; set; }
        public string Url { get; set; }
    }

    public enum SitemapFrequency
    {
        Never,
        Yearly,
        Monthly,
        Weekly,
        Daily,
        Hourly,
        Always
    }

There is another approach if I wanted to use serialization, which would look a bit clearer, but it takes twice as many lines of code, so I skipped it.

[XmlRoot("urlset", Namespace = "http://www.sitemaps.org/schemas/sitemap/0.9")]
public class SitemapUrlSet
{
    [XmlElement("url")]
    public List<SitemapNode> SitemapNodes { get; set; } = new List<SitemapNode>();
}

public class SitemapNode
{
    [XmlElement("loc")]
    public string Url { get; set; }

    [XmlElement("lastmod")]
    public DateTime? LastModified { get; set; }

    [XmlElement("changefreq")]
    public SitemapFrequency? Frequency { get; set; }

    [XmlElement("priority")]
    public double? Priority { get; set; }
}

public enum SitemapFrequency
{
    [XmlEnum("never")]
    Never,

    [XmlEnum("yearly")]
    Yearly,

    [XmlEnum("monthly")]
    Monthly,

    [XmlEnum("weekly")]
    Weekly,

    [XmlEnum("daily")]
    Daily,

    [XmlEnum("hourly")]
    Hourly,

    [XmlEnum("always")]
    Always
}

Step 3: Setting Up the Method to Generate Sitemap Nodes

I needed a service or method that will generate the sitemap nodes based on my website's content. To generate URLs for Razor Pages, you typically could use PageLink or LinkGenerator. I used the last one:

    public class SitemapModel : PageModel
    {
        private readonly LinkGenerator _linkGenerator;

        public SitemapModel(LinkGenerator linkGenerator)
        {
            _linkGenerator = linkGenerator;
        }

        // ... rest of the code
    }

Creating a sitemap for static pages is easier by hardcoding them. Blog pages or other dynamic content are taken from a database or file system (in my case) based on where they are stored.

Method GetUriByPage from provides an absolute URL based on page name - handy.

    public class SitemapModel : PageModel
    {   
        // ... rest of the code   
        public IReadOnlyCollection<SitemapNode> GetSitemapNodes()
        {
            var nodes = new List<SitemapNode>
            {
                new()
                {
                    Url = _linkGenerator.GetUriByPage(HttpContext, "/Index"),
                    Priority = 1,
                },
                new()
                {
                    Url = _linkGenerator.GetUriByPage(HttpContext, "/Tools/CreateCode"),
                    Priority = 0.9
                },
                new()
                {
                    Url = _linkGenerator.GetUriByPage(HttpContext, "/Legal/Privacy"),
                    Priority = 0.6
                },
                new()
                {
                    Url = _linkGenerator.GetUriByPage(HttpContext, "/Legal/TermsOfService"),
                    Priority = 0.6
                }
            };

            foreach(...)
            {
                // fill nodes from blog index
            }

            return nodes;
        }
    }

Step 4: Creating the Sitemap Page

Then I added a new Razor Page Sitemap.cshtml that will be responsible for generating the sitemap.xml. When users or search engine bots access this page, it should return the sitemap in XML format. This is an elegant trick: we return the razor page Sitemap.cshtml as an XML file.

@page
@model Xakpc.Project.Pages.SitemapModel
@{
    Layout = null;
    Response.ContentType = "text/xml";
}
<?xml version="1.0" encoding="UTF-8" ?>
@Html.Raw(Model.RawXmlData)

Another required step is to set up the app to return this file by /sitemap.xml path. It is done in options of AddRazorPages method like this:

builder.Services.AddRazorPages()
    .AddRazorPagesOptions(options =>
{
    options.Conventions.AddPageRoute("/sitemap", "Sitemap.xml");    
});

Step 5: Formatting the XML

On the sitemap Razor Page model, I formatted the sitemap nodes into XML format. For that, I manually built XML file with XElements

    public class SitemapModel : PageModel
    {   
        // ... rest of the code  

        /// <summary>
        /// Serializes to raw XML
        /// </summary>
        public string GetSitemapDocument(IEnumerable<SitemapNode> sitemapNodes)
        {
            XNamespace xmlns = "http://www.sitemaps.org/schemas/sitemap/0.9";
            var root = new XElement(xmlns + "urlset");

            foreach (var sitemapNode in sitemapNodes)
            {
                var urlElement = new XElement(
                    xmlns + "url",
                    new XElement(xmlns + "loc", Uri.EscapeUriString(sitemapNode.Url)),
                    sitemapNode.LastModified == null ? null : new XElement(
                        xmlns + "lastmod",
                        sitemapNode.LastModified.Value.ToLocalTime().ToString("yyyy-MM-ddTHH:mm:sszzz")),
                    sitemapNode.Frequency == null ? null : new XElement(
                        xmlns + "changefreq",
                        sitemapNode.Frequency.Value.ToString().ToLowerInvariant()),
                    sitemapNode.Priority == null ? null : new XElement(
                        xmlns + "priority",
                        sitemapNode.Priority.Value.ToString("F1", CultureInfo.InvariantCulture)));
                root.Add(urlElement);
            }

            var document = new XDocument(root);
            return document.ToString();
        }
    }

The other option is to use serialization (if you map classes with attributes - I skipped that part)

public string GetSitemapDocument(IEnumerable<SitemapNode> sitemapNodes)
{
    var sitemapUrlSet = new SitemapUrlSet { SitemapNodes = sitemapNodes.ToList() };

    var xmlSerializer = new XmlSerializer(typeof(SitemapUrlSet));

    using var stringWriter = new StringWriterUtf8();
    using var xmlTextWriter = XmlWriter.Create(stringWriter, new XmlWriterSettings { Indent = true });

    xmlSerializer.Serialize(xmlTextWriter, sitemapUrlSet);

    return stringWriter.ToString();
}

And all that is left is to call methods in OnGet method and set bound property

    public class SitemapModel : PageModel
    {   
        // ... rest of the code  

        /// <summary>
        /// Gets the raw XML data
        /// </summary>
        [BindProperty(SupportsGet = true)]
        public string RawXmlData { get; set; }

        public void OnGet()
        {
            var nodes = GetSitemapNodes();
            RawXmlData = GetSitemapDocument(nodes);
        }
    }

Step 6: Register the Sitemap with Search Engines

After the sitemap is successfully set up, it's a good idea to register it with major search engines like Google and Bing. This will ensure that they know your sitemap's existence and can crawl your website more effectively.

1. Google: Use Google Search Console to submit your sitemap.

2. Bing: Use Bing Webmaster Tools to submit your sitemap.

Conclusion

Final Sitemap.xml

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
        <loc>https://contoso.com/</loc>
        <priority>1.0</priority>
    </url>
    <url>
        <loc>https://contoso.com/make-code</loc>
        <priority>0.9</priority>
    </url>
    <url>
        <loc>https://contoso.com/legal/privacy</loc>
        <priority>0.6</priority>
    </url>
    <url>
        <loc>https://contoso.com/legal/termsofservice</loc>
        <priority>0.6</priority>
    </url>
    ...
</urlset>

As you can see, setting up a sitemap.xml in a Razor Pages application is straightforward. Following the steps above and adding the appropriate code ensures your website is more visible and accessible to search engines.