I thought I had found a convenient way to expand my website: generate a service page for each city, change the location in the URL and headings, then let the sitemap introduce everything to search engines.
About 400 URLs later, I had plenty of pages. What I did not have was 400 different reasons for someone to read them.
Some pages appeared in the index, then many started disappearing. Cleaning up the structure took longer than generating it. This is what I learned from that experiment on my own website.
First, look at what you actually built
Before opening another SEO tool, I make a simple inventory: URL, title, description, H1, main section headings, and the purpose of the page.
This gives me a skeleton of the site. Are two pages answering the same question? Does the title match the content? Is there a useful difference between the regional versions?
A separate URL does not answer those questions. Neither does a different city name in the H1.
Astro made the technical part easier
I chose Astro because, in the static setup I used, the build produces HTML containing the page text, headings and links. The main content does not need a client-side application to assemble it after loading.
I can also control titles, descriptions, canonical URLs, internal links and the sitemap. That made the generated structure look tidy.
But static HTML is not a guarantee of indexing or high rankings. Astro cannot make an almost identical page useful just because it renders quickly. I still have to give the page something worth reading.
How my regional pages multiplied
I started with Russian service pages for Cheboksary, Kazan, Moscow and other cities. For a landing-page service, the addresses looked like this:
/cheboksary/catalog/ai-landing
/kazan/catalog/ai-landing
/moscow/catalog/ai-landing
The city changed in the title, headings and a few sentences. Most of the offer stayed the same.
Then I extended the idea to English pages for Almaty, Dubai, Abu Dhabi and other locations. The same template produced another set of regional versions.
At the time, automation looked like the solution. Looking back, it also made it easy to avoid asking whether each page needed to exist.
Discovery was not the same as indexing
My sitemap exposed the generated URLs to search engines. It did not make them accept every page. Google's sitemap documentation makes that distinction: a sitemap helps discovery, but does not guarantee crawling and indexing.
Initially, some pages appeared in search. A few days later, many began dropping out, especially in Yandex.
The similar regional content seemed like a plausible explanation. I cannot prove it was the only cause. Internal links, canonical settings, technical problems, demand and the usefulness of individual pages could also have affected the outcome. This was an observation on my site, not a controlled test or a confirmed search-engine penalty.
Keep the indexing controls separate
robots.txt and noindex
robots.txt controls crawler access. Blocking a URL there does not reliably remove it from Google's search results.
noindex is an indexing instruction in a meta tag or HTTP response header. Google must be able to crawl the page to see it. Blocking the page in robots.txt can prevent that. Google explains this dependency.
Canonical URLs
A canonical identifies the preferred version of duplicate or very similar content. It is not an indexing ban, and Google can choose a different canonical. I should not treat it as a replacement for removing a page. Google's canonicalisation guide describes it as a signal of preference.
What I changed
I stopped automatically expanding into more cities. I kept a general Russian version, an English version, and a focused Cheboksary section. I live in Cheboksary, which gave me a practical reason to develop that direction.
I restricted unwanted regional pages and later deleted them. I also separated the general and local catalogue content rather than changing only the city name.
Old URLs did not disappear from webmaster reports immediately. Cleaning the site and seeing search engines reflect the changes were separate stages.
Now I start with one useful page. Another regional version needs a real difference: relevant examples, conditions, services or information for that audience. I no longer generate a city page just because the template makes it possible.
Roman Mikhailov, author of RM Systems.
Adapted from my original article in Russian.
Top comments (0)