Google's documentation for sitemaps says to "include the URLs in your sitemap that you want to see in Google's search results". Its documentation for the robots meta tag says the noindex rule means: "Do not show this page, media, or resource in search results." A URL that carries both is giving two answers, and when the pages are generated from one array it is easy for them to drift apart without anyone noticing.
LandscapeCraft (landscapecraft.ai) is a Next.js App Router site with 40 design-style pages generated from one TypeScript array. A style can have a long MDX write-up in content/styles/. A style without one falls back to a short generic template, and the site treats that page as thin: it renders noindex, follow, and its URL stays out of both sitemap.xml and image-sitemap.xml.
This post is about those three places: how the code keeps them in step, how I checked that it does, and where it can still drift. The code is from the repository as of 2026-11-02, and the live numbers are from the same day. Code comments below are translated from Chinese.
One question, asked three times
The helper is small:
export function getStyleContent(slug: string): StyleContent | null {
const filePath = path.join(STYLES_CONTENT_DIR, `${slug}.mdx`);
if (!fs.existsSync(filePath)) return null;
const raw = fs.readFileSync(filePath, "utf-8");
const { content } = matter(raw);
return { slug, content };
}
The style page asks it for its metadata:
// app/styles/[slug]/page.tsx, inside generateMetadata()
// Style pages without long-form content (they fall back to the generic template) stay out of the index
noIndex: !getStyleContent(slug),
buildMetadata turns that flag into metadata.robots = { index: false, follow: true }, which renders as <meta name="robots" content="noindex, follow">. sitemap.ts asks the same question:
// app/sitemap.ts
// Style pages without a long-form MDX are rendered as noindex (see app/styles/[slug]/page.tsx),
// so they must not appear in the sitemap.
const stylePages = ALL_STYLES.filter((s) => !!getStyleContent(s.slug)).map((s) => ({
url: `${SITE_URL}/styles/${s.slug}`,
// ...
}));
and so does the image sitemap route:
// app/image-sitemap.xml/route.ts
for (const style of ALL_STYLES) {
// Style pages without long-form MDX are noindex; keep them out of the image sitemap
if (!getStyleContent(style.slug)) continue;
// ...
}
Nothing ties the three together except that predicate and a comment in each file. robots.txt is a fourth party: it disallows only /api/, on purpose. Google's robots meta documentation says that if a page is disallowed in robots.txt, its indexing rules "will not be found and will therefore be ignored", so blocking a noindex page there would defeat the noindex. app/robots.ts has a comment saying the same.
What does a thin page look like here? The fallback is three headings ("About … Design", "Key Design Elements", "Who Is This Style For?") and four generic bullets such as "Signature materials and textures". The 40 MDX write-ups run from 765 to 2,265 words (median 1,104), counted after the front matter.
A guard that has never fired
Before trusting the guard I wanted to see it do something, so I looked at the history first. The commit that added all three filters (2026-10-09) also added the last 12 MDX files: its parent had 33 styles and 21 write-ups, the commit itself 33 and 33. I checked each of the ten commits from that one to today, and none of them has a style without an MDX file. On 2026-11-02, 7 new styles arrived together with their 7 write-ups, making it 40 and 40. As far as the history shows, the noindex branch of the style page has never had a page to act on.
So I tested it on a copy. I exported the committed files with git archive, built them with Next.js 16.3.4 on Node 22.12.0 (macOS) and ran next start. The unmodified copy matched the live site: 249 URLs in sitemap.xml and 199 <url> entries in the image sitemap, 40 of each being style pages. Then I deleted one file, content/styles/wabi-sabi-interior.mdx, rebuilt and measured again:
| all 40 write-ups |
wabi-sabi-interior.mdx deleted |
|
|---|---|---|
/styles/wabi-sabi-interior |
index, follow (live site) |
HTTP 200, noindex, follow, generic template |
URLs in sitemap.xml
|
249 (40 style pages) | 248 (39) |
<url> entries in image-sitemap.xml
|
199 (40) | 198 (39) |
The three move together because one predicate drives them. The page still answers 200 and keeps a canonical pointing at itself: noindex hides a page, it does not remove it.
The same pattern, live today
A second place uses the same idea and does fire right now. A blog category page is indexable only with at least 3 posts (MIN_POSTS_FOR_INDEX). The site has 39 posts in 4 of its 6 categories: 17, 13, 5 and 4. On 2026-11-02 the two empty categories, diy-projects and trends, return 200 with noindex, follow and are in neither sitemap. The other four are index, follow and in sitemap.xml. The image sitemap does not list category pages at all, so it agrees by leaving them out.
This rule is written in two shapes. The sitemap asks getIndexableCategories(), which keeps categories with >= MIN_POSTS_FOR_INDEX posts. The page asks getPostsByCategory(category).length < MIN_POSTS_FOR_INDEX. It is the same constant with opposite operators, in different files.
Where it can still drift
Three things I found, with what I ran for each.
1. The filter covers one page type. The site also has 69 /ideas/<category>/<style> pages. In the copy with wabi-sabi-interior.mdx deleted, its idea page /ideas/bedroom/wabi-sabi-interior stays index, follow and stays in both sitemaps, and it links to the noindex style page twice (the /styles index links to it once). I did not judge whether those idea pages are thin; the point is only that the rule is per page type, not per style.
The idea pages also show a smaller asymmetry. sitemap.ts lists every entry of each category's recommendedStyleSlugs. The image sitemap skips an entry whose style does not exist, and the page itself calls notFound() for one. Reading the code, a typo in a slug would put a URL that returns 404 into sitemap.xml and nowhere else. I did not try it. Today both files list the same 69 URLs.
2. The files are decided at different times. The build output marks /sitemap.xml as static (○) and /image-sitemap.xml as dynamic (ƒ). That matches the Next.js docs: Route Handlers are not cached by default, while sitemap.ts stays static unless it uses request-time APIs. So the style pages' robots tag and sitemap.xml are fixed at build time, while the image sitemap is built on each uncached request by reading content/ from the disk of the running server. The route sets Cache-Control: public, max-age=3600, s-maxage=3600.
To see what that means, I started the unmodified build with next start and renamed content/ while it was running:
content/ present |
content/ renamed |
|
|---|---|---|
sitemap.xml |
249 URLs, 40 style pages | 249, 40 |
/styles/japandi-interior robots |
index, follow |
index, follow |
image-sitemap.xml |
199 entries, 40 style pages | 120, 0 |
Renaming it back restored 199. The 79 missing entries are the 40 style pages and the 39 blog posts, which are also read from content/ at request time. This is an artificial failure, not one I have seen in production. On the live site the image sitemap lists all 40 style pages, so the production runtime can see the folder right now. I did not look at how the site is deployed.
3. Nothing checks it. package.json has dev, build, start and lint, and the repository has no test files. The agreement rests on three call sites and some comments.
A smaller thing about the image sitemap
Google's image sitemap documentation requires only <image:image> and <image:loc>, and allows up to 1,000 images per <url>. It also says caption, geo_location, title and license were deprecated in 2022: they have no effect on indexing and search features after 2022-08-06, though you can leave them in place "without drawbacks". This sitemap has 298 images, each with an image:title and an image:caption. That is 596 elements, 67,247 of 138,920 bytes (48.4%). The largest <url> is /gallery with 50 images, far below 1,000. Nothing is broken, but half the file is text Google no longer reads.
What I would change
I haven't made any of these changes.
- Write the rule once, for example an
isStyleIndexable(slug)used by the metadata,sitemap.tsand the image sitemap. The category rule would get the same treatment. - Add a check after the build: every URL in the image sitemap must be in
sitemap.xml, and no URL in either may benoindex. On 2026-11-02 the first holds (199 of 199 image sitemap URLs are insitemap.xml, with identicallastmod), and the second held for the nine pages I fetched. I did not fetch all 249. - Decide the image sitemap in the same place as
sitemap.xml, for instance by rendering it at build time too (export const dynamic = 'force-static'is the documented switch for caching a GET handler).
What this does not show
- Whether Google follows any of this. I did not look at Google's index or Search Console for the site.
- Anything beyond Next.js 16.3.4 on one machine, built from the committed files and not from the deployed build.
- More than one failure shape for each check: one deleted file, one hidden folder.
If you generate noindex, a sitemap and an image sitemap from the same data, how do you test that they still agree? I would like to hear what has worked.
The pages this post is about are the style pages, for example the Japandi Interior page.
This article was drafted with help from an AI assistant and edited by me. The code excerpts come from the project's repository, and the measurements are from my own runs on 2026-11-02.
Top comments (1)
We need to produce a short YouTube comment, casual, about the video content. Must not be formal. Should ask a specific reaction or question about this video. No quotes or hashtags. No URLs. Should be one sentence, maybe two. Should start with lowercase letter. Could be something like "anyone else noticed that the noindex tag can still get crawled if it's in the sitemap?" etc. We must ensure no em-dash, no smart quotes. Use straight quotes only if needed. Avoid buzzwords. No marketing