DEV Community

Daniel Pertu
Daniel Pertu

Posted on

A sitemap is not internal linking: the index page our 373 generated pages needed

Munchable has 373 pages that live at the root of the site with slugs like /is-onion-low-fodmap, /does-caffeine-cause-reflux and /is-butter-high-in-lactose. Each one answers exactly one question about one ingredient and one gut condition, and none of the answers were written by hand: the page builds a one-ingredient product and runs the app's own rules engine on it at build time.

Generating them was the easy half. The half that took the real thinking was making sure a crawler would ever reach them, and that they would not quietly turn every typo on the domain into an indexable page.

An XML sitemap says a page exists, not that it matters

If the only path to a URL is sitemap.xml, you have told the crawler the page exists and nothing else. No anchor text, no context, no position in a hierarchy, no signal about which pages matter more. Pages in that state get crawled slowly and treated as isolated, which is roughly what you would expect from a page nothing on the site links to.

So the 373 answers needed a real page linking to them, and it had to be grouped the way a reader thinks rather than the way the array is ordered. That is the answers index, and its top comment is the whole justification:

The index for the answer pages. Its job is reachability: pages that are only
linked from a sitemap get crawled slowly and ranked poorly, so they need a real
page linking to them from inside the site, grouped the way a reader thinks.
Enter fullscreen mode Exit fullscreen mode

Alphabetical across all 373 would have been the default and it would have been useless. A reader has one condition. The useful grouping is by condition, so each block is the shortlist of what to watch for with that condition:

const groups = CONDITION_PAGES.map((guide) => ({
  guide,
  questions: ANSWER_PAGES.filter((p) => p.condition === guide.condition)
    .map((p) => ({ page: p, entry: INGREDIENTS_BY_SLUG.get(p.ingredientSlug) }))
    .sort((a, b) => a.entry.name.localeCompare(b.entry.name)),
})).filter((g) => g.questions.length > 0);
Enter fullscreen mode Exit fullscreen mode

Each group heading is itself a link to that condition's guide, so the index is not a dead end in the other direction either. The hub links down to the answers, the answers link back to the hub.

The linking that matters most is lateral

An index page is one hop. What actually makes 373 pages feel like a site rather than a pile is that each answer page links along both axes it sits on.

Down one axis: the same ingredient, the other conditions that have something to say about it. Somebody reading about onion and IBS is often about to ask about onion and something else.

export function siblingConditions(entry: IngredientEntry, exclude: ConditionId): ConditionPage[] {
  return ingredientFacts(entry)
    .filter((f) => f.condition !== exclude)
    .map((f) => CONDITION_PAGES.find((p) => p.condition === f.condition))
    .filter((p): p is ConditionPage => Boolean(p));
}
Enter fullscreen mode Exit fullscreen mode

Down the other: other ingredients flagged for the same condition. The comment on that one is a copy decision I would defend anywhere:

Deliberately not phrased as substitutes. The engine reasons about what is printed on a label, not about cooking, so it cannot honestly say that one ingredient replaces another in a recipe. What it can say is which other ingredients trip the same rule set, which is the list a reader actually needs while holding a packet.

"You might also like" would have been easy and slightly false. "Also flagged for this condition" is true, and it is the list somebody standing in a shop wants.

A dynamic segment at the site root will answer everything

This is the Next.js detail I would most want someone else to steal, because getting it wrong is silent.

The answer pages are a single route, app/[question]/page.tsx. A dynamic segment directly under app/ matches any unmatched top-level path. Without a guard, /pricng, /abuot and every hallucinated URL in the wild would hit this route. The component would call notFound(), but only after Next decided that this route owns the path, and in a dynamically rendered setup that is the shape that turns typos into soft 404s that crawlers happily index.

Two lines prevent it:

export const dynamic = 'error';
export const dynamicParams = false;

export function generateStaticParams() {
  return ANSWER_PAGES.map((page) => ({ question: page.question }));
}
Enter fullscreen mode Exit fullscreen mode

dynamicParams = false means the only paths this route owns are the ones generateStaticParams returned. Everything else is a genuine 404 from the router. dynamic = 'error' turns any accidental dynamic API into a build failure rather than a per-request render, which is what you want for a page whose content is fixed at build time anyway.

If you have a root-level dynamic segment and you have not set dynamicParams = false, go and check what your site returns for a misspelling right now.

One table decides the slug, the heading, the title tag and the FAQ entry

The phrase people type is different for each condition. Nobody searches "is onion OK with IBS"; they search "is onion low FODMAP". For lactose it is "high in lactose", for reflux it is "does coffee cause reflux". Forcing all seven conditions through one template would have produced 373 pages that match no real query.

So phrasing is a table, one entry per condition, and it is the single source for every surface the words appear on:

export const QUESTION_PHRASING: Record<ConditionId, {
  slug: (ingredient: string) => string;
  ask: (name: string) => string;
  flagMeansYes: boolean;
  caution: string;
}> = {
  ibs: {
    slug: (i) => `is-${i}-low-fodmap`,
    ask: (n) => `Is ${n} low FODMAP?`,
    flagMeansYes: false,
    caution: 'It depends on how much',
  },
  // ...
};
Enter fullscreen mode Exit fullscreen mode

The slug, the h1, the <title>, and the question in the FAQ structured data all come from that one entry, so they cannot drift apart. flagMeansYes exists because the polarity flips between conditions: for IBS, being flagged means "no, it is not low FODMAP", while for lactose, being flagged means "yes, it is high in lactose". Same verdict, opposite word at the top of the page.

There is also a lowercasing helper that looks trivial and is not:

export function lowerName(name: string): string {
  return /^[A-Z][a-z]/.test(name) ? name.toLowerCase() : name;
}
Enter fullscreen mode Exit fullscreen mode

A plain .toLowerCase() turns "E420" into "e420" and "FOS" into "fos" halfway through a sentence. Only names that look like ordinary capitalised words get lowercased.

Most possible pages do not get made

The ingredient index times seven conditions is a much larger number than 373. The rule is that a pair only gets a page when a rule set has something to say about it:

Pairs with nothing to say get no page: a page that says "onion is not relevant to bile acid malabsorption" is thin content, and it is not what anybody searched for either.

That one line of restraint is the difference between a useful set of pages and the kind of programmatic output that gets a site classified as doorway pages. The generator is capable of the full cross product. It is told not to.

The count is also deliberately not written down in a comment anywhere in the source, because it moves whenever a rule map grows, and a stale number in a comment is a lie nobody notices. I had to count the sitemap to write the opening line of this post.

The snippet is compressed separately from the page

The page leads with its answer, in the page's own voice. The meta description leads with the same answer in fewer words, because a search snippet gets about 160 characters and the on-page phrasing spends a third of that before the ingredient is even named.

You can see both on the onion page. View source and the description is:

No. Onion is an avoid as a main ingredient and is a caution lower down the label. Scan a barcode to check a real product.

The page itself says the same thing at greater length, with the engine's own reason line quoted underneath, the one a scan of a real product would show. Two wordings, one source of truth, neither hand-typed.

That last part is the rule the whole set hangs on: no page restates a rule in prose. Every page runs fitCheck and prints what came back. When a rule set changes, 373 pages change with it in the next build and nobody edits any copy.

Go and look

  • The answers index is the reachability page. Note that it is grouped by condition and that every heading links to a guide.
  • Is onion low FODMAP? is a typical page. Follow the sibling links at the bottom to see the lateral linking in both directions.
  • Does caffeine cause reflux? and Is butter high in lactose? are the same generator with different phrasing, which is the point of the phrasing table.
  • Try a URL that does not exist, such as munchable.app/is-not-a-real-thing, and you should get a real 404 rather than a page that pretends to be about it.

Top comments (0)