One React component cost me 98 unindexed pages

Google had indexed 21 pages of my site. Another 98 sat in Search Console under Discovered - currently not indexed, and the number was climbing.

I spent a week on the wrong explanations. Thin content, maybe. A new domain with no authority. Too many similar pages. All plausible, all wrong.

The cause was four characters in a React component.

The Symptom

-	Reason	Pages
-	Discovered — currently not indexed	98
-	Page with redirect	7
-	Alternate page with canonical tag	6
-	Excluded by noindex tag	2
-	Duplicate without user-selected canonical	1
-	Crawled — currently not indexed	1

Everything except that first row is benign. The redirects and canonicals are working as intended. The 98 are the problem, and the label is precise in a way that’s easy to skim past.

Discovered means Google knows the URL exists. It found it in my sitemap.
Currently not indexed means it chose not to spend crawl budget fetching it.

Google has not read those pages, so this isn’t a verdict on the content. It decided they weren’t worth fetching, and the strongest signal I can control there is internal links.

The Cause

The piece of code that

{categories.map(({ label, tools }) => {
  const isOpen = open === label;
  return (
    <div key={label}>
      <button onClick={() => setOpen(isOpen ? null : label)}>
        {label}
      </button>

      {isOpen && (
        <nav>
          {tools.map((tool) => (
            <Link key={tool.slug} href={`/${tool.slug}`}>{tool.name}</Link>
          ))}
        </nav>
      )}
    </div>
  );
})}

{isOpen && <nav>}

If the category is closed, the nav is not hidden. It does not exist. React never creates the element, so the links are not in the DOM, so they are not in the HTML a crawler receives.

One category is open by default. Five links. The other 25 tool pages had zero links from the homepage - the only page on the site with any external authority at all.

I confirmed it the way I should have started, by reading what is actually served rather than what the browser shows:

curl -s https://papyrio.net/ | grep -c 'href="/[a-z-]*-pdf"'

How I solved this problem

<nav hidden={!isOpen}>
  {tools.map((tool) => (
    <Link key={tool.slug} href={`/${tool.slug}`}>{tool.name}</Link>
  ))}
</nav>

hidden is an HTML attribute. The element renders, the links are in the markup, the browser does not paint it. Crawlers read markup.

Add aria-expanded on the button while you are there, so the state is exposed to assistive technology rather than implied by what is missing.

The result

-		Before	After
-	Tool links on the homepage	5	30
-	Tool pages with no internal links	3	0
-	Internal links site-wide	42	722

Why this is easy to miss

I found these 3 hidden areas

Everything worked. Not a bug. No error, no warning, no failing test. The
accordion behaved exactly as designed in every browser I opened.

My tooling agreed. Lighthouse was green. The pages had titles, meta
descriptions, canonical tags, hreflang, structured data. Every SEO checklist
item I could tick was ticked. None of them check whether the page is reachable.

The browser lied to me. Inspecting the homepage in DevTools, I saw thirty
links, because interacting with a page is how you end up looking at it. The
crawler never clicks anything.

My next analyze

1. curl your pages and count the links. Not DevTools. Not "view source" after you have clicked around. A cold fetch, the way a crawler sees it.
2. Grep for && before JSX that contains a Link. Conditional rendering and conditional visibility are different things, and the first one deletes content from the document.
3. Check tabs, modals, carousels and "load more" buttons. Same pattern, same consequence. Anything that mounts on interaction is invisible to a crawler.
4. Read the Search Console reason strings literally. "Discovered" and "Crawled" are different diagnoses. The first is a linking problem, the second is a content problem. I treated a linking problem as a content problem for a week.

I still annoy on this part

The actual fix was {isOpen && <nav>} to <nav hidden={!isOpen}>

By the way I forgot to mention the product. papyrio

Thanks for reading !!!