Deindexing is the quietest way to lose traffic. Nothing 404s, nothing looks broken - pages just slip out of Google's index and the rankings go with them. We run 10+ web products and these are the eight causes we see over and over. Each one is invisible until you go looking.
1. A stray noindex on a shared template
The classic. Someone adds <meta name="robots" content="noindex"> to a layout or component to hide a staging page, and it ships to a template used by hundreds of pages. Whole sections vanish from search.
Catch it: grep your built output for noindex. It should only appear where you genuinely want it (thank-you pages, internal search results). Anything else is a leak.
2. Canonical tags pointing to the wrong URL
If page A says <link rel="canonical" href="B">, you're telling Google "index B instead of me." Get this wrong across a template - e.g. every paginated page canonicalising to page 1 - and you deindex pages 2+.
Catch it: every page should self-reference its canonical unless you have a deliberate reason not to. Watch especially for canonicals pointing to a redirect or a 404 - Google ignores those and picks its own, unpredictably.
3. robots.txt blocking what you want indexed
A Disallow: rule meant for one folder can match more than you think. And blocking a URL in robots.txt doesn't deindex it - it just stops Google re-crawling, so a page can stay indexed with no snippet, or get stuck.
Catch it: test your important paths against your robots.txt rules. Make sure your sitemap is declared with a Sitemap: line too.
4. Non-canonical or redirected URLs in your sitemap
Your sitemap is a list of "please index these." If it's full of redirected URLs, non-canonical variants, or 404s, you send mixed signals and waste crawl budget.
Catch it: your sitemap should contain only canonical, indexable, 200-status URLs. Audit it.
5. Soft 404s
A page returns HTTP 200 but has no real content ("No results found", an empty category). Google calls these soft 404s and quietly drops them - or worse, loses trust in that section.
Catch it: thin/empty pages that return 200 are the tell. Either add content, noindex them deliberately, or return a real 404.
6. Redirect chains and loops
A -> B -> C -> D wastes crawl budget and dilutes signals at every hop. A redirect loop (A -> B -> A) is worse - Google gives up.
Catch it: collapse every chain to a single hop. A -> D, done.
7. Canonical-to-redirect
A subtle one: page A canonicalises to B, but B redirects to C. Now your canonical signal is broken and Google has to guess. This commonly happens after an HTTP-to-HTTPS or trailing-slash migration where canonicals weren't updated.
Catch it: make sure every canonical target returns 200 directly, not a redirect.
8. Orphan pages
A page with zero internal links pointing to it. Google can technically find it via the sitemap, but with no internal links it reads as low-priority and often won't get or keep an index slot.
Catch it: build your internal link graph and find pages with zero incoming links. One warning from experience: if your crawler times out while resolving the link graph, it will report every page as an orphan. Make sure the incoming-link count actually computed before you trust it - a false "everything is orphaned" result has sent people chasing ghosts.
How to monitor this without a $100/mo suite
- Google Search Console -> Pages report is the source of truth: it tells you exactly what's indexed and, for everything that isn't, why. Free. Check it weekly.
- Screaming Frog free tier crawls 500 URLs and surfaces most of the above for small sites.
- For site-wide crawls without the subscription, we built our own (Kalenux SEO Audit) - it flags noindex, canonical errors, soft 404s, redirect chains, sitemap mismatches and orphans across the whole site, one-time pricing instead of a monthly bill. It's the technical-audit piece only, not a keyword tool.
Deindexing is sneaky precisely because nothing looks broken. Set up the monitoring, run a crawl after every deploy that touches templates or redirects, and you'll catch these before they cost you a quarter of rankings.
Top comments (0)