DEV Community

Max
Max

Posted on Originally published at storecanary.io

Shopify 'crawled not indexed': triage guide for agency devs

#shopify #webdev #seo #tutorial

TL;DR

  • Most URLs in the "Crawled - currently not indexed" report on a Shopify store are expected duplicates (collection-scoped product URLs, ?variant= parameters); their absence from the index is consolidation, not a bug.
  • A real /products/handle URL that survives triage usually has one of three causes: a canonical tag pointing elsewhere, thin content shared across many stores, or broken Product structured data.
  • The fix starts with filtering the export, not reading the raw count.

When you open Search Console for a client's Shopify store and see dozens of product pages sitting under "Crawled - currently not indexed," the immediate question is: platform quirk or actual indexing failure? The distinction matters because Shopify's URL architecture generates legitimate duplicates by default, and most of what fills this report falls into that category.

Google's own reference defines this status in one sentence: "The page was crawled by Google but not indexed." Compare it with the neighboring statuses. "URL blocked by robots.txt" is an access refusal you can switch off. "URL marked 'noindex'" is an explicit directive you can remove. This one is a judgment about the page's content and signals, and there is no switch for it.

This article gives you the triage process: which URL shapes to filter out immediately, the three root causes worth investigating on surviving URLs, and a five-step inspection checklist.

Why does this report fill up with URLs that aren't the problem?

A Shopify storefront generates far more URLs than it has products. Before reading anything into the count, export the report and filter to URLs containing /products/. Everything else is a separate question.

Two shapes in particular tend to dominate this list, and both are duplicates by design:

Collection-scoped product URLs. Shopify serves the same product at /products/handle and again at /collections/name/products/handle. Shopify's own documentation for the within filter says it directly: "Because a standard product page and a product page in the context of a collection have the same content on separate URLs, you should consider the SEO implications of using the within filter." When Google finds several URLs with identical content, it picks one canonical and leaves the others out. That is expected consolidation.

Variant parameters. A ?variant=12345 URL is the same product page with a different option preselected. There is nothing distinct to index.

If the entire list is these two shapes plus non-product URLs, the right action is none.

What causes a real /products/ URL to stay unindexed?

Once you have filtered the list and real product URLs remain, there are three causes to check in order.

Does the canonical point somewhere else?

This is the one that looks like a content problem but is mechanical. If the page's <link rel="canonical"> points to a URL other than itself, Google treats the page as a duplicate. Run the URL through URL Inspection in Search Console and compare the User-declared canonical with the Google-selected canonical. If they differ, that is your answer. The canonical article covers why the collection URL often wins this comparison.

Does the page have unique content?

A product page carrying a supplier description shared verbatim across hundreds of stores gives Google little reason to add another copy to the index. This rarely has a code fix; it is frequently the real answer on dropshipped or catalog-imported stores. Sizing detail, materials, genuine reviews, and shipping specifics are the content that changes the calculation.

Is the structured data parsing correctly?

Worth eliminating before concluding anything about content quality. If the Product block has a syntax error, the whole declaration is invisible to Google, usually with no visible symptom on the page itself. The unparsable structured data article shows how a single stray character silently discards an entire block.

How do you triage this in five minutes?

  1. Open Search Console > Indexing > Pages, click Crawled - currently not indexed, and export the list.
  2. Filter to URLs containing /products/, then remove anything containing /collections/ or ?variant=. What remains is the list that matters.
  3. Take one surviving URL and run URL Inspection on it. Compare User-declared canonical with Google-selected canonical. If they differ, you have the cause.
  4. If the canonicals agree, open the page and assess what content exists there that does not exist anywhere else. If the honest answer is nothing, that is the finding.
  5. Run the same URL through the Rich Results Test to confirm the Product structured data still parses.

The pattern behind most of these silent failures: the Shopify admin shows a healthy, active, priced product, while the public URL tells Google something different. This status is a per-URL decision, and that decision usually has a mechanical cause one layer below it.

Related:

Have you hit this on a client store? What was the root cause when you dug in?

The full version of this article - with screenshots and ongoing updates - lives on the StoreCanary blog.

Top comments (0)