TL;DR
Shopify auto-generates hundreds of low-value URLs (filter params, pagination, collection-product variants). Google crawls those instead of your new collection pages. Three targeted fixes - robots.txt disallow, Search Console URL Parameters config, and canonical checks - can cut indexing time from two months down to two weeks.
The Problem in Plain Terms
You launch 40 new collection pages. Two months later, half still aren't indexed. Meanwhile, Googlebot has visited /collections/dining-tables?color=oak&sort_by=price-ascending&page=3 seventeen times.
That's a crawl budget problem.
Google allocates a finite number of URL visits per day to your store - based on server crawl rate limits and perceived page demand. On a Shopify store with 2,000 products, 200 collections, and active filter navigation, that budget fills up fast. If 300 of your 500 daily crawl visits go to parameter garbage, your real pages are competing for the remaining 200 slots.
Fix the waste, and new collection pages index in two weeks instead of two months. That delta is organic revenue sitting on the table - collection pages target category-level keywords with 1,000 to 50,000+ monthly searches and convert at two to three times the rate of product pages.
Step 1: Block Collection-Product URL Variants in robots.txt
This is the highest-impact fix and the simplest to implement.
Shopify gives every product a canonical URL at /products/product-handle. But whenever a shopper navigates to a product through a collection, Shopify also generates:
/collections/collection-handle/products/product-handle
A product in three collections has four crawlable URLs. On a store with 1,500 products across 80 collections, that's a massive crawl surface - all of it duplicate. Shopify adds canonical tags pointing back to /products/, but Googlebot still crawls the alternates before processing the canonical signal.
The fix: Edit your robots.txt (Online Store > Themes > Edit Default Theme Content, or via robots.txt.liquid) and add:
Disallow: /collections/*/products/
This eliminates the duplicate crawl path entirely. Products stay indexed through /products/ URLs. Collections stay indexed through /collections/ URLs. Nothing breaks.
After adding the rule: Check that no installed Shopify app is overwriting your robots.txt automatically. Some review, loyalty, or SEO apps do this. Verify the disallow is actually present in your live file at yourdomain.com/robots.txt.
Step 2: Configure Filter and Sort Parameters in Search Console
Shopify's collection filtering appends query strings to URLs:
/collections/dining-tables?sort_by=price-ascending/collections/dining-tables?color=oak/collections/dining-tables?color=oak&material=solid-wood&sort_by=best-selling
With eight filter options and three sort options, you're looking at hundreds of parameter combinations per collection - all pointing at the same underlying products, none worth indexing.
The fix: In Google Search Console, open the URL Parameters tool (Legacy Tools > URL Parameters). For each filter and sort parameter your store uses, set the behavior to "No: don't crawl URLs with this parameter." This tells Googlebot to skip variants and focus on base collection URLs.
Also check your theme's filter UI. If filtering generates <a href="/collections/dining-tables?color=oak"> anchor tags, Googlebot follows them as links. A developer can convert these to JavaScript history API state changes that update the URL bar without creating crawlable hrefs - eliminating the problem at the source.
Step 3: Handle Pagination Parameters the Same Way
Paginated collection pages - ?page=2, ?page=3 - contain products that also have their own /products/ pages. They rarely rank for anything meaningful, but they're crawlable by default and Googlebot visits them repeatedly.
The fix: In the same URL Parameters tool in Search Console, mark page as a parameter Google should not crawl separately. Base collection URLs stay indexed. Paginated variants are skipped.
Secondary check: inspect your paginated pages in source and confirm the <link rel="canonical"> tag points to the base collection URL (/collections/dining-tables), not a self-referencing URL. If paginated pages self-canonicalize, Google treats them as independent and may crawl them anyway. Some Shopify themes get this right automatically; many don't.
How to Audit Before and After
Before implementing fixes, get a baseline:
- Search Console Coverage Report - Filter for "Crawled - currently not indexed." A large volume here signals crawl strain. Google is discovering pages faster than it's indexing them.
- Search Console URL Inspection - Pull up specific new collection pages. Check last crawl date. If Googlebot visited once weeks ago and never returned, crawl waste is the likely culprit.
- Screaming Frog - Run a crawl with JS rendering enabled. Sort by URL. If parameter-appended URLs make up more than 20 - 30% of your crawlable inventory, this is an immediate priority.
After implementing all three fixes, monitor Search Console Coverage weekly for 30 days. Watch for a drop in "Crawled - currently not indexed" URLs and faster indexing of newly published pages.
Implementation Priority Order
Work through these in sequence:
-
robots.txt- block/collections/*/products/(biggest volume reduction, zero downside) - URL Parameters - configure filter and sort params in Search Console
- URL Parameters - configure the
pageparam - Canonical audit - verify pagination canonicals point to base collection URLs
- App audit - confirm no installed app is overriding your
robots.txtrules - Monitor Coverage weekly for 30 days post-implementation
What This Won't Fix
Getting Googlebot to your collection pages faster only helps if those pages are worth indexing. If your collections have auto-generated thin copy, no H1, and no targeted keyword in the title tag, fixing crawl budget gets Google there faster to find content that still won't rank.
The technical fix and on-page work have to run together. But for stores that have done the on-page work and are still watching new pages sit unindexed for weeks - crawl budget is almost always the explanation.
For a deeper look at how this fits into a full Shopify SEO strategy, visit New Seas - an SEO and content agency focused specifically on Shopify and ecommerce brands.
Top comments (0)