TL;DR
Shopify auto-generates hundreds of low-value URLs (filter params, pagination, collection-product variants). Google crawls those instead of your new collection pages. Three targeted fixes - robots.txt disallow, Search Console URL Parameters config, and canonical checks - can cut indexing time from two months down to two weeks.
The Problem in Plain Terms
You launch 40 new collection pages. Two months later, half still aren't indexed. Meanwhile, Googlebot has visited /collections/dining-tables?color=oak&sort_by=price-ascending&page=3 seventeen times.
That's a crawl budget problem.
Google allocates a finite number of URL visits per day to your store - based on server crawl rate limits and perceived page demand. On a Shopify store with 2,000 products, 200 collections, and active filter navigation, that budget fills up fast. If 300 of your 500 daily crawl visits go to parameter garbage, your real pages are competing for the remaining 200 slots.
Fix the waste, and new collection pages index in two weeks instead of two months. That delta is organic revenue sitting on the table - collection pages target category-level keywords with 1,000 to 50,000+ monthly searches and convert at two to three times the rate of product pages.
Step 1: Block Collection-Product URL Variants in robots.txt
This is the highest-impact fix and the simplest to implement.
Shopify gives every product a canonical URL at /products/product-handle. But whenever a shopper navigates to a product through a collection, Shopify also generates:
/collections/collection-handle/products/product-handle
A product in three collections has four crawlable URLs. On a store with 1,500 products across 80 collections, that's a massive crawl surface - all of it duplicate. Shopify adds canonical tags pointing back to /products/, but Googlebot still crawls the alternates before processing the canonical signal.
The fix: Edit your robots.txt (Online Store > Themes > Edit Default Theme Content, or via robots.txt.liquid) and add:
Disallow: /collections/*/products/
This eliminates the duplicate crawl path entirely. Products stay indexed through /products/ URLs. Collections stay indexed through /collections/ URLs. Nothing breaks.
After adding the rule: Check that no installed Shopify app is overwriting your robots.txt automatically. Some review, loyalty, or SEO apps do this. Verify the disallow is actually present in your live file at yourdomain.com/robots.txt.
Step 2: Configure Filter and Sort Parameters in Search Console
Shopify's collection filtering appends query strings to URLs:
/collections/dining-tables?sort_by=price-ascending/collections/dining-tables?color=oak/collections/dining-tables?color=oak&material=solid-wood&sort_by=best-selling
With eight filter options and three sort options, you're looking at hundreds of parameter combinations per collection - all pointing at the same underlying products, none worth indexing.
The fix: In your theme's robots.liquid file, add Disallow rules for the parameter patterns your filters and sort options generate — for example, disallowing paths that include ?sort_by= or ?filter.. Then make sure every filtered URL that shoppers can actually reach carries a canonical tag pointing back to the clean collection URL, so any that do get crawled pass their signals to the right page. For filter variants you want to keep accessible but out of the index entirely, add a noindex tag on top of the canonical. Together, these three layers — robots.txt disallows, canonicals, and noindex — give Googlebot a clear signal to spend its time on your base collection and product pages rather than endless parameter combinations.
Also check your theme's filter UI. If filtering generates <a href="/collections/dining-tables?color=oak"> anchor tags, Googlebot follows them as links. A developer can convert these to JavaScript history API state changes that update the URL bar without creating crawlable hrefs - eliminating the problem at the source.
Step 3: Handle Pagination Parameters the Same Way
Paginated collection pages - ?page=2, ?page=3 - contain products that also have their own /products/ pages. They rarely rank for anything meaningful, but they're crawlable by default and Googlebot visits them repeatedly.
The fix: In the same URL Parameters tool in Search Console, mark page as a parameter Google should not crawl separately. Base collection URLs stay indexed. Paginated variants are skipped.
Secondary check: inspect your paginated pages in source and confirm the <link rel="canonical"> tag points to the base collection URL (/collections/dining-tables), not a self-referencing URL. If paginated pages self-canonicalize, Google treats them as independent and may crawl them anyway. Some Shopify themes get this right automatically; many don't.
How to Audit Before and After
Before implementing fixes, get a baseline:
- Search Console Coverage Report - Filter for "Crawled - currently not indexed." A large volume here signals crawl strain. Google is discovering pages faster than it's indexing them.
- Search Console URL Inspection - Pull up specific new collection pages. Check last crawl date. If Googlebot visited once weeks ago and never returned, crawl waste is the likely culprit.
- Screaming Frog - Run a crawl with JS rendering enabled. Sort by URL. If parameter-appended URLs make up more than 20 - 30% of your crawlable inventory, this is an immediate priority.
After implementing all three fixes, monitor Search Console Coverage weekly for 30 days. Watch for a drop in "Crawled - currently not indexed" URLs and faster indexing of newly published pages.
Implementation Priority Order
Work through these in sequence:
-
robots.txt- block/collections/*/products/(biggest volume reduction, zero downside) - URL Parameters - configure filter and sort params in Search Console
- URL Parameters - configure the
pageparam - Canonical audit - verify pagination canonicals point to base collection URLs
- App audit - confirm no installed app is overriding your
robots.txtrules - Monitor Coverage weekly for 30 days post-implementation
What This Won't Fix
Getting Googlebot to your collection pages faster only helps if those pages are worth indexing. If your collections have auto-generated thin copy, no H1, and no targeted keyword in the title tag, fixing crawl budget gets Google there faster to find content that still won't rank.
The technical fix and on-page work have to run together. But for stores that have done the on-page work and are still watching new pages sit unindexed for weeks - crawl budget is almost always the explanation.
For a deeper look at how this fits into a full Shopify SEO strategy, visit New Seas - an SEO and content agency focused specifically on Shopify and ecommerce brands.
Frequently Asked Questions
What is crawl budget and why does it matter for my Shopify store?
Google limits how many URLs it visits on your site each day, and that limit is your crawl budget. On a Shopify store with a large product catalog and active filter navigation, auto-generated URLs from filters, sort options, and collection-product paths can consume the majority of those daily visits, leaving your actual collection pages waiting weeks or months to be indexed.
Why are my new Shopify collection pages taking so long to show up in Google?
When Shopify generates hundreds of duplicate or low-value URLs — such as filtered collection pages and collection-scoped product URLs — Googlebot spends its crawl visits on those instead of your new pages. The fix is blocking that URL waste through robots.txt disallow rules, Search Console parameter configuration, and correct canonical tags, which frees up crawl visits for the pages you actually want indexed.
How do I stop Googlebot from crawling duplicate product URLs in Shopify?
Every product accessed through a collection gets its own crawlable URL at /collections/collection-handle/products/product-handle, on top of the canonical /products/product-handle URL. Adding Disallow: /collections/*/products/ to your robots.txt eliminates that duplicate crawl path entirely, while leaving both your product and collection pages indexed through their proper URLs.
Do Shopify filter and sort parameters hurt my crawl budget?
Yes. Each combination of filter options and sort settings creates a separate crawlable URL pointing at the same underlying products. With multiple filter options per collection, this can produce hundreds of parameter combinations per collection. Blocking these patterns in robots.txt, adding canonical tags pointing to the clean collection URL, and applying noindex tags where needed gives Googlebot a clear signal to skip them.
How do I know if crawl budget waste is actually the problem on my store?
In Search Console, check the Coverage Report filtered for "Crawled – currently not indexed" — a large volume there points to crawl strain. You can also use URL Inspection to check when Googlebot last visited a specific new collection page. If the last crawl was weeks ago with no return visit, crawl waste is the likely cause.
Will fixing crawl budget issues automatically get my collection pages to rank?
Not on its own. Getting Googlebot to your collection pages faster only helps if those pages have solid on-page content — a targeted title tag, an H1, and meaningful copy. If the pages are thin or missing key on-page signals, resolving crawl budget gets Google there sooner to find content that still won't rank. The technical and on-page work need to happen together.
Top comments (0)