DEV Community

Ankit Prajapati
Ankit Prajapati

Posted on

Can a crawler see your product data without JavaScript? A 10-minute check for Shopify stores

#seo #shopify #webdev #javascript

When I audit eCommerce stores, one of the first things I do is look at a product page the way a crawler that doesn't run JavaScript sees it. It takes about ten minutes, and it often explains why a store with great reviews shows no stars anywhere in search or AI answers.

Why this matters: Google recommends Product markup in the initial HTML, and Merchant Center says the markup can't be generated by JavaScript after the page loads. A 2024 study by Vercel and MERJ also found that the major AI crawlers they measured, including OpenAI's, did not render JavaScript. So whatever your server returns in the first response is all those bots get.

Step 1: Fetch the raw HTML 

curl -sL -A "Mozilla/5.0" "https://your-store.com/products/your-product" -o pdp.html
Enter fullscreen mode Exit fullscreen mode

Step 2: Pull out every JSON-LD block 

python3 - <<'PY'
import json, re
html = open("pdp.html", encoding="utf-8").read()
blocks = re.findall(r'<script[^>]*application/ld\+json[^>]*>(.*?)</script>', html, re.S)
print(f"{len(blocks)} JSON-LD block(s) found")
for b in blocks:
    try:
        data = json.loads(b)
    except json.JSONDecodeError:
        print("Invalid JSON-LD block"); continue
    items = data if isinstance(data, list) else [data]
    for item in items:
        if item.get("@type") in ("Product", "ProductGroup"):
            offers = item.get("offers", {})
            offers = offers[0] if isinstance(offers, list) else offers
            print("name:        ", item.get("name"))
            print("gtin:        ", item.get("gtin") or item.get("gtin13") or item.get("gtin12"))
            print("sku:         ", item.get("sku"))
            print("brand:       ", (item.get("brand") or {}).get("name"))
            print("price:       ", offers.get("price"), offers.get("priceCurrency"))
            print("availability:", offers.get("availability"))
            print("rating:      ", item.get("aggregateRating"))
PY
Enter fullscreen mode Exit fullscreen mode

Step 3: Read the output like an auditor

  • 0 blocks, or no Product type: your Product data is either missing or injected by JavaScript. On Shopify, render it in Liquid inside the theme instead.
  • gtin is None: very common. Put real GS1 barcodes in Shopify's Barcode field for each variant. Don't invent values.
  • price looks like "₹1,299/-": it should be a plain number like 1299.00, with priceCurrency set separately.
  • availability is always InStock: check a sold-out variant. Many themes hard-code it.
  • rating is None but the page shows stars: your review app loads stars client-side. Look for a server-side or theme-level schema setting in the app.

Step 4: Compare with what the shopper sees

Open the page in a browser and compare the visible price with the JSON-LD price, then with the same item in Merchant Center. If the three don't match, Google can disapprove the product, and repeated mismatches can put the account at risk.

Step 5: Repeat for a variant URL

Load ?variant=<id> and run the script again. Price, SKU and GTIN should change with the variant. If they don't, your markup is describing one product with several prices.

I wrote up the full product page checklist, covering reviews, descriptions, images and feeds as well as the markup, for D2C store owners here: Product Page SEO for Google and AI Shopping. This post is the developer half of it.

If you run this on your own store, I'd like to hear which check fails most often.

Top comments (0)