Can a crawler see your product data without JavaScript? A 10-minute check for Shopify stores

Ankit Prajapati — Sat, 31 Oct 2026 13:38:17 +0000

When I audit eCommerce stores, one of the first things I do is look at a product page the way a crawler that doesn't run JavaScript sees it. It takes about ten minutes, and it often explains why a store with great reviews shows no stars anywhere in search or AI answers.

Why this matters: Google recommends Product markup in the initial HTML, and Merchant Center says the markup can't be generated by JavaScript after the page loads. A 2024 study by Vercel and MERJ also found that the major AI crawlers they measured, including OpenAI's, did not render JavaScript. So whatever your server returns in the first response is all those bots get.

Step 1: Fetch the raw HTML

curl -sL -A "Mozilla/5.0" "https://your-store.com/products/your-product" -o pdp.html

Step 2: Pull out every JSON-LD block

python3 - <<'PY'
import json, re
html = open("pdp.html", encoding="utf-8").read()
blocks = re.findall(r'<script[^>]*application/ld\+json[^>]*>(.*?)</script>', html, re.S)
print(f"{len(blocks)} JSON-LD block(s) found")
for b in blocks:
    try:
        data = json.loads(b)
    except json.JSONDecodeError:
        print("Invalid JSON-LD block"); continue
    items = data if isinstance(data, list) else [data]
    for item in items:
        if item.get("@type") in ("Product", "ProductGroup"):
            offers = item.get("offers", {})
            offers = offers[0] if isinstance(offers, list) else offers
            print("name:        ", item.get("name"))
            print("gtin:        ", item.get("gtin") or item.get("gtin13") or item.get("gtin12"))
            print("sku:         ", item.get("sku"))
            print("brand:       ", (item.get("brand") or {}).get("name"))
            print("price:       ", offers.get("price"), offers.get("priceCurrency"))
            print("availability:", offers.get("availability"))
            print("rating:      ", item.get("aggregateRating"))
PY

Step 3: Read the output like an auditor

0 blocks, or no Product type: your Product data is either missing or injected by JavaScript. On Shopify, render it in Liquid inside the theme instead.
gtin is None: very common. Put real GS1 barcodes in Shopify's Barcode field for each variant. Don't invent values.
price looks like "₹1,299/-": it should be a plain number like 1299.00, with priceCurrency set separately.
availability is always InStock: check a sold-out variant. Many themes hard-code it.
rating is None but the page shows stars: your review app loads stars client-side. Look for a server-side or theme-level schema setting in the app.

Step 4: Compare with what the shopper sees

Open the page in a browser and compare the visible price with the JSON-LD price, then with the same item in Merchant Center. If the three don't match, Google can disapprove the product, and repeated mismatches can put the account at risk.

Step 5: Repeat for a variant URL

Load ?variant=<id> and run the script again. Price, SKU and GTIN should change with the variant. If they don't, your markup is describing one product with several prices.

I wrote up the full product page checklist, covering reviews, descriptions, images and feeds as well as the markup, for D2C store owners here: Product Page SEO for Google and AI Shopping. This post is the developer half of it.

If you run this on your own store, I'd like to hear which check fails most often.

Measuring AI Search Visibility Like a Developer: GA4 Segments, Prompt Suites and Tracker APIs

Ankit Prajapati — Sat, 24 Oct 2026 19:25:58 +0000

If you build or maintain sites for clients, sooner or later someone asks: "Are we showing up in ChatGPT?" Here's a lightweight, data-first way to answer that, plus what to look for if you want the numbers in your own dashboards.

Step 1: Segment AI referral traffic in GA4

AI trackers report mentions, but they don't report visits. Visits live in GA4. Create a custom segment or exploration that filters session source on AI assistant referrers such as:

chat.openai.com
perplexity.ai
gemini.google.com

Save it as "AI referral traffic" and watch the trend month over month. It's free, and it gives you the outcome metric that sits alongside any mention data.

Step 2: Build a prompt set like a test suite

Treat prompts like test cases. Write 15 to 25 conversational prompts in three groups:

Direct comparison: "X vs Y for small teams"
Category recommendation: "best X for agencies"
Problem-based: "how do I fix X"

Run them against ChatGPT, Gemini, Perplexity and Google AI Overviews or AI Mode. AI output varies between runs, so a single run is noisy. Repeat on a schedule and compare rates, not one-off results.

Step 3: If you buy a tracker, check the data path first

For developers, export and API matter more than the dashboard. From the tools I compared:

Need	Options
Full API + CSV/JSON export	Profound (Enterprise)
API + CSV at a low price	LLMrefs ($79/month, 500 prompts)
API on a lightweight tool	AmIOnAI
CSV only	Semrush (10 CSV exports a day)

Also ask how the tool collects data, whether through live APIs or simulated browser sessions, because that affects how closely it matches what real users see.

Step 4: Validate before trusting

Run 5 to 10 prompts manually, then run the same prompts in two trial tools. If a tool disagrees with your manual results on most prompts, don't pipe its data anywhere.

I've written up the full comparison, including engine coverage and prompt limits for each tool, in my guide to AI visibility trackers with API and export options.

Written by Ankit Prajapati, SEO & GEO consultant.

DEV Community: Ankit Prajapati