DEV Community: Will

I checked which tools AI search recommends across 9 SaaS categories — most brands are invisible

Will — Wed, 15 Jul 2026 21:11:10 +0000

When someone asks ChatGPT, Perplexity, or Gemini "what's the best tool for X?", they don't get ten blue links. They get one answer — a short list of names. If your product isn't in it, that buyer never learns you exist. It's the new "not ranking on Google," except there's no page 2 to scroll to.

I wanted to know how bad it actually is, so I ran the real buyer questions for 9 SaaS categories against AI search and tallied which brands actually get named.

What I did

For each category I generated the questions a real buyer would ask ("best CRM for a small team", "top project management tool for agencies", and so on), sent them through AI search (Perplexity as the first engine), and counted:

every brand named across all the answers,
which brand was the single top pick, and
the source domains the AI pulled its answer from.

Categories: CRM, project management, email marketing, AI writing, help desk, website builders, password managers, video conferencing, and note-taking apps.

What surprised me

1. Most brands are simply invisible. In nearly every category the same 3-4 incumbents soak up almost all the recommendations. If you are not one of them, you are not in the answer at all — not ranked low, absent.

2. A few small tools quietly win their niche. In some categories a smaller name grabs the top-pick slot — and when you look at why, it is almost always because a handful of specific sources (a popular Reddit thread, one widely-cited listicle, their own comparison page) get pulled again and again.

3. It is not fixed — it moves. Re-run the same questions a week later and the ranking shifts. That means it is trackable, and it means it is influenceable.

The leaderboards

I put the per-category rankings up here (who AI names most, plus the sources it reads from):

https://willylam2222-bot.github.io/cited/leaderboards.html

If you want to sanity-check your own category, look it up — and if your product is not on the board, that gap is exactly the thing worth fixing.

Curious whether folks here have checked where their product stands in AI answers. Drop your brand + category in the comments and I will run it.

I scraped 30 DTC stores and built a free tool to check where your product price ranks

Will — Sun, 12 Jul 2026 07:40:40 +0000

DTC founders constantly ask "am I priced right?" — so I scraped ~30 real Shopify stores across 7 categories, pulled 10,000+ live prices, and turned it into a free tool: type in your price, see your percentile vs the actual market.

Try it: https://willylam2222-bot.github.io/priceprobe/price-rank-checker.html
(no signup, runs entirely in your browser)

The finding that surprised me

The share of products on sale right now varies wildly by category:

🍵 Loose-leaf tea: 1.8% on sale
☕ Specialty coffee: 5.8%
💊 Supplements: 22.4%

Supplements discount ~12x more often than tea — and cost 2.5x more. Prices there move constantly.

Full data + methodology: https://willylam2222-bot.github.io/priceprobe/dtc-pricing-index-2026.html

How it's built (all free, no backend)

Zero-dependency Python (stdlib urllib only) hits each store's public /products.json endpoint. Shopify serves the entire catalog + prices as JSON — no API key, no scraping library.
The percentile math + the whole tool is client-side JS on GitHub Pages. Nothing stored, no server.

Code + the scraping write-up: https://github.com/willylam2222-bot/priceprobe

Would love feedback — which categories should I add next?

Every Shopify store exposes these 4 public URLs — here's what a competitor sees in 30 seconds

Will — Tue, 07 Jul 2026 16:13:59 +0000

Most Shopify merchants assume their catalog, pricing logic, and store structure are only visible through their storefront. They're not. Shopify serves several JSON and XML endpoints publicly by design — the same data its own theme uses — and anyone can read them without a login, an API key, or a scraping tool.

This isn't a hack or a vulnerability. It's public, intentional, and useful (it's how apps and themes work). But most store owners have never looked at what it reveals. Here are the four endpoints, what each one shows, and how to check your own store in 30 seconds.

1. `/products.json` — your entire catalog and live prices

https://any-store.com/products.json?limit=250

Returns every product, every variant, every price, in clean JSON. Titles, options, inventory-ish signals, images, and the exact price a competitor would need to undercut you. Paginate with ?page=2 and you get the whole catalog, however big.

I covered the pricing angle in detail in a previous post — the short version: a competitor can snapshot your full price list in seconds and diff it daily.

2. `/collections.json` — how you merchandise

https://any-store.com/collections.json

Every collection you've built: "Best Sellers," "Sale," "New Arrivals," seasonal drops. This is your merchandising strategy laid out — which products you group, what you're pushing, how you segment. A competitor learns what you think sells, not just what you stock.

3. `/meta.json` — your store's identity

https://any-store.com/meta.json

Returns the store's name, ID, and location fields (city, province, country). Usually harmless — it's business info that's often on your contact page anyway — but worth knowing it's queryable programmatically, which is how tools bucket and geo-tag stores at scale.

4. `/sitemap.xml` — every URL, including things you forgot were live

https://any-store.com/sitemap.xml

Shopify auto-generates a sitemap of every product, collection, and page. It's meant for Google, but it's also a complete map of your store for anyone. Occasionally it surfaces pages a merchant assumed were "hidden" (unlinked but still published). If it's not truly unpublished, it's in here.

Check your own store right now

Open each URL with your own domain. Whatever you see is exactly what every competitor, analyst, and bot can see too. That's the point — these power legitimate integrations — but you should know what's on the table.

If you want to see it programmatically, this reads all four with no dependencies:

import json, ssl, urllib.request, urllib.error

def get(url):
    req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
    for ctx in (ssl.create_default_context(), ssl._create_unverified_context()):
        try:
            return urllib.request.urlopen(req, timeout=25, context=ctx).read().decode("utf-8", "replace")
        except urllib.error.URLError:
            continue

store = "https://onyxcoffeelab.com"   # <- put any Shopify store here
for path in ["/products.json?limit=1", "/collections.json", "/meta.json", "/sitemap.xml"]:
    body = get(store + path)
    print(path, "->", len(body), "bytes")

What to actually do with this

If you sell: you can't wall these off (Shopify serves them), so the play isn't hiding — it's watching. Since your competitors' data is equally open, monitor it. Know when a rival drops a price or sells out, because that's your window to react.

If you're technical: the free, open-source version that snapshots and diffs a competitor over time is here — github.com/willylam2222-bot/priceprobe.

If you'd rather not touch code: send me a few competitor URLs and I'll send back a clean report of their current prices, stock, and recent moves — willyverse188.gumroad.com/l/kphhe.

The data is already public on both sides. The only question is who's actually looking.

I pulled 800+ competitor prices from 4 Shopify stores in 8 seconds — with 20 lines of Python (no libraries)

Will — Tue, 07 Jul 2026 14:57:39 +0000

If a competitor drops a price on Tuesday and you notice on Friday, you've lost three days of sales. Most small store owners find out by accident. It turns out you don't need a $99/month tool or a scraping framework to fix that — almost every Shopify store hands you its entire catalog and live prices for free, if you know the one URL.

I ran it across four real coffee roasters this morning. Here are the actual numbers, the 20-line script, and what it means if you sell anything online.

The one URL nobody tells you about

Every Shopify storefront exposes a public JSON feed of its products at:

https://<any-shopify-store>/products.json?limit=250

No API key. No login. No headless browser. It's the same data the store's own theme uses to render pages — titles, variants, and live prices — returned as clean JSON. Open one in your browser right now and you'll see it.

The result (real data, pulled today)

I pointed a tiny script at four well-known roasters and pulled every product + price in about 8 seconds total:

Store	Products	Priced bags	Bag price range	Median
Onyx Coffee Lab	248	171	$10–40	$26.00
Counter Culture	169	201	$10–39	$25.00
Verve Coffee	168	460	$10–40	$26.50
Ruby Coffee	213	494	$10–40	$24.25

That's 798 products and their live prices, for free, in less time than it took to read this paragraph. The median bag sits in a tight $24.25–$26.50 band across all four — which is itself the insight: in a mature niche your price anchor is set by the pack, and being 15% off it (either way) is a decision you want to make on purpose, not by accident.

The entire script

Zero dependencies — Python standard library only:

import json, ssl, urllib.request, urllib.error, statistics

STORES = {
    "Onyx Coffee Lab": "https://onyxcoffeelab.com",
    "Counter Culture": "https://counterculturecoffee.com",
    "Verve Coffee":    "https://www.vervecoffee.com",
    "Ruby Coffee":     "https://rubycoffeeroasters.com",
}

def fetch(url):
    req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
    # 2nd context is a fallback for macOS machines missing root certs
    for ctx in (ssl.create_default_context(), ssl._create_unverified_context()):
        try:
            return urllib.request.urlopen(req, timeout=25, context=ctx).read()
        except urllib.error.URLError:
            continue
    raise RuntimeError("fetch failed: " + url)

def prices(base):
    data = json.loads(fetch(f"{base}/products.json?limit=250"))
    out = []
    for p in data["products"]:
        for v in p["variants"]:
            try: pr = float(v["price"])
            except (TypeError, ValueError): continue
            if 10 <= pr <= 40:          # single-bag band
                out.append(pr)
    return out

for name, base in STORES.items():
    p = prices(base)
    print(f"{name:20} {len(p):3} bags  ${min(p):.2f}-{max(p):.2f}  median ${statistics.median(p):.2f}")

That's the whole thing. Swap in your own competitors' domains and you have a competitor price monitor.

Turning it into an actual monitor

The snippet above is a one-shot snapshot. To catch changes, you save each run and diff against the last one:

On each run, store {product_title: price} to a small JSON file.
Next run, compare: flag anything where the price moved, or a product that appeared/disappeared (a competitor selling out is your window to hold or raise price).
Schedule it once a day with cron (mac/Linux) or Task Scheduler (Windows).

Daily is plenty — be polite, one request per store.

A few honest caveats

Only works on Shopify stores. Roughly a quarter of the stores you compete with run Shopify; check by opening the /products.json URL. Other platforms (VTEX, BigCommerce, WooCommerce) have their own public endpoints; marketplaces like Amazon actively block this and need a different approach.
Prices only, not promotions. Cart-level discounts won't show. The list price still tells you 90% of what you need.
Be respectful. This is public data the store already serves to every browser, but hammering it is rude and pointless. Once a day covers real pricing moves.

If you don't want to touch a terminal

I packaged this into a tiny open-source tool that does the save-and-diff for you: github.com/willylam2222-bot/priceprobe — free, run it on your own laptop.

And if you'd rather just get the answer without setting anything up, I'll run it for you: send me your competitors' URLs and I'll send back a clean report of their current prices, stock, and any recent moves. Details here: willyverse188.gumroad.com/l/kphhe.

Either way — the URL at the top of this post is free and yours to use. Go look at what your competitors are charging right now.

I built a zero-dependency competitor price tracker in pure Python (no pip, no API keys)

Will — Wed, 01 Jul 2026 06:26:27 +0000

Most competitor price-monitoring tools charge $40–100/month to watch a handful of URLs. I only wanted to track a few competitors — so I built a small tool I own and run myself. One Python file, no pip install, no API keys, no account.

It's open source (MIT): github.com/willylam2222-bot/priceprobe

Here's how it works and the couple of things that turned out to matter.

The goal

Point it at competitor product URLs
Get price, stock, and what changed since last run
Zero dependencies — runs anywhere Python 3.8+ runs
Own it, run it locally, no monthly bill

1. Fetching (the boring part that bites you)

Standard library only — urllib with polite headers, gzip, and retries. The gotcha that cost me time: macOS Python.org builds ship without root certificates, so urlopen throws CERTIFICATE_VERIFY_FAILED on plenty of machines. Rather than make users run "Install Certificates.command", I fall back to an unverified context only after a verified attempt fails — we are reading public product pages, not sending secrets:

contexts = [ssl.create_default_context(), ssl._create_unverified_context()]
for ctx in contexts:
    try:
        with urllib.request.urlopen(req, timeout=20, context=ctx) as r:
            ...
    except urllib.error.URLError as e:
        if not isinstance(getattr(e, "reason", None), ssl.SSLError):
            break  # non-TLS error: unverified will not help

That one fallback is the difference between "works on my machine" and "works on the buyer's machine".

2. Extracting price / stock

A regex that handles the currencies real stores use plus title and stock keywords:

PRICE_RE = re.compile(
    r"(?:S\$|US\$|£|€|\$|USD|SGD|GBP|EUR)\s?\d{1,3}(?:[.,]\d{3})*(?:[.,]\d{2})?", re.I)
OOS_RE = re.compile(r"out[\s-]?of[\s-]?stock|sold[\s-]?out|currently unavailable", re.I)

Per-target custom regex is supported for sites that hide the price somewhere weird.

3. Diffing against the last run

Each run saves a JSON snapshot. The next run compares and flags ▲ price up / ▼ price down (with %) / ✕ out-of-stock / ✓ restock — normalising numbers so £1,299.00 and £1.299,00 compare correctly.

4. A report you would actually send someone

It renders a clean HTML report with changes highlighted.

Takeaways

Zero-dependency is underrated for tools you want strangers to actually run. No env, no version hell.
The TLS-verify fallback removed the #1 support question before it happened.
Regex price extraction is "good enough" for a huge share of stores; JS-rendered prices are the long tail.

Try it

Free & open source: github.com/willylam2222-bot/priceprobe — stars welcome ⭐

Not into the terminal? There is a one-time $19 packaged version (double-click runner + step-by-step guide) for non-devs who just want the report: gumroad.com/l/ptzxo. Same engine — you pay for zero setup, not a subscription.

Happy to answer questions about the extraction or diffing logic in the comments.