Competitor Pricing Intelligence: SaaS + Shopify Combined

Every pricing team eventually faces the same question from the CEO: "are the competitors still charging the same thing?" The honest answer, nine times out of ten, is "we think so, we checked two months ago." The pricing pages the team saved in a Notion doc are stale. The screenshots from the last pricing review are useless once a competitor adds a fourth tier. The AE who mentioned offhand that Competitor X raised enterprise pricing heard it from a prospect who heard it from a discovery call.

Competitive pricing intelligence is a surveillance problem that gets done badly because nobody wants to own the surveillance. It is tedious, it is never-finished, and the output — "well, three competitors nudged prices upward and one added a lite tier" — rarely lands as a high-visibility quarterly deliverable. So it doesn't happen. Meanwhile the market drifts.

A scheduled scraping pipeline fixes this. You define 20-40 competitors across your SaaS and e-commerce horizons, run a weekly snapshot, compare against the previous snapshot, and surface only the diffs. You get a weekly email that says "3 changes this week: Competitor A added a new $99 tier, Competitor B removed annual discount, Competitor C installed a new upsell app." That email is actionable. That email is what the pricing team should have had all along.

This post walks through building that pipeline for a mixed portfolio — SaaS competitors where pricing pages change, and e-commerce competitors where the signal is app stack and product mix rather than a published price list. The two flows are different but the delivery is the same.

Grounding Numbers

Some numbers to frame how much change you are actually surveilling.

OpenView's 2024 SaaS Pricing & Monetization Report surveyed 700 SaaS companies and found median pricing-page changes of 2.7 per year, with the 75th percentile at 5 and the top decile at 11+. That means a typical competitor updates pricing roughly quarterly; a growing, aggressive competitor monthly. Most updates are tier changes (new add-on, changed included limits) rather than headline price moves. Headline price changes — the actual dollar number next to the plan — happen at median 0.9 times per year, per OpenView.

For e-commerce, the relevant signal is not "did they change prices" (Shopify merchants change SKU prices constantly), but "did they change their app stack." BuiltWith data across Shopify stores shows median churn of 1.2 apps installed or uninstalled per month per active store. That is your rate of change for the e-comm side.

If you track 20 competitors mixed 60/40 SaaS/e-com, a typical week produces: 1-2 SaaS pricing-page changes to look at, 2-4 e-commerce app-install deltas, and 1 or 2 false-positive diffs from layout tweaks. Across a 90-day snapshot history you accumulate roughly 40 real changes worth reviewing — about three per competitor per quarter. That is the signal density to expect. If your pipeline surfaces 30 changes per week, something is miscalibrated.

Why This Is Hard

Four reasons this is harder than "scrape the pricing page weekly."

1. Pricing pages are A/B tested. Stripe, Notion, and most sophisticated SaaS companies run geo-segmented and cookie-segmented pricing. Hitting the page from one IP at one time gives you one view; the real pricing for your persona may be different. Mitigation: sample from multiple IPs, log headers, and accept some noise.

2. Dollar amounts are often images, component props, or pulled from a pricing API. Chargebee, Metronome, and increasingly home-rolled pricing systems mean the price <p>14.99</p> doesn't exist in the HTML. You have to either render JS (Playwright/Puppeteer) or hit the same JSON endpoint the client hits.

3. Layout changes produce false diffs. Marketing teams rewrite copy and reorganize tiers without changing actual prices. A pipeline that naively diffs HTML will scream every time the pricing page gets a design refresh. Need structural diffing at the tier/price level, not raw HTML.

4. E-commerce and SaaS require different surveillance. SaaS has a canonical /pricing page. E-commerce has product catalog changes, app install deltas, and sometimes a revenue-estimate shift — no single page. The unified pipeline needs to cover both while treating them differently under the hood.

Architecture

Two actors, one diff engine, one alerting layer:

  [competitor list]
  name, url, type={saas|shopify}
       |
       +---- SaaS branch -------+
       |                        |
       |                        v
       |            +------------------------+
       |            | saas-pricing-tracker   |
       |            | (structured plan+tier  |
       |            |  extraction)           |
       |            +------------------------+
       |                        |
       |                        v
       |                [plans_snapshot.json]
       |
       +---- Shopify branch ----+
                                |
                                v
                    +------------------------+
                    | shopify-analyzer       |
                    | (apps, theme, revenue  |
                    |  band, product count)  |
                    +------------------------+
                                |
                                v
                       [shop_snapshot.json]

                Then, weekly:

       [current_snapshot] -- diff --> [previous_snapshot]
                                |
                                v
                       [change_log entries]
                                |
                                v
                       [ranked digest + Slack alert]

Snapshots are stored in a simple JSON-per-competitor-per-week layout in an S3 bucket or Apify key-value store. The diff engine is structural: it compares plan objects (name, price, included limits, features) and app-install lists rather than raw HTML.

Code: Weekly Snapshot Across 20 Competitors

The two actors: saas-pricing-tracker for the SaaS side and shopify-analyzer for the e-commerce side.

from apify_client import ApifyClient
import json, os
from datetime import datetime

client = ApifyClient("APIFY_TOKEN")

competitors = [
    # SaaS
    {"name": "notion",     "type": "saas",    "url": "https://notion.so/pricing"},
    {"name": "airtable",   "type": "saas",    "url": "https://airtable.com/pricing"},
    {"name": "monday",     "type": "saas",    "url": "https://monday.com/pricing"},
    {"name": "clickup",    "type": "saas",    "url": "https://clickup.com/pricing"},
    # ... add 8 more SaaS
    # Shopify / e-com
    {"name": "allbirds",   "type": "shopify", "url": "https://www.allbirds.com"},
    {"name": "beardbrand", "type": "shopify", "url": "https://www.beardbrand.com"},
    # ... add 6 more shopify
]

saas_urls = [c["url"] for c in competitors if c["type"] == "saas"]
shop_urls = [c["url"] for c in competitors if c["type"] == "shopify"]

saas_run = client.actor("nexgendata/saas-pricing-tracker").call(run_input={
    "urls": saas_urls,
    "extract_plans": True,
    "extract_addons": True,
    "render_js": True,
    "countries_to_sample": ["US", "GB", "DE"],
})

shop_run = client.actor("nexgendata/shopify-analyzer").call(run_input={
    "urls": shop_urls,
    "include_apps": True,
    "include_theme": True,
    "include_revenue_estimate": True,
    "include_product_summary": True,
})

snapshot = {
    "date": datetime.utcnow().strftime("%Y-%m-%d"),
    "saas": list(client.dataset(saas_run["defaultDatasetId"]).iterate_items()),
    "shopify": list(client.dataset(shop_run["defaultDatasetId"]).iterate_items()),
}

path = f"snapshots/{snapshot['date']}.json"
os.makedirs("snapshots", exist_ok=True)
with open(path, "w") as f:
    json.dump(snapshot, f, indent=2)
print(f"Wrote {path}")

A single saas-pricing-tracker result for Notion looks roughly like:

{
  "url": "https://notion.so/pricing",
  "country_sampled": "US",
  "plans": [
    {"name": "Free", "monthly_price_usd": 0, "users_included": 1,
     "features": ["unlimited_pages", "basic_ai", "7_day_page_history"]},
    {"name": "Plus", "monthly_price_usd": 12,
     "billing_cycle": "monthly", "annual_discount_pct": 25,
     "features": ["unlimited_pages", "unlimited_files", "30_day_page_history"]},
    {"name": "Business", "monthly_price_usd": 18, "features": [...] },
    {"name": "Enterprise", "monthly_price_usd": null, "contact_sales": true}
  ],
  "addons": [
    {"name": "Notion AI", "price_per_user_monthly": 10}
  ],
  "raw_html_hash": "a1f3..."
}

And a shopify-analyzer result for Allbirds:

{
  "url": "https://www.allbirds.com",
  "apps": ["Klaviyo", "ReCharge", "Gorgias", "Judge.me", "Yotpo", ...],
  "app_count": 24,
  "theme": {"name": "custom", "paid": true},
  "revenue_estimate": {"band": "high", "monthly_usd": "1M+"},
  "product_summary": {"total_skus": 180, "median_price": 98, "new_last_30d": 12}
}

Diff Engine

Structural diff, not HTML diff. For SaaS:

import json, glob
from deepdiff import DeepDiff

snaps = sorted(glob.glob("snapshots/*.json"))
if len(snaps) < 2:
    print("Need at least 2 snapshots to diff"); exit()

with open(snaps[-2]) as f: prev = json.load(f)
with open(snaps[-1]) as f: curr = json.load(f)

changes = []

# SaaS plan-level diff
for c in curr["saas"]:
    p = next((x for x in prev["saas"] if x["url"] == c["url"]), None)
    if not p: continue
    prev_plans = {x["name"]: x for x in p.get("plans", [])}
    curr_plans = {x["name"]: x for x in c.get("plans", [])}

    added = set(curr_plans) - set(prev_plans)
    removed = set(prev_plans) - set(curr_plans)
    for n in added:
        changes.append({"type": "plan_added", "url": c["url"], "plan": n,
                        "price": curr_plans[n].get("monthly_price_usd")})
    for n in removed:
        changes.append({"type": "plan_removed", "url": c["url"], "plan": n})

    for n in set(prev_plans) & set(curr_plans):
        pp, cp = prev_plans[n], curr_plans[n]
        if pp.get("monthly_price_usd") != cp.get("monthly_price_usd"):
            changes.append({
                "type": "price_change", "url": c["url"], "plan": n,
                "prev": pp.get("monthly_price_usd"),
                "curr": cp.get("monthly_price_usd"),
            })

# Shopify app-install diff
for c in curr["shopify"]:
    p = next((x for x in prev["shopify"] if x["url"] == c["url"]), None)
    if not p: continue
    prev_apps = set(p.get("apps", []))
    curr_apps = set(c.get("apps", []))
    for a in curr_apps - prev_apps:
        changes.append({"type": "app_installed", "url": c["url"], "app": a})
    for a in prev_apps - curr_apps:
        changes.append({"type": "app_uninstalled", "url": c["url"], "app": a})

for ch in changes:
    print(ch)

Typical output for a week where a few things happened:

{'type': 'price_change', 'url': 'https://monday.com/pricing', 'plan': 'Pro', 'prev': 16, 'curr': 19}
{'type': 'plan_added', 'url': 'https://clickup.com/pricing', 'plan': 'AI Unlimited', 'price': 9}
{'type': 'app_installed', 'url': 'https://www.allbirds.com', 'app': 'Rebuy'}
{'type': 'app_uninstalled', 'url': 'https://www.beardbrand.com', 'app': 'Privy'}

Four actionable items for the pricing team. A 90-second read.

Worked Example: 90-Day Competitive Review

The pricing PM runs the pipeline weekly for a quarter. At the end she has 12 weekly snapshots and an accumulated change log. Preparing for the quarterly pricing review, she asks three questions:

Q1: Which competitors raised prices, and by how much?

Grep the change log for type=price_change. Aggregate by competitor. Over 90 days: Monday raised Pro from $16 to $19 (19% bump). ClickUp restructured the free tier to include fewer features. Airtable left prices untouched but added an "AI credit" add-on at $20/user. Notion launched an enterprise SKU that wasn't previously listed.

Q2: Which e-commerce competitors shifted their app stack?

Filter for type=app_installed and app_uninstalled. Three of eight Shopify competitors installed a retention app (two picked Rebuy, one picked Bold). Two uninstalled Privy (possibly because Klaviyo now includes Privy-equivalent pop-ups). One installed a new subscription app — they are moving toward a subscription model, worth pitching for.

Q3: Which competitors are structurally growing vs. stable?

Cross-reference: app count increase > 3, new plan added, and revenue-estimate band upgrade all point at a scaling operation. Two of eight pass all three screens. These are the competitors most likely to expand into adjacent markets (yours) over the next two quarters.

She walks into the review with a 2-page memo: four recommendations, each backed by specific observed changes with dates and URLs. The "we should match Monday's 19% increase on our Pro tier" recommendation is grounded in data; without the pipeline it would have been a hunch.

Cost: ~$30 in Apify credits over the quarter. Time: about 30 minutes/week to skim the digest, 3 hours to prepare the quarterly memo. Compare to Competera or Prisync at $3k-12k/year and you see why a built pipeline pays back within a quarter.

Gotchas

Real-world friction you will encounter:

• Geo-dependent pricing. Notion shows different prices in Europe vs. US. Stripe's pricing page literally renders different numbers based on IP country. The tracker samples multiple countries; you should explicitly record country_sampled in your diffs to avoid spurious "price changed" alerts when your proxy rotated countries.

• JS-rendered pricing tables. Many modern pricing pages pull from a CMS or pricing API client-side. The actor uses Playwright with render_js=True; if you see empty plans arrays, confirm the target uses SSR or add a wait-for-selector in the run config.

• A/B tests. If a competitor is running a pricing A/B test, you will see prices flip between two values weekly. Set a cooldown threshold: only alert if the new price persists for 2+ consecutive snapshots. Saves a lot of noisy Slack messages.

• App fingerprint drift. Shopify app detection relies on fingerprints (script URLs, cookie names, DOM patterns). When a vendor rebrands or changes its script CDN path, your detection misses it until the actor fingerprint database catches up. Expect 2-5% false negatives on any given week.

• Feature-list diffs are noisy. Features like "priority support" or "advanced analytics" are marketing copy, not product definitions. Diffing at the feature-string level produces false positives from copy rewrites. Focus the diff on price and tier count; downweight feature changes to "notable" rather than "alert."

• Annual discounts. Competitors sometimes change the effective annual price without changing monthly. Capture both; diff the annual price separately. A silent change from "20% off annual" to "25% off annual" is a real pricing move many trackers miss.

• Enterprise tiers are opaque. "Contact sales" pricing cannot be diffed. The best you get is whether a contact-sales tier exists and whether the marketing language changed (from "enterprise" to "enterprise+ with SSO" for example).

• Scheduled-run reliability. Running scheduled scrapes weekly works 95% of the time. Set up a second "did the snapshot land" check: if this Monday's snapshot is missing, alert yourself. Missing snapshots break the diff chain and you will notice it two weeks later if you don't watch for it.

FAQ

How many competitors can I track?
The two actors scale to hundreds. 20-40 is the sweet spot for a weekly review you will actually read; beyond that you need categorization and filtered digests.

Is scraping competitor pricing pages legal?
Public marketing pages are generally fair game. hiQ v. LinkedIn and subsequent rulings in the US broadly support scraping public data. Avoid bypassing paywalls, authentication, or ToS-explicit prohibitions. If a competitor's robots.txt disallows /pricing, respect it. In practice, almost no one does.

What about international currencies?
The tracker returns monthly_price_usd normalized, with the raw value + currency also included. Diffs should compare normalized USD to avoid false alerts from currency fluctuations.

How do I detect stealth pricing changes (e.g. silent usage-cap reductions)?
Capture features and included limits as structured fields, not free text. Many competitors tighten limits (reduce AI credits from 1000 to 500 at the same price) without touching the headline number. Diffing structured limits catches this.

Can I track pricing via API if the competitor exposes one?
When a competitor's pricing page hits a JSON endpoint client-side, the tracker follows that endpoint automatically. For deeply custom pricing engines (usage-based, negotiated), no. There is no substitute for a human lead getting a quote.

What if two competitors use the same pricing platform (Stripe Billing, Chargebee)?
The extracted plans normalize well across platforms. The tracker recognizes common embed widgets and parses them the same way. You will occasionally get Stripe-hosted pricing pages that render identically — that's fine, they produce comparable structured output.

How do I surface the changes to the team?
A weekly Slack post or email is the most-adopted option. Some teams dump the change log into a Notion database with a filtered view "changes this month." Whichever format your team actually opens on Monday is the right one.

Can I also track review sentiment or support SLAs?
Out of scope for this pipeline. Pair with app-review scraping or customer-review monitoring for sentiment. For support SLAs, there is no public signal unless the competitor publishes a status page.

Conclusion

Competitive pricing intelligence fails at most companies because it's a recurring task nobody owns. A scheduled scraping pipeline — two actors, a structural diff, and a weekly digest — makes it own itself. You stop having to ask "what are they charging now" because you already have a change log with the answer.

The real payoff is not the Slack pings. It's walking into the quarterly pricing review with 90 days of ground truth instead of three screenshots and a vibe. Product and pricing decisions grounded in a kept log of competitor moves are measurably better than decisions grounded in last quarter's vague memory of the market.

Run your first weekly snapshot through the saas-pricing-tracker and shopify-analyzer on Apify. Store snapshots, diff weekly, surface only the changes. The setup is one afternoon; the payoff is permanent.