DEV Community

SPBros
SPBros

Posted on

What do Shopify apps charge? Scraping App Store pricing and reviews with Python (70-app snapshot)

#python #webscraping #shopify #datascience

Disclosure: I run SPBros. I sell a dataset and run a hosted scraper, both linked at the end. Everything before that works without them.

If you're building a Shopify app, or sizing up a category before you start, it helps to know what competitors charge and how many reviews the apps on page 1 have. Category pages show ratings and review counts, but the plan details are on each app's own page. Here's how to pull both into pandas.

1. Find stable hooks in the HTML

Open a category page, for example https://apps.shopify.com/categories/marketing-and-conversion-marketing-email-marketing, and inspect an app card. Skip the class names. Each card carries data-* attributes (example card, trimmed):

<div data-controller="app-card"
     data-app-card-handle-value="klaviyo-email-marketing"
     data-app-card-name-value="Klaviyo: Email Marketing & SMS"
     data-app-card-intra-position-value="1">
Enter fullscreen mode Exit fullscreen mode

I keyed on these instead of the class names, which look auto-generated and likely to change.

2. Parse the cards

pip install httpx beautifulsoup4 pandas (Python 3.8+ for the :=), then:

import re, time, httpx
from bs4 import BeautifulSoup
import pandas as pd

UA = {"User-Agent": "Mozilla/5.0 (compatible; my-research-script; you@example.com)"}

def cards(category):  # page 1 of the category only
    html = httpx.get(f"https://apps.shopify.com/categories/{category}", headers=UA,
                     follow_redirects=True, timeout=30).text
    out, seen = [], set()
    for c in BeautifulSoup(html, "html.parser").select('[data-controller="app-card"]'):
        handle = c.get("data-app-card-handle-value")
        if not handle or handle in seen:
            continue  # skip repeat placements
        seen.add(handle)
        lines = c.get_text("\n", strip=True).split("\n")
        rating = next((float(t) for t in lines if re.fullmatch(r"\d\.\d", t)), None)
        reviews = next((int(m.group(1).replace(",", "")) for t in lines
                        if (m := re.fullmatch(r"\(([\d,]+)\)", t))), None)
        out.append({"category": category, "rank": len(out) + 1, "handle": handle,
                    "name": c.get("data-app-card-name-value"),
                    "rating": rating, "reviews": reviews})
    return out
Enter fullscreen mode Exit fullscreen mode

Two gotchas:

  • Sponsored cards. The App Store can show ads, and you should drop them if you want organic rank. I don't have a reliable marker to share. If one shows up, inspect it and filter on whatever distinguishes it from the organic cards.
  • Repeat placements. The same app can appear more than once on a page. Dedupe on the handle before you number anything.

Shopify also reorganizes categories. For example, the old store-design-page-builders URL now redirects to store-design-storefronts. Keep follow_redirects=True and log the final URL you actually scraped.

3. Get the cheapest paid plan from each app page

The full plan list is on the app's own page (https://apps.shopify.com/<handle>). A regex over the page text catches most of them. Spot-check a few apps, though, because anything else on the page that looks like $X/month will match too. Yearly prices are converted to monthly, and free plans are ignored:

PRICE = re.compile(r"\$([\d,]+(?:\.\d+)?)\s*/\s*(month|year)")

def cheapest_paid(handle):
    html = httpx.get(f"https://apps.shopify.com/{handle}", headers=UA,
                     follow_redirects=True, timeout=30).text
    text = BeautifulSoup(html, "html.parser").get_text(" ", strip=True)
    monthly = [float(p.replace(",", "")) / (12 if per == "year" else 1)
               for p, per in PRICE.findall(text)]
    paid = [p for p in monthly if p > 0]
    return min(paid) if paid else None  # None = no paid plan found
Enter fullscreen mode Exit fullscreen mode

Sleep between requests, cache the HTML while you develop, and check robots.txt first (see section 5).

4. Analyze 

rows = []
for cat in ["marketing-and-conversion-marketing-email-marketing"]:  # add more categories
    for app in cards(cat):
        app["lowestPaidPlanUsd"] = cheapest_paid(app["handle"])
        rows.append(app)
        time.sleep(1.5)  # be polite

df = pd.DataFrame(rows)
print(df.groupby("category").agg(apps=("handle", "count"),
                                 median_reviews=("reviews", "median"),
                                 median_cheapest_paid=("lowestPaidPlanUsd", "median")))
Enter fullscreen mode Exit fullscreen mode

For comparison, here's a snapshot I collected on Oct 16, 2026 (UTC). It covers page 1 of three categories: 70 organic apps and 197 pricing-plan rows. Your numbers will differ. Listings change, and a page-wide regex can pick up prices that aren't plans.

Category (page 1, Oct 16) Apps Median reviews Median cheapest paid plan
email marketing 12 1,942.5 $15
page builders¹ 26 239 $27
returns & exchanges² 32 209 $9.99

¹ URL now redirects to store-design-storefronts. ² Page mixed support/contact/returns apps; URL now redirects to store-management-support.

A few observations:

  • In email marketing, the median review count on page 1 was about 1,940. A new app there competes with a lot of social proof.
  • Page builders had the highest median cheapest paid plan of the three ($27).
  • 69 of the 70 apps listed a free plan.

It's one snapshot of page 1 in three categories. Treat it as a starting point.

5. Keep it polite

Check robots.txt (Python's urllib.robotparser handles it in a few lines), collect only public business data, skip reviewer names, keep request rates low, and identify yourself in the User-Agent. Read the site's terms before you reuse or redistribute the data.

If you'd rather not run it yourself

Both are mine (SPBros). Questions about the parsing are welcome in the comments.

Top comments (0)