DEV Community

Tiandu Tech
Tiandu Tech

Posted on

Monitor your competitors' Chrome extension reviews with a scheduled scraper

#chrome #extensions #automation #python

If you build a Chrome extension, your competitors' 1-star reviews tell you what their users want and aren't getting. The Chrome Web Store has no export or alerts for other people's extensions, so the usual option is checking by hand.

Here's a small setup: a scheduled scraper collects the latest reviews of a handful of competing extensions every day, and a short Python script flags the new low-rated ones.

We'll use the App Store, Google Play & Chrome Web Store Reviews Scraper on Apify for collection and Apify's built-in Schedules for timing.

The idea

  1. Pick 3–10 competing extensions.
  2. Run the scraper on a schedule (say, daily) with a modest maxReviewsPerApp. Chrome reviews are fetched newest first, so a small cap still catches everything posted since the last run, as long as fewer reviews than the cap arrive in a day.
  3. After each run, compare reviewIds with the ones you've already seen, and report new reviews with rating <= 2.

The scraper itself doesn't track what you've seen before. Each run simply returns up to your limit. The "only new" part is a few lines of your own code, shown below.

Step 1: Collect the extension IDs

The ID is the 32-character string at the end of an extension's Web Store URL, e.g. https://chromewebstore.google.com/detail/<name>/ddkjiahejlhfcafbddmgiahcphecmpfh. You can paste either the full URL or just the ID.

Step 2: The input 

{
  "apps": [
    "ddkjiahejlhfcafbddmgiahcphecmpfh",
    "<competitor-extension-id-2>",
    "<competitor-extension-id-3>"
  ],
  "language": "en",
  "maxReviewsPerApp": 50
}
Enter fullscreen mode Exit fullscreen mode
  • countries isn't needed: Chrome Web Store reviews aren't split by country, and country comes back as null.
  • language filters to reviews in that language. Change it (for example to de) if your market isn't English-speaking. Leaving it out doesn't give you all languages: the default is en. One run covers one language.
  • maxReviewsPerApp: 50 is one store request per extension (the scraper fetches 50 per request, at most one request per second), which keeps each run small. This still holds with language set, because the store applies the language filter in that same request: you get up to 50 reviews in that language (fewer if the extension doesn't have that many).

Each review comes back with rating, text, appVersion (the extension version the reviewer used), thumbsUp (the store's "helpful" count), reviewedAt and developerReply. title is always null for Chrome because the store has no review titles.

Step 3: Schedule it

Two ways, both standard Apify features:

  • Saved task + Schedule: in Apify Console, save the input above as a task for the Actor, then create a Schedule (Console → Schedules) with a cron expression such as 0 7 * * * (every day at 07:00 in the schedule's time zone) and add the task to it.
  • From your own cron: call the Actor from your own scheduler with the API client, as in the script below.

Step 4: Flag new low-rated reviews

This script reads the dataset of the latest successful run of the task, keeps a local file of seen review IDs, and prints new 1–2 star reviews:

import json, os, pathlib
from apify_client import ApifyClient

TASK_ID = os.environ["APIFY_TASK_ID"]        # the saved task from Step 3
SEEN = pathlib.Path("seen_review_ids.json")

client = ApifyClient(os.environ["APIFY_TOKEN"])
items = list(client.task(TASK_ID).last_run(status="SUCCEEDED").dataset().iterate_items())

seen = set(json.loads(SEEN.read_text())) if SEEN.exists() else set()
new = [r for r in items if r["reviewId"] not in seen]
alerts = [r for r in new if (r.get("rating") or 5) <= 2]

for r in sorted(alerts, key=lambda r: r.get("reviewedAt") or ""):
    print(f'[{r["appId"]}] {r["rating"]}★  v{r.get("appVersion") or "?"}  '
          f'{(r.get("reviewedAt") or "")[:10]}\n  {r.get("text") or ""}\n')

SEEN.write_text(json.dumps(sorted(seen | {r["reviewId"] for r in items})))
print(f"{len(new)} new reviews, {len(alerts)} low-rated")
Enter fullscreen mode Exit fullscreen mode

Pipe the output into Slack, email or an issue tracker however you like. On the first run every review is "new"; after that you only see the delta.

What to look for

  • Repeated complaints after a competitor's update. Group alerts by appVersion; a cluster of 1-star reviews on one version usually means a regression.
  • Unanswered complaints. developerReply is null on reviews the competitor hasn't answered. Those show what they aren't handling.
  • Most-agreed-with reviews. Sort by thumbsUp to see which complaints other users marked as helpful.

Limits to keep in mind

  • The scraper uses the same anonymous data request that the Web Store's public reviews page makes. It isn't a documented API, so a site change can break parsing. If two extensions in a row can't be parsed, the remaining Chrome tasks are skipped and the run summary says so, rather than writing wrong data.
  • An unknown or removed extension ID returns no rows, and the summary notes it.
  • Emails and phone numbers in review text and replies are redacted, but reviewers can still type other personal details. Keep the data internal.

The same setup works for Google Play and App Store apps too: add their IDs to apps and set countries.

Set up your own monitor: https://apify.com/tiandu-tech/apify-002-app-reviews

Top comments (0)