Find user pain points in app and extension reviews with Python

Star ratings tell you that users are unhappy. The review text tells you why. This post shows a short Python workflow (pandas plus the standard library) that turns a dataset of App Store, Google Play and Chrome Web Store reviews into a ranked list of complaints, broken down by store and app version.

We'll start from a dataset exported by the App Store, Google Play & Chrome Web Store Reviews Scraper on Apify, which gives every review the same fields no matter which store it came from. Any review export with similar columns will work.

1. Get the data

Run the Actor with your apps and extensions (see the companion tutorial for input details), and turn on the optional sentiment switch if you want a quick baseline label:

{
  "apps": ["com.whatsapp", "ddkjiahejlhfcafbddmgiahcphecmpfh"],
  "countries": ["us"],
  "language": "en",
  "maxReviewsPerApp": 500,
  "sentiment": true
}

Then either export the dataset as JSON from the run's Output tab, or fetch it with the API (this uses apify-client 3.x, where call() returns an object, so it's run.default_dataset_id; on the old 2.x client it's run["defaultDatasetId"]):

import os, json
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("tiandu-tech/apify-002-app-reviews").call(run_input={
    "apps": ["com.whatsapp", "ddkjiahejlhfcafbddmgiahcphecmpfh"],
    "countries": ["us"], "language": "en",
    "maxReviewsPerApp": 500, "sentiment": True,
})
items = list(client.dataset(run.default_dataset_id).iterate_items())
with open("reviews.json", "w", encoding="utf-8") as f:
    json.dump(items, f, ensure_ascii=False)

2. Load and clean

import pandas as pd

df = pd.read_json("reviews.json")
df["reviewedAt"] = pd.to_datetime(df["reviewedAt"], errors="coerce", utc=True)
df["text"] = df["text"].fillna("")
df = df[df["text"].str.len() > 0]

print(df.groupby(["store", "appId"]).size())

Useful columns: store, appId, rating, text, appVersion, thumbsUp, reviewedAt, developerReply, and sentiment if you turned it on.

Note: the Actor's sentiment label is a simple rule (English keywords plus star rating), and for non-English reviews it mostly follows the rating. Treat it as a baseline, not ground truth.

3. Ratings at a glance

summary = (df.groupby(["store", "appId"])
             .agg(reviews=("reviewId", "count"),
                  avg_rating=("rating", "mean"),
                  share_1_2_star=("rating", lambda s: (s <= 2).mean()),
                  replied=("developerReply", lambda s: s.notna().mean()))
             .round(2))
print(summary)

replied is the share of reviews with a developer reply, a quick proxy for how actively a team handles feedback.

4. Rating by app version

Releases often explain rating drops. appVersion is the version the reviewer was using:

by_version = (df.dropna(subset=["appVersion"])
                .groupby(["appId", "appVersion"])
                .agg(n=("reviewId", "count"), avg=("rating", "mean"))
                .query("n >= 10")
                .sort_values(["appId", "avg"]))
print(by_version.head(20))

5. Pull out complaint phrases

Take the negative reviews and count the most common two- and three-word phrases. No ML needed:

import re
from collections import Counter

STOP = set("""a an the and or but if to of in on for with is it this that was are be
i me my we you your they them have has had not no just so very app extension
""".split())

def phrases(text, n):
    words = [w for w in re.findall(r"[a-z']+", text.lower()) if w not in STOP]
    return [" ".join(words[i:i+n]) for i in range(len(words) - n + 1)]

neg = df[df["rating"] <= 2]
counts = Counter()
for t in neg["text"]:
    counts.update(phrases(t, 2))
    counts.update(phrases(t, 3))

for phrase, c in counts.most_common(30):
    print(f"{c:4d}  {phrase}")

Read the top phrases and group them into themes by hand. On a few hundred reviews this takes minutes, and you see exactly why each phrase landed in a theme.

6. Tag themes and rank them

Turn your themes into keyword rules and score them by frequency and by how many users agreed (thumbsUp):

THEMES = {
    "crash / freeze": ["crash", "freez", "stuck", "not responding"],
    "login / account": ["login", "log in", "sign in", "account", "password"],
    "performance": ["slow", "lag", "battery", "memory"],
    "ads / paywall": ["ads", "subscription", "paywall", "premium"],
    "update broke it": ["update", "latest version", "new version"],
}

def tag(text):
    t = text.lower()
    return [name for name, kws in THEMES.items() if any(k in t for k in kws)] or ["other"]

neg = neg.assign(theme=neg["text"].map(tag)).explode("theme")
ranking = (neg.groupby(["store", "theme"])
              .agg(reviews=("reviewId", "count"),
                   agreed=("thumbsUp", lambda s: s.fillna(0).sum()))
              .sort_values(["store", "reviews"], ascending=[True, False]))
print(ranking)

The agreed column weights each complaint by how many other users marked the review as helpful, a rough signal of how many users share the problem.

7. Export for the team

ranking.to_csv("pain_points_by_store.csv")
neg.sort_values("thumbsUp", ascending=False)[
    ["store", "appId", "appVersion", "rating", "thumbsUp", "theme", "text"]
].head(50).to_csv("top_negative_reviews.csv", index=False)

Two CSVs: one ranked list of themes per store, and the 50 most-agreed-with negative reviews as evidence.

Where to go next

• Swap the keyword rules for an embedding model or an LLM once you have more data.
• Compare your app against a competitor by adding their IDs to the same run.
• Run it on a schedule and track theme counts week over week (see the monitoring post).

A reminder on data: review text can contain names or details that reviewers typed themselves. The Actor redacts emails and phone numbers, but keep the data internal and follow the privacy rules that apply to you.

Get the scraper here: https://apify.com/tiandu-tech/apify-002-app-reviews

A runnable example project (collect + analyze) is on GitHub: https://github.com/DongmeiHerman/app-reviews-pain-points