Someone ran the analysis on a popular planner last week and sent back three praise points: fast, clean, syncs everywhere. No complaints at all. Their read was that the tool was for finding strengths, not gaps — useful for marketing copy, not for deciding what to build.
That is a fair reading of the output, and it is the most useful piece of feedback the tool has received. It also points at something that is not a bug in the analysis. It is arithmetic.
The complaints were in the sample
Before blaming the model, check the input. I pulled the 200 most recent US App Store reviews for a well-known to-do app from the public RSS feed and counted stars. 62 of the 200 were 1–3 stars. For a well-known language app, on the same day, 47 of 200.
So on a well-liked app, roughly a quarter to a third of the recent window is unhappy. The complaints are in the data. If the report leads with three strengths, the sample did not fail you. The ranking did.
Praise pools, complaints scatter
Read the titles in the to-do app's 200 reviews and the asymmetry is hard to miss.
About half of its four- and five-star titles are one of a dozen words: great, love it, best app ever, lifesaver, works well. They are not describing the product. They are describing the relief of having finished a task, and they reach for the same vocabulary to do it. Group them and you get one enormous theme with a hundred mentions.
The one- and two-star titles do not share a vocabulary. In the same 200: sorting, a paywall on basic features, tasks deleted, crashes after the last iOS update, too complicated to set up. Each one names a different surface of the product. Group them and you get six small themes, none of which wins a count.
Ranking themes by mention count is therefore a praise amplifier on any app people like. The largest cluster wins by construction, and on a well-liked app the largest cluster is "it's good". The complaints are not absent. They are split across exactly the specificity that made them worth reading.
That is the trap: the output is not wrong, it is ordered. And ordering decides what a reader sees first.
"200 reviews" is not a unit of time
Worth knowing before you trust any count. Same setting, two apps, two completely different windows:
- Popular to-do app: 200 reviews covered nine months, February to November.
- Popular language app: 200 reviews covered a single day.
The control says 200 either way. On one app that is a year of opinion, on the other it is one news cycle. Whichever you got, you are reading a window you did not consciously choose. Choose the window first — last 90 days, say — then take as many reviews as fit inside it.
The three-star band is the under-read part
The obvious fix is to filter to one and two stars. That overcorrects. People writing one star are often leaving, and their complaint is frequently about the exit rather than the product: the price change, the removed feature, the update that broke a habit they had already built.
The three-star reviews are where the addressable friction lives. Someone still using the app, still willing to write four paragraphs, telling you precisely which task the product made harder. It is the least flattering band and the least quoted one, because it is neither a success story nor a dramatic failure.
If you change one thing in how you sample, sample this band deliberately instead of waiting for it to out-rank the praise.
Four changes that fix the first screen
- Read the distribution before the themes. Star counts by band, one line. If 30% of the sample is 1–3 stars and the report shows no complaints, the ranking is the problem, not the market.
- Give praise one row and complaints as many as they need. Symmetric output is dishonest here, because the underlying data is not symmetric. One "what people praise" block, then one row per blocked task.
- Name the blocked task, not the sentiment. "Sync conflict resolved by deleting the local copy" is a row. "Sync issues" is a heading pretending to be a row.
- Show the cutoff. "Top 3 of 17 themes, covering 43 of 200 reviews" tells a reader there is more behind the wall, and roughly how much. "See every theme" tells them nothing, which also removes the reason to unlock.
What this means if you are building an alternative
The opportunity is almost never in the top theme of a popular app. It sits in themes four through ten: too small to win a volume contest, specific enough to name a job, and usually described by people who are still there.
A preview that shows the top three by count will show the incumbent at its best. That result is flattering, and it is the one that makes an indie developer close the tab and conclude there is no room.
What this does not prove
One app, one storefront, one week, titles only. Title-level counting is crude — plenty of complaints are vague too ("annoying", "boo"), so this method understates the scatter rather than overstating it. The RSS feed returns recent reviews, not the store's full history, and the two windows above are what Apple returned on the day I looked, not a stable property of either app.
Treat it as a reason to check the sample and the ordering before trusting a conclusion, not as a benchmark.
I'm building ReviewFold, which turns App Store and Play reviews into a comparison across competing apps — strengths, gaps and disagreements, each line linked back to the original review. There's a public example report if you want to see the shape of the output before reading the method. The starred-versus-theme question has its own write-up at sentiment analysis vs theme analysis, and deciding which complaint to chase first is covered at how to prioritize app review feedback. No signup needed for either.
Written by me. Research and editing were done with an AI assistant, and anything I could not support with a review quote is marked as a question rather than a finding.
Top comments (0)