We pulled the 200 most recent App Store reviews for 24 well-known apps, all within about two minutes of each other, all in the US storefront, all newest-first.
For Spotify, those 200 reviews covered 7.8 hours.
For Things 3, those same 200 reviews covered 668 days — back to January 2025.
Same count. Same method. A 2,057× difference in how much time the sample actually represents.
This matters because almost every review analysis you have seen — including ours — reports a count. "We analyzed 200 reviews." "Top themes from recent feedback." Nobody reports the window, because the count feels like the honest part. But the count tells you how much text you read. It says nothing about what period you are talking about.
What we measured
One question per app: how far back does review number 200 reach?
- Reviews taken from the App Store's own review endpoint, 100 rows per request, two requests per app, newest first, US storefront.
- App IDs resolved through the public iTunes Search API, so nothing was guessed from a web page.
- All 24 collections ran inside a two-minute window on 7 November 2026.
- The window is simply newest review date minus oldest review date across those 200 rows.
The endpoint is the one the store's own front end uses. It is undocumented and has no availability guarantee, which is why this is a bounded public sample rather than a full export.
The spread
| App | Window for 200 reviews | Reviews/day |
|---|---|---|
| Spotify | 0.33 days | 616 |
| YouTube | 0.36 days | 563 |
| Duolingo | 0.70 days | 286 |
| TikTok | 1.13 days | 177 |
| Uber | 2.09 days | 96 |
| 2.15 days | 93 | |
| Amazon Shopping | 3.67 days | 55 |
| Netflix | 4.20 days | 48 |
| 5.09 days | 39 | |
| Google Maps | 5.53 days | 36 |
| 8.06 days | 25 | |
| Discord | 8.15 days | 25 |
| Gmail | 13.6 days | 15 |
| MyFitnessPal | 18.0 days | 11 |
| Airbnb | 20.7 days | 9.7 |
| Calm | 23.6 days | 8.5 |
| Slack | 27.0 days | 7.4 |
| Strava | 51.6 days | 3.9 |
| Forest | 57.5 days | 3.5 |
| Notion | 141.5 days | 1.4 |
| Headspace | 176.6 days | 1.1 |
| Todoist | 265.2 days | 0.75 |
| Trello | 421.2 days | 0.47 |
| Things 3 | 668.5 days | 0.30 |
Median window: 10.9 days. Ten apps fit 200 reviews into under a week. Two needed more than a year.
The middle column is the finding. The right column explains it: review volume, not the sample size, sets the window.
Three things this changes
1. At eight hours, you are reading a mood.
Spotify's sample is one day of App Store traffic. Whatever tops that list can be gone by the weekend — a bad release, an outage, a pricing change. That is a legitimate question ("what is happening right now"), but it is not a backlog. If you write it into a roadmap document, you are treating a weather report as climate.
2. At two years, you are reading an archive.
Things 3's 200 reviews reach back to January 2025. A bug reported then and fixed in March sits in the same list as a complaint from last week. Nothing in the count separates them. The theme table will happily merge them into one confident row.
3. In between, the count is silently a rate.
For an app at 25 reviews a day, 200 reviews is eight days. Whether eight days is "recent" depends entirely on how fast the product changes — which the count cannot tell you.
This is the same arithmetic failure as ranking praise and complaints in one list: complaints scatter across many phrasings, praise pools into a few, so a count-based ranking flatters a high-rated app. Here the same mechanism operates one level up — a count-based ranking over samples with unstated windows compares eight hours to two years and calls the result a leaderboard.
Three rules we now apply
None of them requires a bigger sample. They require writing down something the sample already knows.
State the window, not just the count. "200 reviews, 30 October – 7 November 2026" is checkable. "200 recent reviews" is not, because two readers will assume two different periods.
Treat anything under a week as sentiment, not backlog. A short window is excellent for "what broke this week" and poor for "what should we build next". Label it for the first job and do not promote it to the second.
For slow apps, split by version before counting. When the window runs past a release, group by app version first. Otherwise you are summing problems from two different products.
If you have the data, there is a fourth: report a 30-day and a 90-day window side by side and show where the ranking changes. The change is usually more informative than either list.
What this data does not tell you
- One storefront. US App Store only. Other regions differ in volume and tone.
- One sort order. Newest first, not Apple's default and not "most helpful".
- One snapshot. 7 November 2026. For the fastest apps, the window is not reproducible tomorrow.
- No review text in the dataset. We stored IDs, dates, ratings and versions. Theme-level claims cannot be checked against it.
- Not a random sample. The 24 apps were chosen for recognisability. This is not an estimate of the App Store as a whole.
The dataset is a 24-row CSV — app ID, window in days, reviews per day, 1–2 star share, date range, total review count — free to reuse with attribution. The script that produced it is published next to it: Node 18+, no dependencies, public Apple endpoints only, so you can rerun the whole thing and compare windows yourself.
Why publish this at all
We build a review analysis tool, so the obvious question is why we would publish a result that makes every review sample look shakier, including ours.
Because the alternative is worse. A tool that reports "top themes" without saying which period it looked at is making a claim it cannot support. The count was never the honest part; the window is. Our own reports state the storefront, the collection date and that period coverage is not guaranteed — this study is the argument for going further and printing the window itself.
If you take one thing from this: before you act on a theme list, ask what period it covers. If the answer is "the latest 200 reviews", you do not have an answer yet.
Disclosure: I work on ReviewFold, which analyzes app reviews. Our reports state the storefront, the collection date and that period coverage is not guaranteed; printing the window itself is the next step, and this study is the argument for it. The measurements above come from public App Store responses, collected with a script published alongside the dataset. AI assistance was used for editing; the data collection and the numbers are reproducible from the script.
Originally published at reviewfold.com
Top comments (0)