DEV Community

quiethand098
quiethand098

Posted on

Scraping App Store and Google Play reviews without an API key

#webscraping #javascript #mobile #tutorial

Product teams and indie devs keep asking the same thing: how do I pull reviews for my app (or a competitor's) into a spreadsheet? Both stores have endpoints you can call without any key.

Apple App Store

Apple still serves customer reviews as JSON:

https://itunes.apple.com/us/rss/customerreviews/page=1/id=310633997/sortby=mostrecent/json
Enter fullscreen mode Exit fullscreen mode
  • 50 reviews per page, pages 1 to 10, so at most 500 recent reviews per app and country.
  • Change us to gb, de, jp and so on to get each storefront separately.
  • The first entry in the feed is app metadata, not a review. Skip entries without im:rating.
  • App name: https://itunes.apple.com/lookup?id=310633997.

Google Play

There is no public API, but the Play website loads reviews through an internal batchexecute call (rpc id UsvDTd):

const inner = JSON.stringify([null, null, [2, /*sort: 2 newest, 1 helpful*/ 2, [150, null, token], null, []], ['com.whatsapp', 7]]);
const body = 'f.req=' + encodeURIComponent(JSON.stringify([[['UsvDTd', inner, null, 'generic']]]));
const r = await fetch('https://play.google.com/_/PlayStoreUi/data/batchexecute?hl=en&gl=us', {
  method: 'POST', body,
  headers: { 'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8' } });
const data = JSON.parse(JSON.parse((await r.text()).split('\n')[2])[0][2]);
const reviews = data[0];      // each: [id, [name,...], rating, , text, [epochSeconds], thumbsUp, , [, replyText, [replyEpoch]], ... , version at index 10]
const nextToken = data[1]?.[1]; // pass back as `token` for the next page
Enter fullscreen mode Exit fullscreen mode

Page until nextToken is empty. Expect thousands of reviews per app. The format is undocumented and shifts now and then, so check the array indexes when it breaks.

Useful filters

The highest-value slice is usually 1 and 2 star reviews: that is where bug reports and feature requests hide. Filter on rating after fetching, or sort by helpfulness on Google Play.

Prefer not to maintain it

I wrapped both in one Apify actor with the same output shape for both stores, multi-country, star filters and developer replies. About $2 per 1,000 reviews.

Other small data actors by me: Google News with real article URLs and YouTube transcripts with RAG chunks.

Top comments (0)