Introduction
This tutorial will teach us to scrape Google Play Apps results using Node JS with Unirest and Cheerio.
Requirements:
Web Parsing with CSS selectors
Searching the tags from the HTML files is not only a difficult thing to do but also a time-consuming process. It is better to use the CSS Selectors Gadget for selecting the perfect tags to make your web scraping journey easier.
This gadget can help you to come up with the perfect CSS selector for your need. Here is the link to the tutorial, which will teach you to use this gadget for selecting the best CSS selectors according to your needs.
User Agents
User-Agent is used to identify the application, operating system, vendor, and version of the requesting user agent, which can save help in making a fake visit to Google by acting as a real user.
You can also rotate User Agents, read more about this in this article: How to fake and rotate User Agents using Python 3.
If you want to further safeguard your IP from being blocked by Google, you can try these 10 Tips to avoid getting Blocked while Scraping Google.
Install Libraries
To start scraping Google Play Apps Results we need to install some NPM libraries, so that we can move forward.
So before starting, we have to ensure that we have set up our Node JS project and installed both the packages - Unirest JS and Cheerio JS. You can install both packages from the above link.
Target:
Process:
Let's begin the process of scraping the Google Play Apps Results. We will be using Unirest JS to extract the raw HTML data and parse this data with the help of Cheerio JS.
Note: We will not scrape the "Top Charts" results from Google Play as it is loaded by Javascript, which requires you to use Puppeteer JS to scrape these results, which is very CPU intensive and a slow method.
Open the below link in your browser, so we can start selecting the HTML tags for the required elements.
https://play.google.com/store/apps
Let us make a GET request using Unirest JS on the target URL.
const unirest = require("unirest");
const cheerio = require("cheerio");
const getGooglePlayData = async () => {
const url = "https://play.google.com/store/apps";
let head: {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"
}
const response = await unirest
.get(url)
.header(head)
const $ = cheerio.load(response.body);
Step-by-step explanation:
- In the first and second lines, we declared the constant for the Unirest and Cheerio libraries.
- In the next line, we declared a function to get the Google Play Data.
- After that, we declared a constant for the URL and a head object which consist of the User Agent.
- Next, we made the request on the URL with the help of Unirest.
- In the last line, we declared a cheerio instance variable to load the response.
Now, we will prepare our parser by searching the tags with the help CSS selector gadget, stated above in the Requirements section.
All the title tabs you saw above are inside an HTML tag .kcen6d. So, its parser will look like this:
let category = [];
$(".kcen6d").each((i,el) => {
category[i] = $(el).text()
})
We just declared an array for storing the titles in it. And then with the help of Cheerio constant $, which we declared above, parsed all the titles with the matching selector tag.
Now, we will scrape the Apps under these titles.
These apps are under the container called section here. So its parser would look like this:
let organic_results = [];
$("section").each((i,el) => {
let results = [];
if(category[i] !== "Top charts" && category[i] !== undefined)
{
$(el).find(".ULeU3b").each((i,el) => {
results.push({
app_name: $(el).find(".Epkrse").text(),
rating: $(el).find(".LrNMN").text(),
thumbnail: $(el).find(".TjRVLb img").attr("src"),
link: $(el).attr("href")
})
})
organic_results.push({
category: category[i],
results: results
})
}
})
First, we selected the tag for the container as shown in the above image. Then we declared an array to store the Apps results inside it.
After that, we defined a condition that if an element in category array consist of the title "Top Charts" and an element in the category array is undefined, then we will not do anything. If all the conditions are true, we select the tag for each App and loop over it to get the needed details and store it in the results array.
Here you can see which parent tag I selected for collecting the data:
After that, we stored the results and the current title in the category array in the new array named as organic_results.
Now, our results should look like this:
{
category: 'Recommended for you',
results: [
{
app_name: 'WhatsApp Messenger',
rating: 4.1,
thumbnail: 'https://play-lh.googleusercontent.com/bYtqbOcTYOlgc6gqZ2rwb8lptHuwlNE75zYJu6Bn076-hTmvd96HH-6v7S0YUAAJXoJN=s256-rw',
link: 'https://play.google.com/store/apps/details?id=com.whatsapp'
},
{
app_name: 'YouTube',
rating: 4.1,
thumbnail: 'https://play-lh.googleusercontent.com/lMoItBgdPPVDJsNOVtP26EKHePkwBg-PkuY9NOrc-fumRtTFP4XhpUNk_22syN4Datc=s256-rw',
link: 'https://play.google.com/store/apps/details?id=com.google.android.youtube'
},
{
app_name: 'Instagram',
rating: 4.3,
thumbnail: 'https://play-lh.googleusercontent.com/c2DcVsBUhJb3UlAGABHwafpuhstHwORpVwWZ0RvWY7NPrgdtT2o4JRhcyO49ehhUNRca=s256-rw',
link: 'https://play.google.com/store/apps/details?id=com.instagram.android'
},
.....
Here is the complete code:
const unirest = require("unirest");
const cheerio = require("cheerio");
const getGooglePlayData = async() => {
let url = "https://play.google.com/store/apps"
let response = await unirest
.get(url)
.headers({
"User-Agent":
"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36",
})
const $ = cheerio.load(response.body)
let category = [], organic_results = [];
$(".kcen6d").each((i,el) => {
category[i] = $(el).text()
})
$("section").each((i,el) => {
let results = [];
if(category[i] !== "Top charts" && category[i] !== undefined)
{
$(el).find(".ULeU3b").each((i,el) => {
results.push({
app_name: $(el).find(".Epkrse").text(),
rating: parseFloat($(el).find(".LrNMN").text()),
thumbnail: $(el).find(".TjRVLb img").attr("src"),
link: "https://play.google.com" + $(el).find("a").attr("href"),
})
})
organic_results.push({
category: category[i],
results: results
})
}
})
console.log(organic_results[0]);
};
getGooglePlayData();
Serpdog Google Search API
If you don't want to code and maintain the scraper in the long run and don't want to work with complex URLs and HTML, then you can try this Google Search API.
Serpdog | Google Search API solves all the problem of captchas and proxies and allow developers to scrape Google Search Results smoothly. Also, the pre-cooked structured JSON data can save you a lot of time.
const axios = require('axios');
axios.get('https://api.serpdog.io/search?q=coffee&api_key=APIKEY')
.then(response => {
console.log(response.data);
})
.catch(error => {
console.log(error);
});
Register to get the API Key with 100 free credits.
Results:
{
"meta": {
"api_key": "APIKEY",
"q": "coffee",
"gl": "us"
},
"knowledge_graph": {
"title": "Coffee",
"type": "Drink",
"header_images": [
{
"source": "https://en.wikipedia.org/wiki/Coffee",
"image": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"source": "https://www.tastingtable.com/718678/coffee-brands-ranked-from-worst-to-best/",
"image": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"source": "https://www.healthline.com/nutrition/top-evidence-based-health-benefits-of-coffee",
"image": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"source": "https://www.bbcgoodfood.com/recipes/collection/coffee-recipes",
"image": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"description": "Coffee is a brewed drink prepared from roasted coffee beans, the seeds of berries from certain flowering plants in the Coffea genus. From the coffee fruit, the seeds are separated to produce a stable, raw product: unroasted green coffee.",
"source": {
"name": "Wikipedia",
"link": "https://en.wikipedia.org/wiki/Coffee"
},
"Acidity level": "4.85 to 5.10",
"list": {
"Total Fat": [
"0 g",
"0%"
],
"Saturated fat": [
"0 g",
"0%"
],
"Trans fat regulation": [
"0 g",
""
],
"Cholesterol": [
"0 mg",
"0%"
],
"Sodium": [
"5 mg",
"0%"
],
"Potassium": [
"116 mg",
"3%"
],
"Total Carbohydrate": [
"0 g",
"0%"
],
"Dietary fiber": [
"0 g",
"0%"
],
"Sugar": [
"0 g",
""
],
"Protein": [
"0.3 g",
"0%"
],
"Caffeine": [
"95 mg",
""
],
"Vitamin C": [
"",
"0%"
],
"Calcium": [
"",
"0%"
],
"Iron": [
"",
"0%"
],
"Vitamin D": [
"",
"0%"
],
"Vitamin B6": [
"",
"0%"
],
"Cobalamin": [
"",
"0%"
],
"Magnesium": [
"",
"1%"
]
},
"Compounds In Coffee": [
{
"title": "Chlorogenic acid",
"link": "https://google.com/search?gl=us&hl=en&q=Chlorogenic+acid&stick=H4sIAAAAAAAAAONgFuLUz9U3MCorTMtVAjPNizNy4rVEspOt9JPzc3Pz86xS8svzyhOLUopXMQo6Z6TmZiYn5jjn5xbkl-alFC9iFXDOyMkvyk9PzctMVkhMzkzZwcoIAJ5I0VJYAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhlEAU",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Trigonelline",
"link": "https://google.com/search?gl=us&hl=en&q=Trigonelline&stick=H4sIAAAAAAAAAONgFuLUz9U3MCorTMtVAjNNqwwq47VEspOt9JPzc3Pz86xS8svzyhOLUopXMQo6Z6TmZiYn5jjn5xbkl-alFC9i5QkpykzPz0vNycnMS93ByggAa_A38FQAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhlEAc",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Quinic acid",
"link": "https://google.com/search?gl=us&hl=en&q=Quinic+acid&stick=H4sIAAAAAAAAAONgFuLUz9U3MCorTMtVAjPNi5NNLbREspOt9JPzc3Pz86xS8svzyhOLUopXMQo6Z6TmZiYn5jjn5xbkl-alFC9i5Q4szczLTFZITM5M2cHKCAAvEyRDUwAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhlEAk",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Kahweol",
"link": "https://google.com/search?gl=us&hl=en&q=Kahweol&stick=H4sIAAAAAAAAAONgFuLUz9U3MCorTMtV4gIxTdLMDLJNtESyk630k_Nzc_PzrFLyy_PKE4tSilcxCjpnpOZmJifmOOfnFuSX5qUUL2Jl907MKE_Nz9nByggA3TsOH1AAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhlEAs",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"Starbucks Coffee": [
{
"title": "Starbucks Roast Dark Ro...",
"link": "https://google.com/search?gl=us&hl=en&q=Starbucks+Roast+Dark+Roast+Ground+Coffee&stick=H4sIAAAAAAAAAC3IMQrCMBQAUEQKOriIByhO4vKtLtJVwV0PUNLkJ8Yk_8ekteBxHD2Bx9PB7fEm4_kUAmy2j7sOyxkYqKqblrun27frhZM1SA6BqVY80CCSyu9R9TvvUXaWCRzx4FEZbKIg9BnylWO0ZBoXG_TW2Nbjq1hdOpHaXrpcnlnkrjyK5P48Je5JlQfWGvFTjL5shC6JlAAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhmEAU",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Starbucks Frappuccino, Caramel...",
"link": "https://google.com/search?gl=us&hl=en&q=Starbucks+Frappuccino,+Caramel,+13.7+Oz&stick=H4sIAAAAAAAAAA3IMQrCMBQAUIoUdHARDxBcBCmmqYLQVXB18AAlTX9jyP9JTFoLHsfRE3g8feObz1YLTrysno-eNkuuuRC6OpZEttqtraq58kTe1Z2f3CRjlz6Z-B8iqMF4x63zE0KnoQnSASae7j4E43RjQwNotGkR3vn2NsjYjsomdokyhFEp43zBzjJKAiyYOOxP7Pr65tkPqGF6KJMAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhmEAc",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Starbucks Coffee Drink Sw...",
"link": "https://google.com/search?gl=us&hl=en&q=Starbucks+Coffee+Drink+Sweet+Cream&stick=H4sIAAAAAAAAAA3IMQ7CIBQA0DSmiQ4uxgOQji7fmtSYrvUGPQCh9BcJ8KGAch9HT-Dx9I1vuznswMH58loX1-xBQdtSl1d55bfT0cgepHfOUz_7QkXEOX2q9n_WoszaExjyxeKskAdBaBOkhw9Bk-ImcLRa6cniu27GLOL0lCaxwS8LIrtHTYaNBTGzIaJw37r6AeTgLQeOAAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhmEAk",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Starbucks Roast Coffee...",
"link": "https://google.com/search?gl=us&hl=en&q=Starbucks+Roast+Coffee+Whole+Bean&stick=H4sIAAAAAAAAAA3IMQ6CMBQA0BBDooOL8QDV0eUDgwOj3kAHR1LKpzRt_y98lPs4egKPp29869VuAxGK6jX28bgFC2UpRZLxnOJp700NhmNkqjteaNFTJ5-s_F8IaGbHBJ54CdhZbJImDAIycEqObONTg8FZ1wZ854f7rKf2abyoG2uZ1ZX7HlE9Bg6oLqjpm2c_as3ybY0AAAA&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhmEAs",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"People also search for": [
{
"title": "Tea",
"link": "https://google.com/search?gl=us&hl=en&q=Tea&si=AC1wQDCwN61-ebmuwbQCO5QCrgOvEq5bkWeIzJ5JczItzAKNdTku-1rX0tcMn1LKdXOIpKJwlLmVSfa90_0znjAMcL2sAzX4J_UwMexVra0SvN9zBEkwGwA5u6HSaoI47SyRCD9nb-8Glzx2c5rfIJe-aizf7IAxGOhJh_kG0qL9FIdiWvdohKRx4gRfu0e1hMv_cus-OR01faxKayI3akdtDTPtZF4-BCaT_0qctWJy1y5xtkAOr0_dsLhobE2rbdSGteM1GLxNZDARFa9umM8-6UkPFTmf4Q%3D%3D&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhnEAU",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Espresso",
"link": "https://google.com/search?gl=us&hl=en&q=Espresso&si=AC1wQDCwN61-ebmuwbQCO5QCrgOvEq5bkWeIzJ5JczItzAKNdeHFN2RiKqNUOGcF0pmVdz3rIrXiuMEhNdO-SfahftpqQIYK3-hB1rA_kOlkAeWqSr6HTBB8G84PquMrIFHofy3tS0AENRg2iT4kQhARW-cqiwd5mwe4HWoM7NfCdXPHKL-9vcPCuQWsMQH0QsiUPQef1wfYHgpF76EoCEd9WgK2RCOVSWy2_UvrDO9ycBWVfnNdmkvoeclcDeyjTxdu3vRmri1-sGC3Vxg1Xf8C1P-tSU7Ac6Db7sVDHZWBhhei19_FP0o%3D&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhnEAc",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Drink",
"link": "https://google.com/search?gl=us&hl=en&q=Beverage&si=AC1wQDC0KliLIoeyh35S2UVSh9d0at62ormA4JRB9xLdTgvleVEPuIJW3qN0rEFDMoRO5TaIi-eUtrO7i-yBcPsfUzSls6PoTU-buveIcXrKc8j6j8RTk6levipGBVfMDb2AmQYFIlFivgjEWtrw2TTOjzXWO7h0VnyOV-MXRLUeB3vIgKfdyVVgegWm85plPQ3qoUo00QwqxhwByVeRhWtTwWxm0nV7Mhs4OnmQz88vCcewdr1PGbonYzW4YfEznNahmd_nLsKERiRzf7PW6RoKR9MqwLl7tKt3KMgvOj5nUbDOmdXibLM%3D&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhnEAk",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Iced coffee",
"link": "https://google.com/search?gl=us&hl=en&q=Iced+coffee&si=AC1wQDDagiMg03ncxeOQZbwVe-CJxRCchC-jr2hCPTxjc9wbgDb1tUqijDJX85Pg7-Fd4h4ECiqrTLXgOQJbaZmoXfWb8MZKyZ7uj6FykoR0HPdYz7APuA16Lh3IDaqobb6GC1g6vA53Q0ZorgA-FFa18joGMuV0h3kKB2XsyLL0UeBqUl5Ia_r_Auk9F-wZzVIhpHKkmIq6gXo1DP_vcZa8jyTzXy4VXG7vcXdoDmzCjZTsJtClzIOpna_wgbYFRtYbRWqfGISfDDvhjFbeGQDR5bN1gsLT_ra5gs0FdkoknXCL4O-AXnM%3D&sa=X&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQxA16BAhnEAs",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"see_results_about": [
{
"title": "Coffee",
"snippet": "Plant",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
]
},
"inline_videos": [
{
"title": "Miguel - Coffee (Official Video)",
"source": "YouTube",
"date": "Jun 16, 2015",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Kelly Rowland - COFFEE (Official Music Video)",
"source": "YouTube",
"date": "Apr 17, 2020",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Beabadoobee - Coffee",
"source": "YouTube",
"date": "Mar 26, 2020",
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"local_results": [
{
"position": 1,
"title": "Scooter's Coffee",
"place_id": "5225762576898740739",
"rating": "4.4",
"address": "Coffeyville, KS",
"hours": "Closed ⋅ Opens 7AM",
},
{
"position": 2,
"title": "Terebinth Coffee House & Roastery",
"place_id": "18371722425790879519",
"rating": "4.9",
"address": "Coffeyville, KS",
"hours": "Closed ⋅ Opens 6:30AM Mon",
},
{
"position": 3,
"title": "Ane Mae's Coffee and Sandwich Shop",
"place_id": "1687994686344788766",
"rating": "4.7",
"address": "Independence, KS",
"hours": "Closed ⋅ Opens 7AM Mon",
},
{
"more_locations_link": "https://www.google.com/search?gl=us&hl=en&q=coffee&npsic=0&rflfq=1&rldoc=1&rllag=37129806,-95662854,11423&tbm=lcl&sa=X&ved=2ahUKEwiGzIer7qL5AhXCKEQIHdh0D5YQtgN6BAgmEAE"
}
],
"recipes_results": [
{
"title": "Coffee recipes",
"link": "https://www.bbcgoodfood.com/recipes/collection/coffee-recipes",
"source": "BBC Good Food",
"rating": "",
"reviews": "",
"total_time": "",
"ingredients": [
"Instant coffee"
],
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "20 Great Coffee Drinks From Around the World",
"link": "https://insanelygoodrecipes.com/coffee-recipes/",
"source": "Insanely Good Recipes",
"rating": "4.5",
"reviews": "8",
"total_time": "",
"ingredients": [
"Turkish coffee",
" vietnamese coffee",
" white chocolate mocha",
" cold brew coffee",
" frappuccino"
],
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
},
{
"title": "Bulletproof Coffee Recipe",
"link": "https://www.bulletproof.com/recipes/bulletproof-diet-recipes/bulletproof-coffee-recipe/",
"source": "Bulletproof",
"rating": "",
"reviews": "",
"total_time": "5 min",
"ingredients": [
"Mct oil",
" bulletproof coffee",
" grass fed"
],
"thumbnail": "data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
}
],
"peopleAlsoAskedFor": [
"What are benefits of coffee?",
"Which is the best coffee to drink?",
"Is it healthy to eat coffee?",
"Is coffee made from poop?"
],
"organic_results": [
{
"title": "9 Health Benefits of Coffee, Based on Science - Healthline",
"link": "https://www.healthline.com/nutrition/top-evidence-based-health-benefits-of-coffee",
"displayed_link": "https://www.healthline.com › Wellness Topics › Nutrition",
"snippet": "Coffee is a popular beverage that researchers have studied extensively for its many health benefits, including its ability to increase energy levels, promote ...",
"rank": 1
},
{
"title": "The Coffee Bean & Tea Leaf | CBTL",
"link": "https://www.coffeebean.com/",
"displayed_link": "https://www.coffeebean.com",
"snippet": "Born and brewed in Southern California since 1963, The Coffee Bean & Tea Leaf® is passionate about connecting loyal customers with carefully handcrafted ...",
"rank": 2
},
{
"title": "Peet's Coffee: The Original Craft Coffee",
"link": "https://www.peets.com/",
"displayed_link": "https://www.peets.com",
"snippet": "Since 1966, Peet's Coffee has offered superior coffees and teas by sourcing the best quality coffee beans and tea leaves in the world and adhering to strict ...",
"rank": 3
},
{
"title": "The History of Coffee - National Coffee Association",
"link": "https://www.ncausa.org/about-coffee/history-of-coffee",
"displayed_link": "https://www.ncausa.org › ... › History of Coffee",
"snippet": "Coffee grown worldwide can trace its heritage back centuries to the ancient coffee forests on the Ethiopian plateau. There, legend says the goat herder ...",
"inline_sitelinks": [
{
"title": "An Ethiopian Legend",
"link": "https://www.ncausa.org/about-coffee/history-of-coffee#:~:text=An%20Ethiopian%20Legend"
},
{
"title": "The Arabian Peninsula",
"link": "https://www.ncausa.org/about-coffee/history-of-coffee#:~:text=The%20Arabian%20Peninsula,-Coffee%20cultivation%20and%20trade%20began"
},
{
"title": "Coffee Comes To Europe",
"link": "https://www.ncausa.org/about-coffee/history-of-coffee#:~:text=Coffee%20Comes%20to%20Europe"
}
],
"rank": 4
},
{
"title": "coffee | Origin, Types, Uses, History, & Facts | Britannica",
"link": "https://www.britannica.com/topic/coffee",
"displayed_link": "https://www.britannica.com › ... › Food",
"snippet": "coffee, beverage brewed from the roasted and ground seeds of the tropical evergreen coffee plants of African origin. Coffee is one of the three most popular ...",
"rank": 5
},
{
"title": "Starbucks Coffee Company",
"link": "https://www.starbucks.com/",
"displayed_link": "https://www.starbucks.com",
"snippet": "More than just great coffee. Explore the menu, sign up for Starbucks® Rewards, manage your gift card and more.",
"rank": 6
},
{
"title": "#coffee hashtag on Instagram • Photos and videos",
"link": "https://www.instagram.com/explore/tags/coffee/",
"displayed_link": "https://www.instagram.com › explore › tags › coffee",
"snippet": "156M Posts - See Instagram photos and videos from 'coffee' hashtag.",
"rank": 7
}
],
"relatedSearches": [
{
"query": "coffee near me",
"link": "www.google.com/search?gl=us&q=coffee near me"
},
{
"query": "coffee shop",
"link": "www.google.com/search?gl=us&q=coffee shop"
},
{
"query": "coffee benefits",
"link": "www.google.com/search?gl=us&q=coffee benefits"
},
{
"query": "best coffee",
"link": "www.google.com/search?gl=us&q=best coffee"
},
{
"query": "coffee - wikipedia",
"link": "www.google.com/search?gl=us&q=coffee - wikipedia"
},
{
"query": "coffee origin",
"link": "www.google.com/search?gl=us&q=coffee origin"
}
],
"pagination": {
"current": "1",
"next": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=10&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8NMDegQIAxBK",
"page_no": {
"2": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=10&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxA4",
"3": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=20&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxA6",
"4": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=30&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxA8",
"5": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=40&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxA-",
"6": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=50&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBA",
"7": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=60&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBC",
"8": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=70&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBE",
"9": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=80&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBG",
"10": "https://www.google.com/search?q=coffee&gl=us&hl=en&ei=R0TmYuSlH6KdkPIPvs6lqA8&start=90&sa=N&ved=2ahUKEwik5fC64aL5AhWiDkQIHT5nCfUQ8tMDegQIAxBI"
}
},
"serpdog_pagination": {
"current": "1",
"page_no": {
"2": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=10",
"3": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=20",
"4": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=30",
"5": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=40",
"6": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=50",
"7": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=60",
"8": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=70",
"9": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=80",
"10": "https://api.serpdog.io/search?api_key=APIKEY&q=coffee&gl=us&start=90"
}
}
}
Conclusion:
In this tutorial, we learned to scrape Google Play Results with Node JS. Feel free to message me if I missed something. Follow me on Twitter. Thanks for reading!
Additional Resources
- Web Scraping Google With Node JS - A Complete Guide
- Web Scraping Google Without Getting Blocked
- Scrape Google Organic Search Results
- Scrape Google Shopping Results
- Scrape Google Maps Reviews
Author:
My name is Darshan, and I am the founder of serpdog.io. I love to create scrapers. I am currently working for several MNCs to provide them with Google Search Data through a seamless data pipeline.
Top comments (0)