Serpstack. Easy Search Engine Results

Demo code here

This is a sponsored blog post by serpstack. All reviews and opinions expressed here are, however, based on my personal experience.

SERP. I’ll admit, I didn’t know what this meant until a few weeks ago. That’s okay because, appropriately for this post, google is a bro.

Serpstack is an API for search results, including google. Getting google search results can be incredibly valuable and google is quite difficult to scrape just because they are very good at detecting automated attempts and prompt for a recaptcha quickly. Check out the serpstack landing page for more details here.

serpstack is easy

office space is one of the greats

This is my starting code:

(async () => {
    const baseURL = `http://api.serpstack.com/search?access_key=${process.env.serpstackAPIKey}`;

    const axiosResponse = await axios.get(`${baseURL}&query=commercial insurance`);

    console.log('axios response data', axiosResponse.data);

})();

That’s it. Incredibly easy. Now, check how comprehensive the results are. Want to know the ads that show up when using this term? serpstack has you covered:

Interestingly enough, it looks like the ads think we are from Ireland. Serpstack allows you to change your location that you are searching from and we’ll discuss that later.

Now, how about organic results? Beautiful, well formatted results:

How about the local map that often appears when you’re searching? Check it. Fully featured with the link and an image!

local_map: {
    url: 'https://www.google.com/search?safe=off&gl=us&hl=en&q=commercial+insurance&npsic=0&rflfq=1&rldoc=1&rlha=0&rllag=37416705,-95407905,38381&tbm=lcl&sa=X&ved=2ahUKEwj2_vTPwcblAhVhTRUIHZ2dBIMQtgN6BAgKEAQ',
    coordinates: { latitude: 37.416705, longitude: -95.407905 },
    image_url: 'https://static.srpstatic.com/d5ef9ddad87ea3d67d4dcdddc0a47fd17e9e1cbd.png'
  }

Related searches is also there:

Locations

Mind boggling, isn’t it

The results also include local results. In my case, these were not helpful. I’m not in Kansas and I’m not sure why Kansas was arbitrarily choosen.

Fear not. After a little bit of look at the documentation you can see that serpstack has me covered. You can add a location query! Let’s try it.

(async () => {
    const baseURL = `http://api.serpstack.com/search?access_key=${process.env.serpstackAPIKey}`;

    const axiosResponse = await axios.get(`${baseURL}&query=commercial insurance&location=boise`);

    console.log('axios response data', axiosResponse.data);

})();

I just went ahead and added an additional query parameter, location=boise. Bam. Results are different, especially the local ones.

Little plug for UCI. I know the owner and they are great.

serpstack also has an official Locations API that allows you to search for the exact location you would like to use when making a request.

http://api.serpstack.com/locations
    ? access_key = YOUR_ACCESS_KEY
    & query = new york

Example response:

[
    {
        "name": "New York",
        "canonical_name": "New York,New York,United States",
        "country_code": "US",
        "target_type": "City",
        "reach": 23600000,
        "latitude": -74.0059413,
        "longitude": 40.7127837
    },
    {
        "name": "New York, NY",
        "canonical_name": "New York,NY,United States",
        "country_code": "US",
        "target_type": "DMA Region",
        "reach": 23600000,
        "latitude": -74.0059413,
        "longitude": 40.7127837
    },
    {
        "name": "New York",
        "canonical_name": "New York,United States",
        "country_code": "US",
        "target_type": "State",
        "reach": 23600000,
        "latitude": -74.0059413,
        "longitude": 40.7127837
    },
    {
        "name": "Bronx County",
        "canonical_name": "Bronx County,New York,United States",
        "country_code": "US",
        "target_type": "County",
        "reach": 23600000,
        "latitude": -73.8648268,
        "longitude": 40.8447819
    },
    {
        "name": "Kings County",
        "canonical_name": "Kings County,New York,United States",
        "country_code": "US",
        "target_type": "County",
        "reach": 23600000,
        "latitude": -73.9441579,
        "longitude": 40.6781784
    }
]

From this, you can just use the canonical_name in your location parameter for the location you want.

Search types

Serpstack offers a lot of different options of what you can request. You can do a search for images, videos, news, or shopping. Here is the code used for images:

(async () => {
    const baseURL = `http://api.serpstack.com/search?access_key=${process.env.serpstackAPIKey}`;

    const axiosResponse = await axios.get(`${baseURL}&query=commercial insurance&location=boise&type=images`);

    console.log('axios response data', axiosResponse.data);

})();

The results are as the below, including image urls and the source of the image for all of the images.

Serpstack Documentation

The documentation is very robust. They make it easy to use. Here’s an image showing how it defines the results.

I realize this is a sponsored post but I am really impressed with serpstack. Getting reliable search engine results, especially from google, is difficult with web scraping. Google is quick to show a recaptcha and very good at detecting automated use. I wrote a post about trying to get around this manually with proxying and it was no joke.

serpstack gets those results very quickly and in a very well formatted and easy to consume API. I highly recommend them. Pricing is very reasonable and they offer a base plan with 1,000 API calls free each month.

Sign up for serpstack here – https://serpstack.com/

Looking for business leads?

Using the techniques talked about here at javascriptwebscrapingguy.com, we’ve been able to launch a way to access awesome business leads. Learn more at Cobalt Intelligence!

The post Serpstack. Easy Search Engine Results appeared first on JavaScript Web Scraping Guy.