DEV Community

Gio Rich
Gio Rich

Posted on

How to get Amazon search results as JSON (rank, price, rating, sponsored flag) on any marketplace with Python or Node

#webscraping #python #api #ecommerce

I built this actor; it's a paid tool on Apify with a free trial credit.

Amazon's search page is the data everyone in e-commerce wants and nobody enjoys collecting: which products rank for a keyword, at what price, with how many ratings, how many sponsored slots sit above the first organic result, and how that changes by marketplace. The HTML is big, the card layout differs between marketplaces, prices come in a different format on every one of them, and Amazon blocks datacenter IPs with captchas, AWS WAF challenges and bot-manager interstitials.

I'm an 18-year-old engineering student, and I wrapped all of that in an actor called Amazon Search Scraper. This post shows how to call it from Python and Node, what the rows look like, and a small project: a daily keyword rank tracker.

How it works, briefly

You give it keywords (plus a marketplace, department, sort order and price range) or full Amazon search URLs with any filters already applied. It fetches the result pages over plain HTTP through residential proxies in the marketplace's country, no browser, and parses every product card: position, ASIN, title, price and list price, currency, rating, ratings count, "10K+ bought in past month", Best Seller / Amazon's Choice badges, deal and coupon text, delivery text, image and URL, with sponsored listings flagged and numbered separately from organic ones. It follows pagination until your limit, deduplicates ASINs per search, and retries blocked pages on a fresh session. Turn on includeDetails and it also opens each product page for brand, bullet points, seller, Best Sellers Rank, category path, images and the spec table.

Python example 

pip install apify-client
Enter fullscreen mode Exit fullscreen mode 
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run = client.actor("rel8ble/amazon-search-scraper").call(run_input={
    "keywords": ["wireless earbuds", "bluetooth speaker"],
    "domain": "amazon.com",
    "department": "electronics",
    "sort": "relevance",
    "maxItemsPerSearch": 50,
    "includeSponsored": True,
})

for p in client.dataset(run["defaultDatasetId"]).iterate_items():
    flag = "AD " if p["isSponsored"] else "   "
    print(f'{flag}{p["position"]:>3} {p["asin"]} {p["price"]!s:>8} {p["rating"]!s:>4} {p["ratingsCount"]!s:>8}  {p["title"][:60]}')
Enter fullscreen mode Exit fullscreen mode

Output (shortened):

AD    1 B09XPX59JK    19.99  4.3    19200  Wireless Earbuds, Bluetooth 5.4 Headphones in Ear with 4 Noise
AD    2 B0D2HJDH94    28.99  4.3    12700  Wireless Bluetooth 5.4 Earbuds, LED Power Display Headphones
      3 B0FQFB8FMG   199.00  4.4    16100  Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation
      4 B0H5J3FDR2    20.98  4.6       48  Wireless Earbuds, Sports Bluetooth Headphones, Running Headphones
Enter fullscreen mode Exit fullscreen mode

Node example 

npm install apify-client
Enter fullscreen mode Exit fullscreen mode 
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });

const run = await client.actor('rel8ble/amazon-search-scraper').call({
    searchUrls: ['https://www.amazon.de/s?k=kopfh%C3%B6rer&i=electronics&rh=p_36%3A2000-5000'],
    maxItemsPerSearch: 30,
    includeDetails: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const p of items) console.log(p.asin, p.price, p.currency, p.details?.brand, p.details?.bestSellersRank?.[0]);
Enter fullscreen mode Exit fullscreen mode

What a row looks like 

{
    "position": 3,
    "organicPosition": 1,
    "asin": "B0FQFB8FMG",
    "title": "Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation Live Translation, Heart Rate Sensing, ...",
    "price": 199,
    "priceText": "$199.00",
    "listPrice": 249,
    "currency": "USD",
    "rating": 4.4,
    "ratingsCount": 16100,
    "boughtLastMonth": 10000,
    "boughtLastMonthText": "10K+ bought in past month",
    "isSponsored": false,
    "isBestSeller": true,
    "isAmazonChoice": false,
    "dealText": null,
    "offersText": "7+ used & new offers",
    "deliveryText": "Join Prime to get FREE delivery Sat, Oct 10 Or Non-members get FREE delivery Tue, Oct 13",
    "imageUrl": "https://m.media-amazon.com/images/I/61solmQSSlL.jpg",
    "url": "https://www.amazon.com/dp/B0FQFB8FMG",
    "page": 1,
    "keyword": "wireless earbuds",
    "marketplace": "amazon.com",
    "scrapedAt": "2026-10-08T18:02:11.420Z"
}
Enter fullscreen mode Exit fullscreen mode

Small project: a keyword rank tracker

Track where a set of ASINs rank for a keyword, organic and overall, once a day, and append to a CSV. Run it from cron or an Apify schedule.

import csv, datetime
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
KEYWORD = "wireless earbuds"
WATCH = {"B0FQFB8FMG", "B0D2HJDH94", "B09XPX59JK"}   # your ASINs and competitors

run = client.actor("rel8ble/amazon-search-scraper").call(run_input={
    "keywords": [KEYWORD], "domain": "amazon.com", "maxItemsPerSearch": 200,
})
items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
today = datetime.date.today().isoformat()
ads_above_first_organic = sum(1 for p in items if p["isSponsored"] and p["position"] < next(q["position"] for q in items if not q["isSponsored"]))

with open("ranks.csv", "a", newline="") as f:
    w = csv.writer(f)
    for p in items:
        if p["asin"] in WATCH:
            w.writerow([today, KEYWORD, p["asin"], p["position"], p["organicPosition"], p["price"], p["rating"], p["ratingsCount"], p["isSponsored"]])
    w.writerow([today, KEYWORD, "ADS_ABOVE_FIRST_ORGANIC", ads_above_first_organic, "", "", "", "", ""])

print(f"{len(items)} products scanned, {len([p for p in items if p['asin'] in WATCH])} watched ASINs found")
Enter fullscreen mode Exit fullscreen mode

A 200-product scan costs $0.40 at $2 per 1,000 results, so a 25-keyword daily tracker is about $10 a month.

Things to know

  • ~300 products per search is where Amazon stops paginating for most keywords. Narrow by department or price band to see more.
  • Sponsored results rotate per request, so two runs minutes apart show different ads. organicPosition is the stable number.
  • Prices follow the visitor's country. The default residential proxy country matches the marketplace, so amazon.de quotes EUR; with a US proxy country it quotes export prices in USD. The currency field follows the text shown.
  • Brand is usually only on the product page, so it's null on search cards unless Amazon prints a brand row; use includeDetails when you need it.
  • Residential proxies are the default. Datacenter IPs work for a while and then hit captchas and WAF challenges; the actor retries on a fresh session, but residential is cheaper in retries than it costs in bandwidth (about 0.4 MB per search page).

If something breaks or a field is wrong, the Issues tab on the actor page reaches me directly.

Amazon Search Scraper on Apify

This article was drafted with AI and published by me.

Top comments (0)