Scrape the full Amazon Best Sellers Top 100 (a chart page only renders 30 of 50)

Amazon Best Sellers is the one ranking on the site that comes from actual sales, and for amazon product research it is the fastest read on a category: what it charges, how crowded the top is, how many ratings a top-10 product carries. Two chart pages are the Top 100, and most scrapers return something that looks like it and is not.

1. Where the missing ranks go

A chart page holds 50 ranks, but the server renders only 30 as HTML; the browser lazy-loads the other 20 as you scroll. Read the page as served and you get ranks 1–30, then 51–80 from page 2. That scraper reports "the Top 100" as 60 rows, and the gaps are silent — nothing in the output says 31–50 are missing.

The page does carry the complete ranked list, though, because its own lazy-loader needs it: every ASIN with its rank, in the served HTML. The Amazon Best Sellers Scraper on Apify reads that list, then makes the same load-more call the browser makes, which returns the remaining 20 entries rendered like the first 30: title, price, rating, rating count, image. If that call is refused, the missing entries are completed from their product pages; if that fails too, the row still arrives with rank, asin and url, and the free status row counts it under partial. pages: 2 is ranks 1–100, contiguous, and the run says how complete it is.

curl -X POST "https://api.apify.com/v2/acts/kestrel~amazon-best-sellers-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"categories": ["electronics"], "chart": "bestsellers", "pages": 2, "domain": "com"}'

A product row — rank 31, the first lazy-loaded entry:

{ "type": "product", "rank": 31, "asin": "B0CFPJYX7P", "title": "Example Wireless Earbuds, Bluetooth 5.4",
  "chart": "bestsellers", "category": "electronics", "page": 1,
  "price": 24.99, "price_display": "$24.99", "currency": "USD", "rating": 4.5, "ratings_count": 18342,
  "image": "https://m.media-amazon.com/images/I/example.jpg", "url": "https://www.amazon.com/dp/B0CFPJYX7P",
  "domain": "com", "marketplace": "US", "fetched_at": "2026-08-29T06:20:11+00:00" }

fillRanks: false turns the completion off: one request per page, 30 rows, the gaps back. maxResultsPerCategory: 20 caps each category and minRating: 4.5 drops the rest; neither is billed. chart: "new_releases" pulls New Releases with the same slugs.

2. Python: a BSR tracker in a CSV

rank is the product's amazon best sellers rank within that category node; keep the runs and rank history is a group-by. This appends today's chart and prints new top-20 entrants.

import csv, os
from pathlib import Path
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("kestrel/amazon-best-sellers-scraper").call(run_input={
    "categories": ["electronics", "electronics/headphones"], "chart": "bestsellers", "pages": 2, "domain": "com"})
rows = [r for r in client.dataset(run["defaultDatasetId"]).iterate_items() if r["type"] == "product"]

path = Path("bsr_history.csv")
seen = {(r[1], r[2]) for r in csv.reader(path.open())} if path.exists() else set()
with path.open("a", newline="") as f:
    w = csv.writer(f)
    for r in rows: w.writerow([r["fetched_at"][:10], r["category"], r["asin"], r["rank"], r["price"], r["rating"], r["ratings_count"], r["title"]])
new = [r for r in rows if r["rank"] <= 20 and (r["category"], r["asin"]) not in seen]
for r in new: print(f"new in top 20: #{r['rank']} {r['category']} {r['asin']} {r['price_display']} {(r['title'] or '')[:60]}")

The change in ratings_count between runs is a better velocity signal than rank alone, and only exists if you keep the runs. Join a chart: "new_releases" run on asin: a product that reaches the Best Sellers top 50 within a month of appearing there is a launch that worked.

3. Rank history in Google Sheets, alerts in Slack (n8n)

The n8n template Amazon Best Sellers rank tracker → Google Sheets (n8n/ folder of kestrel-actors-examples) is seven nodes: a 06:00 Schedule trigger, an HTTP Request to the run-sync endpoint for the Top 50 of one category, a Code node that looks up yesterday's rank per category|chart|asin in workflow static data and adds previous_rank and change (+3, -2, 0, new), a Google Sheets append of all 50 rows, an IF for change = new and rank ≤ 10, Slack, and a NoOp. The first run only seeds the history; pages: 2, maxResultsPerCategory: 100 makes it the Top 100.

4. From rank to reviews

Every row carries asin, which is the input the Amazon Reviews Scraper wants:

asins = [r["asin"] for r in rows if r["rank"] <= 10]
reviews = client.actor("kestrel/amazon-reviews-scraper").call(run_input={"asins": asins, "domain": "com", "minRating": 3, "requireText": True})

That returns the reviews Amazon shows on the public product page — roughly 8–13 per product. The full archive sits behind a login and the actor does not attempt it: enough for "what sells, and what its buyers complain about", not a complete review corpus.

5. Cost and limits

• $0.004 per delivered product row; status rows, filtered rows and empty categories are free. Top 100 of one category: $0.40. Ten categories daily: $4 a day. The reviews actor is $0.005 per review row plus $0.003 per product row.
• rank is chart position, not units sold; Amazon does not publish sales volume.
• Prices are what the page rendered, in the currency it rendered; price is null when the page prints a range.
• Movers & Shakers is rendered entirely client-side — the served page carries no entries — so it is not offered.
• Slugs differ between marketplaces; on no_results, copy the segment after /gp/bestsellers/ from that marketplace's URL.
• Apify Proxy is required; the run rotates IP on a captcha.

That is the whole thing: one run for a complete chart, a schedule for the rank history, one join for the reviews. Full input/output reference and FAQ on the actor page: apify.com/kestrel/amazon-best-sellers-scraper.

Every actor's inputs, output fields, a sample row and its honest limits are documented at mtedj.github.io/kestrel-actors-examples, including a side-by-side comparison of every review scraper.