DEV Community: Albin Johansson

Scrape Amazon in Python without proxies: best sellers, niche research and a price tracker

Albin Johansson — Sun, 27 Sep 2026 09:39:20 +0000

Scraping Amazon yourself means rotating proxies, CAPTCHAs and HTML that changes every few weeks. Amazon's official API (PA-API, replaced by the Creators API in 2026) only works for Amazon Associates with 10 qualifying sales in the last 30 days. And seller tools cost $29/month (Jungle Scout) to $99/month (Helium 10), both billed annually.

Here's the other way: Amazon product data as JSON from Python, paid per result. Below are three scripts I actually use: best sellers in a niche, a quick niche check, and a price tracker.

1. Setup

pip install "apify-client>=3"
export APIFY_TOKEN=...   # free account at console.apify.com

I'm using the Amazon Product Scraper on Apify (disclosure: I built it). It reads a commercial Amazon data API instead of loading Amazon in a browser, covers 18 marketplaces and costs $1.50 per 1,000 search results.

2. What actually sells? Rank by "bought in past month"

Amazon shows "5K+ bought in past month" on many listings. It's the closest thing to public sales data, and it comes back as boughtPastMonth:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("jesting_grass/amazon-product-scraper").call(run_input={
    "searchQueries": ["yoga mat"],
    "marketplace": "us",
    "maxResultsPerQuery": 100,
})
rows = [r for r in client.dataset(run.default_dataset_id).iterate_items() if "error" not in r]
rows.sort(key=lambda r: r.get("boughtPastMonth") or 0, reverse=True)
for r in rows[:6]:
    badge = "Best Seller" if r["isBestSeller"] else "Amazon's Choice" if r["isAmazonChoice"] else ""
    print(f'{r.get("boughtPastMonth") or 0:>6,}/mo  ${r["price"] or "-"!s:<7} {r["rating"]}★ {r["reviewsCount"]:>7,} reviews  {badge:<15} {r["title"][:40]}')

Real output (amazon.com, September 2026):

10,000/mo  $-       4.6★  70,600 reviews                  Amazon Basics Extra Thick Exercise Yoga
 8,000/mo  $22.49   4.2★   1,100 reviews                  18 Pcs EVA Foam Floor Tiles, Puzzle Exer
 7,000/mo  $15.99   4.6★   3,800 reviews                  CAP Barbell 1/2-Inch High Density Exerci
 7,000/mo  $36.97   4.7★   1,400 reviews  Best Seller     CAP Barbell Folding Exercise Mat – Durab
 6,000/mo  $21.23   4.6★  70,600 reviews                  Amazon Basics Extra Thick Exercise Yoga
 6,000/mo  $28.16   4.4★  10,600 reviews                  Gruper Yoga Mat Non Slip, Eco Friendly E

Two things stand out. The "Best Seller" badge is per category, so the product selling most in this search (10,000 a month) doesn't have it. And a foam floor tile set with only 1,100 reviews sells 8,000 a month in a "yoga mat" search. That's the kind of gap a seller looks for. ($- means the listing only shows a price after picking a variant.)

3. Check several niche ideas at once

The same data answers "is this niche worth entering?". For each idea: total monthly sales of the top 50 results, median price, median review count, and "low-review winners" (1,000+ sales a month with fewer than 500 reviews):

from statistics import median

IDEAS = ["dog water bottle", "cable organizer", "bamboo cutting board"]
run = client.actor("jesting_grass/amazon-product-scraper").call(run_input={
    "searchQueries": IDEAS, "marketplace": "us", "maxResultsPerQuery": 50,
})
by_query = {}
for r in client.dataset(run.default_dataset_id).iterate_items():
    if "error" not in r:
        by_query.setdefault(r["searchQuery"], []).append(r)

for q in IDEAS:
    rows = by_query[q]
    sold = sum(r.get("boughtPastMonth") or 0 for r in rows)
    winners = [r for r in rows if (r.get("boughtPastMonth") or 0) >= 1000 and (r.get("reviewsCount") or 0) < 500]
    print(f'{q:<22}{sold:>9,}/mo  ${median(r["price"] for r in rows if r.get("price")):.2f}'
          f'  {median(r.get("reviewsCount") or 0 for r in rows):>6,.0f} reviews  {len(winners)} low-review winners')

Real output:

dog water bottle         22,608/mo  $14.98   2,200 reviews  3 low-review winners
cable organizer         177,150/mo  $13.68   3,850 reviews  4 low-review winners
bamboo cutting board     70,511/mo  $21.99   1,300 reviews  3 low-review winners

Cable organizers move almost 8x the volume of dog water bottles at a similar price, and bamboo cutting boards have the lowest review barrier. The sales numbers add up Amazon's rounded labels ("1K+"), so read them as a floor, not an exact count. That run returned 150 products for about $0.23.

4. A price tracker in 20 lines

For price monitoring, pass ASINs (or product URLs) and append the result to a CSV. Schedule it daily and the CSV becomes your price history:

import csv
from datetime import date
from pathlib import Path

ASINS = ["B0B41YH9B6", "B0F8MHPVPH", "B0BLCW7MBR"]
COLUMNS = ["date", "asin", "price", "listPrice", "discountPercent", "isAvailable", "rating", "reviewsCount", "title"]

run = client.actor("jesting_grass/amazon-product-scraper").call(run_input={"asins": ASINS, "marketplace": "us"})
out = Path("prices.csv")
new_file = not out.exists()
with out.open("a", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=COLUMNS, extrasaction="ignore")
    if new_file:
        w.writeheader()
    for row in client.dataset(run.default_dataset_id).iterate_items():
        if "error" not in row:
            w.writerow({"date": date.today().isoformat(), **row})

First rows of the real CSV:

date,asin,price,listPrice,discountPercent,isAvailable,rating,reviewsCount,title
2026-09-27,B0F8MHPVPH,99.99,112.99,12,True,4.5,4857,"FEZIBO Standing Desk 48 × 24 Inch Electric Height Adjustable, Maple"
2026-09-27,B0B41YH9B6,99.99,119.99,17,True,4.5,12020,"ErGear 48 X 24 Inch Height Adjustable Electric Standing Desk, Black"

Full product pages cost $0.008 each, so tracking 3 products daily is about $0.72 a month. The same call also returns brand, specs, all images, variant ASINs and, when Amazon shows them, top reviews.

5. No Python? One HTTP call

curl -X POST "https://api.apify.com/v2/acts/jesting_grass~amazon-product-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchQueries":["standing desk"],"marketplace":"us","maxResultsPerQuery":20}'

It also runs from n8n, Make and Zapier, and AI agents can call it through the Apify MCP server. Switch marketplace to uk, de, jp or 14 others for other Amazon stores.

All examples (best sellers, niche research, price tracker, ASIN details, Node.js, cURL): github.com/emiohr/amazon-product-scraper-api. Questions or feature requests? Drop a comment.

This article was written by an AI agent under my direction, and all code was tested against the live API. Not affiliated with Amazon, Jungle Scout or Helium 10.

Find link-building prospects with Python: a competitor backlink gap for $1 per 1,000 links

Albin Johansson — Sat, 26 Sep 2026 12:17:31 +0000

The best link-building prospects are sites that already link to your competitors but not to you. They cover your niche, they link out, and you're missing from the list. Ahrefs calls this Link Intersect and Semrush calls it Backlink Gap, and both sit behind subscriptions starting at $129/month (Ahrefs Lite) and $139/month (Semrush SEO plan).

Here's how to get a competitor backlink gap, plus the full backlink list of any site, from Python, paid per result.

1. Setup

pip install "apify-client>=3"
export APIFY_TOKEN=...   # free account at console.apify.com

I'm using the Backlink Checker on Apify (disclosure: I built it). It reads a commercial backlink index and costs $1 per 1,000 backlinks. Filters run at the source, so you only pay for rows you keep.

2. The link gap: who links to them, but not to you?

Say you're Allbirds, and your competitors are Rothy's and Vivobarefoot:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("jesting_grass/backlink-checker").call(run_input={
    "mode": "linkGap",
    "yourDomain": "allbirds.com",
    "competitors": ["rothys.com", "vivobarefoot.com"],
    "minCompetitorsLinked": 2,     # links to BOTH competitors
    "minDomainRank": 50,
    "maxResultsPerTarget": 10,
})
for row in client.dataset(run.default_dataset_id).iterate_items():
    print(f'{row["referringDomain"]:<30} rank {row["domainRank"]:<4} links to {", ".join(row["linksToCompetitors"])}')

Real output (September 2026):

4.bing.com                     rank 77   links to rothys.com, vivobarefoot.com
medium.com                     rank 75   links to rothys.com, vivobarefoot.com
app.welcometothejungle.com     rank 65   links to rothys.com, vivobarefoot.com
storeleads.app                 rank 64   links to rothys.com, vivobarefoot.com
prnewswire.com                 rank 64   links to rothys.com, vivobarefoot.com
shows.acast.com                rank 62   links to rothys.com, vivobarefoot.com
castbox.fm                     rank 59   links to rothys.com, vivobarefoot.com
zoominfo.com                   rank 59   links to rothys.com, vivobarefoot.com
retailcraft.transistor.fm      rank 57   links to rothys.com, vivobarefoot.com
player.fm                      rank 57   links to rothys.com, vivobarefoot.com

It compares the top 1,000 referring domains of each site. Not every row is a pitch: a search-engine redirect (4.bing.com) and data directories (zoominfo.com, storeleads.app) are noise. But look at the pattern: both competitors got links from podcasts (acast, castbox, player.fm, a retail podcast on transistor.fm) and from press releases (prnewswire.com). That's a concrete plan: pitch the same retail podcasts and publish a press release.

The run above scans three sites ($0.25 each) and returns 10 prospects ($0.002 each), about $0.77 in total.

3. Every backlink of a site → CSV

For audits or anchor-text analysis, switch to backlinks mode:

import csv

run = client.actor("jesting_grass/backlink-checker").call(run_input={
    "mode": "backlinks",
    "targets": ["allbirds.com"],
    "maxResultsPerTarget": 200,
    "onePerDomain": True,      # best link from each linking website
    "dofollowOnly": True,
})
cols = ["domainRank", "pageRank", "sourceUrl", "anchor", "dofollow", "targetUrl", "firstSeen", "lastSeen"]
with open("backlinks.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=cols, extrasaction="ignore")
    w.writeheader()
    for row in client.dataset(run.default_dataset_id).iterate_items():
        if "error" not in row:
            w.writerow(row)

First rows of the real CSV:

domainRank,pageRank,sourceUrl,anchor,dofollow
88,17,https://www.pinterest.com/weareallbirds/,allbirds.com; Opens a new tab,True
84,0,https://goo.gl/iBwLNC,,True
84,0,https://pay.amazon.com/blog/for-shoppers/possible-with-pay-9-eco-friendly-brands-for-more-sustainable-shopping,AllBirds,True

Other filters: minDomainRank, anchorContains (e.g. only links mentioning your brand), and referringDomains mode for one row per linking site.

4. No Python? One HTTP call

curl -X POST "https://api.apify.com/v2/acts/jesting_grass~backlink-checker/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"mode":"backlinks","targets":["allbirds.com"],"maxResultsPerTarget":20,"onePerDomain":true}'

It also runs from n8n, Make and Zapier, and AI agents can call it through the Apify MCP server.

A note on the numbers

Every backlink tool runs its own crawler, so Ahrefs, Semrush, Moz and this index each find a different set of links, and the authority scores differ too. Compare sites within one tool rather than across tools.

All examples (backlinks CSV, link gap, Node.js, cURL): github.com/emiohr/backlink-checker-api. Questions or feature requests? Drop a comment.

This article was written by an AI agent under my direction, and all code was tested against the live API. Not affiliated with Ahrefs, Semrush or Moz.

Keyword research in Python: search volume, difficulty and AI Overviews in bulk (2026)

Albin Johansson — Sat, 26 Sep 2026 11:39:11 +0000

In 2026 the first question about a keyword isn't only "how many searches?" but also "does Google answer it with an AI Overview?". Those keywords often send fewer clicks, and they're exactly the ones to target if you want to be cited in AI answers (AEO).

Google Keyword Planner is free, but it shows volume ranges unless you run active ads, and it has no keyword difficulty or AI Overview data. Ahrefs and Semrush have all of it, starting at $129/month and $139/month.

Here's how to get search volume, keyword difficulty, intent, CPC and the AI Overview flag for a whole list of keywords, paid per keyword, in a few lines of Python.

1. Setup

pip install "apify-client>=3"
export APIFY_TOKEN=...   # free account at console.apify.com

I'm using the Keyword Research Tool on Apify (disclosure: I built it). Search volume, CPC and the 12-month trend come from Google Ads data; difficulty, intent and SERP features come from a commercial SEO database. Keywords without data aren't charged.

2. A keyword list → CSV

import csv, os
from apify_client import ApifyClient

KEYWORDS = ["protein powder", "cold brew coffee", "keyword research tool", "standing desk", "how to make cold brew"]
COLUMNS = ["keyword", "searchVolume", "keywordDifficulty", "intents", "hasAiOverview"]

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("jesting_grass/keyword-research-tool").call(run_input={"keywords": KEYWORDS, "country": "us"})

with open("keywords.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=COLUMNS, extrasaction="ignore")
    w.writeheader()
    for row in client.dataset(run.default_dataset_id).iterate_items():
        if row.get("searchVolume"):
            w.writerow({**row, "intents": ", ".join(row.get("intents") or [])})
            print(f'{row["keyword"]:<24} vol {row["searchVolume"]:>8,}  KD {row["keywordDifficulty"]:<4} AI Overview: {row["hasAiOverview"]}')

Real output (US, September 2026):

protein powder           vol  368,000  KD 32   AI Overview: True
cold brew coffee         vol   40,500  KD 23   AI Overview: True
keyword research tool    vol    4,400  KD 81   AI Overview: True
standing desk            vol  135,000  KD 38   AI Overview: False
how to make cold brew    vol   12,100  KD 4    AI Overview: True

Each row also has cpc (with a low/high bid range), adsCompetition, serpFeatures (featured snippet, People Also Ask, local pack…) and monthlySearches for the last 12 months, so you can spot seasonality.

3. Find easy wins from one seed keyword

Give it a seed and it returns related keyword ideas with the same metrics. Then filter for decent volume and low difficulty:

run = client.actor("jesting_grass/keyword-research-tool").call(run_input={
    "seedKeywords": ["cold brew coffee"],
    "ideasPerSeed": 100,
    "country": "us",
})
ideas = [r for r in client.dataset(run.default_dataset_id).iterate_items() if r.get("source") == "idea"]

easy, groups = [], set()
for r in sorted(ideas, key=lambda r: -(r.get("searchVolume") or 0)):
    # Google reports close variants with identical numbers: keep one keyword per group
    group = (r.get("searchVolume"), r.get("cpc"), r.get("keywordDifficulty"))
    if (r.get("searchVolume") or 0) >= 500 and (r.get("keywordDifficulty") or 100) <= 40 and group not in groups:
        groups.add(group)
        easy.append(r)
for r in easy[:10]:
    print(f'{r["keyword"]:<40} vol {r["searchVolume"]:>7,}  KD {r["keywordDifficulty"]}')

aio = [r for r in ideas if r.get("hasAiOverview")]
print(f"{len(aio)} of {len(ideas)} ideas trigger a Google AI Overview")

Real output:

cold brew coffee maker                   vol  49,500  KD 1
coffee for cold brew                     vol  40,500  KD 23
stōk cold brew coffee                    vol  40,500  KD 4
stok cold brew coffee                    vol  40,500  KD 2
cold brew coffee stok                    vol  40,500  KD 1
cold brew coffee from starbucks          vol  33,100  KD 13
starbucks cold brew coffee               vol  33,100  KD 7
how make cold brew coffee                vol  27,100  KD 11
how to brew cold coffee                  vol  27,100  KD 23
nitro cold brew coffee starbucks         vol  27,100  KD 20
33 of 100 ideas trigger a Google AI Overview

Two things stand out:

A third of this niche gets an AI Overview. For a coffee blog, those pages need to be written to be cited in the AI answer, not just to rank.
Google groups close variants. Keywords like "cold brew coffee maker" and "coffee cold brew maker" come back with identical numbers, which is why the code keeps one per group. Treat each group as one topic and write one page for it; the brand queries (Stōk, Starbucks) are better left to the brands.

4. No Python? One HTTP call

curl -X POST "https://api.apify.com/v2/acts/jesting_grass~keyword-research-tool/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"keywords":["protein powder","cold brew coffee"],"country":"us"}'

It works for 17 countries (US, UK, Canada, Australia, the big EU markets, the Nordics, Brazil, Mexico, India), runs from n8n, Make and Zapier, and AI agents can call it through the Apify MCP server.

A note on the numbers

Every SEO tool estimates volume and difficulty its own way, so Ahrefs, Semrush and this will not match exactly. Volumes here are Google Ads 12-month averages. Compare keywords within one tool rather than across tools.

All examples (bulk CSV, easy-win finder, Node.js, cURL): github.com/emiohr/keyword-research-api. Questions or feature requests? Drop a comment.

This article was written by an AI agent under my direction, and all code was tested against the live API. Not affiliated with Ahrefs, Semrush or Google.

Check domain authority for 1,000 sites at once with Python (a pay-per-use Ahrefs alternative)

Albin Johansson — Sat, 26 Sep 2026 11:37:34 +0000

You have a list of 500 guest-post prospects, expired domains or client competitors, and you want to know which ones are actually strong. Browser extensions check one site at a time. Ahrefs and Semrush can do it in bulk, but they start at $129/month (Ahrefs Lite) and $139/month (Semrush SEO plan).

If you only need it now and then, or inside a script, a pipeline or an AI agent, here's how to get domain authority and backlink numbers for a whole list, paid per domain, in a few lines of Python.

1. Setup

pip install "apify-client>=3"
export APIFY_TOKEN=...   # free account at console.apify.com

I'm using the Bulk Domain Authority & Backlink Checker on Apify (disclosure: I built it). For every domain it returns a domain rank (0–100), referring domains, backlinks, the dofollow/nofollow split, a spam score and optional 30-day changes. Invalid domains and domains without backlink data aren't charged.

2. A list of domains → CSV

import csv, os
from apify_client import ApifyClient

DOMAINS = ["dev.to", "allbirds.com", "gymshark.com", "hashnode.com", "indiehackers.com"]
COLUMNS = ["domain", "domainRank", "referringDomains", "backlinks", "dofollowRatio", "spamScore", "brokenBacklinks"]

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("jesting_grass/bulk-domain-authority-checker").call(run_input={"domains": DOMAINS})

with open("domain_authority.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=COLUMNS, extrasaction="ignore")
    w.writeheader()
    for row in sorted(client.dataset(run.default_dataset_id).iterate_items(), key=lambda r: -(r.get("domainRank") or 0)):
        if "error" not in row:
            w.writerow(row)
            print(f'{row["domain"]:<18} rank {row["domainRank"]:<4} ref. domains {row["referringDomains"]:>8,}  backlinks {row["backlinks"]:>10,}')

Real output (September 2026):

dev.to             rank 82   ref. domains   93,830  backlinks 17,292,820
hashnode.com       rank 82   ref. domains   88,946  backlinks 10,021,858
indiehackers.com   rank 68   ref. domains   14,704  backlinks    709,281
gymshark.com       rank 59   ref. domains    9,663  backlinks     99,941
allbirds.com       rank 58   ref. domains    9,310  backlinks     91,766

You can paste full URLs: https://, www. and paths are stripped and duplicates removed, so a raw export from a spreadsheet works as-is.

3. Link building: keep only strong prospects

Add minDomainRank and weaker sites are dropped from the results:

run = client.actor("jesting_grass/bulk-domain-authority-checker").call(run_input={
    "domains": ["dev.to", "hashnode.com", "indiehackers.com", "allbirds.com", "gymshark.com"],
    "minDomainRank": 60,
})
for row in client.dataset(run.default_dataset_id).iterate_items():
    if "error" not in row:
        print(row["domain"], "rank", row["domainRank"], "| dofollow ratio", row["dofollowRatio"])

dev.to rank 82 | dofollow ratio 0.812
hashnode.com rank 82 | dofollow ratio 0.951
indiehackers.com rank 68 | dofollow ratio 0.946

Two more columns help when screening expired domains or link sellers:

dofollowRatio: a site whose backlinks are almost all nofollow passes little link equity.
spamScore: a 0–100 spam signal for the backlink profile, a quick toxic-link check (brokenBacklinks also shows links you could reclaim).

4. No Python? One HTTP call

curl -X POST "https://api.apify.com/v2/acts/jesting_grass~bulk-domain-authority-checker/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains":["dev.to","allbirds.com"]}'

It also runs from n8n, Make and Zapier, and AI agents can call it as a tool through the Apify MCP server. Schedule it weekly and the referringDomainsChange column shows who is gaining or losing links.

A note on "domain authority"

There's no single universal DA. Moz has Domain Authority, Ahrefs has Domain Rating, and this tool uses its backlink index's own 0–100 rank. Each is computed from the vendor's own backlink index, so the same site gets different numbers in different tools. What matters is comparing domains with one score consistently, not matching another tool's number.

All examples (Python CSV export, link-building filter, Node.js, cURL): github.com/emiohr/bulk-domain-authority-checker. Questions or feature requests? Drop a comment.

This article was written by an AI agent under my direction, and all code was tested against the live API. Not affiliated with Ahrefs, Semrush or Moz.

Find any website's tech stack in bulk with Python (a pay-per-use BuiltWith / Wappalyzer alternative)

Albin Johansson — Sat, 26 Sep 2026 10:01:09 +0000

Want to know which of 500 prospects run Shopify, which use HubSpot, or what analytics and payment tools your competitors use? That's technographics — and the usual tools are subscriptions: BuiltWith starts at $295/month (API access only on higher plans) and Wappalyzer Pro is $250/month.

If you only need it now and then, or want it inside a script, a pipeline or an AI agent, here's how to do it pay-per-website in a few lines of Python.

1. Setup

pip install "apify-client>=3"
export APIFY_TOKEN=...   # free account at console.apify.com

I'm using the Tech Stack Detector on Apify (disclosure: I built it). It fetches each live website and matches it against 7,600+ open-source technology fingerprints (the Wappalyzer format), and invalid, offline or undetectable sites aren't charged.

2. Tech stack of a list of domains → CSV

import csv, os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("jesting_grass/tech-stack-detector").call(run_input={
    "domains": ["allbirds.com", "gymshark.com", "klarna.com", "hubspot.com", "ikea.com"],
})

cols = ["domain", "cms", "ecommerce", "payments", "analytics", "crm", "hosting", "technologyNames"]
with open("tech_stacks.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=cols, extrasaction="ignore")
    w.writeheader()
    for row in client.dataset(run.default_dataset_id).iterate_items():
        if "error" not in row:
            w.writerow(row)
            platform = row["ecommerce"] or row["cms"] or "-"
            print(f'{row["domain"]:<14} platform: {platform:<20} payments: {row["payments"] or "-"}')

Real output (September 2026):

gymshark.com   platform: Shopify              payments: Apple Pay, PayPal
ikea.com       platform: Astro                payments: -
allbirds.com   platform: Shopify              payments: Apple Pay, PayPal
hubspot.com    platform: HubSpot CMS Hub      payments: -
klarna.com     platform: Contentful           payments: Klarna Checkout

Besides the full technologies list, every row has flat columns — cms, ecommerce, payments, analytics, marketingAutomation, crm, hosting, cdn, jsFrameworks — so the CSV drops straight into Sheets or a CRM.

3. Lead generation: keep only sites using a technology

Selling a Shopify app? Only want prospects on HubSpot? Add one field:

run = client.actor("jesting_grass/tech-stack-detector").call(run_input={
    "domains": ["allbirds.com", "gymshark.com", "klarna.com", "hubspot.com", "ikea.com"],
    "onlyDomainsUsing": ["Shopify"],
})
for row in client.dataset(run.default_dataset_id).iterate_items():
    if "error" not in row:
        print(row["domain"], "uses", row["matchedTechnologies"])

gymshark.com uses ['Shopify']
allbirds.com uses ['Shopify']

Feed it domains from anywhere — a lead list, a CRM export, Google Maps or search results.

4. No Python? One HTTP call

curl -X POST "https://api.apify.com/v2/acts/jesting_grass~tech-stack-detector/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains":["stripe.com","allbirds.com"]}'

It also runs from n8n, Make and Zapier, and AI agents can call it as a tool through the Apify MCP server.

A note on accuracy

Any tech-stack detector — BuiltWith and Wappalyzer included — can only see what a website exposes publicly: scripts, headers, cookies, markup. Back-office tools that leave no trace on the site can't be detected by anyone. Treat results as strong signals, not a complete inventory.

All examples (Python CSV export, Shopify lead filter, Node.js, cURL): github.com/emiohr/builtwith-wappalyzer-alternative. Questions or feature requests — drop a comment.

This article was written by an AI agent under my direction, and all code was tested against the live API. Not affiliated with BuiltWith or Wappalyzer.

Google Trends API in Python without pytrends (2026 guide)

Albin Johansson — Sat, 26 Sep 2026 09:07:45 +0000

pytrends was archived in April 2025. If you still depend on it, you've probably met this:

pytrends.exceptions.TooManyRequestsError: The request failed: Google returned a response with code 429

pytrends loads the Google Trends website like a browser, and Google rate-limits that traffic. Sleeping, rotating proxies and retrying helps a bit — until it doesn't. And Google's official Trends API is still an application-only alpha (announced July 2025, no release date yet).

Here's how I get the same data (interest over time, related queries, interest by region) reliably in a few lines of Python.

1. Setup

pip install "apify-client>=3" pandas
export APIFY_TOKEN=...   # free account at console.apify.com

I'm using the Google Trends Scraper & API on Apify (disclosure: I built it). It gets the data from commercial Google Trends data providers, so you never hit Google from your own IP or fight 429s yourself, and failed lookups aren't charged.

2. Interest over time → pandas (the `interest_over_time()` replacement)

import os
import pandas as pd
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("jesting_grass/google-trends-api").call(run_input={
    "keywords": ["bitcoin", "ethereum"],   # up to 5 compared in one query
    "timeRange": "past_5_years",
})

series = [
    pd.Series({p["date"]: p["value"] for p in row["timeline"]}, name=row["keyword"])
    for row in client.dataset(run.default_dataset_id).iterate_items()
]
df = pd.concat(series, axis=1)
df.index = pd.to_datetime(df.index)
print(df.tail())

            bitcoin  ethereum
2026-08-23       26         3
2026-08-30       23         2
2026-09-06       19         2
2026-09-13       19         2
2026-09-20       22         3

Each row also comes with averageInterest, peakInterest, peakDate and latestInterest, so you often don't need pandas at all.

3. Related queries — the keyword-research goldmine

"Rising" and Breakout related queries are where new keyword ideas come from. Flip one switch:

run = client.actor("jesting_grass/google-trends-api").call(run_input={
    "keywords": ["protein powder"],
    "location": "United States",
    "searchType": "youtube",          # web | youtube | news | images | froogle
    "includeRelatedQueries": True,
    "includeInterestByRegion": True,
})
for row in client.dataset(run.default_dataset_id).iterate_items():
    for q in row["relatedQueries"]["rising"][:5]:
        print(f'{q["query"]:<35} {q["value"]}')

lead in protein powder              +550%
protein powder scam                 +350%
flavorless protein powder           +200%
costco protein powder               +170%
orgain protein powder               +160%

That's YouTube search trends — handy if you make videos. Same for News, Images and Google Shopping.

4. pytrends → API cheat sheet

pytrends	API input / output
`build_payload(kw, timeframe='today 12-m', geo='US')`	`keywords`, `timeRange: "past_12_months"`, `location: "United States"`
`interest_over_time()`	`row["timeline"]`
`related_queries()`	`includeRelatedQueries: true`
`related_topics()`	`includeRelatedTopics: true`
`interest_by_region()`	`includeInterestByRegion: true`
`gprop='youtube'`	`searchType: "youtube"`

5. Not using Python?

One HTTP call, JSON back:

curl -X POST "https://api.apify.com/v2/acts/jesting_grass~google-trends-api/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"keywords":["chatgpt","claude"],"location":"United States","timeRange":"past_90_days"}'

It also works from n8n / Make / Zapier, and AI agents can call it as a tool through the Apify MCP server.

A note on the 0–100 values

Google Trends numbers are relative: 100 is the peak within the keywords, place and time you compared. Two separate queries aren't on the same scale — if you need that, include a shared "anchor" keyword in each group.

All examples (Python, pandas, Node.js, cURL) are in this repo: github.com/emiohr/google-trends-api. Questions or feature requests — drop a comment.

This article was written with AI assistance and all code was tested against the live API.

DEV Community: Albin Johansson

Scrape Amazon in Python without proxies: best sellers, niche research and a price tracker

1. Setup

2. What actually sells? Rank by "bought in past month"

3. Check several niche ideas at once

4. A price tracker in 20 lines

5. No Python? One HTTP call

Find link-building prospects with Python: a competitor backlink gap for $1 per 1,000 links

1. Setup

2. The link gap: who links to them, but not to you?

3. Every backlink of a site → CSV

4. No Python? One HTTP call

A note on the numbers

Keyword research in Python: search volume, difficulty and AI Overviews in bulk (2026)

1. Setup

2. A keyword list → CSV

3. Find easy wins from one seed keyword

4. No Python? One HTTP call

A note on the numbers

Check domain authority for 1,000 sites at once with Python (a pay-per-use Ahrefs alternative)

1. Setup

2. A list of domains → CSV

3. Link building: keep only strong prospects

4. No Python? One HTTP call

A note on "domain authority"

Find any website's tech stack in bulk with Python (a pay-per-use BuiltWith / Wappalyzer alternative)

1. Setup

2. Tech stack of a list of domains → CSV

3. Lead generation: keep only sites using a technology

4. No Python? One HTTP call

A note on accuracy

Google Trends API in Python without pytrends (2026 guide)

1. Setup

2. Interest over time → pandas (the interest_over_time() replacement)

3. Related queries — the keyword-research goldmine

4. pytrends → API cheat sheet

5. Not using Python?

A note on the 0–100 values

2. Interest over time → pandas (the `interest_over_time()` replacement)