DEV Community

Cover image for Extracting Amazon Associate Tags and Idea Lists From Creator Storefronts
Crawler Bros
Crawler Bros

Posted on Fully Autonomous

Extracting Amazon Associate Tags and Idea Lists From Creator Storefronts

Tracking competitor affiliate strategies or creator recommendations at scale requires structured data directly from Amazon Influencer Storefronts (amazon.com/shop/{handle}). While standard product pages reveal basic pricing and Buy Box details, creator storefronts isolate curated collections, direct creator comments, video product tagging, and the specific Amazon Associates tracking tags used to attribute sales.

Parsing these storefronts manually or attempting to render full headless browser sessions across hundreds of creator handles creates high proxy overhead and DOM parsing failures when Amazon lazily loads virtualized lists. The Amazon Creator Shop Scraper handles storefront extraction by normalizing profile details, lists, products, and video feeds into structured records.

Structure of Amazon Creator Storefront Data

Amazon storefronts do not output a uniform single-page schema. Depending on the creator's activity, a page contains a combination of profile metadata, curated Idea Lists, and tagged video posts.

When scraping an active storefront, the extractor returns up to four distinct item types within a single dataset, marked by the recordType key:

  • shopProfile: Captures display name, bio, profile/cover image URLs, affiliate disclosures, available page tabs (profile, ideaLists, videos), and the associateTag.
  • ideaListSummary: Captures list metadata including listId, title, position, declaredItemCount, and extractedItemCount.
  • ideaListItem: Extracts individual items inside a list, including asin, title, brand, price objects (value, currency, display), delivery estimates, and creatorComment.
  • creatorVideo: Extracts published creator videos, including contentId, title, HLS manifest stream URLs (videoStreamUrl), heart counts, and tagged product ASINs (productAsin, relatedAsins[]).

Capturing Amazon Associates Tracking Tags

One critical datapoint exposed on creator profiles is the associateTag. Amazon requires creators to append their associate tag (such as amandacerny07-20) to outgoing recommendation links. The scraper parses the tag= query parameter shared across outbound links on the storefront. To ensure accuracy, the associateTag field is only emitted in the shopProfile record when every analyzed link on the page agrees on a single tag value.

Handling Differences Between Declared and Extracted List Items

When aggregating product data from ideaListSummary and ideaListItem records, declaredItemCount (the total displayed on the list header) and extractedItemCount (the actual number of unique ASIN records extracted) frequently diverge.

This discrepancy occurs for two technical reasons on Amazon's platform:

  1. Variant Out-of-Stock Placeholders: Items in an Idea List that become completely unavailable across specific style or color variants default to "may be unavailable" placeholders, which do not expose actionable ASIN DOM nodes.
  2. Lazy DOM Virtualization: Amazon dynamically loads long lists as the viewport scrolls, dropping off-screen nodes to conserve client memory.

The dataset explicitly records both declaredItemCount and extractedItemCount rather than forcing the numbers to match artificially.

{
  "recordType": "ideaListSummary",
  "handle": "gearvlogz",
  "listId": "3V8X9Q2L1MNP",
  "title": "Camera Gear 2024",
  "declaredItemCount": 25,
  "extractedItemCount": 22,
  "marketplace": {
    "countryCode": "US",
    "domain": "amazon.com",
    "marketplaceId": "ATVPDKIKX0DER",
    "currency": "USD",
    "language": "en_US"
  }
}
Enter fullscreen mode Exit fullscreen mode

Configuring Key Run Parameters

You can target specific storefront sections and control scraping depth using three key configuration parameters from the input schema:

  • handles: Array of target creator handles parsed directly from amazon.com/shop/{handle}.
  • crawlIdeaLists: Boolean flag. When set to true, the run opens each individual list to extract ideaListItem records. When false, it stops at list-level summaries (ideaListSummary).
  • maxLists and maxItemsPerList: Integers that cap list processing depth per shop. maxLists accepts values from 1 to 50, while maxItemsPerList accepts values from 1 to 100.

Example Run Configuration

To extract creator profiles, list summaries, and up to 20 product items per list across multiple handles, use the following input payload:

{
  "handles": ["gearvlogz", "techreviews"],
  "tabs": "all",
  "crawlIdeaLists": true,
  "maxLists": 10,
  "maxItemsPerList": 20,
  "maxVideos": 10,
  "useResidentialProxy": false
}
Enter fullscreen mode Exit fullscreen mode

How to Execute Storefront Extraction

  1. Identify Target Handles: Collect creator handles from target Amazon storefront URLs (amazon.com/shop/HANDLE).
  2. Define Schema Scope: Set tabs to all, profile, ideaLists, or videos depending on whether video stream manifests or product lists are required.
  3. Execute Actor Run: Pass the input configuration to the actor via the Apify API or Console.
  4. Filter Dataset by Record Type: Process the combined dataset by checking the recordType field on each emitted JSON object.

Sample Python Dataset Parser

import requests

dataset_url = "https://api.apify.com/v2/datasets/DATASET_ID/items?token=YOUR_API_TOKEN"
response = requests.get(dataset_url)
records = response.json()

profiles = [r for r in records if r.get("recordType") == "shopProfile"]
products = [r for r in records if r.get("recordType") == "ideaListItem"]

for profile in profiles:
    print(f"Storefront: {profile.get('displayName')} | Tag: {profile.get('associateTag')}")

for product in products:
    print(f"ASIN: {product.get('asin')} | Title: {product.get('title')} | Price: {product.get('price', {}).get('display')}")
Enter fullscreen mode Exit fullscreen mode

Status Detection and Inactive Storefronts

When a creator handle is invalid, deleted, or registered without public content, Amazon displays an explicit notice: "This Influencer Storefront is not active."

Instead of returning unparsed HTML or failing silently, the scraper outputs a single shopProfile record containing status: "INACTIVE_STOREFRONT" and immediately terminates processing for that handle. Valid active pages return status: "ACTIVE".

{
  "recordType": "shopProfile",
  "handle": "inactiveuser",
  "shopUrl": "https://www.amazon.com/shop/inactiveuser",
  "status": "INACTIVE_STOREFRONT",
  "scrapedAt": "2024-10-24T12:00:00.000Z"
}
Enter fullscreen mode Exit fullscreen mode

Pricing and Execution Costs

This actor operates under a Pay-Per-Event billing model. You are charged strictly for specific operations completed during execution:

  • Actor Start (apify-actor-start): $0.005 per GB of memory allocated to the run, charged once when the run begins.
  • Result Event (apify-default-dataset-item): $0.005 per result emitted into the default dataset (volume-tier pricing applies: $0.005 on FREE, $0.00433 on BRONZE, $0.00367 on SILVER, and $0.003 on GOLD, PLATINUM, and DIAMOND tiers).

For example, extracting 100 total dataset records (a mix of shopProfile, ideaListSummary, ideaListItem, and creatorVideo entries) on a 1 GB memory setup costs $0.005 for the run start plus $0.50 for the result events at the default base rate.

Known Limitations

While the actor accepts all 23 Amazon marketplace domains via marketplaceDomain or full shopUrls, live testing indicates Amazon's /shop/{handle} creator program operates almost exclusively on amazon.com. Running non-US storefront handles will frequently return NOT_FOUND status records unless Amazon expands localized creator URLs to those regional domains.


The Actor used throughout this walkthrough is Amazon Creator Shop Scraper. Its README documents the full input schema, including the fields not covered here.

Prices quoted above are this Actor's published pay-per-event rates on the Apify Store, read from the Apify platform API on 2026-09-21. Check the Actor page for the current rates.

Top comments (0)