Connect Etsy Scraper to AI Agents via MCP

Exposing an Apify Actor to an AI agent converts an automated web crawler into a structured, callable tool. The Model Context Protocol (MCP) defines how Large Language Models query external systems, construct payloads, and process results. When wrapping an e-commerce scraper like Etsy Scraper as an MCP tool, the language model needs a well-defined input signature, a precise understanding of execution parameters, and guardrails to handle execution delays and bot detection.

Exposing the Actor as a tool an AI agent can call via MCP requires understanding the scoped ?tools= configuration, what the input schema becomes as a tool signature, and the platform limits an agent will hit.

Checked against the Actor's input schema and Apify docs on 2026-09-09.

How do you expose an Apify Actor as an MCP tool?

You expose an Apify Actor as an MCP tool by pointing your client to https://mcp.apify.com?tools=crawlerbros/etsy-scraper with a valid Apify API token in the Authorization header. The server parses the schema to create a tool definition for the model. Unauthenticated access works only for metadata tools, so executions require proper HTTP header credentials.

To configure this in an MCP-compliant client, add the server definition to your MCP settings file:

{
  "mcpServers": {
    "etsy-scraper": {
      "url": "https://mcp.apify.com?tools=crawlerbros/etsy-scraper",
      "headers": {
        "Authorization": "Bearer YOUR_APIFY_API_TOKEN"
      }
    }
  }
}

The Apify MCP server dynamically inspects the Actor's input schema and converts it into standard JSON Schema tool definitions. Running Actors via MCP always requires an API token. While unauthenticated requests can access metadata tools like search-actors or fetch-actor-details, execution requires valid credentials passed in the HTTP request header.

By restricting the endpoint with ?tools=crawlerbros/etsy-scraper, you prevent the MCP server from loading hundreds of public platform tools into the agent's prompt context. This reduces token overhead and prevents context window dilution.

What is the generated tool input schema for the LLM?

The input schema generated by the MCP server maps the Actor's structural inputs directly to standard JSON primitives, exposing properties like startUrls, searchQueries, maxItems, includeDetails, and country. The model must explicitly pass either searchQueries or startUrls in every invocation because prefill fields are ignored during API calls.

When an AI agent inspects the tool, the schema exposes parameters defined in the Actor's specification:

{
  "type": "object",
  "properties": {
    "startUrls": {
      "type": "array",
      "description": "Etsy URLs to scrape. Supports search results, shop pages, and category pages.",
      "items": {
        "type": "object",
        "properties": {
          "url": { "type": "string" }
        },
        "required": ["url"]
      }
    },
    "searchQueries": {
      "type": "array",
      "description": "Keywords to search on Etsy. Each query generates a separate search.",
      "items": {
        "type": "string"
      }
    },
    "maxItems": {
      "type": "integer",
      "description": "Maximum number of items to extract per URL or search query.",
      "default": 100
    },
    "includeDetails": {
      "type": "boolean",
      "description": "When enabled, attempts to visit each product page for additional details.",
      "default": false
    },
    "country": {
      "type": "string",
      "description": "Target Etsy market. Sets proxy exit country, browser locale, and search URL prefix.",
      "default": "US"
    }
  }
}

A common issue in automated agent integration stems from UI defaults vs API defaults. Platform interfaces often apply prefill values visually to fields in the console UI. However, prefill is ignored during direct API or MCP calls. Only fields specified as default in the schema are automatically filled by the platform if omitted by the model.

Because startUrls and searchQueries lack a default value in the schema, the model must explicitly provide at least one of these two keys in every tool invocation call payload.

{
  "searchQueries": ["handmade leather wallet"],
  "maxItems": 50,
  "country": "US",
  "includeDetails": false
}

How does the LLM receive and process the scraped dataset output?

The tool execution returns structured listing records from the default dataset containing listingId, price, rating, badges, and shop details. Fields such as originalPrice and discountPercent are omitted or empty when items are not on sale. Agents should validate missing fields and parse nested structures like the badges object cleanly.

The JSON payload returned to the caller maps listings to an array of objects matching the output schema definition:

{
  "listingId": "1588968371",
  "shopId": "10932069",
  "url": "https://www.etsy.com/listing/1588968371/rainbow-moonstone-necklace",
  "title": "Rainbow Moonstone Necklace Copper Wire Wrapped Necklace Natural Gemstone Jewelry",
  "price": "31.18",
  "originalPrice": "124.73",
  "currency": "$",
  "discountPercent": "75",
  "imageUrl": "https://i.etsystatic.com/10932069/r/il/5d01ee/5450878877/il_300x300.5450878877_qp6p.jpg",
  "shopName": "tanaygemsandjewels",
  "shopUrl": "https://www.etsy.com/shop/tanaygemsandjewels",
  "rating": "4.6",
  "reviewCount": "19100",
  "badges": {
    "starSeller": false,
    "freeShipping": true,
    "bestseller": false,
    "etsyChoice": false,
    "ad": false
  },
  "scrapedAt": "2026-03-23T07:10:18.494241+00:00"
}

Agent workflows invoking this tool must anticipate null or omitted values within this structure. For instance, originalPrice and discountPercent only contain string values when an item has an active promotional discount. Furthermore, rating and review count fields are extracted from search result listing cards but are missing when targeting seller shop pages directly.

How do you call the scraper programmatically in Python?

You call the Actor in Python by instantiating ApifyClient with your API key and running client.actor("crawlerbros/etsy-scraper").call(run_input=payload). This executes a synchronous call and waits for the run to complete before returning dataset items. It works best for small item counts using direct search queries.

When building a custom agent wrapper outside standard MCP middleware, you can call Etsy Scraper directly using official SDK libraries.

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_API_TOKEN")

run_input = {
    "searchQueries": ["vintage brass lamp"],
    "maxItems": 64,
    "country": "US",
    "includeDetails": False
}

# Run the Actor synchronously
run = client.actor("crawlerbros/etsy-scraper").call(run_input=run_input)

# Fetch results from the run's default dataset
dataset_items = client.dataset(run["defaultDatasetId"]).list_items().items

for item in dataset_items:
    listing_id = item.get("listingId")
    title = item.get("title")
    price = item.get("price")
    badges = item.get("badges", {})

    print(f"[{listing_id}] {title} - {price} (Star Seller: {badges.get('starSeller')})")

If the agent needs to fetch extended metadata, setting "includeDetails": True alters the execution path. Instead of relying purely on search card payloads, the Actor visits each listing page using HTTP impersonation or fallback browser instances.

How do you implement asynchronous execution for long-running tool calls?

You implement asynchronous execution by starting the Actor with client.actor().start(), capturing the run ID, and polling client.run(run_id).get() until completion. Synchronous API calls timing out past 300 seconds return an HTTP 408 error, making async polling essential for large product runs or detail page extraction.

Because Etsy search listing pages cap out at 64 products per page, retrieving high volumes of items forces the scraper to execute search variations using different sort orders and price filters. Combining search variant rotation with includeDetails: true causes the execution time to easily surpass the 300-second threshold.

import time
from apify_client import ApifyClient

def run_etsy_scraper_async(client: ApifyClient, run_input: dict, timeout_secs: int = 600):
    # Start the actor run asynchronously without blocking
    run = client.actor("crawlerbros/etsy-scraper").start(run_input=run_input)
    run_id = run["id"]
    dataset_id = run["defaultDatasetId"]

    start_time = time.time()
    while time.time() - start_time < timeout_secs:
        status_obj = client.run(run_id).get()
        status = status_obj.get("status")

        if status == "SUCCEEDED":
            return client.dataset(dataset_id).list_items().items
        elif status in ["FAILED", "ABORTED", "TIMED-OUT"]:
            raise RuntimeError(f"Actor run ended with status: {status}")

        time.sleep(10)

    raise TimeoutError("Client-side polling timed out before Actor finished.")

For production agentic architectures, polling can be replaced by event-driven webhooks. However, platform webhooks support exactly one action: issuing an HTTP POST to an external URL upon status change.

How do you process category pages and specific product URLs?

You process category pages and product URLs by supplying formatted Etsy links inside the startUrls input array. The Actor accepts URLs for search results, shop storefronts, category hierarchies, or individual product pages. Each item inside startUrls must be formatted as an object containing a url string.

When an AI agent needs to analyze specific links rather than executing keyword queries, it uses the startUrls field.

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_API_TOKEN")

run_input = {
    "startUrls": [
        {"url": "https://www.etsy.com/c/jewelry-and-accessories"},
        {"url": "https://www.etsy.com/listing/1808282840/sterling-silver-ring"}
    ],
    "maxItems": 50,
    "includeDetails": False,
    "country": "US"
}

run = client.actor("crawlerbros/etsy-scraper").call(run_input=run_input)
items = client.dataset(run["defaultDatasetId"]).list_items().items

for item in items:
    print(f"Scraped URL: {item.get('url')} - Title: {item.get('title')}")

Passing a direct listing URL extracts the listing information for that specific item. If includeDetails is set to true, the scraper attempts to visit the product page directly to scrape descriptions, materials, variations, and tags.

How do you automate scraping schedules with platform constraints?

You automate schedules by creating a schedule object with a 6-field cron expression, referencing an Actor that has already been executed at least once. Schedules are created disabled by default, so you must explicitly set isEnabled to True in the payload for automated agent triggers to work properly.

When setting up automated agent workflows that run periodically, developers must account for schedule initialization requirements.

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_API_TOKEN")

# Construct schedule payload using 6-field cron syntax
schedule_data = {
    "name": "daily-etsy-trends",
    "cronExpression": "0 0 12 * * *",  # Every day at 12:00 UTC
    "isEnabled": True,  # Schedules default to DISABLED unless explicitly set
    "actions": [
        {
            "type": "RUN_ACTOR",
            "actorId": "crawlerbros/etsy-scraper",
            "runInput": {
                "searchQueries": ["trending handmade gifts"],
                "maxItems": 64
            }
        }
    ]
}

# Create the schedule via Apify client
schedule = client.schedules().create(schedule=schedule_data)
print(f"Created schedule ID: {schedule['id']}")

If you attempt to create a schedule for an Actor or task that has never been executed, the platform API rejects the request. Execute a manual run first to register the target configuration.

What real operational limitations and failure modes will an agent encounter?

Deploying Etsy Scraper through an autonomous agent introduces specific operational constraints dictated by target site defenses and platform infrastructure.

Anti-Bot Challenges and Session Warmup

Etsy uses aggressive anti-bot protection. To establish valid sessions, the scraper launches a real browser, navigating to Etsy's homepage prior to processing search queries or URLs. This session warmup takes several seconds before data extraction begins.

When scraping product pages with includeDetails: true, the Actor uses an HTTP-first approach featuring Chrome TLS fingerprint impersonation. If a session is blocked, the Actor automatically retries with a fresh browser session. It attempts up to 5 sessions per task, first without proxy and then with residential proxy, before skipping the item. If an agent passes an invalid target or gets blocked across all attempts, specific fields like ratings or detailed descriptions return empty.

Request Queue Processing Isolation

If an agent architecture attempts to scale processing by triggering parallel runs over a shared state, it hits a request queue limitation: a request queue can only be processed by one Actor or task run at a time. While multiple runs can append requests to a shared queue, fan-out parallel processing against a single queue instance is unsupported.

Storage Retention Limits

Data saved by ephemeral run executions is subject to platform expiration rules:

def verify_dataset_retention(client: ApifyClient, dataset_id: str):
    dataset_metadata = client.dataset(dataset_id).get()
    # Unnamed dataset storages automatically expire.
    # On Free plan tiers, only the 10 most recent runs are retained, up to 4 months.
    # Named storages are permanently exempt from automatic deletion.
    print(f"Dataset Name: {dataset_metadata.get('name', 'Unnamed (Ephemeral)')}")

If an agent relies on past execution data, it must assign a named storage identifier during execution or extract and save records to an external system. Storage rate limits enforced by the platform allow up to 60 requests per second for individual storage objects, and 400 requests per second for dataset item pushes or request queue CRUD operations.

How does execution cost scale across platform pricing tiers?

Platform cost scales based on container compute unit duration, proxy bandwidth, and active subscription pricing tiers. Compute consumption follows the formula CU = (memory_mb / 1024) * duration_hours, where doubling memory is CU-neutral only for autoscaling runs.

Autoscaling applies exclusively to solutions running multiple tasks or URLs continuously for at least 30 seconds each. Single-page agent lookups do not trigger autoscaling benefits.

Compute Unit and Proxy Pricing Rates

Platform execution pricing varies depending on the active platform plan:

Plan Tier	Monthly Fee	Price per Compute Unit (CU)	Residential Proxy Cost
Free	$0	$0.20 / CU	$8 / GB
Starter	$19 / mo	$0.20 / CU	$8 / GB
Scale	$199 / mo	$0.16 / CU	$7.50 / GB
Business	$999 / mo	$0.13 / CU	$7 / GB

Proxy usage is billed per gigabyte of transferred data. While datacenter proxy sessions persist for up to 26 hours, residential proxy sessions expire after approximately 30 minutes, requiring automatic session rotation during long extraction jobs.

To prevent an autonomous agent loop from consuming budget unexpectedly, append the maxTotalChargeUsd parameter to execution runs:

curl -X POST "https://api.apify.com/v2/acts/crawlerbros~etsy-scraper/runs?token=YOUR_APIFY_API_TOKEN&maxTotalChargeUsd=2.00" \
     -H "Content-Type: application/json" \
     -d '{
           "searchQueries": ["handmade ceramic mug"],
           "maxItems": 200
         }'

The maxTotalChargeUsd parameter is passed into the execution container as ACTOR_MAX_TOTAL_CHARGE_USD. When this spending cap is triggered, the execution terminates. Because termination involves graceful shutdown signaling, container resource consumption continues briefly during the 30-second abort window rather than instantly dropping to zero.

How do you handle schema validation errors in agent tools?

You handle validation errors by wrapping the tool execution call in a decorator that sanitizes parameters before sending them to the API. This layer converts raw string inputs to expected array structures and caps maxItems within valid schema ranges.

An LLM generating tool calls may pass invalid parameters, such as supplying a bare string for startUrls instead of an array of objects.

import functools
from typing import Dict, Any

def validate_etsy_tool_input(func):
    @functools.wraps(func)
    def wrapper(payload: Dict[str, Any], *args, **kwargs):
        sanitized = {}

        # Ensure searchQueries is a list of strings
        if "searchQueries" in payload:
            queries = payload["searchQueries"]
            if isinstance(queries, str):
                sanitized["searchQueries"] = [queries]
            elif isinstance(queries, list):
                sanitized["searchQueries"] = [str(q) for q in queries]

        # Fix startUrls formatting if LLM passes raw string URLs
        if "startUrls" in payload:
            urls = payload["startUrls"]
            if isinstance(urls, str):
                sanitized["startUrls"] = [{"url": urls}]
            elif isinstance(urls, list):
                sanitized["startUrls"] = [
                    {"url": u} if isinstance(u, str) else u for u in urls
                ]

        # Enforce bounds on maxItems (schema supports 1-1000)
        max_items = payload.get("maxItems", 100)
        sanitized["maxItems"] = max(1, min(int(max_items), 1000))

        # Pass through remaining options
        sanitized["includeDetails"] = bool(payload.get("includeDetails", False))
        sanitized["country"] = str(payload.get("country", "US"))

        if not sanitized.get("searchQueries") and not sanitized.get("startUrls"):
            raise ValueError("Validation Error: Must provide either searchQueries or startUrls.")

        return func(sanitized, *args, **kwargs)
    return wrapper

@validate_etsy_tool_input
def execute_agent_tool_call(sanitized_payload: Dict[str, Any]):
    print("Payload validated successfully:")
    print(sanitized_payload)

Validating types, coercing common model hallucinations (like single strings instead of arrays), and enforcing maxItems limits ensures reliable tool execution within automated agent pipelines.

How do you build a complete multi-search seller analysis agent?

You build a multi-search seller analysis agent by executing keyword extraction, aggregating shop occurrences, and fetching catalog listings from top sellers. This pattern allows an LLM tool pipeline to identify dominant merchants across product categories and extract their listing details in sequence.

import time
from typing import List, Dict, Any
from apify_client import ApifyClient

def extract_top_shops_for_keywords(
    client: ApifyClient, 
    keywords: List[str], 
    items_per_query: int = 64
) -> Dict[str, Any]:
    """
    Step 1: Execute keyword searches across multiple terms to find top shops.
    """
    search_payload = {
        "searchQueries": keywords,
        "maxItems": items_per_query,
        "country": "US",
        "includeDetails": False
    }

    print(f"Triggering search queries: {keywords}")
    run = client.actor("crawlerbros/etsy-scraper").call(run_input=search_payload)
    items = client.dataset(run["defaultDatasetId"]).list_items().items

    # Aggregate shop frequencies and listing count
    shop_counts = {}
    shop_urls = {}

    for item in items:
        shop_name = item.get("shopName")
        shop_url = item.get("shopUrl")
        if shop_name and shop_url:
            shop_counts[shop_name] = shop_counts.get(shop_name, 0) + 1
            shop_urls[shop_name] = shop_url

    # Sort shops by occurrence frequency
    sorted_shops = sorted(shop_counts.items(), key=lambda x: x[1], reverse=True)
    return {
        "raw_items_count": len(items),
        "top_shops": sorted_shops[:3],
        "shop_urls": shop_urls
    }

def scrape_shop_catalog(client: ApifyClient, shop_url: str, max_items: int = 50) -> List[Dict[str, Any]]:
    """
    Step 2: Scrape specific shop listings using the startUrls parameter.
    """
    shop_payload = {
        "startUrls": [{"url": shop_url}],
        "maxItems": max_items,
        "country": "US",
        "includeDetails": False
    }

    print(f"Scraping catalog for shop URL: {shop_url}")
    run = client.actor("crawlerbros/etsy-scraper").call(run_input=shop_payload)
    return client.dataset(run["defaultDatasetId"]).list_items().items

# Demonstration execution flow
if __name__ == "__main__":
    apify_client = ApifyClient("YOUR_APIFY_API_TOKEN")

    # Step 1: Discover active shops across category keywords
    analysis = extract_top_shops_for_keywords(
        apify_client, 
        ["minimalist ring", "stacking rings"], 
        items_per_query=30
    )

    print(f"Discovered {len(analysis['top_shops'])} top performing shops.")

    # Step 2: Extract inventory catalog from the top identified shop
    if analysis["top_shops"]:
        top_shop_name, frequency = analysis["top_shops"][0]
        target_url = analysis["shop_urls"][top_shop_name]
        print(f"Targeting top shop '{top_shop_name}' ({frequency} items in search results)")

        catalog = scrape_shop_catalog(apify_client, target_url, max_items=20)
        print(f"Retrieved {len(catalog)} listings from {top_shop_name}")

This multi-step pipeline demonstrates how an AI agent uses searchQueries for broad discovery, processes the returned dataset to select strategic targets, and uses startUrls to retrieve targeted shop catalog data.

How do you build a category and competitor pricing monitoring agent?

You build a category and pricing monitoring agent by defining target category URLs, executing search runs across specific markets using the country parameter, and comparing base prices against discounted rates. This allows an autonomous process to track price trends and discount percentages across localized market segments.

When building an automated monitoring pipeline, the agent consumes both standard listing fields and discount metrics to track dynamic marketplace shifts.

from typing import List, Dict, Any
from apify_client import ApifyClient

def monitor_category_pricing(
    client: ApifyClient,
    category_url: str,
    target_country: str = "US",
    max_items: int = 100
) -> Dict[str, Any]:
    """
    Extracts category items and calculates promotional pricing distribution.
    """
    run_input = {
        "startUrls": [{"url": category_url}],
        "maxItems": max_items,
        "country": target_country,
        "includeDetails": False
    }

    print(f"Fetching category data for {category_url} in market {target_country}...")
    run = client.actor("crawlerbros/etsy-scraper").call(run_input=run_input)
    listings = client.dataset(run["defaultDatasetId"]).list_items().items

    discounted_items = []
    regular_items = []

    for item in listings:
        listing_id = item.get("listingId")
        title = item.get("title")
        price = item.get("price")
        original_price = item.get("originalPrice")
        discount_percent = item.get("discountPercent")
        badges = item.get("badges", {})

        record = {
            "listingId": listing_id,
            "title": title,
            "price": price,
            "shopName": item.get("shopName"),
            "isBestseller": badges.get("bestseller", False),
            "isStarSeller": badges.get("starSeller", False)
        }

        # Check if item has active promotional discount metadata
        if discount_percent and original_price:
            record["originalPrice"] = original_price
            record["discountPercent"] = discount_percent
            discounted_items.append(record)
        else:
            regular_items.append(record)

    return {
        "totalScraped": len(listings),
        "discountedCount": len(discounted_items),
        "regularCount": len(regular_items),
        "discountedItems": discounted_items,
        "sampleRegular": regular_items[:5]
    }

if __name__ == "__main__":
    apify_client = ApifyClient("YOUR_APIFY_API_TOKEN")

    # Scrape category hierarchy page
    pricing_report = monitor_category_pricing(
        apify_client,
        category_url="https://www.etsy.com/c/jewelry-and-accessories",
        target_country="US",
        max_items=50
    )

    print(f"Scraped {pricing_report['totalScraped']} total items.")
    print(f"Found {pricing_report['discountedCount']} items with active discounts.")
    if pricing_report["discountedItems"]:
        sample = pricing_report["discountedItems"][0]
        print(f"Sample discounted item: {sample['title']} at {sample['price']} (was {sample.get('originalPrice')})")

Setting the country parameter ensures that the proxy exit location, browser locale, and localized search URL prefix align with the designated target market. This gives agents accurate currency symbols and region-specific item pricing.

The Actor's README is the source of truth for its inputs, outputs and limits. Need a hand wiring this into your stack? Email [e-mail]