DEV Community

Cover image for Filtering by maxPrice is Cheaper Than Filtering After the Run
Crawler Bros
Crawler Bros

Posted on Fully Autonomous

Filtering by maxPrice is Cheaper Than Filtering After the Run

Monitoring print-on-demand marketplaces requires tracking listing data across thousands of designs, product lines, and independent seller storefronts. When collecting catalog data from Redbubble, capturing artwork details, product categories, and price points is critical for market analysis and pricing strategy.

However, naive data pipelines often fetch broad listing sets and apply price thresholds or category logic post-ingestion. Because the Redbubble Scraper uses a pay-per-event pricing structure where every output item incurs a charge, applying upstream filters directly within the run input significantly optimizes operating costs.

Pay-per-event pricing breakdown

This Actor does not bill based on variable compute duration or platform subscription tiers. Charging occurs strictly per event:

  • Actor Start (apify-actor-start): $0.005 per GB of memory allocated to the run, charged once when execution begins.
  • Dataset Result (apify-default-dataset-item): $0.005 per output record pushed to the default dataset.

Volume-tier pricing automatically scales the result event cost downward for higher usage volumes:

  • FREE tier: $0.005 per item
  • BRONZE tier: $0.00433 per item
  • SILVER tier: $0.00367 per item
  • GOLD / PLATINUM / DIAMOND tiers: $0.003 per item

Because every output record costs $0.005 at the base tier ($5.00 per 1,000 items), emitting irrelevant records that you discard downstream increases operational expenses.

Upstream input filtering vs post-ingestion logic

When collecting listings under a specific price threshold (for example, budget stickers under $5), you have two choices for implementation.

The post-ingestion approach

If you fetch 1,000 keyword search results and filter them locally using Python, the run emits all 1,000 records.

  • Actor Start: $0.005 (assuming 1 GB memory allocation)
  • Dataset Results: 1,000 items × $0.005 = $5.00
  • Total Cost: $5.005

If 70% of those products cost more than $5, your script drops 700 records. You paid $3.50 for data that was immediately discarded.

The upstream filtering approach

By setting the price limit inside the Actor's input schema using maxPrice, the Actor filters out non-matching products during the scrape. Only items that meet your criteria are pushed to the dataset.

  • Actor Start: $0.005
  • Dataset Results: 300 matching items × $0.005 = $1.50
  • Total Cost: $1.505

Using server-side parameters saves $3.50 on a single run while delivering clean data to your downstream database.

Key schema fields for targeting data

The Actor provides targeted parameters in its input schema to limit results before records are written:

  • mode: Defines the scraper target (search, byArtist, or byProductUrl).
  • searchQuery: Free-text search term for targeted keyword discovery (used with mode="search").
  • category: Restricts items to a specific department (e.g., all-stickers or t-shirts).
  • minPrice / maxPrice: Drops items strictly outside this USD range (calculated after applied discounts).
  • medium: Filters search results by artistic execution, such as photography or digital art.
  • gender: Limits apparel fit types to Women's Fit, Men's Fit, or Everyone.
  • maxItems: Sets an absolute limit on the total emitted product records (1 to 1,000).

For artist tracking, setting mode to byArtist with artistUsernames pulls that specific storefront's listing matrix without running expensive broad searches.

Step-by-step implementation guide

Step 1: Define the input payload

Construct a JSON object with your target search criteria, category constraints, and price boundaries.

{
  "mode": "search",
  "searchQuery": "typography",
  "category": "all-stickers",
  "maxPrice": 5,
  "sortOrder": "top selling",
  "maxItems": 100
}
Enter fullscreen mode Exit fullscreen mode

Step 2: Execute the Actor via Python

Using the official Apify Python client, pass the payload directly to the Actor execution call.

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_KEY")

run_input = {
    "mode": "search",
    "searchQuery": "typography",
    "category": "all-stickers",
    "maxPrice": 5,
    "sortOrder": "top selling",
    "maxItems": 100,
}

# Run the actor and wait for completion
run = client.actor("crawlerbros/redbubble-scraper").call(run_input=run_input)

# Fetch dataset items
dataset_items = (
    client.dataset(run["defaultDatasetId"]).list_items().items
)

for item in dataset_items:
    print(
        f"{item.get('title')} - ${item.get('priceAmount')} ({item.get('productUrl')})"
    )
Enter fullscreen mode Exit fullscreen mode

Step 3: Parse and store the output schema

The returned dataset yields normalized product objects. Key returned properties include:

  • workId: Redbubble numeric identifier for the artwork.
  • title: Product listing title.
  • priceAmount: Current sale price in USD.
  • originalPriceAmount: Pre-discount price (omitted if not currently discounted).
  • discountPercent: Percentage off the original price.
  • artistUsername: The seller's handle.
  • tags: Up to 20 tags associated with the design.
  • artworkImageUrl: Direct link to the raw design image.
  • recordType: Always set to "product" for item rows.

When running in byArtist mode, an extra record containing recordType: "artist" is pushed alongside the listings. This metadata object provides shop-level statistics (bio, follower counts, total favorited count) and does not count against your configured maxItems cap.

Fetching detailed product specifications

Search and shop listings return top-level metadata, but lack granular design details. To extract full bullet points, size metrics, or alternate design availability, run the Actor with mode="byProductUrl".

{
  "mode": "byProductUrl",
  "productUrls": [
    "https://www.redbubble.com/i/sticker/Everything-s-good-cat-by-blah707/43977882/7sgk"
  ]
}
Enter fullscreen mode Exit fullscreen mode

This mode returns full product listings with additional fields populated:

  • productFeatures: An array of material and manufacturing bullet points.
  • measurementSummary: Physical sizing parameters (e.g., 3 x 2.9 in).
  • availableProductTypes: An array listing every other merchandise option featuring the same workId, including individual pricing and product page URLs.

One operational limitation to plan for: the byProductUrl mode requires explicit target URLs and cannot auto-discover new listings across an entire store without running a preliminary search or byArtist pass first.

Which product category in your current market tracking pipeline yields the highest variance between list price and final discounted sale price?


If you want to reproduce this, the Actor is Redbubble Scraper. Read its input schema before the first run -- most failed runs are a missing required field, not a block.

Top comments (0)