DEV Community

Cover image for How to Scrape Sephora Prices, Sales and Ratings with Python
ReventaData
ReventaData

Posted on Originally published at apify.com

How to Scrape Sephora Prices, Sales and Ratings with Python

#python #webscraping #apify #tutorial

By the end of this tutorial, you will have a short Python script that turns a Sephora US category page into structured JSON: one row per product with its price in US dollars, sale price and sale type, the full list-price range, rating, review count, number of sizes or shades, and Sephora's own Bestseller, New, Limited Edition, Online Only and Exclusive flags.

We use the Apify Actor Sephora Scraper (reventadata/sephora-scraper). It is for anyone who tracks beauty prices, watches sales in a category, or compares brands by rating, and would rather read rows than product tiles.

⏱️ Time: the setup is a few minutes. Run time depends on how many pages you ask for: the Actor waits 5 seconds between page requests, the Crawl-delay in Sephora's robots.txt.

You will learn how to configure the input, run the Actor from Python, and read every field it returns.

Prerequisites

Install the client:

pip install "apify-client~=3.0"
Enter fullscreen mode Exit fullscreen mode

No scraping code is needed: the Actor handles pages, proxy and parsing. You only send an input and read rows.

What you'll achieve

You will send one or more Sephora category URLs (for example https://www.sephora.com/shop/moisturizing-cream-oils-mists or https://www.sephora.com/shop/lipstick) and receive one clean row per product. A category page holds 60 products, and the Actor walks as many pages as you allow.

Every row comes from the product data Sephora embeds in the category page, so the Actor does not open product pages one by one. The flip side: a row holds what the tile holds. There is no description, ingredient list or per-shade price table.

Example use case: you want to know which moisturizers are on sale right now, cheapest first. Set sort to PRICE_LOW_TO_HIGH and onlyOnSale to true, and a small maxItems returns the top of that list.

Step 1: Access Sephora Scraper

Open the Sephora Scraper page on Apify Store. In Apify Console, an Actor opens on its Input tab, where you can fill the same fields this tutorial sets from code.

We will run it from Python, so keep your API token at hand and export it:

export APIFY_API_TOKEN="<your token>"
Enter fullscreen mode Exit fullscreen mode

Step 2: Configure the category and the caps

The one required field is startUrls: Sephora US category pages of the form https://www.sephora.com/shop/<category>. Query strings are dropped, because the Actor sets page, sort and filters itself. Search pages, /browse/ pages and product pages are refused before any page is requested: Sephora's robots.txt disallows search and browse, and the Actor stays on the category pages it leaves open.

Here is the script. Its input is the Actor's own prefill: one category, 20 rows, one page.

import os

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_API_TOKEN"])

run_input = {
    "startUrls": [{"url": "https://www.sephora.com/shop/moisturizing-cream-oils-mists"}],
    "maxItems": 20,
    "maxPages": 1,
    "sort": "RELEVANCE",
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
        "apifyProxyCountry": "US",
    },
}

run = client.actor("reventadata/sephora-scraper").call(run_input=run_input)
if run is None or run.status != "SUCCEEDED":
    raise SystemExit(f"Run did not succeed (status: {run.status if run else 'unknown'})")

for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item["brand"], "|", item["title"], "|", item["price"], item["currency"], "|", item["saleType"], "|", item["rating"])
Enter fullscreen mode Exit fullscreen mode

maxItems counts rows saved after the filters, across all start URLs. maxPages caps pages per start URL. The defaults are 100 and 5.

Step 3: Add sort order and filters

Parameter Type Default What it does
sort string RELEVANCE RELEVANCE, BEST_SELLING, TOP_RATED, PRICE_LOW_TO_HIGH, PRICE_HIGH_TO_LOW or NEW. Applied by the site.
brands array none Brand names as Sephora spells them (Tatcha, LANEIGE). Several brands are OR. Applied by the site.
minPrice, maxPrice number none Price band in USD, applied by the site.
onlyOnSale boolean false Keeps rows whose saleType is FULL or PARTIAL.
minRating number none Minimum average rating, 0 to 5. Unrated products are dropped when set.
includeKeywords array none Every word must appear in the name, brand or category.
excludeKeywords array none Drops a row when any word appears there.
proxyConfiguration object residential, US Apify residential proxy in the United States.

Three details worth knowing:

  • The site applies sort, brands and the price band before the Actor reads the page, so a small maxItems gets the top of the right list, not a filtered sample of the wrong one.
  • The price band covers the whole list-price range. A product listed as a range is kept only when both ends fall inside your band. It is not an overlap test.
  • Keywords ignore case and accents and look at the name, brand and category together.

Step 4: Run the Actor

Run the script with python sephora.py. .call() starts the Actor on Apify and waits until it finishes. Behind the scenes the Actor fetches each category page through an Apify residential proxy in the US, waits 5 seconds between page requests, reads the product data, applies your filters and saves the rows.

It stops at the first empty page, or when it has read as many products as the category holds.

⚠️ Tip: .call() blocks until the run ends. To fire and forget, use a webhook instead (see the pro tips below).

Step 5: View and download results

The script prints one line per row. In Apify Console, results appear in the run's Output tab, and Export downloads them. Datasets export as JSON, CSV, XML, Excel, HTML table, RSS or JSONL.

Here is the first row of a moisturizer page, a sample from a capture dated 2026-10-21, parsed with the Actor's own row builder. listPriceText, sourceUrl and scrapedAt are left out here:

{
  "id": "P519160",
  "skuId": "2898419",
  "title": "Glazing Milk Hydrating Ceramide Facial Essence",
  "brand": "rhode",
  "url": "https://www.sephora.com/product/glazing-milk-P519160?skuId=2898419",
  "imageUrl": "https://www.sephora.com/productimages/sku/s2898419-main-zoom.jpg?imwidth=270&pb=clean-at-sephora",
  "price": 20.0,
  "priceMin": 20.0,
  "priceMax": 32.0,
  "salePrice": null,
  "valuePrice": null,
  "onSale": false,
  "saleType": null,
  "currency": "USD",
  "rating": 3.94,
  "reviewCount": 2921,
  "variantCount": 1,
  "variationType": "Size",
  "isNew": false,
  "isBestseller": true,
  "isLimitedEdition": false,
  "isOnlineOnly": false,
  "isSephoraExclusive": true,
  "badge": "clean-at-sephora",
  "category": "Moisturizers",
  "categoryUrl": "https://www.sephora.com/shop/moisturizing-cream-oils-mists",
  "categoryTotal": 727,
  "page": 1,
  "position": 1
}
Enter fullscreen mode Exit fullscreen mode

Understanding your results

  • price is the lowest current price: the sale price when there is one, else the low end of the list price.
  • priceMin and priceMax are the two ends of the list price; listPriceText keeps it exactly as Sephora prints it.
  • salePrice, onSale and saleType describe a markdown. FULL means every size or shade is marked down, PARTIAL only some.
  • valuePrice is the value Sephora states, shown on sets and some sizes.
  • rating is null when a product has no reviews.
  • categoryTotal is how many products the category holds under your sort and filters.

Here is a sale row from page 2 of the same capture, trimmed to the price fields:

{
  "id": "P455369",
  "title": "Peptide Moisturizer",
  "brand": "The INKEY List",
  "price": 12.6,
  "priceMin": 18.0,
  "priceMax": 18.0,
  "salePrice": 12.6,
  "onSale": true,
  "saleType": "FULL",
  "rating": 3.72,
  "reviewCount": 979,
  "position": 25
}
Enter fullscreen mode Exit fullscreen mode

On page 1 of that capture, all 60 products were rated, 32 listed a price range rather than a single price, 52 carried the Bestseller flag, and none was on sale.

Pro tips and advanced usage

  1. Let the site sort for you. PRICE_LOW_TO_HIGH with maxItems: 20 returns the first 20 products in Sephora's own low-to-high order, because the site applies the order before the Actor reads the page.
  2. Narrow by brand before you widen by pages. In the 2026-10-21 recon, filtering moisturizers to Tatcha took the category from 727 products to 13. Fewer pages means a shorter run.
  3. Combine start URLs. Put several categories in startUrls. A product that sits in two of them is saved once, de-duplicated by id.
  4. Hook it to your pipeline. An Apify webhook fires when a run finishes; its resource is "the response of the Get Actor run API endpoint" (docs.apify.com/integrations/webhooks/actions), which carries defaultDatasetId.
  5. Use the flags as columns. isBestseller, isNew and isSephoraExclusive come straight from Sephora, so you can count them per brand without guessing from titles.

For current pricing and details, see the Store page.

Troubleshooting common issues

  • "is not a category page". The start URL is a search, /browse/ or product page. Use a https://www.sephora.com/shop/<category> URL.
  • "0 rows for ...". The run fails on purpose when nothing is saved: the brand has no products in that category, the price band is too narrow, or your filters dropped every row.
  • "minPrice ... is above maxPrice". Swap the two values.
  • Sort rejected. sort takes only the six values above; anything else is refused before the run starts.
  • A moved category. The Actor follows a moved category for up to two hops when it points to another /shop/ page; a category that does not exist is logged and skipped. Check the run log.

Next steps

Save your input as an Apify task and add a schedule: schedules use cron expressions, so a daily sale check is one line (docs.apify.com/actors/running/schedules). The Apify Python client docs cover the rest of the API, and you can start small with the prefill input above and grow from there.

What are you planning to scrape on Sephora? Drop a comment! 💬

Top comments (1)

Collapse
 
lucas_him_f7e05624dd9cc96 profile image
Lucas Him •

The FULL versus PARTIAL saleType distinction is useful here. A daily BrowserAct Cloud Bot should keep that field beside priceMin and priceMax, rather than alerting on the lowest price alone. With no per-shade price table, a changed tile minimum does not tell you whether the same size or shade became cheaper.