How to Scrape SHEIN Products with Python

Ivan — Fri, 06 Nov 2026 14:39:10 +0000

A product's headline price is only part of the information needed for a useful shortlist. Currency, availability and the number of reviews can change which items are worth inspecting. This tutorial shows how to scrape SHEIN products into JSON or CSV, then turn the results into a small spreadsheet of in-stock products priced below $20 with a rating of at least 4.5.

The tool is 2scraper/shein-scraper, an MIT-licensed Python project. The example below uses its bundled sample data so you can try the filtering step without making a live request. Those records are dated September 30, 2026; they are not current prices or results from a new test.

How to scrape SHEIN products into a budget shortlist

Use the Playwright engine to collect a small search result from us.shein.com. Save the records as JSON, inspect the run's metadata, and filter products by currency, stock, price and rating. Export that selection to CSV. Treat it as a snapshot of the collected products, not a complete catalogue or variant inventory.

Install Python 3.9 or newer and clone the repository.
Install the Playwright dependencies and Chromium.
Configure any credentials in .env.
Run one search with a small result limit.
Check the exit code and the accompanying metadata file.
Filter the JSON records and write a shortlist to CSV.

Install once, collect a small search

The repository recommends Playwright. Use a separate virtual environment for it: the project's Playwright, Selenium and Puppeteer dependency sets should not be installed together.

These setup commands are from the README and use a Bash-compatible shell, such as Linux, macOS or WSL:

git clone https://github.com/2scraper/shein-scraper.git
cd shein-scraper
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements-playwright.txt
playwright install chromium
cp .env.example .env

The .env file contains TWOCAPTCHA_KEY for CAPTCHA solving, SHEIN_CDP_ENDPOINT for a remote browser profile, and SHEIN_PROXY for an optional proxy. Add only the values you use. Keep credentials out of commands, screenshots and published files.

A local browser can start without a CAPTCHA key, but that does not mean it will get product data: SHEIN may challenge or rate-limit it. For regular use, the README recommends a 2Captcha Scraping Browser API profile plus a solver key. Browser access and CAPTCHA solving are separate services.

For a small search, run:

python3 playwright_scraper.py --query "linen shirt" --max-results 20

The query is an example input, not a claim that it was tested for this article. When rows are written, the default files are shein_results.json and shein_results.json.meta.json. The second file describes the run, including its status and why it stopped.

To request CSV directly, use the documented output options:

python3 playwright_scraper.py --query "linen shirt" --max-results 20 --format csv --out linen-shirts.csv

This is another collection run, not a conversion of the previous JSON file. For an offline conversion and filter, use the Python example further down.

What each input actually returns

Input	Where the data comes from	Useful output
`--query "linen shirt"`	Embedded data on a search listing	Product IDs, titles, prices, discounts, ratings and stock flags when present
`--category "Women Jeans-c-1934.html"`	Embedded data on a category listing	The same product-level fields for that category
`--url` with a product-page address	The page's structured JSON-LD data	Product identity and price; other fields can be missing

For a search-to-product workflow, start with the listing and take a returned product_url. Pass that address to --url when you want to inspect that specific product page. A listing run does not automatically open every product card and build a complete variant inventory.

Three field groups matter for the shortlist:

price, currency, original_price and discount_pct describe the price and displayed discount.
rating and review_count describe the average rating and its review count. These are not individual review texts.
in_stock is a product-level flag. It does not prove that every size or colour is available.

The README identifies sku as SHEIN's goods_id, shared across a product's colours and sizes. On a product page with differently priced variants, price_source: "json_ld_min_variant" means the parser selected the lowest variant price. Do not present that number as the price of every variant.

Keep the market and price source visible

This implementation targets us.shein.com. Other regional sites have not been tested, so replacing the domain is not a documented way to switch markets. The sample records use USD; the filtering script below explicitly keeps that currency.

Listings are read from window.gbRawData, the product data embedded in the page. Product pages are read from schema.org JSON-LD. The price_source field preserves that distinction. It is useful when a listing price and a product-page price do not appear to agree.

Missing values also carry meaning. A missing rating is not a zero-star review, and an unknown stock flag is not confirmed availability. The shortlist leaves those records out rather than guessing.

Why a browser can stop before returning products

The project handles JavaScript-rendered product data, an infinitely scrolling listing and SHEIN's risk gateway. A basic HTML request does not cover that whole workflow.

The README distinguishes two gateway responses:

Response	Documented behaviour
`/risk/challenge`	The scraper attempts the checkbox, then can send an image grid or icon-sequence puzzle to 2Captcha when configured.
`/risk/action/limit`	This is a rate limit with no puzzle to solve. A run with no rows exits with code `3`.

Image grids use GridTask; icon sequences use CoordinatesTask. The default --solve-captcha mode is when-blocked. --max-solves defaults to 8 and limits paid solve attempts per run; 0 prevents paid solves. That cap does not include remote-browser or proxy charges.

Solving a challenge is not guaranteed to work. The README's September 30 test reported success for three of five fresh US browser profiles. That small historical test is not a reliability promise. The maintainers' full guide provides the broader setup and option reference.

Inspect the run before trusting the file

The repository's current documented release is 0.2.0, dated October 16, 2026. Its exit codes distinguish a completed run from incomplete data:

Exit code	Meaning for the collected data
`0`	The run completed. A requested result limit can still make it only a selection.
`6`	Some rows were written, but the run is partial. Check `stop_reason`.
`3`	Blocked or rate-limited, with no rows.
`4`	The page loaded but had no products.
`5`	Content was not obtained or a listing known to contain products could not be parsed.

A zero-row run normally preserves previous output rather than overwriting it. That is helpful, but an existing file may therefore belong to an earlier run. Check the latest command's exit status and the records' scraped_at values before using it as fresh data.

The README also says collection beyond the first roughly 20 listing products has not been verified. Increasing --max-results does not prove that you collected the whole category.

Build a CSV shortlist from the bundled sample

Here are selected fields from one record in sample_output.json, unchanged:

{
  "sku": "499485260",
  "brand": "jotten",
  "price": 8.55,
  "currency": "USD",
  "price_source": "embedded_json",
  "scraped_at": "2026-09-30T12:56:49Z",
  "original_price": 13.79,
  "discount_pct": 38.0,
  "rating": 4.8,
  "review_count": 143,
  "in_stock": true
}

The bundled file contains three dress records. They demonstrate the processing step; they are not results for the linen shirt query above.

Save this code as shortlist.py inside the repository. It uses only Python's standard library and fields present in the sample:

import csv
import json
import sys
from pathlib import Path

source = Path(sys.argv[1] if len(sys.argv) > 1 else "sample_output.json")
rows = json.loads(source.read_text(encoding="utf-8"))
selected = [
    row for row in rows
    if row.get("currency") == "USD"
    and row.get("in_stock") is True
    and isinstance(row.get("price"), (int, float))
    and 0 <= row["price"] < 20
    and isinstance(row.get("rating"), (int, float))
    and row["rating"] >= 4.5
]
selected.sort(key=lambda row: row["price"])
fields = ["sku", "title", "price", "currency", "original_price",
          "discount_pct", "rating", "review_count", "product_url"]

def csv_value(value):
    if isinstance(value, str) and value.startswith(("=", "+", "-", "@", "\t", "\r", "\n")):
        return "'" + value
    return value

with Path("budget_shortlist.csv").open("w", encoding="utf-8", newline="") as file:
    writer = csv.DictWriter(file, fieldnames=fields)
    writer.writeheader()
    writer.writerows({key: csv_value(row.get(key)) for key in fields} for row in selected)
print(f"Saved {len(selected)} products to budget_shortlist.csv")

Run it on the repository sample:

python3 shortlist.py sample_output.json

All three bundled records meet these conditions. In ascending price order, the CSV contains SKUs 499485260, 458057728 and 430743850, priced at $8.55, $13.03 and $17.51. These are reproducible calculations from the supplied sample, not live offers.

Illustration of the bundled sample after filtering and sorting; not a screenshot of a live scraping run.

After checking a live run's status and freshness, process its JSON instead:

python3 shortlist.py shein_results.json

The script overwrites budget_shortlist.csv each time. It retains the review count for context but does not impose a minimum count. A rating of 4.8 from a few reviews and the same rating from many reviews deserve different interpretations.

Questions to resolve before extending the workflow

Can this collect all SHEIN products?

The repository does not establish complete catalogue coverage. Start with the small documented scope, inspect partial-run metadata and keep the unverified scrolling behaviour in mind.

Can I export every size and colour?

The output is one row per product, not one row per variant. A product-page minimum variant price is useful context, but it is insufficient for a size-by-size stock report.

Can I turn this into a SHEIN price tracker in Python?

The repository includes diff_runs.py for comparing compatible, complete runs. Keep the same search and market; a product missing from a capped selection is not evidence that SHEIN removed it.

Does this need an official SHEIN product API?

The documented browser workflow reads data embedded in SHEIN pages. It does not require access to a separate official product API, and this article makes no claim about whether such APIs are available to particular partners.

How much does a collection run cost?

There is no fixed per-run price established by this example. CAPTCHA solve counts, remote-browser usage and optional proxy usage are separate inputs; consult the services' current pricing before a paid run.

Use this workflow only where you have permission and in line with applicable site terms, robots.txt and data-protection requirements. Keep request rates modest and stop when access is refused. This example concerns product information and does not require customer accounts or personal data.

The useful output here is a small, traceable shortlist: every price has a currency, every record has a collection time, and missing fields remain visible. Start with the bundled sample, then decide whether the scraper's documented market and product-level scope fit your own task.

Creation note: this article was drafted with generative AI from the project's documentation and bundled sample data. The offline filtering example was checked against that sample. No live SHEIN scraping was performed for this article.

DEV Community: Ivan