DEV Community

Алексей Спинов
Алексей Спинов

Posted on

How to Scrape Wayfair, Target, and Other E-Commerce Sites

Beyond Amazon and Walmart, here are more e-commerce scraping targets.

Target.com

Target has a product search API:

https://redsky.target.com/redsky_aggregations/v1/web/pdp_client_v1?key=PUBLIC_KEY&tcin=PRODUCT_ID
Enter fullscreen mode Exit fullscreen mode

Wayfair

Wayfair uses GraphQL internally — intercept with Playwright:

page.on("response", async (res) => {
  if (res.url().includes("graphql")) {
    const data = await res.json();
    // Product data in GraphQL response
  }
});
Enter fullscreen mode Exit fullscreen mode

Best Buy

Best Buy has a free Products API:

https://api.bestbuy.com/v1/products?apiKey=KEY&format=json
Enter fullscreen mode Exit fullscreen mode

General E-Commerce Approach

  1. Check for JSON API first (DevTools > Network > XHR)
  2. Look for JSON-LD structured data in HTML
  3. Fall back to HTML parsing with Cheerio
  4. Use Playwright only for JS-heavy SPAs

All E-Commerce Guides


Need e-commerce data from any store? $20. Email: Spinov001@gmail.com | Hire me

Top comments (0)