DEV Community

Crawler Bros
Crawler Bros

Posted on

How to Scrape Woolworths Australia Grocery Data and Prices

Australian grocery pricing shifts rapidly with weekly promotions, inflation adjustments, and brand competition. For market researchers, developers building grocery comparison tools, or analysts tracking FMCG trends, gathering structured data from major retailers is essential. Manually tracking thousands of SKUs across Australian supermarkets is inefficient and prone to errors.

The woolworths-au-scraper solves this by automating data extraction directly from Woolworths Australia's public search API. It retrieves real-time prices, discounts, nutrition facts, and stock information without requiring user accounts or session cookies.

What Data Can You Extract from Woolworths Australia?

The actor extracts clean, structured product data directly into dataset records. Empty fields are omitted, meaning records only include the data Woolworths actually publishes for that specific SKU.

Key output fields include:

  • Product identity: productId (Woolworths stockcode), barcode (EAN/GTIN), name, brand, and variant
  • Pricing and specials: price (in AUD), originalPrice, discount, discountPercent, pricePerUnit, and isOnSale
  • Categorisation: category (department name) and subcategory (e.g., Full Cream Milk)
  • Availability: inStock, isNew, and purchaseLimit
  • Product information: description, imageUrl, productUrl, and unit
  • Health and dietary specs: ingredients, allergens, dietaryAttributes (such as Gluten Free or Vegetarian), healthStarRating, and the per-100g nutrition panel (energyKj, protein, fat, saturatedFat, carbohydrates, sugars, sodium)

Real-World Use Cases for Grocery Data Scraping

1. Competitive Price Monitoring

Retail analysts and FMCG brand managers can monitor how competing products are priced across departments. By tracking fields like price, originalPrice, and pricePerUnit, businesses can identify when competitors run promotions or adjust base retail prices.

2. Supermarket Price Comparison Apps

Developers building comparison platforms can ingest Australian supermarket data to help consumers compare basket costs. Using identifiers like barcode and productId, teams can match products across multiple datasets accurately.

3. Deal Aggregation and Promotion Tracking

With dedicated deal scanning, marketers and deal aggregation sites can surface weekly supermarket specials. By filtering for discounted products, platforms can alert consumers to half-price deals or category-wide sales.

4. Nutrition and Diet Analysis

Researchers and health app developers can use published dietaryAttributes, healthStarRating, and the nutrition panel to analyse health metrics, ingredient profiles, and allergen declarations across thousands of grocery items.

How to Configure the Scraper: Core Modes and Inputs

The actor supports three primary operational modes via the mode parameter:

  • search: Performs a free-text search using the searchQuery input (e.g., milk, chocolate, or chicken breast).
  • byCategory: Browses any of Woolworths' 19 real departments, such as Bakery, Fruit & Veg, Pantry, Freezer, or Poultry Meat & Seafood, configured through the category input.
  • weeklyDeals: Scans a selected department specifically for products Woolworths currently flags as on special, half-price, or Everyday Rewards specials.

Refining Your Search with Input Filters

You can refine the dataset directly at the run level using these input options:

  • onSaleOnly: A boolean setting that ensures only discounted products are saved to the dataset (automatically active in weeklyDeals mode).
  • minPrice and maxPrice: Numeric thresholds in AUD to drop items outside your target price window.
  • brand: A case-insensitive text filter to collect items matching specific brand names.
  • maxItems: An integer cap (between 1 and 200) defining how many products to emit. The actor automatically handles pagination through the search results until it reaches the cap or exhausts matching products.
  • proxyConfiguration: Configures proxies, defaulting to Apify's free datacenter proxy pool with an automatic fallback to direct requests.

Step-by-Step Walkthrough: Running the Actor

  1. Navigate to the woolworths-au-scraper on the Apify platform.
  2. Select your desired mode (search, byCategory, or weeklyDeals).
  3. Set your target query or department using searchQuery or category.
  4. (Optional) Apply filters such as maxPrice, brand, or set onSaleOnly to true.
  5. Set maxItems to define how many product records you want to retrieve.
  6. Click Start and let the actor automatically paginate through Woolworths' product listings.
  7. Export your dataset in JSON, CSV, or Excel formats, or connect the output directly to downstream workflows and webhooks.

Frequently Asked Questions

Do I need a Woolworths login or session cookies?

No. The actor connects directly to Woolworths' public search API—the same data returned during standard browser searches—without requiring accounts or authentication cookies.

How does the scraper handle pagination?

The actor features automatic pagination. It steps through consecutive pages of results until your specified maxItems limit is reached or no further matching products exist.

Can I browse specific grocery departments?

Yes. You can select any of the 19 standard departments using the category dropdown in byCategory or weeklyDeals mode, allowing targeted data collection across sections like Dairy, Eggs & Fridge or Drinks.


Ready to try it yourself? Run *Woolworths Australia Scraper** on the Apify Store -- no setup required.*

Top comments (0)