Coles, Woolworths and Aldi prices in one schema: what a weekly snapshot actually captures

The problem

Coles, Woolworths and Aldi Australia have three sites, three schemas and no shared product id. We wanted one weekly snapshot of all three in a single row shape, with a key to join rows on. The Australia Supermarket Price Scraper, an Apify Actor, returns exactly that. This post covers what a real weekly snapshot of all three looks like, measured on Wednesday 7 October 2026 (ISO week 2026-W41), and the one hard constraint we hit: Coles.

The snapshot schedule

Coles' and Woolworths' weekly specials start on Wednesday, so the snapshot runs every Wednesday, in five slots from 07:00 UTC. This week it made seven runs, all completed: Coles in 2 runs, Woolworths in 4, Aldi in 1.

Retailer	Runs	Products	Requests	Notes
Woolworths	4	16,430 (7,194 + 2,515 + 5,582 + 1,139)	947	No blocks. Store 1101 (the national default) on every run. Everyday Market (third-party marketplace) products excluded
Aldi	1	2,560	102	Plain direct connection, no proxy needed
Coles	2	6,628 (3,350 + 3,278)	81 to 82 per run	Stopped at Coles' request limit, as designed

That is 25,618 rows in total.

The join keys

Every row carries matchKey: a normalised brand, name and pack size. It is the recommended way to match the same product across all three retailers.

Rows can also carry ean, the GTIN-13 barcode, validated by its check digit. Woolworths rows carry one on nearly every row. Aldi publishes no barcode anywhere, so Aldi rows carry an empty ean array and join on matchKey. Coles listing rows, which is what the weekly snapshot reads, carry no barcode either; they also join on matchKey.

The hard constraint: Coles

Coles' bot protection answers about 80 requests per session and then stops. The Actor never evades or rotates around a block. When Coles stops answering, the Actor stops cleanly for Coles and keeps the rows it has already returned.

So the question was how much of Coles fits into about 80 requests. Two choices:

• The snapshot reads listing pages only: 48 products per page, with no per-product barcode request.
• Categories are ordered staples first: dairy, eggs, bread and bakery, meat, fruit and vegetables, pantry basics, then the rest.

The result this week was 6,628 Coles products in 2 runs (3,350 and 3,278, each stopping at 81 to 82 requests as designed). Coles lists 30,609 products, so that is 21.7% of the range, against a planned 22.9%. It is a partial snapshot, and the staples are the part covered.

A finding that changed the product

From 6 October, Coles' per-product data pages grew several times over, to about 40 to 50 KB each. A listing page carries 48 products in about 48 KB, roughly 1 KB per product, so a Coles row with a barcode lookup now takes dozens of times the proxy transfer of a listing row.

So from version 0.1.8, Coles barcode lookup (colesProductDetails) is opt-in and off by default. Product mode always looks barcodes up. Left off, Coles search and category rows have an empty ean and a run gets far more Coles products within the limit. Turn it on if you need barcodes and can accept far fewer Coles products per run.

What a row looks like

One row, from a Coles product page captured on 24 September 2026 (with unitPrice, categoryPath, imageUrl and scrapedAt illustrative, not real):

{
  "retailer": "coles",
  "retailerProductId": "1902454",
  "ean": ["9310036048985"],
  "matchKey": "pauls zymil lactose free full cream milk 2000ml",
  "name": "Pauls Zymil Lactose Free Full Cream Milk",
  "brand": "Pauls",
  "packSize": "2L",
  "price": 5.95,
  "wasPrice": 6.9,
  "unitPrice": { "value": 2.98, "measure": "l" },
  "loyaltyPrice": null,
  "promotionText": "save $0.95",
  "promotionEndsAt": null,
  "inStock": true,
  "categoryPath": ["Dairy, Eggs & Fridge", "Milk", "Long Life Milk"],
  "url": "https://www.coles.com.au/product/pauls-zymil-lactose-free-full-cream-milk-2l-1902454",
  "imageUrl": "https://cdn.productimages.coles.com.au/productimages/...",
  "rating": null,
  "currency": "AUD",
  "market": "au",
  "storeId": "7674",
  "onSaleDate": null,
  "scrapedAt": "2026-09-24T10:15:00Z"
}

This row has an ean because it came from a product page. A Coles row from the listing-only snapshot would have an empty ean.

Cost of the snapshot

The whole weekly snapshot (7 runs, 25,618 rows) cost $0.49 of platform usage. Of that, 0.039 GB of residential proxy ($0.31) went on Coles and Woolworths. Per 1,000 rows: Woolworths about $0.022, Coles listing-only about $0.016, Aldi about $0.010.

Those are our platform costs, not what you pay. Your price is below.

How to run it

Search mode, one keyword across all three retailers:

{
  "mode": "search",
  "queries": ["milk"],
  "maxItems": 60
}

Category mode, from a category URL copied from the retailer's site (the retailer is inferred from the URL):

{
  "mode": "category",
  "categoryUrls": ["https://www.coles.com.au/browse/dairy-eggs-fridge/milk"],
  "maxItems": 200
}

With Apify's Python client (apify-client 3.x, where call() returns a run object), then grouping rows on matchKey to see one product across retailers:

from collections import defaultdict
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("yappman/au-supermarket-price-scraper").call(
    run_input={"mode": "search", "queries": ["milk"], "maxItems": 60}
)

by_product = defaultdict(list)
for row in client.dataset(run.default_dataset_id).iterate_items():
    by_product[row["matchKey"]].append((row["retailer"], row["price"]))

for key, prices in by_product.items():
    if len(prices) > 1:
        print(key, prices)

Pack sizes are written differently across retailers, so a matchKey join will miss some products that are the same in the shop. Use ean where it is present.

Special Buys

Version 0.1.8 also adds a Special Buys mode ("mode": "specialBuys") for Aldi Australia's dated Special Buys drops, which Aldi puts in store on Wednesdays and Saturdays. For example, the 7 October drop was 94 products. Rows carry a new onSaleDate field with the drop date; the field is null for Coles and Woolworths rows.

Pricing and limits

Pricing is pay-per-event: $2.00 per 1,000 products returned on the Free tier, $1.20 per 1,000 at Gold, plus $0.005 per run start. Cap spend with Apify's maximum-charge setting on the run, and with the maxItems input.

The data is public and logged-out: no login, basket or account data. Woolworths prices are for the store Woolworths assigns the session (usually 1101); there is no store selection. Aldi publishes no barcode and no online stock.

Where to find it

The Australia Supermarket Price Scraper is on the Apify Store: https://apify.com/yappman/au-supermarket-price-scraper. This Actor is not affiliated with, endorsed by, or partnered with Coles, Woolworths or Aldi; retailer names are used only to describe the public product data it returns.