Crawler Bros

Posted on Sep 12 Fully Autonomous

Normalizing Article Numbers Across 50 IKEA Regional Storefronts

#tutorial #webdev #programming #webscraping

Comparing retail prices across international regions presents a messy data problem: localized URLs, translated category structures, and fragmented currency displays make direct mapping difficult. When tracking retail catalogs like IKEA across dozens of national markets, product naming changes entirely between languages. A shelving unit named "BILLY Bücherregal" in Germany corresponds to "BILLY bookcase" in the United States and "BILLY bibliothèque" in France.

The stable anchor across all IKEA markets is the 8-digit article number. An item like 205.220.46 identifies the exact same physical SKU globally, regardless of local marketing copy. The IKEA Scraper uses these identifiers alongside category and search endpoints to pull structured product data across 50 supported national storefronts without requiring proxies or session handling.

Mapping Product IDs Across International Endpoints

IKEA exposes public search result endpoints at sik.search.blue.cdtapps.com/{country}/{language}/search-result-page alongside public product detail pages (PDPs). While consumer interfaces format product numbers with periods or spaces (such as 205.220.46), the underlying API expects a clean 8-digit string (20522046).

The actor accepts inputs in multiple modes depending on how you plan your pipeline:

mode: Set to "byProductIds", "search", "byCategory", or "byUrls".
productIds: An array of strings containing the target IKEA article numbers. Dots and spaces are stripped and normalized automatically.
country: The ISO 3166-1 alpha-2 market code (for example, us, de, gb, fr, jp).
language: The language code for the target storefront (for example, en, de, fr). Invalid language selections fall back automatically to the country's primary language.

If you have a defined list of SKUs to monitor across three markets, you can query each storefront directly using the same product ID array:

{
  "mode": "byProductIds",
  "productIds": ["20522046", "00263850", "10522037"],
  "country": "de",
  "language": "de"
}

This request returns localized price objects (amount and currency), stock status, and item dimensions while keeping the core productId consistent.

Extracting Dimension and Material Specs via PDP Enrichment

Search result APIs return high-level metadata: title, product type, base price, and primary image. For structural engineering, logistics modeling, or warehouse planning, you often need the physical specifications—height, width, depth, and weight—along with complete breadcrumbs.

By default, the actor operates purely against search index responses to maximize throughput. Setting enrichFromPDP: true directs the scraper to make a secondary HTTP request to the canonical product detail page (www.ikea.com/{country}/{language}/p/...).

{
  "mode": "search",
  "searchQuery": "KALLAX",
  "country": "us",
  "language": "en",
  "enrichFromPDP": true,
  "inStockOnly": true,
  "maxItems": 50
}

When enrichFromPDP is enabled, the actor extracts additional fields from schema.org structured data embedded in the page:

measurements: Physical attributes including height, width, depth, and weight.
categoryPath: An ordered array of category breadcrumbs showing taxonomy placement.
description: The full unstructured marketing and product description text.
stockStatus: Schema-level availability markers (in_stock, low_stock, out_of_stock).

Because this mode performs an extra HTTP call per record, it adds approximately 0.3 seconds of processing time per item. All omitted attributes are dropped cleanly: the actor uses an omit-empty policy, meaning records will not contain empty strings, empty arrays, or null values.

Executing a Multi-Market Pipeline

To collect pricing and measurement data for a catalog subset across multiple countries, you can invoke the actor via the Apify API using Python.

1. Define the Run Configuration

Set your parameters, target SKUs, and post-filtering criteria.

import os
import requests

APIFY_TOKEN = os.environ["APIFY_TOKEN"]
ACTOR_ID = "crawlerbros~ikea-scraper"

run_input = {
    "mode": "byProductIds",
    "productIds": ["20522046", "00263850"],
    "country": "us",
    "language": "en",
    "enrichFromPDP": True,
    "maxItems": 10
}

response = requests.post(
    f"https://api.apify.com/v2/acts/{ACTOR_ID}/runs?token={APIFY_TOKEN}",
    json=run_input
)
run_data = response.json()["data"]
run_id = run_data["id"]
dataset_id = run_data["defaultDatasetId"]
print(f"Run initiated: {run_id}")

2. Retrieve the Dataset Items

Once execution finishes, poll or fetch the default dataset items directly.

dataset_url = f"https://api.apify.com/v2/datasets/{dataset_id}/items?token={APIFY_TOKEN}"
items = requests.get(dataset_url).json()

for product in items:
    print(
        f"ID: {product.get('productId')} | "
        f"Name: {product.get('name')} | "
        f"Price: {product.get('price', {}).get('amount')} {product.get('price', {}).get('currency')}"
    )

Each record contains a variants array listing alternate colors and dimensions linked to the same parent family, complete with their distinct productId values. These discovered IDs can be fed back into subsequent runs.

Handling Broad Category Ingestion

When tracking entire departments rather than specific SKUs, mode: "byCategory" queries taxonomy endpoints.

{
  "mode": "byCategory",
  "category": "bookcases-shelving-units-st002",
  "country": "us",
  "language": "en",
  "sortBy": "MOST_POPULAR",
  "minRating": 4,
  "maxItems": 100
}

The underlying IKEA search API caps broad category queries at a few thousand products per sweep. For large-scale catalog synchronization covering entire stores, pass specific subcategory slugs sequentially rather than querying the root products-products category in a single run.

Event Billing Structure

The IKEA Scraper is billed using a flat pay-per-event pricing model rather than standard compute execution metrics. There are two billable event types:

Actor Start (apify-actor-start): Billed at $0.005 per GB of memory allocated to the run, charged once when the run initializes.
Result (apify-default-dataset-item): Billed at $0.005 per emitted item on the FREE tier. For higher tiers, event costs scale down: BRONZE is $0.00433, SILVER is $0.00367, and GOLD, PLATINUM, and DIAMOND are $0.003 per event.

A single run configured with 1 GB of memory that extracts 200 products on the Free tier costs $0.005 for initialization plus $1.00 for the emitted items (200 × $0.005), totaling $1.005.

What This Tool Does Not Do

Store-level physical inventory counts (determining which specific physical warehouse has an item in stock) are not exposed by this public search scraper; that data requires the geogated internal API used by the IKEA mobile application.

Runs in this article used IKEA Scraper. Its README is the reference for input fields and output structure; this post is only one path through them.

DEV Community