DEV Community

Cover image for How to Scrape Idealo Prices and Offer Counts with Python
ReventaData
ReventaData

Posted on Originally published at apify.com

How to Scrape Idealo Prices and Offer Counts with Python

#python #webscraping #apify #ecommerce

By the end of this tutorial, you will have a short Python script that turns an Idealo search into structured rows: product title, the lowest offer price as a number, the number of shop offers, the rating and review count, and a link to the comparison page.

Idealo is a price comparison site with domains in Germany, Austria, Spain, France, Italy and the United Kingdom. It sits behind Akamai: in testing on 2026-10-16, direct and datacenter requests got HTTP 403, while Apify Unblocker and residential proxies returned real pages.

We use the Apify Actor Idealo Scraper (reventadata/idealo-scraper). You will configure a search, run the Actor and filter rows by price and offer count. The row shown below is a sample from a capture dated 2026-10-24, parsed with the Actor's own parser.

⏱️ Time estimate: one sitting: install one package, set one token, run three short Python blocks.

Prerequisites

To follow this tutorial, you will need:

  1. Python 3.11+ (apify-client 3.3.0 requires it).
  2. The Apify API client for Python.
  3. An Apify account and its API token, which sits on the API & Integrations page in Apify Console.

Install the client:

pip install -U "apify-client~=3.0"
Enter fullscreen mode Exit fullscreen mode

The code reads the token from an environment variable:

export APIFY_API_TOKEN="your_apify_token_here"
Enter fullscreen mode Exit fullscreen mode

What you'll achieve

You will turn an Idealo search into one row per product. Each row carries the product id and comparison-page url, the title, price (the lowest offer price as a number) with its currency, offersCount (how many shop offers Idealo lists), rating, reviewsCount, the card's spec line in description, an image URL, the country market and the matchedKeyword.

Note: Rows describe products, not shops: no merchant names or seller details are in the output.

Example use case: Suppose you track iPhone prices on idealo.de and only care about models that many shops sell. You would set searchQueries: ["iphone"], country: "DE" and minOffers: 100, and every saved row would show at least 100 shop offers.

Step 1: Access Idealo Scraper

Go to the Idealo Scraper page on Apify Store. The README lists every input and output field. In this tutorial we drive the Actor from Python, so everything after this step happens in code.

Step 2: Configure the search

The main input is searchQueries: one Idealo search per term, for example iphone or rtx 4070. Two more inputs decide where and in which order the search runs:

  • country: the market, one of DE, AT, ES, FR, IT or UK (default DE). Prices come back in EUR, or GBP on the UK site.
  • sort: DEFAULT, PRICE_ASC, PRICE_DESC, DISCOUNT_DESC or NEWEST, passed to Idealo as its own sort order.

maxPages (default 5) bounds how many result pages each query walks, and maxItems (default 60) caps the rows saved across all queries.

Here is the setup. It searches iphone on idealo.de, reads one result page and saves at most 5 rows:

import os
from apify_client import ApifyClient

# 1. Initialize ApifyClient with the token from the environment
client = ApifyClient(os.environ["APIFY_API_TOKEN"])

# 2. Define the search: one keyword, one market, one result page, at most 5 rows
run_input = {
    "searchQueries": ["iphone"],
    "country": "DE",
    "maxPages": 1,
    "maxItems": 5,
}
Enter fullscreen mode Exit fullscreen mode

💡 Tip: a row is kept only when its title contains every word of the query, ignoring case and accents. iphone 17 pro keeps titles with all three words; iphone alone keeps every iPhone title.

No proxy setting is needed in run_input: when none is given, the Actor uses Apify Unblocker, which returned real pages in testing on 2026-10-16.

Step 3: Add optional filters

These filters run on the rows after each page is read. Rows a filter drops never reach your dataset:

Parameter Type What it keeps
minPrice / maxPrice number Rows whose price is inside the band, in the market's currency. Rows without a price are dropped when either is set.
minOffers integer Rows with at least N shop offers. Rows without an offer count are dropped.
includeKeywords array Rows whose title or description contains every one of these words.
excludeKeywords array Rows whose title or description contains none of these words.

Three combinations that work:

  • Cheapest first under a cap: sort: "PRICE_ASC", maxPrice: 500.
  • Widely sold models only: minOffers: 100.
  • One storage size: searchQueries: ["iphone 17"], includeKeywords: ["256GB"].

⚠️ A country or sort value outside its list is refused before the run starts.

Step 4: Run the Actor

Now start the run with client.actor("reventadata/idealo-scraper").call(run_input=run_input). The call starts the Actor on Apify and waits for it to finish before returning the run:

# 3. Start the Actor run and wait for it to finish
print("Starting Idealo extraction...")
run = client.actor("reventadata/idealo-scraper").call(run_input=run_input)
if run is None or run.status != "SUCCEEDED":
    status = run.status if run else "unknown"
    raise SystemExit(f"Run did not succeed (status: {status}).")

# 4. Fetch the saved products
items = client.dataset(run.default_dataset_id).list_items().items
print(f"Retrieved {len(items)} products.\n")
Enter fullscreen mode Exit fullscreen mode

Behind the scenes:

  • The Actor builds the market's search URL (idealo.de/preisvergleich/MainSearchProductCategory.html?q=iphone here) and fetches each result page through Apify Unblocker.
  • It parses each product card, applies your filters, skips a product it already saved under an earlier query, and saves the rows.
  • If a page comes back as a challenge page, it tries once more, then moves on to the next query.
  • It stops at maxItems, at maxPages, or when a page has no products, whichever comes first.

Step 5: View and download results

Loop over the rows to print the parts you need:

for item in items:
    print(item["title"])
    if item.get("price") is not None:
        print(f"  Lowest price: {item['price']} {item['currency']}")
    print(f"  Shop offers: {item.get('offersCount')}")
    print(f"  Rating: {item.get('rating')} ({item.get('reviewsCount')} reviews)")
    print(f"  Link: {item['url']}")
    print("-" * 40)
Enter fullscreen mode Exit fullscreen mode

Here is one row, a sample from a capture dated 2026-10-24 (page 1 of the iphone search on idealo.de) parsed with the Actor's own parser. A saved row also carries scrapedAt, added at save time:

{
    "id": "207643424",
    "title": "Apple iPhone 17",
    "url": "https://www.idealo.de/preisvergleich/OffersOfProduct/207643424_-iphone-17-apple.html",
    "price": 858.0,
    "currency": "EUR",
    "offersCount": 580,
    "rating": 1.5,
    "reviewsCount": 271,
    "description": "5G Handy, 6,3 Zoll, 120 Hz , 48 MP, 8 GB RAM, A19, iOS 26, 3.692 mAh",
    "image": "https://cdn.idealo.com/folder/Product/207643/4/207643424/s1_produktbild_mittelgross/apple-iphone-17.jpg",
    "country": "DE",
    "matchedKeyword": "iphone"
}
Enter fullscreen mode Exit fullscreen mode

Apify datasets export as JSON, CSV, XML, Excel, HTML table, RSS or JSONL.

Understanding your results

The fields that matter most:

  • price and currency: the card's "ab" (from) price, ab858,00 € on the page, as the number 858.0. In the capture, prices ran from 269.95 to 1,539 EUR across 36 products.
  • offersCount: how many shop offers Idealo lists for the product (580Angebote became 580). It ranged from 47 to 669; 21 of the 36 products had at least 100.
  • rating and reviewsCount: the card's Note ∅ value and its review count. 22 of the 36 rows had a rating. The 14 without one were all storage-size listings such as Apple iPhone 17 256GB Schwarz, and the five Apple iPhone 17 256GB colours showed the same review count (271) as the base Apple iPhone 17.
  • description: the card's spec line, joined as shown, so 120 Hz , 48 MP keeps its stray space.
  • id and url: Idealo's product id and its comparison page, where the shop list lives.
  • country and matchedKeyword: which market and query produced the row.

Pro tips

  1. Sort on the site, filter in the Actor. sort changes the order Idealo serves; minPrice, maxPrice, minOffers and the keyword lists only drop rows after reading. Pair sort: "PRICE_ASC" with maxPrice so the rows you want come first.
  2. Use the title rule. Every query word must be in the title, so a precise query (iphone 17 pro max) is itself a filter.
  3. Batch queries in one run. Put several terms in searchQueries. A product kept under one term is not saved again under the next.
  4. Compare markets with one run each. country takes one market per run. Run the same query once per market; each row's country says where it came from.
  5. Watch for new listings. sort: "NEWEST" asks Idealo for its newest-first order (Neuheiten on idealo.de).
  6. Start small. Keep maxPages: 1 and maxItems: 5 until your downstream code handles the fields, then raise them.

Troubleshooting

  • Zero rows: a run that is blocked or cannot read a page ends FAILED; an over-strict filter or a query no title matches ends SUCCEEDED with an empty dataset and a status message saying so.
  • Fewer rows than expected: every query word must appear in the title, and rows without a price or offer count drop out once a price or offer filter is set. Read the log's summary line, Run finished: pushed N records to dataset.
  • Run refused at start: check country and sort against their lists.
  • Challenge pages in the log: keep the default proxy. If you set your own proxyConfiguration, remember that direct and datacenter requests got HTTP 403 in testing on 2026-10-16.

For help, use the Actor's Store page.

Pricing and next steps

For current pricing and details, see the Store page.

To repeat a search, save the input as a task and add an Apify schedule, which runs an Actor or task on a cron expression (0 8 * * * is every day at 8 AM). To react when a run finishes, add a webhook on ACTOR.RUN.SUCCEEDED; its payload carries the run object, including defaultDatasetId.

Which products or markets are you planning to track on Idealo? Drop a comment! 💬

Top comments (0)