Parsing prices from any country in Python: 1.299,00 TL vs $1,299.00

#python #webscraping #tutorial #beginners

If you scrape shops from more than one country, prices come in different shapes: 1,299.00 in the US, 1.299,00 in Turkey and much of Europe, sometimes with a currency symbol before or after, sometimes with a space as thousands separator. float() fails on all of them. Here is a small function that handles the common cases, using only the standard library.

The idea

Find the first number-like chunk, then decide which separator is the decimal one:

If both . and , appear, the LAST one is the decimal separator.
If only a comma appears and it is followed by one or two digits, it is a decimal comma (49,90). Otherwise it is a thousands comma (1,299).
If several dots appear and no comma, they are thousands separators (1.299.000).

The code

import re

def parse_price(text):
    m = re.search(r"\d[\d.,\s]*", str(text))
    if not m:
        return None
    s = m.group(0).strip().replace(" ", "")
    if "," in s and "." in s:
        s = s.replace(".", "").replace(",", ".") if s.rfind(",") > s.rfind(".") else s.replace(",", "")
    elif "," in s:
        head, tail = s.rsplit(",", 1)
        s = head.replace(",", "") + "." + tail if len(tail) in (1, 2) else s.replace(",", "")
    elif s.count(".") > 1:
        s = s.replace(".", "")
    try:
        return float(s.rstrip("."))
    except ValueError:
        return None

Real output

'£51.77'       -> 51.77
'$1,299.00'    -> 1299.0
'1.299,00 TL'  -> 1299.0
'49,90'        -> 49.9
'1,299'        -> 1299.0
'Price: 7 USD' -> 7.0
'1 299,90'     -> 1299.9
'free'         -> None

The one case you must decide yourself

"1,299" is ambiguous: it is 1299 in English-style formats and 1.299 in a format that uses a decimal comma with three digits. The function treats a comma followed by three digits as a thousands separator. If you scrape a site that really uses three decimals, pass the page language in and adjust this rule.

Testing it

The function is covered by unit tests with all the cases above. The same logic is used in a small price tracker that I tested against a local web server serving pages in each price format and checked once on a public demo shop (it returned 51.77 for a book listed at £51.77).

Get it

This is one of 20 helpers in a small tested collection (CSV encodings, retry, slugify, rate limiter and more) on Gumroad for $6: https://ixilig.gumroad.com/l/yathgth

Built together with Claude (an AI assistant); I ran and checked the code myself.

DEV Community