DEV Community

EcommMastermind
EcommMastermind

Posted on Edited on

423.900 KD is not 423,900 - the price parsing bug almost everyone ships

Most price parsers work fine until you point a scraper at a multi-region catalog. The moment you mix European thousand separators (1.299,00 €), Swiss apostrophes (4'239.00 CHF), and Middle Eastern 3-decimal currencies (423.900 KD), naive regex stripping starts corrupting data silently.

A standard parseFloat(raw.replace(/[^0-9.]/g, '')) or a simple heuristic that treats any trailing .999 group as a thousand separator will turn 423.900 KD (423.90 Kuwaiti Dinars) into 423900, inflating the recorded price by 1,000x.

The 3-decimal ISO 4217 problem

Most codebases assume currencies have either 2 minor units (USD, EUR, GBP) or 0 (JPY, KRW). ISO 4217 also defines 3-decimal currencies subdivided into 1,000 minor units (fils/baisa/millimes): KWD, BHD, OMR, JOD, and TND.

Locale Raw input Naive parser output Actual value
en_US $4,239.00 4239.00 4239.00 USD
de_DE 1.299,00 € 1.299 1299.00 EUR
ar_KW 423.900 KD 423900 423.900 KWD
de_CH 4'239.00 CHF 4 or NaN 4239.00 CHF

If a parser sees three digits after a dot (.900), it cannot know whether that dot is a thousand separator (like 1.299 € in Germany) or a decimal mark (like 423.900 KD in Kuwait) unless it resolves the currency code and its ISO 4217 exponent first.

Python implementation

Resolving the currency alias before inspecting separator positions avoids this ambiguity:

import re
from dataclasses import dataclass
from typing import Optional

THREE_DECIMAL_ISO = {"KWD", "BHD", "OMR", "JOD", "TND"}
CURRENCY_ALIASES = {
    "KD": "KWD", "د.ك": "KWD",
    "BD": "BHD", "د.ب": "BHD",
    "JD": "JOD", "DT": "TND",
    "€": "EUR", "$": "USD", "CHF": "CHF",
}

@dataclass
class NormalizedPrice:
    amount: float
    currency: Optional[str]

def parse_price(raw: str) -> NormalizedPrice:
    text = raw.strip()
    currency = None
    for token, iso in CURRENCY_ALIASES.items():
        if token in text:
            currency = iso
            text = text.replace(token, "").strip()
            break

    num_str = re.sub(r"['\s\u00a0]", "", text)

    if "." in num_str and "," in num_str:
        if num_str.rfind(",") > num_str.rfind("."):
            num_str = num_str.replace(".", "").replace(",", ".")
        else:
            num_str = num_str.replace(",", "")
    elif "," in num_str:
        parts = num_str.split(",")
        if len(parts) == 2 and len(parts[1]) in (1, 2):
            num_str = num_str.replace(",", ".")
        else:
            num_str = num_str.replace(",", "")
    elif "." in num_str:
        parts = num_str.split(".")
        if len(parts) == 2 and len(parts[1]) == 3 and currency not in THREE_DECIMAL_ISO:
            num_str = num_str.replace(".", "")

    return NormalizedPrice(amount=float(num_str), currency=currency)

print(parse_price("423.900 KD"))   # NormalizedPrice(amount=423.9, currency='KWD')
print(parse_price("1.299,00 €"))   # NormalizedPrice(amount=1299.0, currency='EUR')
Enter fullscreen mode Exit fullscreen mode

TypeScript implementation

Here is the equivalent logic in TypeScript for Node or edge runtimes:

const THREE_DECIMAL_CURRENCIES = new Set(["KWD", "BHD", "OMR", "JOD", "TND"]);

const SYMBOL_MAP: Record<string, string> = {
  "KD": "KWD",
  "د.ك": "KWD",
  "BD": "BHD",
  "JD": "JOD",
  "DT": "TND",
  "€": "EUR",
  "$": "USD",
  "CHF": "CHF",
};

export interface PriceResult {
  amount: number;
  currency: string | null;
}

export function parseProductPrice(raw: string): PriceResult {
  let text = raw.trim();
  let currency: string | null = null;

  for (const [symbol, iso] of Object.entries(SYMBOL_MAP)) {
    if (text.includes(symbol)) {
      currency = iso;
      text = text.replace(symbol, "").trim();
      break;
    }
  }

  let cleaned = text.replace(/['\s\u00a0]/g, "");

  if (cleaned.includes(".") && cleaned.includes(",")) {
    if (cleaned.lastIndexOf(",") > cleaned.lastIndexOf(".")) {
      cleaned = cleaned.replace(/\./g, "").replace(",", ".");
    } else {
      cleaned = cleaned.replace(/,/g, "");
    }
  } else if (cleaned.includes(",")) {
    const parts = cleaned.split(",");
    cleaned = parts.length === 2 && parts[1].length <= 2
      ? cleaned.replace(",", ".")
      : cleaned.replace(/,/g, "");
  } else if (cleaned.includes(".")) {
    const parts = cleaned.split(".");
    if (parts.length === 2 && parts[1].length === 3 && (!currency || !THREE_DECIMAL_CURRENCIES.has(currency))) {
      cleaned = cleaned.replace(/\./g, "");
    }
  }

  return { amount: Number(cleaned), currency };
}
Enter fullscreen mode Exit fullscreen mode

Edge cases beyond basic separators

In production scrapers, raw strings also include price ranges ($19.99 - $29.99), promotional strike-throughs (Was €1.299,00 Now €999,00), RTL markers, and VAT suffixes.

I put together an open test corpus of 820 real-world and adversarial price vectors if you want to benchmark your own parser against these edge cases:

  • Test corpus (820 JSON vectors): ecommastermind.xyz/corpus/cases.json
  • Live /v1/normalize playground & OpenAPI spec: ecommastermind.xyz
  • Hosted API on RapidAPI: E-commerce Product JSON Formatter

Top comments (0)