Korean cosmetic labels are in Korean. I mapped 50,000 ingredient entries to English INCI names — here's what matched and what broke.

#datascience #webscraping #showdev #korea

Two days ago I posted an ingredient breakdown of Olive Young's top 100 serums, with one honest caveat at the bottom: the ingredient names came back in Korean, exactly as printed on the label, and I had grouped them by their Korean names myself.

That's fine for one blog post and useless for anyone who wants to compare a Korean label with a US or EU product. 병풀추출물 means nothing to an ingredient checker. Centella Asiatica Extract does. So I added English INCI names to the scraper, and this is what it took.

Step 1: find a dictionary you're actually allowed to use

Korean cosmetic labels use standardized Korean ingredient names, and each one maps to an INCI name. The obvious source is the industry association's online ingredient dictionary. It has a nice search page — and terms of use that forbid using the data commercially without prior consent. I charge for the scraper, so that was out.

The answer turned out to be government open data. Korea's Ministry of Food and Drug Safety publishes its cosmetic ingredient list on the national open-data portal with no usage restrictions: 21,945 standard names, each with the English name, CAS number and a list of synonyms. One API key and 220 paged requests later I had a Korean-to-English map of about 24,000 names (standard names plus synonyms).

If you scrape for a living: read the terms before you build. The second source was better anyway.

Step 2: labels are messier than dictionaries

A plain exact-match lookup isn't enough, because a label is written by a person:

Amounts glued to the name. 나이아신아마이드 20%, 소듐하이알루로네이트(1,000ppm), 글라이콜릭애씨드(7%). Strip trailing amounts and parentheses, but try the raw name first, because some real names end in a parenthesis.
Footnote markers. 토코페롤*PDRN: 0.015% is tocopherol plus a footnote about something else. Everything after the * goes.
Hyphens come and go. The dictionary says 1,2-헥산다이올, labels say 1,2헥산다이올. I keep a second index with hyphens removed.
Latin letters change case. T-부틸 on the label, t-부틸 in the dictionary.
Spaces and stray quotes from copy-pasted spreadsheets.
The English side is messy too. The source gives Water,Aqua and Titanium Dioxide,CI 77891 — alternative names joined by a comma with no space — while 1,2-Hexanediol has a comma that belongs to the name. Splitting on "comma followed by a letter" keeps the first and leaves the second alone.

Result: 96.7% of entries, two thirds of products completely

I tested on 966 popular products from 12 category searches (toner, cream, sunscreen, cleanser, sheet mask, cushion, lip, shampoo, body lotion, eye cream, essence, lotion): 49,843 ingredient entries in total.

96.7% of entries got an English name.
66% of products (641) were translated completely, every ingredient.
The median product is 100% matched; the worst tenth sits around 92%.

What's left unmatched is mostly not a dictionary problem: label typos (폴리그리세릴 for 폴리글리세릴), marketing names that aren't standard names ("plant-derived squalane"), colorant lakes, and leftovers from shade-by-shade lists in makeup. Those come back as null in that position, with the Korean original untouched next to it.

What you can see once it's in English

With English names, the same 966 products become comparable with everything else you know. A few things that jumped out:

Korean sunscreens run on filters the US doesn't have. 68% of the sunscreen results contain Diethylamino Hydroxybenzoyl Hexyl Benzoate and 67% contain Ethylhexyl Triazone — neither is on the FDA's list of permitted sunscreen actives. Titanium Dioxide appears in 46%, Zinc Oxide in 29%.
Cushion foundations are skincare claims in disguise. 90% contain Niacinamide and 86% contain Adenosine, the two ingredients that let a Korean product be labelled brightening and anti-wrinkle.
More than half the shampoos contain Salicylic Acid (55%, versus 4% across the other categories) — scalp care is the selling point.
Fragrance splits the store in two. 82% of cushions, 81% of shampoos and 65% of body lotions list fragrance. Toners: 13%. Essences: 9%.
PDRN is spreading from serums into sheet masks: 28% of the sheet-mask results list Sodium DNA.

And the boring backbone of all of it: Water (94%), Glycerin (84%), Butylene Glycol (81%), 1,2-Hexanediol (81%), Ethylhexylglycerin (68%).

Using it

It's part of the Olive Young Scraper on Apify. Same input as before:

{
    "mode": "search",
    "keywords": ["선크림"],
    "includeDetails": true,
    "maxItems": 100
}

Each product now carries both lists, in the same order:

{
    "ingredientList": ["정제수", "다이부틸아디페이트", "프로판다이올", "…"],
    "ingredientListEn": ["Water", "Dibutyl Adipate", "Propanediol", "…"],
    "ingredientCount": 64,
    "ingredientsEnMatched": 63
}

So the counting script from the last post works unchanged — swap ingredientList for ingredientListEn and the output is readable without a Korean keyboard.

Caveats

The category numbers are for search results on a keyword, not a curated category list; a few off-category products are in there.
The English name is the first name the ministry lists for that ingredient. Where the EU and US names differ (Aqua vs. Water) you get one of them.
96.7% is measured on these 966 products. Makeup with long shade lists matches worse than skincare.

If you work with Korean ingredient data and something maps wrong, tell me — the dictionary is easy to patch.

DEV Community