I wanted a complete list of kaomoji, the ( ͡° ͜ʖ ͡°) and ¯\_(ツ)_/¯ family. There isn't one.
So I merged the public collections instead, and the merge turned up something I
wasn't expecting: they hardly overlap at all.
Seven collections, 21 pairs, 0.3% median overlap
The ones I found:
- fdw/rofimoji, the biggest English-side set
- Allaman/emoji.nvim
- aoguai/rime_kaomoji_dict
- mtripg6666tdr/Kaomoji_proj
- the Japanese IME dictionaries
- kaosute
- an older list of my own
Across the 21 pairwise combinations the median Jaccard overlap is 0.3%. Four
pairs share not a single entry. One pair clears 5%, rofimoji and emoji.nvim,
which are both English-side Linux pickers, so that one is no surprise.
The IME dictionaries are where most kaomoji actually live, and they share almost
nothing with any of the English sets.
Each entry in the merged file keeps the sources it came from, so you can check
this yourself:
import gzip, json, itertools, collections, statistics
by = collections.defaultdict(set)
for line in gzip.open("data/kaomoji.jsonl.gz", "rt", encoding="utf-8"):
r = json.loads(line)
for src in r["sources"]:
by[src].add(r["text"])
o = [len(by[a] & by[b]) / len(by[a] | by[b]) for a, b in itertools.combinations(by, 2)]
print(statistics.median(o), max(o), sum(1 for x in o if x == 0))
# 0.003 0.079 4
None of the collections is close to complete, and after seeing those numbers I
understand why.
Coverage checked at run time
Saying "this has everything" is not worth much on its own, so
scripts/verify_coverage.py downloads each upstream file when you run it and
diffs it against the merged set. Nothing is precomputed.
| Upstream | Entries | Covered | Missing |
|---|---|---|---|
| rofimoji | 1,562 | 100.0% | 0 |
| emoji.nvim | 2,016 | 99.9% | 3 |
| rime_kaomoji_dict | 959 | 99.4% | 6 |
| Kaomoji_proj | 2,166 | 97.8% | 48 |
It prints all 57 misses, so you can look at them and disagree with me about what
counts as a kaomoji. Three rounds of that took rofimoji from 93.0% to 100.0%.
Labels, and where they fall down
Deduplicated it comes to 82,109 entries. Around 70k carry emotion, intent and
subject labels from a controlled vocabulary in en, ja, zh, es, pt and de, so
嬉しい, 开心, feliz and happy return the same entries instead of whichever English
word the original author happened to use.
Three things worth knowing before you use it:
- The labels are coarse. 69,679 labelled entries share only 605 distinct label combinations, because they were applied in bundles rather than judged one at a time. Fine for pulling up a mood, not for finding one specific face.
- About 15% is left unlabelled rather than guessed at.
- Most of the corpus is 顔文字+セリフ, a face plus a line of Japanese dialogue, which is not much use if you cannot read it.
That last one is why the English-language picker at
fontvibe.ai/tools/kaomoji shows 11,020, the
subset that renders safely outside Japanese contexts, while the full 82,109 sits
in the dataset and on the Japanese page.
Getting it
npm i kaomoji-dataset
pip install kaomoji-dataset
Both read a gzip off disk, so they are Node and Python only for now. They will
not bundle for a browser. The repo has the raw JSONL and a CSV as well:
github.com/Funovate/fontvibe-kaomoji
106 of the entries are not collected from anywhere. I drew them, for feelings
that had no kaomoji yet: 躺平, 社死, saudade, mamihlapinatapai. Those are CC0.
If you know of a collection I missed, say so and I will run the coverage script
against it.
Top comments (0)