Our word counter had a bug that no unit test caught: it said the family emoji 👨👩👧👦 is 11 characters. Users see one. If you build anything with a character limit (captions, bios, SMS, form fields), you probably have the same bug, because the default way to count a string in JavaScript counts something users never see.
The test
We ran seven strings through four ways of counting:
| Text |
.length (UTF-16) |
[...s].length (code points) |
Graphemes (what users see) | UTF-8 bytes |
|---|---|---|---|---|
Free tools for photos and PDF |
29 | 29 | 29 | 29 |
Photo ka size kam kaise karein |
30 | 30 | 30 | 30 |
تصویر کا سائز کم کیسے کریں (Urdu) |
26 | 26 | 26 | 47 |
Eid Mubarak 🌙✨🎉 |
17 | 15 | 15 | 23 |
Great job 👍🏽👍🏽 |
18 | 14 | 12 | 26 |
Family time 👨👩👧👦 |
23 | 19 | 13 | 37 |
Pakistan 🇵🇰 zindabad |
22 | 20 | 19 | 26 |
Plain English and Roman Urdu are identical whichever way you count. Everything else drifts.
What is going on
-
.lengthcounts UTF-16 code units. Emoji outside the Basic Multilingual Plane take two units each (a surrogate pair), so 🌙 is 2. -
[...s]counts code points. Better, but many visible characters are built from several code points:- 👍🏽 = 👍 + a skin-tone modifier (2 code points)
- 🇵🇰 = two "regional indicator" letters, P and K
- 👨👩👧👦 = four people joined by three invisible zero-width joiners: 7 code points, 11 UTF-16 units
- Graphemes are what a person would call "one character". Unicode calls them extended grapheme clusters.
- Bytes matter for storage and some limits. Urdu letters take 2 bytes each in UTF-8, so 26 characters became 47 bytes.
The fix: Intl.Segmenter
Every current browser and Node.js has a grapheme segmenter built in:
const seg = 'Segmenter' in Intl ? new Intl.Segmenter() : null;
const chars = s => seg ? [...seg.segment(s)].length : [...s].length;
chars('Family time 👨👩👧👦'); // 13
chars('Great job 👍🏽👍🏽'); // 12
The fallback to code points keeps very old browsers usable without bringing in a library.
Two details from our version:
-
Characters without spaces: run the same function on
s.replace(/\s/g, ''), so emoji are still counted once. -
Words: count with a Unicode-aware regex such as
/[\p{L}\p{N}][\p{L}\p{M}\p{N}]*/gu. The\p{M}part keeps Urdu and Hindi combining marks inside the word instead of splitting it.
Which number should you show?
- Users typing against a limit you control: graphemes. That is what they can see and edit.
- A third-party limit: match how that platform counts. For example, X weights most emoji as 2, and SMS switches from 160 to 70 characters per message as soon as one non-GSM character (such as Urdu) appears.
-
Database columns and byte limits (Amazon's 249-byte search terms, some APIs): UTF-8 bytes via
new TextEncoder().encode(s).length.
Try it
The fixed counter is free and runs entirely in the browser: word counter with characters, reading and speaking time. There is also a plain character counter, and for byte-based limits the Amazon character counter shows both characters and bytes.
Top comments (0)