Your String Length Is a Lie: Four Unicode Traps in Everyday Code

Every character counter you have ever written is wrong for at least one of your users. Here are the four Unicode traps I keep finding in code reviews — each one verified in Node, not recalled from a blog post.

1. .length counts UTF-16 code units, not characters

"\u{1F44D}".length                    // 2  (one thumbs up)
"\u{1F1E8}\u{1F1F3}".length           // 4  (one flag)
"\u{1F468}\u200D\u{1F469}\u200D\u{1F467}\u200D\u{1F466}".length  // 11 (one family)

Code points are closer: [...str].length gives 7 for that family. But only grapheme clusters give the real answer:

[...new Intl.Segmenter("en", {granularity: "grapheme"}).segment(str)].length  // 1

The dangerous part is slicing. str.substring(0, 1) on a thumbs up returns a lone high surrogate. Encode that and you get 3 bytes of replacement character — no exception, no warning, just corrupted data in your database.

2. "é" is sometimes two characters

const nfc = "é".normalize("NFC");   // 1 unit, 2 bytes
const nfd = "é".normalize("NFD");   // 2 units, 3 bytes
nfc === nfd;                        // false

Identical pixels, different strings. new Map([[nfd, 1]]).has(nfc) is false, so dedupe, rate-limit keys, email lookups and "already exists" checks quietly miss. macOS filenames arrive decomposed, so this is not a theoretical case. Normalize to NFC at the boundary, before hashing or comparing.

3. Invisible characters survive trim()

" \u200B ".trim().length is 1 — a zero-width space is not whitespace in JavaScript, and /\s/.test("\u200B") is false. A non-breaking space, by contrast, is stripped. So text pasted from a browser can look perfectly clean and still not match.

Markdown is where it hurts most. I tested rendered output in a GFM previewer: a "blank" line containing only U+00A0 does not split paragraphs, and a line where the space before # is U+00A0 never becomes a heading — it renders as <p> # Title</p>.

4. Bytes are a third number

The same family emoji is 11 UTF-16 units, 7 code points, 1 grapheme and 25 UTF-8 bytes. Column lengths, index key limits (utf8mb4 reserves 4 bytes per character), and base64 payloads all run on bytes, so a 768-character field is not a 768-byte field.

Rule of thumb: units for strings, code points for iteration, graphemes for UI, bytes for storage.

When I need to find which of these is biting me, I dump the string into a character-level inspector — I use Unicode Inspector, which runs entirely in the browser and shows the code point, UTF-16 units, UTF-8 bytes and entity for every character, plus filters for invisible ones. Paste a failing value in, find the U+200B you could not see, done.