The Same Name Typed on Two Keyboards

Two customers signed up
a week apart,
both called José.

Same spelling on the screen.
Same font, same accent,
same everything a person can see.

Your database says
they are different names,
and it is right.

One keyboard typed
the é as one character.

The other typed an e
and then a separate mark
that sits on top of it.

They draw identically.

They are not the same bytes,
and equality in almost every language
compares bytes.

So the search for José
finds one of them.
The unique rule on usernames
lets both register.
The check that looks
for repeated customers
sees two strangers.
The sorted list
puts them in different places.

Nobody wrote a bug.

Every line did
exactly what it says.

The trouble is that text
has more than one way
to be written down,
and your code quietly assumed
it had one.

It gets wider.

A name pasted from a PDF
with a non breaking space in it.
A trailing space
from a spreadsheet cell.
A capital letter
from a phone that capitalises
whatever it likes.
A letter from another alphabet
that looks exactly like ours
and has a different number underneath.

Each of these is a person
who cannot find their own account,
or a person who now has
a twin in your system
that nobody meant to create.

The fix is a decision,
made once,
in one place.

Pick a canonical form.
The Unicode standard already names them.

Normalise to it
the moment text crosses
into your system,
at the edge,
before it is stored,
before it is compared,
before it becomes a key.

Then decide separately
what counts as the same
for each purpose.

For display,
keep what the person typed.
It is their name.

For identity,
compare the normalised form,
case folded if your rules say
case does not matter,
with the spaces trimmed,
and write those rules down
next to the function
that applies them.

One function.
Every comparison goes through it.

Then write the test
with the two Josés in it,
typed both ways,
and watch it fail
before you fix anything.

Text that looks equal
is a claim made to your eyes.

Your code needs
its own reason to agree.

– Serguey Asael Shinder