The supplier sends a stock file
every night.
Four thousand rows,
a product code
and a quantity.
You were careful.
Before turning the quantity
into a number,
you checked it.
if qty.isdigit(),
then int(qty).
Anything else
went to a rejects file
for a person to look at.
It ran for a year.
Then one night
the whole import stopped
on row two thousand and six.
ValueError.
Invalid literal for int.
The cell said twelve,
with a small raised two after it.
A footnote.
Someone at the supplier
had marked the line,
and the note at the bottom
said back order.
isdigit looked at that two
and said yes.
int looked at the same two
and said no.
Both were doing their job.
isdigit asks Python's question,
is every character
something Unicode calls a digit.
The superscript two is one.
So are the circled numbers
and a long list of others.
int asks a narrower question,
is this a decimal number
it can actually read.
The superscript two is not.
The documentation says it,
for anyone who reads it.
Digits include
the compatibility superscript digits.
And the gap runs the other way too.
Arabic Indic digits,
from a branch in Cairo,
pass isdigit,
and int reads them happily
as the number you would expect.
Full width digits,
typed on a Japanese keyboard,
the same.
Even regular expressions
join in.
In Python 3,
backslash d
matches every decimal digit
in Unicode,
not just nought to nine.
So your check
and your parser
were never checking
the same thing.
You had two definitions
of a number,
and only one of them
was going to be used.
The fix is to stop
asking a different function
for permission.
Call int,
catch the ValueError,
and send that row
to the rejects file.
The parser
is the only honest judge
of what the parser accepts.
If you only want
the digits nought to nine,
say so out loud.
isascii and isdecimal together,
or a pattern
of nought to nine,
or the ASCII flag
on your regular expression.
Then add the rows
nobody types on purpose
to your tests.
A superscript.
A circled one.
A digit from another script.
And make sure
one strange cell
sends one row to a person
instead of stopping
the other three thousand
nine hundred and ninety nine.
A check that asks
a different question
from the code it protects
is not a check.
It is a second opinion
nobody asked for.
– Serguey Asael Shinder
Top comments (0)