The supplier sent a price file every night.
One row per product,
one column per day.
Your script read it,
worked out the average price
and wrote the result as JSON
for the dashboard.
For two years
it was boring.
Then one morning
the dashboard was blank.
No error on the server.
The job had finished.
The file was there.
In the browser console,
one line.
Unexpected token N.
Somewhere in the night
the supplier had a gap,
and instead of leaving the cell empty
their export wrote three letters.
NaN.
Python's float
reads that happily.
It reads inf and infinity too,
in any case,
with spaces round them.
So the gap became a number
that is not a number,
and anything added to it
became one too.
The average for that product
was NaN.
Then json.dumps
wrote it out as NaN,
with no quotes.
That is not JSON.
The standard has no way
to write it.
Python writes it anyway,
by default,
and Python reads it back without complaint,
so your own tests,
which loaded the output
with Python,
passed.
The browser follows the standard.
JSON.parse saw the N
and refused the whole document.
Not one product.
All of them.
One missing price
took down every number on the page.
And the check you might have written
would not have caught it.
value == float of nan
is always false,
because NaN is not equal
to anything,
itself included.
So be strict at both ends.
Pass allow_nan=False
to json.dumps,
so a bad value fails loudly
in the job,
where you can see it,
instead of quietly
in a user's browser.
Use math.isnan
when you check for it,
never ==.
Decide at parse time
what a gap means.
Missing is not zero,
and it is not a number.
Make it None
and let it become null.
And test the output
with the parser
your reader actually uses.
A file your own code accepts
is not the same thing
as a file
everyone else can read.
– Serguey Asael Shinder
Top comments (1)
Nice, you can also define a function that raises and pass it as
json.loads(payload, parse_constant=reject), so your Python tests catchNaNandInfinityjust like the browser does.