The new detector had a good brochure.
It caught ninety five percent
of compromised logins.
It wrongly flagged
only two percent of normal ones.
At eleven at night
it flagged the finance director.
You were sure.
Ninety five percent.
You locked her account,
killed her sessions,
and rang her at home.
She was in a hotel in Lisbon,
on the hotel Wi-Fi,
doing exactly what she had told everyone
she would be doing.
It happened twice more that week.
Here is the mechanism.
The question you answered
was how good is the detector.
The question in front of you
was how likely is it
that this login is bad.
Those are different questions,
and the gap between them
has a name.
Base rate neglect.
Psychologists showed it decades ago
with a simple puzzle
about a witness and a taxi,
and almost everybody
gets it wrong.
Do your own numbers.
Say one login in a thousand
is really compromised.
In a hundred thousand logins,
that is a hundred bad ones.
The detector catches ninety five.
It also looks at
ninety nine thousand nine hundred
normal logins,
and two percent of those
is nearly two thousand alerts.
So about two thousand and ninety alerts,
and ninety five of them real.
Fewer than one in twenty.
The detector was exactly as good
as it claimed.
Your alert was still
almost certainly a hotel.
The accuracy of a test
tells you very little
until you know
how rare the thing is
that you are testing for.
The rarer it is,
the more of your positives
are false,
however good the test sounds.
So before you act on a positive,
ask the boring question first.
How often does this
actually happen here.
Then treat the alert
as a reason to look,
not a reason to lock.
Check the device.
Check the calendar.
Send a message
before you send a lockout.
And when someone sells you
a percentage,
ask for the other number,
the one about the people
who were fine.
A good test
on a rare event
is mostly a very confident way
of disturbing the innocent.
– Serguey Asael Shinder
Top comments (0)