Any system that blocks people has two failure modes, and only one of them is loud.
Fraud that gets through arrives as a chargeback, a support ticket, a cleanup job. It has a cost you can put in a spreadsheet and a person whose week it ruined. It gets reviewed.
A real customer wrongly refused closes the tab. They do not file a bug. They do not email support. Nothing in your dashboard records that they existed, and the metric that would have caught it is the absence of an event.
The structural consequence is what bothers me: if the only feedback reaching you is the loud kind, then every review concludes the rules are too loose, and they tighten. Forever. Not because anyone decided to be aggressive, but because one side of the ledger is invisible and the other is not.
Things I have seen actually work against it:
- Count challenges issued against challenges passed. People passing your step-up at a high rate means the band is doing its job; people abandoning it means it is set too wide.
- Run in log-only mode for a period and look at the score distribution before acting on it, so the cutoff comes from your traffic rather than someone else's.
- Sample blocked traffic and actually look at it, on a schedule, as a job someone owns.
None of these are clever. They just require someone to go looking for the thing that does not announce itself.
What do you use? I am genuinely collecting approaches here.
Top comments (0)