Our customers pay us by bank transfer, about nine thousand payments a month, and each one has to be matched to the invoices it pays. Then we bought a cash matching tool. It reads the bank statement each morning, looks for invoice numbers in the payment reference and allocates the money. The supplier's contract and our business case were both measured on one figure, the automatic match rate.
At first the rate was about seventy percent, because references are messy. Customers type their own order numbers, abbreviate ours or write nothing at all. Over the following months the rules were tuned, with the supplier's help, to push the figure up. The most effective rule was a fallback. If no reference matched, take the oldest open invoice for that customer with exactly the same amount. The match rate reached ninety two percent and was reported to the board.
Many of our customers buy the same things every week at the same price. A chain of cafés paying this week's invoice of one thousand two hundred and forty pounds was matched to an identical invoice from March that it had disputed and was withholding. This week's invoice then showed as unpaid. Credit control chased it, the dispute looked settled when it was not, and three accounts went on credit hold while being, in fact, up to date. One of them was our fourth largest customer, whose orders were stopped for a morning.
When we audited a sample of fallback matches, about one in five had gone to the wrong invoice. Every one of them had counted as a success.
The rule had done exactly what the measure asked. A confident wrong match raised the rate and an honest unmatched payment lowered it, so the tuning had rewarded guessing.
The fallback rule is gone. Any match that does not rest on a reference goes to a person, with the candidate invoices shown side by side. We report three numbers instead of one: matched automatically, matched by a person, and matched wrongly, found by sampling fifty allocations a week. The automatic rate is now seventy nine percent, and the wrong rate is below one percent. Credit control has its mornings back.
When automation is measured on how much it decides, it learns to decide things it should have left alone.
– Serguey Shinder
Top comments (0)