Last year we replaced the system our depots use to book inbound deliveries from suppliers. The project plan gave the business four weeks of acceptance testing, and each depot was asked to provide two people for those four weeks.
The people who arrived were not the people we had pictured. Of the twenty two testers, nine were agency staff in their first months with us, five were on light duties after injuries, and several more had been released because their own team was quiet that month. They worked carefully through every script we gave them. All three hundred and forty test cases passed, and the depot managers signed off.
The system went live in September and the goods in offices struggled for six weeks. A delivery of mixed pallets could not be split between purchase orders. A haulier bringing three suppliers' goods on one trailer could only be booked against one of them. The rule that lets a regular supplier keep its standing Tuesday slot did not exist. None of this was in the test scripts, because the scripts had been written from the requirements, and the requirements had missed it in the same way. The people who would have found it in an afternoon, goods in supervisors with ten years of exceptions in their heads, had been too busy to release, which is exactly why they were the people we needed.
I do not blame the depot managers. Asked for two people for a month with no cover, they protected their operation, which is what they are paid to do. The cost of releasing their best people would have fallen on them. The cost of not releasing them fell on the project, six weeks later.
Business testing is now costed in the project budget as backfill, so a depot that releases its most experienced supervisor gets agency cover paid for by the project. The sponsor names the testers at the start, by role and experience, and the project cannot enter testing until they are confirmed. At least half of the testing time is unscripted, with testers working through the previous month's real bookings, exceptions included. And sign off comes from the people who did the testing, not only from their manager.
The next system we replaced went live with two defects of any consequence.
Whoever tests a new system sets the standard it goes live at. We had left that choice to whoever happened to be free.
– Serguey Shinder
Top comments (0)