Most software evaluations fail in the same place: the demo is a performance, and the team grades the performance. The slides are clean, the dashboard in the demo account is full of recoveries, and the questions stay at the level of features — do you retry, do you email, do you integrate. Six months later the surprises arrive, and they are never about features. They are about ownership, counting, and what happens on the bad weeks. A rubric fixes this, because a rubric forces the awkward questions into the room while the vendor is still there to answer them.
Start with classification. Ask how the tool sorts a failed payment before it acts. A serious answer names families — credentials that changed, timing that will pass, verdicts that will not — and explains what each family gets. A vague answer talks about smart retries in the abstract. The distinction matters more than any single feature, because a retry sent to a replaced card and a retry sent to an empty account are opposite acts wearing the same name, and software that cannot tell them apart will spend your customers' patience at random.
Next, score the counting. Who decides that a recovery counts, and by what rule? Does a payment that succeeds on the platform's own native retry, three days after the tool touched the case, land in the tool's column? Does a pause, a skip, a plan change close the case as a save? These are not gotcha questions; they are the difference between a recovery rate you can plan around and one that exists to be renewed against. Write the vendor's answers down verbatim. If the counting rule cannot survive being written down, that is the evaluation.
Then the handover questions, which demos rarely volunteer. What happens to a case mid-sequence if you cancel? Can you export the full failure history, the message templates, the case log — in a file, not a screenshot? What does setup actually require from your team, in hours, named step by step? A tool that is easy to leave is usually easy to trust, and a tool that makes leaving hard has told you something about where its confidence lives.
Price belongs on the rubric too, scored as arithmetic rather than as a number. A percentage-of-recovery fee feels aligned until you model it against your best month: the better the tool works, the more it costs, and the invoice scales with exactly the outcome you bought it for. A flat fee reads boring and stays boring, which for infrastructure is a compliment. Score both models against the same recovered-revenue figure — your figure, from your last quarter — and let the two totals sit next to each other on the page.
Working from a thirty-check scorecard for scoring dunning software before a demo keeps the meeting honest in a specific way: every check is a behaviour you can observe or a sentence the vendor must say out loud, not an impression you carry out of the room. When we ran our own buying decision this way, two well-presented tools lost on counting and export alone — categories that never appear in a feature grid — and the tool that won did so on answers, not polish.
The last check is the quietest: run the rubric again ninety days after signing, against the live system, and compare scores. Software drifts, settings drift, and the team that re-scores on a schedule catches the drift while it is still a settings change rather than a renewal argument. An evaluation is not a gate you pass once. It is a habit — the same habit, pointed at a vendor, that good recovery work points at a billing queue: sort carefully, count honestly, and write it down.
Published by Swipe Again, the flat-fee failed payment recovery service for subscription businesses.
Top comments (0)