“Complete” testing is impossible, so we have to do the best we can with the time and resources we have. Often, that comes down to making sure that there’s a good reason for each test to exist. If there isn’t, then you should remove it and spend that time on something more valuable. When evaluating test suites, I will ask, “why do you have this test?” or “why are you testing this?” To me, a good reason is that the test addresses some specific concern, risk, or functionality that no other test does. But, I’ve heard plenty of bad reasons, too…
To cover all the scenarios
Ok, but which scenario does this test cover? Do you have a list of “all” the scenarios? Is each one unique? What makes them unique? Who gave you that list? Is it ever possible to cover them all if you did have such a list?
To get more confidence
What are you more confident about after running this one test that you weren’t confident about before? Could you run another test to give you that same confidence? Why not? What are you worried about happening if we didn’t run this test?
To see that it works
To see that what works? It can’t be the product or application, because that’s what all the tests do. What could be broken if we didn’t run this test?
Because the Product Owner told us to
What does the Product Owner know that you don’t? Do they have a reason? You should probably know what it is too. Maybe there’s a more effective way to test that aspect, but you have to know what aspect they’re interested in first. If they don’t have a reason, it’s probably part of your job to steer them towards more useful tests and, perhaps, illustrating why this one isn’t necessary.
Why I care
If you don’t know what a test tests, how are you going to interpret it when it fails? Tests are only valuable if they tell us something useful when they fail. Each test costs us in time to design, write, run, analyze, and maintain. The only way to know that we’re making the most of the time and resources we have is to be sure that each test adds specific value. Without that, you’ll never really know which ones are worth including and which should be excluded.
Top comments (0)