My CTO: “We’re really impressed with your testing solution, you’ve gone above and beyond!”
(Immediate feeling of guilt washes over me)
This happened a couple months ago after my CTO heard about a testing solution I put together for our web applications. My solution was to auto-generate the QA Selenium code instead of manually coding it by recording/configuring the tests. (Shameless plug: it’s available for free over at snaptest.io . This tool has dramatically increased coverage - but why did I still feel guilty about accepting this compliment?
It’s because I unreasonably assume testing should be exhaustive.
The guilt comes when I say things like: “I’ve tested the new feature”, or “the tests are passing”. In the back of my mind, I know there are hundreds of thousands of combinations I didn’t test for.
We must give ourselves a break and remember that exhaustive testing is usually not possible. One must reconcile the cost/reward balance sheet of testing… Especially when you start thinking about negative tests.
Negative tests/Destructive tests/error path testing — refers to checking how gracefully a system handles errors or unexpected situations. This is when exhaustive testing gets exhausting… Don’t expect to get them all.
Here are my recommendations for handling negative test cases: