DEV Community

Jesse Phillips
Jesse Phillips

Posted on

Repeatable Test Step Fallacy

If you're a test professional you're often more then just a tester. You're given a pile of functionality and expected to verify it functions, detail out how it functions and do it again with the next piece of functionality along with doings it for the new functionality.

Recently I was asked to produce repeatable test steps as evidence of testing. We are in a position where we could have third party audits and this person has placed themselves as the final gate to say the product is ready to be released. (this is not intended to jab at the benefits of this person's skills or contributions, only to clarify the request is reasonable) this person has no involvement with the product, including defining or knowing what functionality exists.

Rapid Software Testing describes a targeted testing session where the tester will take notes for themselves, the tester can review their sessions with a senior tester to help find gaps and exchange knowledge. These notes are intended as personal, used by their creator, not as repeatable test steps.

If you're in the agile space you're probably thinking, there are no testers, and we have detailed behavior through acceptance criteria. But acceptance criteria aren't the test steps.

When a bug is identified, working on a minimal reproduction step list is critical to having timely resolution and even insight into severity. This is not where this fallacy lies.

The idea that by defining repeatable test steps on new functionality you will have:

  • Full coverage to catch defects
  • Time to execute everything for each release
  • Have more than one release a year
  • documentation to hand off for another tester to pick up
  • An understanding of what was tested

I don't think you get this, especially if you want more than one. Leaving bread crumbs about the testing is useful. There is also a time to cover the test effort in more detail. I recommend focus first

  • How are found issues handled
  • What issues might crop up that are similar or are hiding
  • Understand the issue (root cause, not accusations)
  • determine your risk tolerance
  • can you reproduce closer to time of development
  • could the issue be found in a different manner

Top comments (0)