What is Exploratory Testing? An Alternative to Scripted Testing and Try To Break It Testing

Exploratory testing is a testing approach and mindset. In this post I compare exploratory testing with two different testing approaches:

1. Scripted testing
2. Try to break it testing

Scripted testing

Scripted testing provides a series of checking points for a specific part of a feature. For example, a script might tell you to select the print button and expect print dialogue opens. You might go on to check the default number of pages to print and expect the default is set to "All".

Scripted testing captures a pass or fail for each test:

• Pass: It met the expected result
• Fail: It did not meet the expected result ‍

But what happens when a specific test does something beyond the boundaries of the expected result? Do you pass or fail it? An item under test isn’t so black and white and in the grey lies problems and ideas waiting to be discovered.

Scripted testing takes time to plan and document. Much of this planning and documentation doesn’t provide immediate feedback about the item under test. It’s only during a test execution phase where scripts are put to use and something is discovered.

So why not immediately explore, even without a script? What’s stopping you? Surely you’ll find something interesting even without a script telling you what to do and expect! Go evaluate what’s actually in front of you instead of what a document says it’s supposed to do. You could use try to break it testing.

Try to break it testing

Discard the scripts and use your judgement to explore with the intention of breaking the item under test. Discovery is an alternative way of thinking about breaking. You might discover part of a feature is broken but you didn’t actually break the feature.

With a discovery approach you follow your instincts to learn what happens. You discover a bug with the orientation feature of the print dialogue box, something the script wouldn’t have asked you to check. You share this immediately. Fast feedback and hopefully a fast fix for retesting!

But how do you know you’re testing the most important thing? Where do you choose to start? How do you evaluate how much testing is enough? When do you decide to stop exploring? Scripts give you that sense of organisation and priority. They also give you a set of activities with a stopping point. Try to break it testing – or ad-hoc testing as it's also referred to or incorrectly labelled as exploratory testing – has no explicit prioritisation or focus.

Exploratory testing

So how about using both approaches at the same time? Exploratory testing provides the organised feel of scripted testing along with the rapid feedback of try to break it testing. Identifying what to test next is where exploratory testing comes into its own – by using structure and fast feedback.

Risk areas help focus exploratory testing efforts. Risk areas are things that might be troublesome to the desired outcomes of customers, users and business stakeholders. Risks-to-explore are assumptions and these assumptions might present themselves as actual risks but only if we take the time to run focused exploration to find out. Use risks to guide where and what to explore and you'll provide your team with an ongoing evaluation of test coverage.

So how do you get started? Define a test exploration goal (also referred to as a “charter”) using risks as your trigger – risks are things that threaten the value of the product. Don’t stop at one goal, come up with many so you and your team can decide which goal to explore first. Choose an amount of time you'd like to spend exploring each test exploration goal. Use a timebox to provide focus and a clear stopping point. Any timebox from 30 to 90 minutes works best.

Discover more!

A test exploration goal and timebox is an excellent alternative to following a test script or attempting to break an item under test by playing with it. The balance of structure and freedom is liberating! The output of every goal-driven timeboxed exploratory testing session provides a team with a useful starting point for a conversation. And these conversations help a team answer one of their most important and ongoing questions: What do we need to work on next?

Or better still: What shall we learn next?

Originally posted on the TestBuddy blog.