Exploratory Testing is a black box testing technique which lays out the freedom to testers to think out of the box with a minimal dependency on pre-designed answers and step on a process of investigation and discovery to improve the quality of application under test. In layman terms, Exploratory testing is said to be a detailed series of testing by exploring the platform, designing the test cases, executing those immediately and maintaining the results. The thumb rule of exploratory testing is : minimum planning and maximum execution. Exploratory testing is a crucial aspect of software testing as this is not a script based test, it involves the art of being creative and productive. Nowadays, this type of testing is preferable as this doesn’t define a fixed set of methodologies which a tester has to follow, the tester is completely free to fly on his own path to play with an application and identify the potential bugs. This is actually a simultaneous process that carries out the activities of test design and test execution without formally documenting the test cases and test conditions.
Like every other testing technique, exploratory testing also has some pros and cons, let’s have a quick look at those.
Pros and Cons of Exploratory Testing
Pros:
As this type of testing doesn’t require any extensive test planning, the testing can even be done when requirement documents are not easily available.
It is more efficient in short term projects where minimal planning is required and important bugs are found quickly.
This testing technique uncovers the maximum defects when compared with usual regression testing as the regression testing is only performed according to a fixed set of test cases.
This type of testing can also be done by the management team and stakeholders since this doesn’t require any kind of scripting.
In an agile environment where continuous deployments are done, exploratory testing is a perfect testing style that can share the insights in a limited timeframe.
Cons:
As exploratory testing is a crucial part of testing, it is more dependent on the tester’s prior skillset, knowledge and experience.
As no formal documentation is made before performing exploratory testing, this might create a negative impact in long-term projects.
It is difficult to review the test cases later which creates a possibility of missing the critical bugs.
Techniques of Exploratory Testing
Strategy Based Exploratory Testing
This type of exploratory testing is mostly done by testers that are a bit familiar with an application’s functionalities that need testing. The familiarity is important to define the testing strategies. The strategy to test is usually developed using various analysis such as Boundary Value analysis, equivalence technique and risk-based technique to identify more potential bugs.
Freestyle Exploratory Testing
As the name suggests, this technique provides a free style to the testers to investigate bugs or defects without any detailed planning. This approach is basically considered as the combination of smoke testing and monkey testing. This technique is usually carried out by a tester when there is a need to get familiarized with an application and to perform parallel continuous testing without defining any ground rules for test coverage.
Scenario Based Exploratory Testing
This type of testing is usually carried out after the initial phase of testing to make a variation in testing flow as per the application learning and observations from the initial testing process. The idea is to develop end to end scenarios according to the real user interaction. Testers tend to prosper different sets of possibilities by exploring the platform to match each developed scenario and provide maximum test coverage.
When and How to use Exploratory Testing in testing workflow
Exploratory Testing is said to be done in early phases of Software Development Life Cycle(SDLC). In the agile development environment where the sprints are short and software builds are released very frequently, exploratory testing plays a vital role in discovering and reporting bugs and getting them resolved within the same sprint cycle. The experience gained by testers from exploratory testing can be valuable in later stages of testing and to design in-depth test cases. Once the test cases are designed from such investigation and observation, they can further be planned to be automated to get added in the regression suite.
Till now we are aware that exploratory testing is all about exploring. Exploratory testing is not only done to understand the product functionalities but also to understand the customer requirements. For quality benchmarking, it is important to navigate the software according to the end user perspective. Though there is no defined process for exploratory testing, the below depiction will define a structure of exploratory testing commonly adopted by testers.
Real Time Example of Exploratory Testing
Example #1
What we see is what we believe, right? You might still be wondering how exploratory testing looks like. Well, it looks like wandering out in an app and choosing your own path to fly without depending on someone’s direction. That means, there are no pre-designed test cases and testers have some sort of quick description which defines what exactly needs to be tested.
Let’s take an example of a food delivery app and list few testing modules that strikes in our mind:
Login
Search and filter
Outlets nearby
Restaurant selection
Adding food items to cart
Modifying cart
Promo codes
Payment Gateway
Delivery tracking
In case of product exploratory testing, it is a good practise to start the testing from the initial module like login page and then move on to the next relevant module. Following this testing workflow would also cover the testing according to the end user perspective. Always remember, speed and accuracy are the most important factors in such applications.
Since the food delivery app would be having multiple restaurants and dishes, it is important for an exploratory tester to test with different test data. The login page and payment gateway is a more concerned aspect in terms of security and is important to be tested with multiple test scenarios. However, being creative and analytical is something that exploratory testing requires the most.
Example #2
Let’s take another example of exploratory testing using pCloudy Certifaya AI-Bot.
The new trend of intelligent automation has proposed the new way of exploratory testing. The concept of AI Testbot has been introduced to reduce the burden on the QA team.
In exploratory testing, AI Testbot investigates and reports bugs in case of functionality crashes or unexpected errors pop-ups. For each test iteration, it has a power to store execution logs, video recordings and screenshots for future references.
pCloudy’s Certifaya features a testbot that allows a tester to upload an application that needs to be tested and sit back tight until the test report is handed over. The key feature of this testbot is that it not only performs exploratory testing but also the cross device testing on different device variants. Once an application is uploaded and a session is triggered, the smart bot crawls inside the application without any human intervention and digs for each random corner case with a motive to uncover maximum bugs. As soon as the Certifaya session gets completed, an insightful report gets mailed to the team to document the test results.
Certifaya Report
The comprehensive report includes scenarios performed by the testbot along with logs and snapshots. The functionality performance score is also wrapped up in the report in the form of charts and graphs like battery chart, memory chart, CPU chart in addition to each frame rendering time.
In agile methodology where software is released in multiple small iterations and developers have to wait for rapid feedback from testers, the testbot plays a vital role in performing quick exploratory testing for new software versions and smoke testing for the entire build deployed.
Myths about Exploratory Testing
Let us discuss about some misconception related to exploratory testing that mislead a tester in his way to become an expert:
Myth: Exploratory testing doesn’t require planning and documentation
Many people think that exploratory testing can be done without even minimal planning and without any sort of documentation.
Reality: This is a big myth that the testers need to incinerate before starting off with exploratory testing. It is a structured approach of testing, the way of structuring completely depends on a tester which definitely needs some kind of planning to make a move. Since the probability of change in requirements is so high in agile development, without documenting the testing plans and statuses, exploratory testing cannot be possible.
Myth: Exploratory testing is similar to Ad-hoc testing
Though some of the factors of exploratory testing match with ad-hoc testing, that doesn’t mean that these two approaches are completely similar.
Reality: Ad-hoc testing is a very informal and random approach of testing that doesn’t specify any strategy towards testing. In ad-hoc testing, the agenda is to first learn the background processes of an application and then perform the testing whereas in exploratory testing, the agenda is to understand an application while exploring and testing.
Myth: Exploratory testing is not applicable on complex systems
Sometimes the complexity of the system enforces testers to think that exploratory testing cannot be applied to a complex system which requires in-depth planning and training.
Reality: Exploratory testing gives testers a freedom to choose their own way to test, this helps testers to understand these complex systems more while crawling in-depth. The more complex an application is, the more test cases get designed and implemented which lead to a better coverage of the system.
Myth: Either exploratory testing or scripted testing can be done
Due to the short span of time in sprint, it is obvious for such myths to come around, however, the myths should be clear before it is too late.
Reality: 100% coverage with scripted testing just cannot be possible. Though the implementation of scripted testing requires a lot of prior planning and designing of test scenarios and test cases, exploratory testing cannot be replaced with scripted testing. In exploratory testing, the tester gets a chance to carry out additional tests that were never pre-defined. In the process of exploratory testing, the critical test cases can be dug out to make sure they can be scripted later on for better test coverage.
Achieving success in exploratory testing with test automation
Many people have a question in mind, can exploratory testing be automated?
We can achieve success in testing with a combination of exploratory testing and test automation, however, these two approaches cannot replace each other.
After being aware of the exploratory testing approach, do we still think that sense of creativity be automated? Can human curiosity be automated? Or random inventions automated? I don’t think so.
Though these kinds of activities can be partially done by a testbot but it is actually not possible via test automation. Common things that we can automate in flow of exploratory testing is to generate random test data, functional test execution, output logging, developing reports and sharing insights with the development team.
In an agile development model, the process structure is defined in such a way that the testing can be done parallely with development. Once the new build is deployed on a testing environment, testers get a go ahead from dev to start off the testing.
To start exploratory testing, initial minimal plans are developed and testing is continued accordingly. Each edge case, critical case, flaky test case and other anonymous doubtable test cases are recorded separately so that they can be retested.
It is very complex and time consuming for the QA team to retest such test cases everytime a new build is deployed in a testing environment. Hence, once the exploratory testing of a particular module comes to an end, a new plan is developed to automate such test cases which can have direct impact on the performance of the production environment. In this way, we can achieve success in exploratory testing with test automation which further leads to better test coverage by covering all corner cases and uncovering all major defects and serving continuous improvement in quality.
Wrapping Up
Without any doubt, exploratory testing brings out the creativity of a tester and has an extreme effect in improving the product quality. Investing too much time in exploring and discovering edge case scenarios might affect the regular testing, getting a AI bot like Certifaya to do some of the testing is definitely helpful. Also, It is important to maintain a healthy balance between the two. The kind of reporting done in exploratory testing matters a lot. The tester may find out only one bug in exploratory testing but the way the reporting is done can change the decision made by the management to enhance the growth of the product.
Top comments (0)