We need more automated testing as software gets more sophisticated. Recently, top QA consultancy companies QASymphony and TechWell surveyed mid-size and large-size enterprises for insights into the QA Automation trends of 2019. They found that most of these organizations(76%) were automating less than half of their test cases in 2018.
However, almost half of these enterprises expect to automate 50% of the test cases by this year. In general, 92% of the organizations, including startups and, large enterprises perform some automation with the help of tools. You can find the full report here.
The benefits are huge and rewarding if automated testing is practised diligently with proper planning. The ROI increases exponentially with time if we follow top automated testing practices.
However, it does not mean that we can depend on automated testing completely and bid goodbye to manual testing. The overall software testing process will always include a mix of manual and automation test cases so that we achieve high quality, the better speed with optimum use of resources.
So, let’s have a look at some top automated testing practices we can follow while automating the test cases for a happy automated testing experience!
Automation is the second step, not the first
We cannot directly start automating new software. The first step is to have a stable and well-designed manual testing process and right test cases in place.
When the environment, development code and manual test processes gain stability, we can start thinking about bringing automated testing in the project. Hence, the test cases which are ideal for automated testing can be selected with the help of manual testers and put in automated testing which frees up manual test engineers time for more human attention specific tasks, such as usability testing, exploratory testing, and ad-hoc testing.
So, how do we identify which cases to be automated? Let’s find out on the next point.
Identify ideal test cases for automated testing
Though 100% of the test cases present in any project can be automated, it does not imply that they should be automated. We need to be very careful while selecting the ideal cases for automation. Otherwise, we may lose time and money on creating scripts for the cases which are rarely run.
Below test cases could be picked up for automation because of their repetitive nature:
a) Regression Tests
We need to ensure that the new code changes have not introduced any new defect in the code. Execution of the regression test cases manually could be time-consuming and a mundane task for the manual test engineer. That is why regression test cases become the best candidates for automated testing. The test engineer can use the time in other more useful tasks that require manual attention e.g. for exploratory testing.
b) Smoke Tests
Whenever a new build is released for testing, smoke testing is performed to ensure that the critical functionalities of the product are checked. Since these tests are run after each build, they are good candidates for automated testing. Mostly smoke tests are integrated with Continuous Integration(CI) tools like Jenkins to automatically run them whenever the development team releases a new build.
c) Data-Driven Tests
Data-driven testing involves running data stored in several file formats like CSV, Excel, DB tables, JSON files through the same code multiple times.
(i) Reduction in testing time
Data-driven testing is time-consuming when done manually. With automated testing, the testing time is considerably reduced. Whenever there is a change in the test scenario, only the file where data is stored needs to be changed.
(ii) High-quality test data
The test data which we use for data-driven testing should be of high quality and easily maintainable. Otherwise, if we are using just filler data or using data that is difficult to maintain later with requirement changes, our purpose of automation will be lost.
d) Difficult manual test cases
The manual test cases which are difficult to execute, for instance, cases involving many numerical calculations, are ideal for automated testing. Since they consume the manual tester’s time, it is better to leave such calculations to the automation scripts and make better use of the saved time. In short, the test cases which are repeating in nature are the best choice for automated testing.
Stability of environment and application under test(AUT)
The environment and application should be stable so that the automation scripts provide consistent results when run.
a) Stable AUT– If the AUT is not stable and keeps on changing, the QA team will have to rework on automation scripts in every release. This will diminish the purpose of automated testing. Hence, make sure that the application is stable before starting with automated testing.
b) Stable Environment– If the environment is unstable, the actual result will vary from the expected result on every run of the automated test. In this case, it will be really difficult to verify the correctness of the test run and then eventually debug it.
Choose the appropriate automated testing tool
There are plenty of automated testing tools at present, and it is very important to analyse the below points before selecting the automated testing tool.
a) Application technology
Is our application web-based, desktop, mobile-based? This analysis is important because this will help in the selection of the correct tool.
b) Platforms and OS
Will the application run on which platforms and Operating Systems? Does your automated testing tool support Android and iOS or both? Think about your requirements and the features of the automated testing tool.
c) Skillset available
Today, the boundary between manual tester and automation test engineer has diminished. It is required for a QA team member to be well versed in manual and automated testing both. Automated testing means knowledge of scripting/programming language like Java, Python, Ruby, Perl, etc. One test engineer may be well versed in one language and unaware of the other. Hence, it is important to discuss with the QA team if they are comfortable with the language and the automated testing tool.
Below options can be considered if there is a mismatch between the skillset available and the skill set required:
- Training – The QA team can be trained with the required scripting language.
- Hire resources – New resources with knowledge of the automated testing tool and corresponding scripting language can be hired.
- Scriptless automated testing tools – Scriptless automation tools remove the dependency on coding for testers. These Scriptless tools can be NLP (Natural Language Processing) based, AI-based, Model-based, etc. Check out the different scriptless automated testing methods in addition to Record and Playback.
d) Cost of the tool
Is your tool open-source? Or are you using some licensed tools? It is wise to calculate the cost of the tool and your budget before the selection of the automated testing tool. Mostly, there are different open source tools available for different applications like mobile, web-based and desktop. So, if the organization does not want to spend on licensed tools, the QA team must have an understanding of all the different tools if the application runs on different platforms.
To escape this, a licensed tool can be used. Because a single licensed tool will provide solutions for all the platforms and technologies. Hence, the QA team will need to learn only a single tool so this can also be a good choice if the application runs on diverse platforms.
Also, the licensed tool provides services and support for the tool which is a plus.
Follow programming design principles
When an automated testing tool with a scripting language is used, test script writing is similar to software programming only and hence, the programming design principles are required to be followed for a simple, maintainable and reusable testing framework.
Today, we have scriptless automated testing tools where we need not worry about the programming design principles.
Simple automation test design
Try to keep the automation test design simple so that it is easy to understand and maintain.
a) Keep the test data and test scripts separate for easy test execution
Whenever there is a change in test data, only test data files should require modification and the test scripts should use variable names to fetch the required data without any change in scripts.
b) Keep the test cases independent
If you have dependent cases in the framework, parallel execution will not be possible and it will hinder the purpose of automation testing.
Screenshots, logs, and detailed reports are important
Choose an automated testing framework which provides screenshots, logs and detailed test reports on fail/pass for debugging. This will help the test engineer to find out where exactly the defect is or if there is an issue with the test script. These screenshots and reports are the deliverables after every testing cycle.
Hence, it is important that they catch the issue easily and the tester takes an immediate necessary step like logging the bug in a defect tracking tool without wasting time in analysis.
In case of open-source automated testing tools, we may have to write scripts ourselves for capturing screenshots and reports using the plug-ins available e.g, TestNG, Junit, Extent library can be used for Selenium.
If a licensed tool is used, the evidence capturing and reporting may come as a part of the tool itself.
Review and remove outdated/ non-usable tests
Developing and executing automation test script is the initial step, the next big task is periodically reviewing and updating the tests.
a) Code Review
Code review is necessary as in the case of software development. Many times, we may miss something which could be found during an effective code review.
b) Remove Tests
Periodically, the test scripts need to be reviewed to find cases which are outdated, unused or irrelevant. These cases should be removed because running them un-necessarily means losing time and resources.
Naming convention
In order to make the test cases readable and maintainable, tests should be clean. The naming convention of test classes, test methods, class fields and variables should be self-explanatory so that it is easy for the test engineer to recognise which particular test has failed/ passed without giving much effort in understanding, the name should explain it all.
e.g. A class method performing a new user registration- when a valid email id is provided with the registration is successful and a success message is displayed. We may use below method name to elaborate what the method does and what is the expected result –
registerNewUser_ValidEmail_ShouldCreateNewUserAndDisplaySuccessMessage()
More on clean code here .
In the case of scriptless automated testing tools, the naming convention and other design-related protocols are automatically taken care of, hence it is very liberating for a QA engineer.
Conclusion
Time, cost, quality and mitigation of risk are the key points to be kept in mind, whenever we are talking about software development. To have a perfect balance of these four factors does the actual magic. Hence, we can conclude that automated testing could be absolutely rewarding if done following the top automated testing practices to receive maximum ROI with time.
We need to understand that automated testing and manual testing will work in tandem and not in isolation. Hence, decide upon the best candidates for automated testing and also identify areas that need to be manually tested.
Top comments (1)
Explained the concepts beautifully !