Automated or Manual Testing: Keeping the Balance Right!

If you are a tester, then you must have had a discussion around automated or manual testing. This is nothing new, and lots of techies have different views around this. Whether you are a big team and already established an automation framework or you are a small team, new to automation, it is always necessary to keep this balance right in order to get maximum efficiency.

Surely automation testing is having the benefits of increasing efficiency, getting faster regressions and thus contributing to timely project deliveries. It also removes the execution of repetitive test cases or regression cases manually and saves a tester’s life.

But before considering automation, there are certain points which you should evaluate. You must have heard a statement “You can not automate everything” which is very true.

Manual testing is required in many cases. In fact the biggest drawback of manual testing itself is its biggest advantage that it requires human intervention! There are certain cases which require human instinct and intuitiveness to test a system. To name a few, these are the following cases where manual testing plays a vital role-

1. Usability Testing- This is testing an application in the view of how easy or difficult is it to understand. This is to test how interactive the application is to the users who are going to use it. These kind of tests cannot be automated and must be performed manually.
2. UI and UX Testing- UI and UX testing cannot be automated and even if you try, it would be only to some extent. Automation scripts can be used to test the layout, CSS errors and HTML structure but the whole user experience cannot be automated as it is very subjective.
3. Exploratory Testing- Cem Kaner who coined the term in 1984, defines exploratory testing as – “A style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.”
4. Ad-hoc testing- This is completely unplanned testing which relies on tester’s insight and approach. There is no script ready for this testing and has to be performed manually.

Pros of Automation Testing

There are certain cases where automation testing is beneficial and can actually reduce efforts and increase productivity. Let’s have a look-

1. Regression Testing– Regression cases are mostly repetitive and we can automate them once and execute in a timely manner.
2. Load Testing– Automation is very much useful in case of load testing. Load testing identifies the bottlenecks in the system under various workloads and checks how the system reacts when the load is gradually increased, which can be achieved by automation.
3. Performance Testing- Performance Testing is defined as a type of software testing used to ensure whether software applications perform well under their expected workload. Automation is very useful in this type of testing.

Apart from that, the test cases which are repetitive can be automated. Keeping in mind the above points, you can decide on what, how and why of automation.

What, Why and How of Automation- To maintain a balance between manual and automation can be very tricky at times. I have seen many aggressive managers pushing to automate everything. But is this the best approach?

Before starting to automate, you need to answer these three questions-

1) What needs to be automated?

Let’s first think of what exactly needs to be automated. Hereby ‘Exactly’, I mean what part of the ‘requirement’/ ‘feature’/ ‘application’ is a candidate looking for automation. Often the application which is going to be automated is termed as AUT (Application Under Test ). It is quite possible that a part of a feature can be automated and the rest be tested manually.

This requires deep-dive into the feature, it’s test cases and effort which will be required. Sometimes knowing how the developer is going to implement that feature plays a vital role in deciding if it can be automated and to what extent.

2) Why automation?

This is very important. Why do you need to automate? Is it because it reduces effort and increases efficiency? Or is it because it would benefit in the long run? Or is it just conventional? During my tenure as QA, I found some managers who aggressively wanted everything to be automated without analyzing that it might increase effort and reap no fruit. You might end up asking a few questions to yourself-

a. Is it a one-time requirement and never coming in future? we probably don’t need to automate then.
b. Is the automation solution complex? Also, you need to understand the complexity of an application under test. If automating it leads to building a parallel application itself, there is no point of automation.
But there can be a case where the solution is complex but the feature or AUT keeps on changing and development is planned for the long term, then you may find automation beneficial in the long run.
c. Time constraint- There might be a time constraint in delivery. At that time a manager’s role is very crucial in deciding to invest in automation or go for manual.
d. Resources and skills of Testing team- This is also an important factor. How many automation engineers are available in the testing team to leverage their bandwidth for AUT? Mostly, for small teams, this is the deciding factor to go for automation.

3) How to automate?

This refers to find the solution. The Why and How are co-related. So you might find answering How and get answered for Why and vice-versa.

HOW is to decide how we are going to automate AUT. Do not confuse with AUT or feature as this stands for both if you are going to automate entire application or a small feature. Sometimes for a small application, you need not require a full-fledged automation framework.

There are various tools available which don’t require coding and can solve the requirement. For e.g. TruBot from CloudQA is one such tool which has many handy tools for different types of testing and is very user-friendly for small applications.

Apart from that, there are various requirements which can be catered by simply writing a shell script.

A full-fledged automation framework is required when AUT is big and there are continuous enhancements going on. At that time, a regression suit can be executed before each feature release and automation can significantly reduce the effort.

Automation framework development requires both coding skills and time, so before jumping into that, a tester should always analyze the ROI (Return On Investment) and then make a plan accordingly.

Also, automation gives a sense of confidence that there is maximum coverage of regression tests and existing features are not broken because of new feature addition.

Conclusion

Let’s conclude that either only manual or only automation is not the right approach. There should be a balance between both and I hope that the above points will be helpful in finding the right balance.