DEV Community: JAYAVEL

10 Points to Help You Choose the Right Test Automation Tool

JAYAVEL — Tue, 07 Apr 2020 16:00:14 +0000

Making a decision to start automation is easy but choosing an appropriate tool for automation is not. There are teams that are spending a lot on hiring new manual testing resources but find it hard to invest in automation. The reasons could be many.

Sometimes, teams spend a lot of time exploring tools and get so overwhelmed with the information out there that they give up on the idea of automation altogether. Other times, choose a generic tool, start with automation but then, never get past the first few test cases.

In this article, we have put together some points that will help such teams navigate their search for the right tool:

1. Project Requirements: There is no point in looking for a solution when you don’t know the problem.

So, before you start exploring the various tools and technologies available in the market for test automation, you need to list down your project requirements and the problems you are looking to solve.

The list, in general, should answer the below questions:

Type of application that needs to be tested: It could be web, mobile, API or a desktop application.
Platforms that need to be tested: If yours is a desktop application, list down the operating systems that should be tested. If yours is a mobile application, then list down the supported mobile operating systems. If yours is a web application, then list down the supported browsers.
Language your application is built in: This can help if you are planning to use a programming language for automation.
Need for cross-browser testing/cross-device testing: If yours is a web application or a mobile application then you will, most probably, need this.

In addition, you could also add requirements that you deem important.

2. Team skills / Learning curve: When selecting a tool for automation there could be 2 types of tools:

A codeless test automation tool
An automation tool that requires coding

If yours is a team that already has people that are skilled in some programming language then you can think of using an automation tool in that programming language. Or, if you plan to hire skilled people for automation then you don’t need to consider this point.

But, if you are planning to have an automation tool that will not need you to look for people with the required skillset, going for codeless automation tools will be a good idea. These tools allow the automation of test cases without the need for knowing a programming language.

Check this guide to know about Codeless Testing in detail.

3. Budget:This is a very important aspect of choosing the correct automation tool. You might easily say that you will want a free tool because you don’t want to spend on automation if you can avoid it.

But, you also need to consider that the amount of time being spent on automation, the number of people working on the tool and the machines being used for automation also constitute the amount you spend on automation. So, consider below points before deciding the budget:

Cost of human resources being used for automation: If there is a tool that does not need you to hire new resources especially for automation, consider it a saving.
Time spent on learning the tool: If there is a tool that has a low learning curve, that is an indirect saving in the cost you might have spent in terms of the time the resources spend on learning the tool. Or hiring resources that are skilled in that particular tool.
Time being spent on automation: If there is a tool that makes it easy to create and maintain test cases, thereby saving time, consider it a saving in cost.
Cost of infrastructure: If there is a tool that allows hosting on cloud, you won’t need to invest in state-of-the-art infrastructure. Consider it a saving.

4. Ease of test case creation and Maintenance: Not every tool is made to handle all kinds of scenarios. So, to make sure that your chosen tool meets your needs, try automating a few test cases of your application to know if the tool suits your needs. That could be done with the trial version of a tool if your search has narrowed down to premium tools.

Also, to avoid spending more time in test case maintenance as compared to test case creation, make sure to choose a tool that fits your budget including the maintenance costs. There are tools that have the ability to self-heal the test cases in case of minor changes in the application.

Such tools help to reduce the cost of test case maintenance. Also, it helps if the tool supports pause and resume of test case execution for a better debugging experience.

5. Reusability: To avoid writing the same code multiple times in multiple test cases and to avoid duplication of efforts, look for tools that allow the reuse of already created test steps in different test cases and projects.

6. Data-driven Testing: If yours is an application that needs testing for a variety of data at multiple interfaces, it is important to choose a tool that supports data-driven testing.

7. Reporting: Test case creation and test case execution would be useless if the reports were not useful so do go through all the features in the reporting supported by a tool. Few of them would be:

Screenshots for failed steps
Video for test execution
Stack trace for the error
A clear indication of failed test cases/steps
Time taken for execution of test steps and test cases is reported

8. Support for collaboration: If you are doing automation of a project for a client, the client will want to review the quality of automated test cases.

It will also be beneficial if other non-technical members of the team are able to automate/review the test cases. So, in such scenarios, look for tools that make collaboration with the management and clients easy.

9. Support for tools for integration: If there are some process improvement or CI/CD tools that you already use or plan on using, make sure that you chose a tool that integrates with them.

10. Training and 24×7 support: Consider a scenario – you started using a tool for automation, and after automating about 10 test cases successfully, you got stuck on the 11th test case; you don’t know how to resolve the problem.

You have looked at all possible forums but you don’t have a solution in sight. If you want to save the time, use a tool that has 24×7 support to resolve any problems you encounter.

Conclusion

Sometimes, teams decide to create their own test automation frameworks because they cannot find a tool that fit their testing requirements.

At the moment, there really are multiple types of test automation frameworks and tools available in the market that support automation on a varied variety of applications and are still being improved.

So, do go through the above points and spend some time looking at available test automation tools before thinking of implementing a framework on your own

15 Must Have Skills For a Top Automation Tester

JAYAVEL — Wed, 22 Jan 2020 12:10:10 +0000

In the accelerated agile way of working, every firm demands a highly-skilled automation tester. Slowly, automation testing is replacing manual testing(though it cannot be done completely). Automation Testing is the key to the business success of the software industry. With automation, you can expand your business to a larger audience saving both time and effort.

As an automation tester, it is very essential to have certain skills which help in testing the application better. Let’s understand in the article, the automation testing skills required to be an efficient and successful automation tester in the software testing industry.

How does test automation benefit the software testing industry?

Replacing manual testing processes with automation testing has helped in the rapid adoption of agile methodology across software development industries
Automated testing benefits large scale projects that undergo frequent changes by speeding up the testing process and making deliveries within the stipulated timeline.
Manual testing in agile requires a lot of human intervention and the overall testing effort becomes tedious, automation testing helps solve this problem by automating the repetitive tasks.
Usually, for repetitive tasks, test scripts need to be created once and these can be reused every time the same scenario needs to be executed.
With every feature enhancement, the scripts can be updated as per the requirement and then can be reused.
Automated testing expedites the overall testing process. Automation testing can reduce the time to run repetitive tests from days to hours saving time and money.
Automation testing increases the depth and scope of tests that improve the software quality and also increases the test coverage of the application modules.
The automated test scripts can be reused multiple times for any frequent change in the software.
Automation testing helps the DevOps team to ship the software to the market at the earliest.

What are the skills needed for an automation tester?

Let us look at the top must-have skills for an automation tester:

i. Focus on analytical thinking :

For an automation tester, an instinct for analytics and logical application of concepts is important. Once the business team provides the business requirement document, the automation testing team should focus on understanding every aspect of the feature very well from an automation perspective.

The automation testers should raise queries in the agile refinement ceremonies to bridge any gap that can be a prerequisite in automating a certain functionality of an application.

The automation testing team needs to think and identify areas of the functionality which can or cannot be automated and define a detailed automation test strategy. The testing team should plan a walkthrough session with all the stakeholders specifically to discuss the automation testing approach for a feature.

Automation testing teams should collect ideas from all the team members and use the same to formulate the test plan document. In the test plan document define the scope of testing, automation testing approach, execution timelines, etc. Analyze how the test cases can be automated as per the priority.

The automation testing team can plan internal training sessions to discuss the automation testing approach. Also, the resources can share knowledge on certain automation tools that can be best utilized for the current project.

The testing team can discuss the estimation effort for automating each module of the software. This includes test designing as well as execution effort. Critical thinking is important to properly test software, write automated tests to cover defects in the software.

ii. Understanding of programming languages :

To be a successful automation tester, the tester should have a good understanding of programming languages.

Mostly the automated test tools use programming languages like Java, Python, Perl, Vb script, etc. The automation tester needs to be proficient in these programming languages. The thought process of the automation tester should be to identify and cover all the possible modules that demand automation.

The automation tester must have good coding skills in order to design the test scripts. Self-learning is a good way to get acquainted with these programming languages that can help in designing the automated test scripts.

Though it is not required today that you code your automated scripts with scriptless test automation, it is good to know the basics.

The testing team needs to ensure that the code conforms to certain designated quality standards. Along with maintaining the overall quality of the software, it is indeed essential to design easy to understand test scripts. Such that they can be understood by everyone in the team at a high level.

The team usually has an added advantage if they have an experienced automation tester in the team. Their previous project experience can be utilized in designing test cases and test suites.

iii. Good functional testing skills :

To excel as an automation tester, the tester should have sound knowledge and experience of functional testing performed manually.

It will be beneficial if the tester along with the application knowledge also understands the domain very well. For e.g., if a wholesale banking application requires payments domain knowledge to design automated test scripts of any given functionality, then it will be an added advantage if the tester along with good scripting skills also is well versed with domain-specific knowledge.

Automation testers should be well aware of the manual testing process and the test techniques that are adopted in the testing phase. It can be challenging and time-consuming for automation testers to design test scripts without having good functional knowledge of the application. If the automation test team has good exposure to the functionality they can achieve good test coverage and better test accuracy.

As manual testers are well acquainted with the STLC, Automation testers also need to understand the STLC (Software test life cycle) very well. This is required to know how the testing is performed at each step. This also makes automation of the manual test cases easy.

Refer manual testing to learn more.

iv. Expertise in creation of test scripts :

In the market, there is a wide range of automation frameworks, out of which some will expect the tester to have sufficient programming knowledge when it comes to writing automated test scripts whereas for some tools the test scripts are written in plain English language and do not require an understanding of backend logic and coding.

Most of the organizations in today’s date are using the Cucumber framework for test automation. In cucumber, as the test scripts are designed using plain English language, sufficient knowledge on Selenium WebDriver is enough to create the test scripts.

Automation testers should have programming knowledge if the organizations are dependent on automated tools like UFT (Unified functional tester) or QTP (Quick Test Professional).

The automation tester should develop the test cases with the aim of saving both time and effort during the test creation as well as the test execution process. An automation tester should have a good grasp of programming languages like Java, Python, Ruby, Perl, Vb script which are widely used, these days, in most of the automated test tools.

For a web-based application, the automation tester should be able to develop efficient test scripts for functionalities such as submitting a form, to generate a popup, to upload a file, etc. Estimating the complexity of an application, an automation tester should be able to create test scripts for a small as well as large module in the application.

Much before designing the test scripts the automation tester should question in the agile refinement sessions if there’s any unclarity on any requirement or functionality. This is the base step before creating the test scripts. Once the tester has better insight into the application, it becomes easier to create test scripts as well.

v. Possess good knowledge on Automated testing tools:

The automation testers should be well-equipped with ample knowledge about the automated testing tools present in the market that eventually help to optimize the overall testing process.

There is a wide range of automated testing tools available in the market that provide extremely superior benefits to an enterprise. To excel in the automation testing field, the tester should have good understanding and exposure of the automation test tools.

Depending on the selection of the automated testing tool, the tester should be able to classify whether certain test cases can be automated or not.

Once the tester is well-versed with the automated tools, he can utilize his automation testing skills for designing the test cases that guarantee good test coverage and speed up the test execution process.

vi. Clear understanding of business requirements :

An automation tester needs to have a clear understanding of the business requirements.

Let’s understand this with an example which all of us must have come across. If you are unwell and you visit a doctor, the doctor while examining you, also wants to know your medical history whether you have suffered from any kind of disease or ailment in the past. Depending on the overall past and present medical condition he can then suggest certain medicines for recovery.

On the same note, a skilled automation tester must know the application inside out, both at frontend and backend level before the testing phase begins.

An automation tester must be familiar with :

The programming language on which the application is developed.
Browser or device requirement where the application is to be accessed by the end-users.
APIs or any web services connected to the application and their working.
Which databases are used for storing the backend information? Working of all the modules and features in the application.
Is there a need for manual testing for testing some areas in the application?
Planned test execution time by the automation tester.
If there are any defects that were deferred in the last release and expected to be fixed in this release and the overall impact of this fix on the application.
Test execution completion delivery date.
Project release timelines.

vii. Well versed with agile, DevOps and continuous delivery :

Automation testing demand in the market is increasing with new-age agile and DevOps methodology replacing waterfall model.

As agile methodology involves frequent changes, it is essential to have an automation testing process in place for the same. Automation testers can automate the test scripts for a module to be able to respond to frequent customer induced requirement changes.

Let’s go through the agile scrum methodology in detail.

To achieve good speed and agility it becomes important for an automation tester to automate the testing process and configure them to automatically run when the deployment is completed in a testable environment.

Automation tools and continuous integration tools such as Jenkins, Git lab can help us achieve this integration. So, the automation tester should be familiar with the usage of these integration tools required for automation. Automation ultimately ensures the success of DevTestOps.

You may refer to a detailed explanation of how to extend test automation to enable continuous delivery.

viii. Maintain good communication and interaction with stakeholders :

Having good communication skills and collaboration is essential for automation testers. This is most important before and during the testing phase as automation testers have to interact with developers, business analysts, feature engineers (possessing excellent domain-specific knowledge) and all other stakeholders.

In the agile refinement sessions, the automation testers can formulate open questions to be raised in front of all the stakeholders to get better clarity on the requirements.

Also, once the automated test scripts are designed, the automation testers need to give a walkthrough to the developers, business and all other stakeholders.

During the test execution process, if any defect retest fails, the testers apart from raising the defect on a specific tool can also communicate the same to the developers in the daily stand up meeting. Sometimes, environmental issues may appear that could block the testing process and delay the delivery. Maintaining good communication about these issues will avoid any end-time surprises related to the time of delivery/release.

The automation testers can interact with all the stakeholders and discuss the overall testing progress and status in the agile scrum meetings.

ix. Curious to learn new technologies and trends :

Being an automation tester you should be willing and curious to learn new technologies in the area of automation.

Considering the history of automation tools being widely used in the organizations, earlier firms preferred using QTP (Quick Test Professional) tool for automation testing for a web-based application whereas now Selenium WebDriver is in huge demand.

In the market, demand for new automated testing tools keeps on changing as per advanced technology and the benefit it provides to the software industry. So, be it QTP, Selenium, Testsigma or some other automated testing tool, an automation tester should be well versed with the automation techniques.

The automation concepts should be very clear to the automation tester so that he is well aware of what needs to be automated, how it needs to be automated and why it needs to be automated.

If you are good at implementing the logic needed for automation then you can apply the same logic in any kind of automation tool and design appropriate test suites suitable for testing a software application.

x. Able to assess and mitigate the risk :

Though test automation seems like a strategic step in the agile world, there is always a risk associated with the test automation process.

If there are changes in the interfaces, after the automation test scripts were prepared, it can cause a problem during the test execution process as irrelevant test results will be generated due to these interface changes. Similar problems might occur in case of changes in business logic. This incurs additional cost to support the changes, also it can involve modification of test data and impact other test cases as well.

There’s a risk of issues related to the environment execution. This increases the total time taken for testing as the application is not stable throughout the testing process. Overall increased testing time results in a delay in delivery of the software.

An automation tester needs to keep a mitigation plan ready for such risks encountered during the testing phase. The testers need to monitor the test execution process and immediately report any obnoxious application behaviour to the developers and the stakeholders.

Early identification of such problems helps to prevent the occurrence of such issues and mitigates the risks.

xi. Good problem-solving skills :

Automation tester needs to have in-depth technical knowledge of the automated testing tools before they actually start using them.

In the test execution phase, testers come across scenarios where the script is generating some errors but the functionality of the application is working on expected lines. On the other hand, the script could be showing success but the application may have some real-time errors. The automation testers must have good problem-solving skills that are required to debug these errors. These issues mostly occur when the automation tester does not have sufficient knowledge for configuring the test scripts.

If you want to become a successful automation tester you need to have proficient knowledge in configuring the tools and steps to resolve them when a situation like false positives and false negatives arise.

xii. Good reporting skills :

An automation tester must possess good reporting skills.
Automation testers should regularly communicate the status of the application under test to all the stakeholders.

The practice of regular reporting leads to better coordination of the overall test project and also gives transparency to the project management in terms of the test execution status, show stopper and critical bugs in the application, defect closure status, release timelines, etc which eventually help them to make better decisions where needed.

xiii. Excellent at time management :

An automation tester must be excellent in time management. This helps in increasing the overall productivity and thus meeting the delivery deadlines. The purpose of time management should be to prioritize the tasks with respect to the creation of automated test scripts and executing the same.

The automation testing team needs to determine the sequence of tasks and plan how the team will spend the available time. The team can divide the tasks into smaller blocks of time. If you keep the work defined in smaller pieces it will be better in understanding the key responsibilities and when can you have them completed. Managing your time allows you to produce higher quality work as well as utilize the extra time in improving the automated testing process.

xiv. Passion for automation :

Being an automation tester, you need to keep on upgrading your skills to be able to deliver an efficient product to the customer.

Learning is an ever going process and also an essential factor for career growth.

An automation tester needs to be passionate about learning new things and thinking of new ideas to improve the automation process and take efforts and implement it.

This promotes automation tester’s adaptability and their ability to switch between different environments and tools whenever the market demands it. Automation is the key to success in the agile software development process. So keep learning to succeed.

xv. Online Automation Testing Courses and Certifications :

A certification always stands as an added advantage for an automation tester.
It helps to build a strong profile for an automation tester and enhance your knowledge in terms of automation testing.

You may also read more about the top online automation testing courses and certifications.

Conclusion

Automation testers can be termed as “Full-stack QA engineers” as their role is to test all aspects of product quality such as functionality, performance, security, etc in combination with various technologies and test techniques to the application under test.

Automation testers must be multiskilled and most importantly possess a combination of domain knowledge, technical expertise and testing skills that allow them to quickly deliver quality software. To become a top automation tester, the testers need to stay current on the top test automation trends. An automation tester can develop the above-defined qualities and skills to be an efficient gem in the software industry.

Beyond the Wall of a Sprint

JAYAVEL — Mon, 26 Aug 2019 16:41:31 +0000

How many times have we started the day with a smile, big mug of coffee and enthusiasm for what is ahead of us...An exciting new project of creative writing, also known as programming! Where every day is a new, exciting adventure; filled to the brim with easy to solve bugs, fun algorithms, and awesome new technologies, we’ve been experimenting with! A blissful day indeed!

Until the coffee washes the sleep out of our eyes and we realize that yesterday we stayed up late, implementing all the fun bits and the only thing left to do today is to write tests. Similar to the effect of hearing yourself speak during a call, you have to persevere and dive deep into your code’s logic. The code you’ve written in a sleepless, caffeine-induced mania. Where variables start off as CamelBack and slowly devolve into meaningless words… The deeper you go, the less characters you see. Like the light of a torch fades, so do our comments.

Not that bad, we say to ourselves – I’ll just test this (obvious) input, catch that (obvious) corner case and make sure it’s handled… maybe add a few extra safety steps... Oh, this one will never be hit? No problem, test coverage is important. It makes my commit message look great. And I can brag about it during the sprint planning this afternoon? Sign me up!

So, we carry on being responsible developers, write up a few more (obvious) test cases and make the coverage report look amazing, brag about it and move on with our day.

A few weeks go by. We resolve some more tickets, implement some more fun features, write some more (obviously) diligent tests. Life is great. Until we get a bug report related to the fun algorithm we did. The one we wrote loads of tests for and made sure we are good developers for? “Oh, sweet!” We think to ourselves, as we chair-dance to our favourite song.

We fix the easy to solve bug, run the debugger and mark the ticket as done. Wait... what are these errors? The tests failed? No… It can’t be. We were such good developers, what happened...?

It’s a common scenario for many developers. And as technology moves at a higher cadence year by year… so does our Jira backlog. Learning new languages, catching up to all the new features of old languages (C++17 anyone?), getting used to the latest Microsoft office product… It’s all too much, too fast.

But we do catch up to the latest, trendiest open source project changing the world over. We are forming a continuous integration team (of one person). We’ve heard of Kubernetes. Maybe some use it constantly since they got hired as full stack developers. Others are turning into full stack developers, but without the fancy title (or paycheck). They’re just staying ahead of the curve and being paid peanuts for it.

The reality of today is that we have to stay on top of things. Including tests. Test driven development is a phrase we all like using and love ignoring the fundamental meaning of. Some of us, rightfully so, turn to automated testing tools driven by AI in order to start the project right, tackling the scariest part during the design phase, rather than at the end.

Going back to the fun algorithm anecdote, which we’ve obviously resolved.

We find ourselves in the sprint planning meeting. Our manager gives everyone the chance to shine, which we’ve eagerly been waiting for. The one thing we’ve been eagerly awaiting for during the past two weeks (or more, if your managers are meanies).

Everyone lists their list of incredible achievements, quickly glances over the tickets they didn’t get around to and… It’s our turn. Oh, the fun algorithm ticket? Of course, we fixed it. Tests? Of course, we wrapped it up, we’re good developers after all! Oh, it’s time to push it to deploy to the clients? Oh… right.

We do our best to look eager, as we take on the task, which we know can be over in an afternoon… or take the rest of the sprint. No matter, we get on with it and head to DevOps(also called DevTestOps or QAOps).

Or, we would’ve, if the team was formed, as was promised, several months ago. No matter, we volunteer our time, as a good developer does, and deploy ourselves.

So we open up our trusty text editor, create the extensionless Jenkinsfile and get on to work. We write the classic “Hello world” send it to the nether and after a few minutes get an email telling us the build failed. No biggie, surely there’s a boilerplate floating around, which we could use.

So we start digging through the proverbial drawer, also known as our Gerrit server and find some commits related to Jenkins. Those look pretty good, let’s make use of them. After stripping out all the capitalised variable and delete all the “NB:”, “TODO:” and function instruction manuals, we are finally ready to implement the fun algorithm.

After a few hard hours, we get beautiful 100% test coverage, beautiful, green PASS-es are seen everywhere. It could only mean one thing. It’s time to deploy. Fearlessly, we hit the deploy button and contemplate going for lunch. It will probably take a while, surely, so we get up, put on our coats and… Bing! A new email from our service ops account? What could it be…?

Oh, look! It’s from Jenkins!... Hm, that’s strange, the word success appears nowhere in the Outlook notification. So we open it up and are greeted in big, bold, red letters – “Build failed.” Fantastic. And we were just about to go for lunch too.

Turns out our manually written, locally grown, organically produced tests don’t fare so well out in the cloud.

We, developers, are creative people. Similar to writers, we turn an empty sheet of paper into a work of art. We wait for the muse to strike, and when it does, we grab it by the horns and fly off into the night, only to wake up the next day with an incredible spawn of love and care. Sure, it might not be the most beautiful kid in the class, but it’s ours, you know? We made this. And it’s (obviously) perfect. Why on earth would we want to change that?

In conclusion, the best way to describe testing is -- complexity. It’s a challenging, grueling task, closely followed by writing documentation. It’s a mechanical process, which we, as artists, tend to avoid. There’s nothing creative about going through your own painting and adding the glass and the frame around it. Of course there’s value to it -- we protect the baby we’ve nurtured over that one exciting night of passion.

But at the end, it’s difficult changing who we are. Add to that the need to stay on top of the latest and greatest frames, the trendy new polyglass that’s all the rage nowadays and the incredible new manuals on paint framing… and it all just turns into an impossible task!