DEV Community: Vladislav

Step by Step Guide to Building a Test Automation Strategy

Vladislav — Mon, 14 Sep 2020 10:13:34 +0000

Automation is about using tools to perform specific testing tasks more efficiently. It means that automated testing isn’t adopted to replace manual one but is always used to enhance it.

When an idea to implement automation crosses your mind, instead of how part, focus on why at first. Automation for the sake of automation often comes along with the following arguments:

it sounds good, so it must be good;
the guys from that other company already use it;
we can cover 100% of functionality by automated tests;

This kind of reasoning is commonly mistaken for green light for the adoption of automated testing. Meanwhile, learning about test automation strategy is a good idea for those who want to make software delivery cycles shorter and leave less mundane tasks for QA engineers.

What Is Test Automation Strategy?

It is the way a QA team applies automation during the quality assurance process. If you start decomposing test automation strategy, you’ll get a list of elements that will look approximately like this:

test plans, both manual and automated;
test environments, with definitions and descriptions;
test cases;
automation scripts;
test data;
test results;
execution logs.

A cost-effective test automation strategy implies a result-oriented approach that helps companies keep up with the market trends and outperform, or at least reach their ambitious business goals.

A test automation plan and strategy are specific for every project. Even when you test similar products, some aspects require adjustments. You can use the same template to create a test automation strategy, but the final version will be at least slightly different.

How to Build a Test Automation Strategy

So you already know that efficient automation starts with setting your goals and objectives. Let me specify what those are. Goals define what you aim to achieve in a long-term perspective, for example:

better product quality and, as a result, higher user satisfaction;
minimizing human effort and testing time;
increasing efficiency of testing while reducing costs for it.

An objective is more like a ‘one step at a time’ thing. These are smaller tasks you have to complete to achieve the goals. The objectives of test automation include:

broader test coverage;
creation of tests that are easy to run and maintain;
enhanced product quality after each iteration;
improved stability and reliability;
increased efficiency of QA engineers and better motivation

If you would like to get a more detailed explanation of each item on the list, you can find it in the post about automated testing goals and objectives. Now, let’s move on to the step-by-step instructions.

Step 1. Decide what to automate

Setting up test automation is a time-consuming process, so consider what you plan to achieve in both short-time and long-time perspectives. Set reasonable expectations. Consider how frequently a test will be repeated and what scope of work it requires. In the perfect case, you will pick tests that match at least several of the following criteria:

time-consuming if done manually;
involve large data sets;
cover stable software components;
require the checkup in multiple systems (browsers, OS, hardware, etc.);
mundane and don’t require creativity;
with clear pass/fail results;
highly-repetitive.

On the other hand, some tests are not suitable for automation. For instance:

tests that will be executed once;
tests based on visual perception;
tests without evident pass/fail results;
anti-automation features, like CAPTCHA;
raw and unstable functionality.

I wouldn’t recommend automating these things. The process will take a lot of time, and the results you’ll get after running tests won’t be accurate. For example, UX testing requires clear human input and estimation. Exploratory testing relies on QA engineer’s knowledge, experience, even intuition and serendipity. Obviously, manual inspection is much more reasonable here.

Step 2. Choose a test approach

A test automation approach defines how you will carry out the tests. A proactive approach initiates the test design process at the early stages of SDLC. A QA team finds defects before a build is created. In a reactive approach, testing starts after the coding is completed. A QA team examines an initial version of a product that is supposedly ready for the release.

A test approach also covers types of testing a team is going to perform. The most common candidates for the automation are:
Unit Testing – a checkup of the smallest functioning elements of code by a development team that is familiar with the backend.

Component testing – a checkup of every object or part of the software separately by a testing team on the frontend.
Integration testing – an inspection of the behavior of unit combinations.
API testing – a segment of integration testing that focuses on API functionalities.
Compatibility testing – an analysis of software behavior in different browsers, OS, network environments, hardware, etc.
Smoke testing – a review of a new build aimed to prove the basic functionality (installation, launching, registration, etc.) works so that the team can move on to testing the rest of the features.
GUI testing – a check aimed to prove that the graphic user interface meets the specifications stated in software requirements.
End-to-end testing – an examination of the complete workflow from the beginning to the end by replicating real user scenarios.
Regression testing – an analysis expected to verify that code changes haven’t affected untouched parts of software features after the latest iteration.

The Test Pyramid Principle

The Test Pyramid is a principle that helps to choose cases for automation smartly. Unit tests that cover the lowest level of software code and have the highest ROI since they help avoid more serious mistakes. Thus, it is reasonable to automate this level to the maximum.

Component, integration, and API tests are the next in terms of priority. Regression is one more candidate for automation, as it covers the familiar functionality that requires revision after every iteration.

And only then goes GUI testing. Though very common, GUI automation is highly impractical. These tests are difficult to maintain, expensive, and with a high number of false positives and negatives.

A traditional approach suggests the opposite: more automation for GUI and less for unit tests, and that is not always efficient.

Factors to Suggest when Choosing a Test Approach

Technology the app is based on. App specifications affect the types of testing required and later – the choice of a platform and tools.
Team roles. Make sure that every team member knows their responsibilities and is qualified to cover a particular aspect of QA on the project.
Risks involved. To eliminate the most probable and damaging risks, you need to cover the vulnerable areas. And for this, you should know the risks. Create a document where you record the following parameters:

risk level – the difficulty to handle the risk if it becomes reality;
probability – the likeness of risk turning into reality;
mitigation – actions to take to resolve the risks;

Step 3. Select an automation framework

A test automation framework is a set of detailed guidelines, coding standards, project hierarchies, reporting mechanisms, etc. that create a scaffolding for the automated test scripts.

Simply put, it is a practice that makes test automation efficient. A QA team needs to choose a suitable framework to create scripts with higher re-usage, easier portability, and reduced maintenance cost.

Some of the most popular practices are:

linear scripting;
library architecture;
modular framework;
data-driven framework;
keyword-driven framework;
hybrid framework.

Record & playback
Also known as linear scripting or record & replay, it is the simplest of all the frameworks. It allows QA engineers to record each step of the testing process and the validation parameters manually so they can play it back anytime later.

Linear scripting is the fastest way to generate a script, and it doesn’t even require automation experience. The downsides are little reuse and difficulties in maintenance.

Structured scripting
This framework also has alternative names also – library architecture or functional decomposition. It takes scripts created using record & playback, groups similar ones into functions, saves them into the library, and calls whenever needed.

The library architecture framework ensures a higher level of reuse that comes along with less costly development and less messy maintenance. On the other hand, it also requires expertise and more planning.

Modular framework
A test automation engineer divides an AUT (application under test) into smaller components – units, functions, sections – and creates a script for each to test them in isolation. Combining these scripts hierarchically, you can create an overall script for a broader test run.

You cannot use multiple datasets since the data are hardcoded, but if an application undergoes changes, you only need to fix an individual test script.

Data-driven framework
This framework separates test data from script logic and stores it externally. Test scripts use this file to read input or output parameters. The data-driven framework is perfect for situations when you need to test the same feature multiple times with different data sets. Thus, we can test scenarios with varying data quickly with a minimum number of scripts.

Setting up the data-driven framework is time-consuming. It is a task for a skilled test automation engineer able to format data properly and write scripts that can utilize all the advantages of this framework.

Keyword-driven framework
Used mainly for GUI automation, this framework also implies storing test data separately from scripts. The instructions are words that describe actions in consecutive order (login, click, etc.).

With the keyword-driven framework, you get reusable code to apply across multiple test scripts that can be built independently of the AUT. It also takes much time to set up and a lot of effort to keep in order. The high cost of the setup is justified if the application is large. Then, you can use the scripts for several years.

Hybrid framework
Every framework comes with its pros and cons. When test automation engineers start to maximize the positive features and minimize the downsides, they may come up with a combination of different frameworks that works the best for a particular project. Such a blend makes a hybrid framework, the most common option for an agile and DevOps test automation strategy.

Step 4. Find qualified people to join your team

Don’t forget that software testing automation is a task for a person hired specifically for it. Some try to make automation a part-time job of manual QA engineers. However, these specialists already have other tasks to cover and, most likely, lack skills to set up automated testing. It would be more efficient to find a person with experience in scripting or at least relevant knowledge to be responsible for this scope of tasks.

Step 5. Select a testing tool (or several tools)

The first thing you pay attention to is the popularity of a tool, and it’s not a mistake. Popular tools usually prove to be reliable and widely discussed online. An automation engineer can find many useful tips and interesting practices on the web.

Also, it is easier to find QA engineers well-versed in Selenium, Appium, or Cucumber than a tool that has just recently appeared and seems good. Besides popularity, there are other criteria to consider.

Tech stack:

The technology software is built on.
Requirements of the system under test.
Programming languages you are familiar with.

Tool features:

Supported OS, platforms, environments (e. g. automated browser testing, native/hybrid app testing, etc.).
Cross-platform testing support.
Complexity vs ease of use.

Budget:

Are you looking for an open-source or a paid option?
Are you looking for a development or runtime license in a paid tool?
How frequently are you going to use this tool?
Does the price justify the functionality it provides?

The experience shows that building a custom framework upon an open-source tool is the best option. Try several solutions. The majority comes with a free trial period so you can weigh all the pros and cons of a commercial tool before paying for the license.

Step 6. Create and execute automated tests

So you know what to automate, what types of testing to apply, what framework and tools to use. It’s time to move from the arrangements to actual work and start writing scripts. Here are some tips that will help you with this:

Outline day-to-day tasks and procedures.
Create test case templates to use in multiple projects.
Before writing a new test case, remember to check if similar cases already exist.
Write tests that are concise and easy to understand, for in the future, other people will work with them, too.
Make test scripts as minimal as possible, so they’ll be simple to understand and maintain.
Break bulky tests into sequences and check each individually. The smaller they are, the less is the probability of a mistake.
Make sure tests aren’t dependent on the UI. Using UI elements and pathways that may change is a sure way to create useless scripts.
Have a clear plan of how to resolve failed test cases (perform feature analysis, just report to a dev team, etc.).
Learn to prioritize test cases.

Step 7. Maintain your scripts

People that have experience in delivering automated software testing services admit that things tend to get messy on the maintenance stage. So first and foremost, categorize the tests according to their purpose or come up with an alternative classification that will allow finding relevant cases quickly and easily in the future.

Don’t forget to categorize new cases. Review and clear up existing cases. Many of them aren’t meant to last forever, and that’s okay. Address product requirements to decide whether old cases are efficient and should be used or or they’re not relevant anymore. Never try to reanimate what is useless.

Don’t Forget About Documentation

Documentation has nothing to do with bureaucracy and delays if you manage it right. On the contrary, documentation is more reliable than memory and helps you keep a clear head. So what to put on paper?

Outline the scope of work, then set milestones and timelines for each task. It will keep every team member on the same page.
Document the approach to automation – with the process, roles, and technology. It will become a set of guidelines you can address when you need to clarify something or resolve a dilemma.
Outline the automation environment. Create a pipeline for software to pass before it is ready for the release. Conduct risk analysis to be prepared for a ‘sth goes wrong’ scenario. Risks always exist, and ignoring the possibility of risk is just careless.
Create an execution plan to ensure bug-free testing. Write down day-to-day tasks and procedures. Run separate tests before launching automated regression. Make sure there are no fake results.

Writing a Test Automation Strategy for Agile Projects

Test automation strategy in agile usually implies a proactive approach and a preference for automated unit testing. These are the basic things that make fast releases possible.
In addition to this, consider using a keyword-driven automation framework. If you choose words that correlate with business requirements, you will be able to use the scripts for as long as these words correspond to known commands. Keywords can also be a part of a hybrid framework, which enables flexibility so essential in agile SDLC.

The automation strategy for agile projects never overlooks unit and component testing. Testing one small feature at a time helps to extract true value from the automation. On the one hand, you can create a script for a feature right after a developer writes a piece of code. On the other hand, compiling smaller pieces into a test script that covers a large part of the functionality helps to minimize mistakes made on the way.

Test Automation Strategy for In-House and Outsource

There is no fundamental difference between a QA automation strategy for in-house and outsource teams. In both cases, you need to decide what test approach and framework to use, select automation tools, and find qualified people to handle the tasks.

The distinction lies in the scope of managerial work. Of course, a project owner (or whoever is in charge) has to manage the process in any case. And if you are reading about how to build a test automation strategy, you probably need to figure out some things about it.

In both cases, you need to start with the hiring process. Outsource testing is a time-saver here. Assembling an in-house team is not a quick process. Hopefully, you will find qualified people who are familiar with the best practices of test automation.

If you go for QA outsource, you get a fully functional team with relevant experience in software test automation services, well-tried automation strategies, favorite tools, and test script templates.

In other words, an in-house team will require more involvement and management, at least at the beginning. With an outsourced QA, you request reports, specify the details that interest you, but the rest is a task for them to handle.

Companies that have QA departments often decide to cover manual software testing with internal QA resources and outsource automation since it is more cost-effective. Some choose to start with an external team and hire automation specialists later if the flow of tasks becomes steady.

Bottom Line

Implementing test automation takes time, effort, and specific skills. Automated testing is not always essential. It doesn’t replace manual testing and doesn’t benefit a project unless you define clear goals and objectives. So how to draw those advantages everyone is talking about?

Know why exactly you need QA automation. Keep reasonable expectations. Start small and focus on a long-term perspective. Don’t give up on automation if the results aren’t immediate. But first and foremost, work carefully on your automation strategy. Hopefully, I helped you with the latter, at least a little.

Answer: How to test your app while not effecting active users?

Vladislav — Fri, 07 Aug 2020 10:08:16 +0000

answer re: How to test your app while not effecting active users?

Aug 6 '20

Google Play’s Core library (on 1.5.0 or higher) has a feature for apps running on devices using Android 5.0 (API level 21) or higher and Chrome OS devices, called In-app updates. If you want users to try a new app feature or apply updates to improve performance or fix bugs…

Open Full Answer

AIOps and Quality Assurance: AI and ML Drive Change

Vladislav — Thu, 11 Jun 2020 11:50:43 +0000

DevOps model facilitates the product delivery process. Still, this dynamic method might be not enough when both development and business teams participate in the decision-making process and the deadlines are tough. That’s where AIOps comes to rescue – a brand new way to optimize SDLC and business processes.

Definition of AIOps

AIOps (artificial intelligence in IT operations) means the use of machine learning and data science for solving IT-related issues. An AIOps platform applies big data to enhance the functions of IT operations and minimize human input. These platforms consume and analyze the data generated by IT to better understand software behavior.

For example, traditional test automation services cover repetitive time-consuming tasks so people can focus on uncommon issues and cross-team collaboration. In AIOps, algorithms collect valuable insights and use data from performance monitoring, event analysis, IT service management, etc. for more efficient automation.

AIOps and DevOps: What’s the Difference?

DevOps automates the path from development to production – with auto tests and readiness checks, in particular. AIOps uses the data it receives to predict the performance, suggest ways of optimization, and process root cause analysis.

With DevOps, we still rely on humans to look at logs, alerts, and metrics to find issues. AIOps is the next step on the path of automation. AI analyses data more accurately. It can correlate performance with code issues to recommend changes or even fix issues immediately based on past experiences.

What Makes AIOps Efficient?

The core advantage of AIOps is a fast-paced delivery of complex apps and distributed systems. Companies that employ DevOps still need to release new code monthly or weekly. It gets difficult for IT teams to keep up with the updates in the products they support, as well as for QA teams to run regular checks.

For AIOps platforms, the big scope of changes is not a challenge. It the future, the range of AI tasks will go further beyond automating regression testing. AI will be able to cover A/B tests, auto healing, automatic alerts, and much more.

Why Adopting AIOps Is a Good Decision?

AIOps helps organizations to dramatically improve service health and productivity. AI and ML can predict load patterns and schedule maintenance works (patches, upgrades, new releases) during low-impact periods. It can roll back the previous build if the new one has failed, increase/decrease CPU based on memory usage, and take other actions to keep software stable.

AIOps systems analyze test traffic and logs automatically, show infrastructure changes and previous incidents. They find issues early by addressing outages and service degradations. These platforms also detect inconsistencies and proactively identifies potential issues before they cause problems. If a problem does happen, a platform presents only a few critical events that have affected the service.

AIOps and QA

AIOps can change the way a software testing company works. With the right tools, test data becomes priceless. AI together with ML build a predictive QA model that converts data into actionable insights that reduce test cycles and allow for faster product delivery.

Large applications imply equally large test suites that take a lot of time to run. There is no guarantee that these tests will cover all relevant elements. The predictive QA model can pick cases that are essential for a particular code modification. Moreover, an AIOps platform would alert the changes that could cause critical bugs.

Implementing AIOps

The scenario of AIOps adoption may differ depending on a project scale, complexity, and specifications. Still, this step-by-step guide will be helpful.

Get familiar with the AI and ML vocabulary, do some research and team training.
Identify and understand data in your operations: logs, metrics, API outputs, device data, etc.
Think about how the data can solve your problems. For example, a system will review past failures and detect the root of the high-priority problems.
Analyze project feasibility. Make sure AIOps helps solve problems and its implementation is relevant.
Select test cases for ML. If there are too many, choose the most suitable.
Use these insights to prepare an AIOps platform for real-time software monitoring.

The system should have enough data to learn from. AI needs to get the full picture to work with high accuracy and make valuable predictions. With time, AI will mature and prevent both known and new issues.

Just don’t forget that AIOps is not an alternative to manual software testing. AI cannot replace humans. It facilitates the work, augments the abilities of the QA specialists, but people still supervise AI and arrange ML.

Best AIOps Tools

If it is difficult to decide upon a platform for AIOps, Gartner Magic Quadrant may become helpful. This is a graph a company came up with to estimate technology providers. It distinguishes four types of companies on the market – Leaders, Visionaries, Niche Players, and Challengers – so you can choose one based on your priorities and business goals.

Speaking about AIOps companies, product reviews on G2 name these five software solutions as the best on the market today:

Dynatrace;
Splunk Enterprise;
AppDynamics;
Splunk Cloud;
Moogsoft.

Bottom Line

Tech companies that apply AIOps to enhance dev and QA resources are more likely to top the leaderboard, but only in case they apply it smartly. AIOps is not a panacea, so always keep in mind the specific nature of your project before rushing to adopt new technologies only because they are having a moment.

Test Automation with Selenium and Cucumber

Vladislav — Thu, 16 Apr 2020 09:13:41 +0000

Selenium has been making huge strides in becoming the automated testing tool of choice for DevOps. Although it does require a degree of technical skill to setup and manage Selenium, it’s one of the most effective methods to automate testing on web browsers. Selenium is also able to integrate with other apps and tools to customize and streamline testing processes. But more recently, we’ve been hearing a lot of hype around the Cucumber tool.

Cucumber Selenium testing is fast becoming a popular combination for web application test automation. While Selenium is primarily used for regression and functional testing, the Cucumber testing tool allows you to write easy test cases that are executed by Selenium on different browsers.

Whether you employ a software testing company to manage your QA or you’re looking to implement new tools in-house, understanding the fundamentals behind different testing approaches will arm you with knowledge to make informed decisions.

Introduction to Cucumber

Cucumber test automtion supports Behavior Driven Development (BDD), which is a technique where you write test cases in simple English sentences regardless of technical ability. In fact, anyone in the organization would be able to understand the test case requirements when written in Cucumber with Gherkin.

The Cucumber test framework incorporates explaining the application’s behavior using the Gherkin language. Gherkin uses a simple syntax and supports multiple programming languages like Java, Ruby, Python etc. so you can use whatever language you’re familiar with. The Cucumber automation tool not only integrates with Selenium but other web-based testing tools as well such as Ruby on Rails and Watir.

P.S. Cucumber is open source, so it’s free to install and download.

What is Behavior Driven Development (BDD)

Cucumber makes it easy to read and understand application flow.

With Behavior Driven Development (BDD), the test scenarios are created before the code is developed. This defines the expected behavior of the system from the user’s perspective and what they want to achieve with the application.

Cucumber is the link between business analysts, non-technical and development resources. Because you don’t need coding experience to write test cases, anyone who understands the application’s logic can contribute to the test process.

Why Cucumber and Selenium?

The Cucumber framework in Selenium allows running automated acceptance tests developed in a BDD format. One of its best features is being able to interpret plain-text descriptions.

Cucumber Selenium testing is run using the tests written in Feature Files, which are non-technical and easy to understand. So, Selenium is actually the testing framework and Cucumber is the BDD tool to test at the user interface layer.

A basic Cucumber test case example is testing a login page that’s based on the following actions:

Given I am on the login page
When I enter valid credentials
Then I should see my account

Cucumber can leverage skills as expertise differs between team members. Some testers could be highly technical, while in the same team, other testers may have little or no technical skill. Regardless of level, every person in the team can still participate in developing automation tests from tests based on business logic and workflow to creating objects and calls, and defining data sources.

Setting up test cases in Selenium can require more programming knowledge whereas Cucumber allows anyone who can read English to write a test case for the web app’s interface. Selenium is an open-source automation testing framework mainly developed on Java. It interacts with the browser interpreting the test instructions from Cucumber.

Differences between Selenium and Cucumber

Selenium is a browser automation testing tool and Cucumber is as a testing tool based on BDD.
Selenium test scripts are written in Java, C#, etc. Cucumber test scripts are written in English text using Gherkin language syntax and is more like documenting procedures.
You can use conditional statements in Selenium but not in Cucumber.
Selenium automates UI testing on browsers while Cucumber enables BDD so you can write the scenarios first starting with the business team.

In Summary

Cucumber and Selenium are both free and available for download. Integrating Cucumber with Selenium leverages skills in DevOps and across business areas for a BDD approach for UI testing. Selenium automates cross-browser testing so having this combination can improve the DevOps CICD process, expedite testing and involve those with more business knowledge.

Automated testing has become essential in reducing repetitive and manual tasks and maintaining software quality. And, because Selenium offers automation, security, usability, and performance, more and more QA testing companies are choosing to go with Selenium to get the advantage on software testing.

Behavior Driven Development is also getting more attention to test important user scenarios with more business input. This approach defines the expected behavior of a feature in plain English text. BDD supports communication between development and business so everyone has a shared understanding of what needs to be developed.

Selenium Automation Testing. Best Practices

Vladislav — Thu, 09 Apr 2020 15:49:36 +0000

What is Selenium Testing

Selenium automated testing is the most popular open-source automated testing tool in the technology industry today. Bringing immense benefits, selenium testing offers robust, browser-based automation for end-to-end testing. So, before you think this is the answer to your testing prayers, its important to remember that great toolsets don’t necessarily guarantee great success.

Ultimately, the best results come from finely tuned implementations and application of best practice methods.

Selenium Best Practices

We could write books about Selenium’s features, but the real value lies in how your DevOps team untaps the value of the powerful Selenium automation testing tool.

Teamwork
Communication and collaboration across DevOps and the business are key to any strategy. Driving the importance of testing with stakeholders support and shared vision promotes increased benefits and opportunities to achieving the best results.

Develop Resources
Selenium can present a steep learning curve without some knowledge of programming, which is recommended to fully grasp and configure its powerful features. Explore the online resources available and discover countless YouTube and tutorial videos to harness Selenium’s potential.

Mentorships
Having a key member skilled and experienced in Selenium is an invaluable resource. Encourage experts to mentor and disseminate their expertise and knowledge to build a solid and valued Selenium testing team.

Establish Robust Processes

Set Naming Conventions
Establishing standard naming conventions for every type of file created helps streamline development and Selenium testing processes. For example:

Don’t use prefixes or suffixes when naming utility classes, instead follow Java format standards.
Differentiate the page object classes from other classes, like the utility or test classes.

Centralized Folder Structures
In a central repository for the Selenium team, organize and define folder structures with established naming conventions to set up a solid foundation for Dev and the Selenium testers. Time wasted on erroneous filenames and code issues has a far greater impact negatively.

Testing Focus

Cross-Browser Testing
Selenium’s cross-browser testing functionality uses the Selenium Web driver to enable the web application to suit different audiences.
In today’s world, there are several popular devices and browsers so testing across all platforms is integral to an application’s all-round market success.

Selenium automated testing executes a test case at one location to run through multiple defined platforms.

Data-Driven Testing
With Selenium testing you can use a Data-Driven Framework, a Keyword Driven Framework or a Hybrid of both. Data-Driven frameworks are more effective in how the data set is separated from actual test code. Input test data is taken from external sources (Excel, CSV, etc.)

Technical Tweaks

Page Object Model (POM)
Using the Page Object Model (POM) is fast becoming a popular design pattern in Selenium automated testing.

POM improves Selenium test maintenance and reduces code duplication and the pattern can be used in any framework. Selenium testing uses the methods from the POM whenever an interaction is required with the User Interface (UI). So, if the UI changes, the tests don’t need to change.

Note: Page Factory initializes the web elements for interactions with the page object when an instance is created.

Benefits of POM:

Achieve code reusability by writing the code once and using it across multiple tests.
A clear distinction between test code and specific code changes.
Readability is improved between test code and page code.

Pause Settings
Browser user operations are asynchronous where it waits for the user to perform an action or for an application to load. Automated testing can easily fail while expecting an event or app, so adding in pause settings helps avoid unnecessary testing failures.
Selenium can handle these time-consuming events through implicit and explicit waiting conditions.

Locators
Locators provide access to HTML elements from a web page. In Selenium, the locators perform actions on links, text boxes, checkboxes.

The Selenium framework interacts with the browser to navigate, click, enter text and select options according to the test using locators that refer to the Class, ID, Link Text, PartPartial Link Textual, Tag Name and XPath. It’s imperative to choose the right locators to avoid tests failing at the smallest change in the user interface.

If locators are present, define unique classes or IDs as Selenium locators.
Using locators is an important strategy to correctly locate elements in Selenium testing.
Testing incorrect components or having Selenium unable to locate them has a disruptive knock-on effect to the automated testing.

Unique IDs
The developer needs to define unique references for web objects that are present on the page. The ID mustn’t be repeated elsewhere although exceptions do apply. The ID is the best used locator in Selenium.

Conclusion

By leveraging effective Selenium automation testing techniques with efficient DevOps processes, the ripple-effect is unstoppable. Marketing efforts are complemented, business objectives are achieved while product stability improves and enhances customer experience.
This allows for more product innovation and less support, which in turn boosts company morale and motivates those who build your products

The Role of Automated Testing in the CI/CD Pipeline

Vladislav — Tue, 22 Oct 2019 13:01:50 +0000

The world is transforming with the upsurge of software applications, mobile devices, and the Internet, so it’s simply not enough to use these elements to support or promote your products and services.

Technology is becoming the lifeblood of every business and the ability to rapidly deliver improved applications and services through systematic software testing is the distinguishing factor. Having the competitive edge means serving customers with better speed and efficiency.

Whether you’re banking, shopping, or looking to travel, you’re probably using a software program to interact with the vendor. And, the vendor is most likely using software to improve operating efficacy and increase productivity.

So, with software being an integral part of our lives, companies need to develop and deliver at a faster pace using refined DevOps practices and automated software testing.

DevOps Environment and Objectives

Conventional development processes have separate teams manage different components of the software lifecycle with focus on delivering a large amount of enhancements and bug fixes on a specified ‘Release Day’.

This process increases the risk product instability and extensive program issues that result in severe business implications.

By contrast, the modern approach has a DevOps team manage the development lifecycle as well as the test and deployment processes.

The focus is to automate slow and manual processes to improve and deliver projects quickly and reliably.

The way to achieve this is through implementing best DevOps practices, such as:

Continuous Integration (CI) is regularly merging code changes into the code repository and then running automated tests and builds. The continuous integration testing process identifies issues quicker, reduces testing validation time and improves the quality of the software.
Continuous Delivery (CD) is an automated process that builds, tests and prepares the software changes for deployment to the testing and/or production environments. The advantage is always being in a deployment-ready state.
Continuous Deployment (CD) extends beyond continuous delivery by constantly deploying code changes to a selected environment.

In essence, frequent and small updates minimize deployment risks and software bugs. So, continuous integration and continuous delivery (CI/CD) validates every code change committed to the repository through an automated and tested build.

Why Automated Testing is Important for the CI/CD Environment

In manual test and release practices, software testing & QA operates as a silo analyzing software changes after all development is complete, which could potentially cause a bottleneck and delays with release. Without any form of automation in the workflow pipeline, code change validation and testing cannot be done instantaneously and continuously.

So, a true CI/CD environment can only truly exist with automated testing in the continuous delivery pipeline.

By moving test and release processes at a faster pace, companies can innovate quicker, be more flexible to change, optimize release to market, and increase efficiency with effective results and business benefits:

Reduce DevOps time spent on manual tasks, improve productivity, and code quality.
Rapid release to market enables a faster response to customer demands.
Enhanced product stability and reliability with earlier bug detection improves release quality.

When implemented properly, automated software testing is secure, scalable, and on-demand, making it an invaluable component in the CI/CD pipeline.

Automated Testing vs Continuous Testing in CI/CD – Key Differences

Automated testing comprises individual test cases automatically executed against newly committed code to achieve a pass or fail result. The first practices of test automation involved executing automated tests after the development phase was complete and program changes were ‘ready for testing’.

This approach is changing as processes evolve with testing in parallel with development. And, that’s where continuous testing comes into the equation.

CI testing (continuous integration testing) is an Agile process where automated tests are constantly run against small but frequent code changes that developers save to the central code repository.

Continuous testing ensures requirements are met, business risks assessed, and product quality is optimized while still in the pipeline.

Tricentis developed the Continuous Testing Maturity Model to analyze and review current testing standards and maturity level within the DevOps processes.

Image source: Tricentis.com

Continuous testing isn’t implemented with a single tool but rather through changes in work habits and behavior, through adopting new technologies and processes.

Moving away from the conventional testing phase after development, automated testing within the CI/CD pipeline provides more value by covering the entire software development cycle.

Ways to Implement Automated Testing in CI/CD

By employing automation testing services or implementing the process internally, the repeatable and scalable tests will improve timeframes for faster product release as bugs are identified quicker and earlier. The test coverage is broad optimizing quality while allowing DevOps to accelerate product innovation.

The following objectives should be considered when implementing automated testing in CI/CD:

Testing is part of the development process from when the first bit of code is saved to the repository through to release.
Every test executes promptly to avoid creating a bottleneck in the pipeline.
Automated tests are comprehensive enough to detect if any code change affects core user functionality that will disrupt the user’s experience.

The tips below will help you get started with defining an automated test strategy. The purpose is to understand all the software’s functionality and distinguish which components are best suited for automated versus manual testing.

Automation Identification It's not realistic to try automate everything at once and automated software testing is not intended to replace every aspect of manual testing. Prioritize repetitive, quick, and simple tests for automation. Additionally, review functionality that is dependent on specific skills and people for testing.
Temporary Test Environments Using cleaned testing environments safeguard the testing integrity by removing any residual data from previous testing.
Managing the CI/CD Pipeline Proper management of the CI/CD pipeline ensures the relevant code change in a central repository goes through automated testing timeously while assessing potential vulnerabilities. A secure, fast and reliable pipeline improves the entire development and release cycles.
Documentation and Versioning Documenting code changes, processes, and test cases is essential to maintaining an accurate depiction of the development and testing infrastructures. Technical documentation and version control are essential to supporting and conserving the CI/CD pipeline.
CI/CD Applications and Tools Choosing and implementing the right tools to manage the CI/CD pipeline is critical to the business. Effective tools are valuable when administered by DevOps in the right environments.

Best Automated Testing Tools for Continuous Delivery

Finding the right tools involves research while making use of product trial periods. Review a tool’s strengths and weaknesses and full capabilities and limitations. It’s important to spend time assessing automated tools and defining the right approach before development starts.

To get started, review the list below for some of the more popular automated testing tools for continuous delivery:

Jenkins is a scalable open source application tool that integrates test execution in the Jenkins pipeline.
Katalon Studio integrates with JIRA and native CI/CD integration and tools such as Bamboo, Jenkins, and Azure.
Bamboo is a continuous integration server used to automatically build, test, and release while integrating with JIRA and Bitbucket.
Selenium is a well-known and powerful open-source framework for automated web testing; however, it requires some programming skill to implement automated testing in the development cycle.
Testsigma is fairly new as a cloud-based tool that supports test automation. It is a user-friendly application aimed at helping manual testers quickly learn automation skills.