DEV Community: Renju Paul Jose

In-Sprint Test Automation Challenges for Regression Testing

Renju Paul Jose — Fri, 10 May 2019 15:59:46 +0000

Introduction of Agile and In-sprint Test Automation

It's been a while since Agile has taken the Software Development realm by storm. Most of the teams adhere(or at least hope to adhere) to the guidelines set by Agile to streamline their Development processes. One of the requisites for that is In-sprint Test Automation. In-sprint test automation aims to develop faster and reliable test automation in parallel to development.

Necessity of In-sprint Test Automation for Regression Testing

The Agile approach doesn't allow much room in the sprint between development and release. So, automating all of the new changes is not possible in the same Sprint might not be easy due to lack of time. Unless these changes are automated, they can't be added to the Regression Test Suite and that hits the coverage. Also, pushing those tasks to the next sprints would increase the burden down the road. Therefore, In-sprint test automation ensures maximum coverage within these short release cycles.

However, there are a few challenges in implementing In-sprint test automation. We'll discuss those in the below section.

Challenges in Implementing In-sprint Test Automation

A. The requirement of a prototype for testing

The UI part is mostly added at the last phase of development after the backend services and features are implemented. We can't waste the time during the development since time is a valuable entity in Sprints. Therefore, the test automation solution should give the freedom to write automated tests even before the prototype is available. This would help in utilizing the whole time that is available between the starting of the development phase and builds.

B. Support for API Tests

As we know, the UI undergoes many changes up until the last minute so it is always recommended to automate the API tests and perform unit tests first so that the bugs can be identified early on in the cycle. Hence, the tools must support different types of tests including unit and API tests along with UI automation.

C. Lack of time to automate in short sprints

As discussed above, All of the development and testing need to be complete in a sprint and sometimes, sprints span for merely a week or two. But as we all know, automation testing demands a significant amount of time not just the automation engineers but the developers and QAs. During short development cycles, it is difficult to spare dedicated time for automation.

D. Lack of a unified continuous testing platform

Credits to modern advancements in development, now the norm is to develop the backend services first and then reuse the same for multiple platforms such as Web, Mobile, REST APIs, OS Native Apps e.t.c. So, every new feature needs to be tested in all these different application types to ensure smooth user experience.

Hence, to automate different application types, it is best to have a unified automated testing framework to avoid managing multiple tools to automate different application types.

E. Tool Extensibility

The modern business is very dynamic and the requirements change very often.
There may be a new design and approach for the product at any time. The automation solution must be flexible enough to enhance and optimize new automation requirements at any time.

F. Demanding technical expertise

Some of the popular automation tools that are still in wide use require niche technical skills for developing the tests. Also, the stakeholders need to be familiar with these to understand the Test scripts.

Most tools demand a steep learning curve and a deep technical understanding. This makes it difficult for the teams to automate testing within a sprint. In order to implement automation testing within short cycles, writing automated tests should be made as simple as possible.

Automation Testing should be easy enough for Functional Testers who are new to the concept or the underlying complexity.

G. Preconfigured Solution

Teams spend a lot of time setting up test environments to execute tests. As the cycles are very short, the timely availability of different tools in the test automation ecosystem is very critical for in-sprint test automation.

Instead, a dynamically scalable pre-configured environment will save a lot of time to focus on the test development process.

An ecosystem with the right set of tools (with important elements like test environments, continuous integration environment, automated build, servers etc.) is a must for In-sprint test automation.

Try Testsigma | Schedule a Demo

How Testsigma addresses all the above challenges

a) Testsigma’s simple English based test script development approach doesn’t depend on application availability. You can start writing automated tests right from the design phase. It also supports both UI and API testing. Here's how this can be done -
In-Sprint Test Automation at Agile and DevOps Speed Using Testsigma

b) Testsigma support REST API Testing as well and you can write Hybrid Test Cases that consist of Web and API Test Steps as required.
REST API steps with normal Test Steps - Example

c) Testsigma’s natural language based approach requires very less time to automate your testing and automating your tests within the sprint now becomes a reality - AI-Driven Automated Testing Tool.

d) Testsigma is a unified continuous testing platform to automate all your applications including Web, Mobile, Native apps and APIs. You don’t need to manage multiple tools to automate different types of applications.

e) Testsigma is simple and flexible and can be customized according to your testing requirements.

f) Testsigma doesn’t demand programming skills to write automated tests but uses simple English to automate your tests. Writing automated tests is as simple as writing manual tests.

g) Testsigma is a scalable pre-configured Cloud-based Testing solution and will save a lot of time since there's no need for additional configurations.

Try Testsigma | Schedule a Demo

Image credit: Business vector created by fullvector – www.freepik.com

Selenium 4.0 alpha released - What to expect from the upcoming stable release!

Renju Paul Jose — Thu, 09 May 2019 14:18:14 +0000

Selenium team has added Selenium 4.0 alpha to mvnrepository two weeks back(24th April) and that has created a buzz in the Automation QA Community.

The QA fraternity had been eagerly waiting for some of the features proposed in this release. The most significant of them would be the W3C Standardization. It would make the Framework much more stable and reduce some of the compatibility issues across different Web Browsers. Another beginner-friendly change that is coming with this major release is updated Documentation. After version 2, Selenium Documentation has not been thoroughly updated. The updated documentation would help the beginners, as well as the experienced, immensely.

We hope some of the drawbacks in the article Pros and Cons of Selenium as an Automation Testing tool will be handled by Selenium 4. Let us look at some of the prominent changes that we can expect from this release. And if you dig in the right place, you can see the complete list of changes in CHANGELOG or CHANGES file - Selenium Github Repo

Note: This is not a complete list and I have deliberately omitted many of the underlying implementation and architectural changes that a regular QA Automation guy like me wouldn't have to deal with on a daily basis 😀

Architectural Changes

1. Now using W3C Protocol for Browser-driver Communications

The earlier versions of Selenium used JSON wire protocol which required encoding and decoding of the API. Whereas, Selenium 4 will now use standard W3C Protocol to communicate between the driver and the browser. The advantage is that W3C protocol doesn’t require encoding or decoding the API and the tests will be able to directly communicate with the Browser.

2. Compliant with W3C WebDriver Specifications

Selenium 4 will be entirely compliant with W3C WebDriver standards and this will include the documentation as well. So, the updated documentation will cover detailed instructions for usage.

Also, The actions API has been revamped to conform with the WebDriver Spec.

3. Removed support for few Browsers

The team will remove native support for Opera and PhantomJS going forward. Still, the users who need to test Opera can rely on Chrome since Opera is based on Chromium(which Chrome is forked from) and for PhantomJS users can use Chrome or Firefox in headless mode.
Selenium Server now no longer includes HtmlUnit by default.

4. Optimized Selenium grid

Alpha version of new Grid Server supports "standalone", "hub",
"node", and completely distributed usage. Also, this new Grid Server can output logs in single-line JSON format to the stdout.

5. Added limited support for Opentracing and Docker

Basic support for OpenTracing landed. Use the --ext flag to the new Grid Server to provide the classpath to an OpenTracing implementation.

Selenium 4 will introduce Basic support for using Docker containers with the new Grid Server. More information will be available once the official documentation is released.

Check out Testsigma to see how most of these things are already considered and made available to all our customers.

Try Testsigma Now!

API changes

1. Changes to Window class

All window manipulation commands are now supported - fullscreen() and minimize() methods will be added.
Added driver.switchTo().parentFrame() - We can use it to go from the child frame to the parent frame directly.
Added getRect() & setRect() and removed getSize() & getPosition() - The getRect() and serRect() methods retrieves and sets respectively, a rect describing the current top-level window's size and position. This will replace the getSize() and getPosition() methods(a change that's not too relevant in common usage)
Added command to open a new window.

2. Changes to Builder class

Added "setChromeService", "setEdgeService", & "setFirefoxService" methods - setChromeService sets the service builders to use for managing the chromedriver child process when creating new Chrome sessions. The others do the same for their respective Browsers

3. Changes to chrome.Driver

Added sendDevToolsCommand() - The sendDevToolsCommand() method sends an arbitrary devtools command to the browser and returns a promise that will be resolved when the command has finished.
Added setDownloadPath() - The setDownloadPath() method sends a DevTools command to change Chrome's download directory and returns a promise that will be resolved when the command has finished.

4. Changes to firefox.Driver

Added installAddon(path) - The installAddon() method installs a new addon within the current session. This function will return an "id" that can be used to uninstall the addon using uninstallAddon().
Added uninstallAddon(id) - Uninstall the Addon using ID.
Added addExtensions() - Add extension to the automation session easily.
Added setPreference() - Add preferences to the session.

5. Changes to Options/Capabilities Class

Options class now extends Capabilities class for Chrome, Firefox, IE, Safari.
Removed the Firefox.Profile class. All of its functionality is now provided directly by Firefox.Options.
setProfile() now only accepts a path to an existing profile
Removed static factory methods android(), ipad(), iphone(), opera(), phantomjs(), htmlunit(), and htmlunitwithjs(). Users can still manually configure capabilities for these, but their use is not recommended and they will no longer be surfaced in the API.

6. Changes to Errors

Added ElementClickInterceptedError - The Selenium code throws ElementClickInterceptedError when a click command cannot complete due to the click target being obscured by other elements on the page.
Added InsecureCertificateError - The Selenium code throws InsecureCertificateError when a navigation event causes the browser to generate a certificate warning. An expired or invalid TLS certificate usually cause this.
Added InvalidCoordinatesError - The Selenium code throws InvalidCoordinatesError when an interaction operation becomes invalid due to the passing of invalid coordinates.
Added NoSuchCookieError - The Selenium code throws NoSuchCookieError when a named Cookie is missing in the cookie jar for the current document.
Removed ElementNotVisibleError
Removed InvalidElementCoordinatesError

7. Changes to WebDriver

Removed touchActions - TouchActions implements actions for touch enabled devices such as doubleTap(), singleTap(), flick(), longPress(), scroll() e.t.c. This might have been moved to Appium library but can be confirmed only once the complete documentation is available.

8. Changes to Alerts

Removed authenticateAs

9. Others

Revamped the actions API to conform with the WebDriver Spec
Element screenshotting might be added
Element screenshots are now possible.
Deleted many of the deprecated methods and classes.

Better and standard Documentation

One of the main issues that haunted the Selenium users was outdated documentation. The Selenium team had not updated the documentation since Selenium 2.0. With Selenium 4, they have updated Selenium's official documentation to explain the latest changes and it conforms to W3C Standards now.

Footnote

While it's great to see the improvements, it would be a hassle for small and medium-sized teams to read through the API docs, analyze the changes, and rectify the tests broken by the updates due to the considerable changes.

Many of the above-discussed changes have already been implemented by us in Testsigma prior to this release as per the feedback or request from our users. That's one of the reasons why cloud-based scriptless tools like Testsigma is more suitable for such teams.
With a cloud-based tool like Testsigma, the updates will be reviewed internally and applied to all the customer's tests automatically without any additional time, resources and efforts from the teams.

Signup to Testsigma for free!

Originally published at Testsigma Blog on May 8, 2019.

Selenium for Test Automation - Aye or Nay?

Renju Paul Jose — Mon, 15 Apr 2019 09:07:05 +0000

Selenium for Test Automation - Aye or Nay?

"Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes but is certainly not limited to just that. Boring web-based administration tasks can (and should!) be automated as well."

As SeleniumHQ stated above, the primary use of Selenium has been to automate Web Applications for Testing purposes and it has been doing the same for a decade or longer. It has been serving Automation Testing needs of testers worldwide for quite a long time and it's a great tool. From Selenium RC and Selenium IDE, Selenium has grown to a full-blown Web Automation library that supports multiple Web Browsers and Platforms.
In this article, let us see some of the factors that led to the widespread adoption of Selenium as the de-facto standard for web automation. Also, we will check the scope for advancement and improving a generic Selenium-based Test Framework.

Reasons for the adoption of Selenium - Pros of Selenium

1. Open Source/ No Licensing costs

Selenium has no upfront licensing costs or payments to be made for its use since its freely available. It is also an Open Source project which allows for extension and modification of the base framework for one's personal or professional usage.

2. Language Independent

Devs or QA Analysts working on a particular language don't have to learn a new language just to start automating their Tests using Selenium. Selenium provides official Language bindings with popular languages such as C#, Java, Ruby, JavaScript, PHP, Python in addition to the unofficial language bindings available for the rest of the languages.

3. Third-party Integrations

Selenium does not restrict a QA's choice of reporting tools, build systems or any other aspect of their Tech Stack. It integrates well with popular tools such as SauceLabs, Selenium-Grid, Extent, JUnit and others.

4. Parallel Testing

Selenium supports parallel Test Executions by integrating with Selenium Grid which helps to considerably bring down the Execution/Feedback time for Test Executions.

5. Cross Browser and Platform Independent

Selenium supports all the popular Web browsers such as Google Chrome, Firefox, Safari, Opera e.t.c and it works on all the available Operating System platforms such as Windows, Mac, and Linux.

6. Community Support

Selenium is backed by a vast community of developers and Testers who actively contribute to the Tool and documentation.

Given above are the reasons which make Selenium a great choice for the Automation Testers in Enterprises all around the world. However, as time flows and technology advances, we need to adapt and adopt newer tools and technologies to keep pace.
One of these advancements that we can't ignore is the impact of Artifical Intelligence(AI). AI is being adopted heavily everywhere and Software Test Automation is one such field that can gain heavily from it. Modern Test Automation Frameworks need to leverage the capabilities of AI better.

That is one of the reasons. Now, let us look into some other reasons that urge us to look for a better alternative to Selenium.

Reasons to start looking for new Test Automation tools - Cons of Selenium

1. High Initial Setup time and costs

It is true that Selenium is free and there are absolutely no licensing costs involved.
However, when it comes to ROI and Initial Cost, we need to think about a couple of other factors.
The setup and configuration of a Selenium-based Test Framework from scratch takes considerable time. That adds up to indirect costs.
For a Testing team that consists mostly of manual Testers at present, the team members need to learn programming or the company needs to hire a bunch of Automation experts to automate those Tests using Selenium.

Proposed Solution: Codeless test automation tools like Testsigma can help the manual testers step up to the automation game easily. And most of these tools come with zero initial setup time as well since they are hosted on Cloud.

2. Not an All-in-One solution - The requirement for 3rd party bindings

Selenium + TestNG/JUnit is not the complete, comprehensive solution to fully automate the testing of your Web Applications. You need different libraries(POI API, GSON, Extent Reports) to make it a complete solution for testing. And managing multiple dependencies is difficult which everyone would love to avoid.

Proposed Solution: A Good dependency management tool such as Maven, Gradle or NPM can make this task a bit easier. ANT is outdated now and I wouldn't suggest it since there are better solutions.

3. The difficulty of managing local Test Infrastructure

Parallel Testing is very much possible with Selenium Grid - However, it is not always feasible to set up and manage a local infrastructure(Test Machines) since the requirements for devices vary with projects.

Proposed Solution: Using a Cloud platform(IaaS) for the Test Environments would avoid the hassle.

4. Eventually becomes a parallel development solution

There's a lot of complexity involved in bringing multiple things together to create an effective and highly functioning test automation ecosystem based on Selenium. It takes huge technical efforts and requires constant updates, improvements, and maintenance of the complete framework. Eventually, it becomes a parallel development project which small and medium-sized companies can not afford in the early stages.

Proposed solution: Use a ready-to-use solution that delegates the maintenance to an external team so that the QA can spend more time on actual issues or bugs.

5. Inability to Integrate Continuous Testing / In-sprint Automation

With its code-based approach and strict dependency on UI, it is difficult for Selenium-based projects to shift the testing more to the left, which is required for continuous testing.

Proposed solution: Frankly, there's no easy solution for this issue since Selenium's dependency on UI is by design and one of the ways to shift left would be to rely on API Tests from the beginning.

6. No participation from non-technical resources

In continuous delivery and DevOps, quality is everyone's responsibility. But with the cryptic code-based approach, only programmers can participate in the test creation activities.

Proposed Solution: Use a Framework or tool that provides highly readable Tests(to be read as BDD or ATDD) and encourages the collaboration of team members.

7. Lack of professional On-demand support for Enterprise Projects

Even though the Selenium community is pretty helping, you can't always rely on Community support for large scale Enterprise projects which demand utmost privacy and quick resolution time.

Proposed solution: Something similar to point 4.

8. Solving Automation/ Maintenance Challenges

There are no inbuilt solutions for specific challenges such as dynamic UI, File handling, API Level tasks, OS-based popup handling, and Basic Authentication
Due to the heavy usage of AJAX and modern web development technologies, Tests are not always very stable. Also, since the Test Scripts are code-based, Test Maintenance is a bit difficult.

Proposed Solution: Adding fallback logic that can handle these challenges with native Javascript code.

9. No Test Management, Test Data management, and Test Reporting Capabilities

No built-in Test management, Test Data management and Test Reporting capabilities for any Selenium plus Test Orchestration library approach.

Proposed Solution: Integrate with a third party Test Management tool and third-party libraries for Test Data management and Test Reporting.

These are some of the reasons that led us(Our team at Testsigma) to think about a new tool that improves upon Selenium. And we have been quite successful in addressing all of the above challenges as well. Learn how - Pros and Cons of Selenium as an Automation Testing tool