DEV Community: DeviQA

Complete Guide on How to Build a Maintainable Selenium Framework

DeviQA — Thu, 28 Jan 2021 10:38:55 +0000

Selenium is second to none when it comes to automation testing tools. Thousands of automation QA engineers efficiently use it every day to facilitate the execution of testing. If you are a newbie and want to get the philosophy of Selenium, there are some tips on how to get the most out of it and create a maintainable and robust Selenium automation framework.

A brief overview of the Selenium framework

Selenium framework can be defined as a set of testing tools, patterns, and practices that provide opportunities for efficient test automation, code readability, and easy code maintenance.

A plenty of automation framework types are differentiated. Still, if you prefer to work with Selenium, you can build three of them:

Data-driven framework splits test script logic off from test data. It gives an opportunity to store test data in external storage. For this purpose, a QA engineer can use MS Excel Sheets, SQL Database, CSV files, text files, XML files, ODBC sources, etc.
Keyword-driven framework speeds up the process of automated testing. Each set of functions and instructions also stored in an external file has a particular keyword. As a result, QA engineers can easily pick up functions for testing.
As its name implies, a hybrid framework is a combination of two previous frameworks. Either test data or keywords are externalized. While keywords are placed in a segregated Java class file, test data is located in an Excel file or Properties file.

When a great volume of data should undergo testing, a data-driven framework is definitely the best choice. If you are going to deal mostly with functionalities, it is better to apply a keyword-driven framework. In situations when a large volume of either data or functions should be tested, it makes sense to leverage a hybrid framework.

Why Do We Need A Selenium Framework?

Selenium framework considerably simplifies the work of automation QA engineers. Code and data would be kept together without a framework, and it would heavily affect reusability and readability. When it is properly developed and implemented, the Selenium framework brings multiple advantages such as code reuse and readability, better portability, easier code maintenance, minimum human intervention during test running, maximum test coverage, easy reporting, and many others.

Merits of the Selenium automation framework

Numerous merits make the Selenium automation framework so popular among QA experts from different corners of the world. Let's have a look at its main advantages to understand why it is an unbeatable automation testing tool:

Availability

Selenium is freely available, and everyone can use it without paying a penny whereas most other solutions are commercial.

Most user actions can be reproduced

Selenium WebDriver can copy the most various user actions. Therefore, QA engineers can automate various inputs like dragging and dropping, mouse-clicking, button pressing, clicking and holding, and so on.

User-friendliness

Selenium automation framework is easily implemented and easy-to-use. Its interface is pretty simple as well. Besides, QA experts can enrich their frameworks with different extensions according to their needs and project requirements.

The most various scenarios can be covered

Selenium provides various options; that is why every QA expert most likely finds something that suits their scenario.

Multilingual capability

It is indeed a helpful feature of Selenium. QA experts enjoy freedom while choosing a programming language. Building the Selenium automation framework they can use Java, C#, JavaScript, Python, Ruby, Perl, or PHP.

Cross-browser compatibility

Selenium supports all commonly used browsers — Safari, Chrome, Opera, Firefox, Edge, Internet Explorer.

Support of various operating systems

Selenium supports such OSs as Windows, Linux, Mac, UNIX, etc.

Integration with numerous tools

To significantly simplify automated testing, Selenium smoothly integrates with numerous tools. For instance, integration with TestNG & JUnit enables test case management and report generation. Integration with Maven, Jenkins, and Docker gives an opportunity for deployment process automation.

Reusability

Created scripts are compatible with different browsers. As a result, QA engineers can run numerous testing scenarios utilizing the same base.

Maintainability

QA experts can quickly modify code and easily maintain scripts due to an opportunity to refactor and group their test cases.

Community Support

The Selenium community is indeed wide and active. As a result, a ton of information can be found, and valuable pieces of advice can be received when QA experts need them.

Unique options

With Selenium WebDriver, QA experts can reproduce clicking on the browser “back” and “forward” buttons. This feature can be pretty useful in some situations and most other solutions do not offer it.

How to build maintainable Selenium framework

A maintainable and reliable test automation framework is 50% of test automation success as it enables QA experts to create effective test scripts. Below you can find some aspects you should take into consideration to build a robust Selenium automation framework.

To create a framework that would satisfy all your needs, it is vital to identify processes that should be automated. That is why you should study the internal structure and workflows of your application to understand which processes can be and must be automated and define requirements for your future test automation framework.

Programming language

As we have already mentioned, Selenium supports plenty of programming languages and you should decide which one to use for your framework design. In this case, we advise you consider a language that has been used for your application development or a language at which most of your colleagues are good.

Framework architecture

Selenium framework architecture consists of the Selenium Core component and the Selenium Test component. It is advised to begin with the Selenium Core component which controls browser instances and element interactions. As soon as your Selenium Core component is ready, you can proceed to the Selenium Test component. You can use a page object model (POM) to design this component. It is indeed very useful due to its ability to eliminate code redundancy and cut down testing expenses.

Unit framework

Unit framework is utilized for test verification and test case grouping and prioritization. Working on your automation framework development you should choose a unit framework that will be integrated. In these terms, TestNG is considered as one of the best unit frameworks for Selenium as it offers multiple benefits, among which are such valuable features as parameterizing, sequencing, annotations, and grouping.

CI/CD implementation

To get the most out of your automation framework you should also consider CI/CD implementation. In these terms, a range of helpful tools should be integrated with your framework:

Dependency managers like Maven, NuGet, Gradle, NPM, and Ant give an opportunity to efficiently manage dependencies and librarie. Build tools also come in handy when it comes to source code writing, dependent library creation, and test running.
Version control systems as GitHub, Bitbucket, or TFS significantly simplify work on the source code of your tests. Still, if your source code should be kept private, it is highly advised to implement your own version control system using Git.
To speed up product delivery and get a competitive advantage, it is essential to integrate automated tests into CI/CD pipelines. That is why, integration with one of the CI tools such as Jenkins, TFS, or Bamboo is a must.
- Report generation

A reliable test automation framework should also provide test results generating accurate testing reports. These reports should be clear, readable, and available for either technical or non-technical staff who need them. As Selenium does not provide a built-in report option, you should integrate an external tool. For instance, you can utilize TestNG or JUnit. However, reports generated by these tools are not readable. To create human-readable reports, external libraries like ExtentReport or Allure should be used.

Conclusion

Test automation can significantly simplify and optimize testing processes. However, to achieve excellent results, it is essential to prepare a strong foundation, i.e., create an effective test automation framework. In these terms, Selenium is the best choice. It provides almost everything you need, while options that are not available still can be used due to integration with multiple tools. While building your Selenium automation framework, you should take into consideration numerous aspects as project requirements, processes that should be automated, programming language, framework architecture, unit framework, report generation, CI/CD implementation, etc. Still, at the end of the day, all efforts will be justified because a well-thought framework enables QA team members to create effective automated scripts and maintain them easily.

Manual Testing Guide

DeviQA — Tue, 29 Sep 2020 08:40:32 +0000

Testing is one of the most important stages of the software development life cycle. These days, there are two main approaches to software testing: manual and automated. Most QA teams skillfully combine both approaches as they are not entirely interchangeable and have either advantages or disadvantages. We are going to consider manual testing in detail. So let's start from scratch and consider the manual testing basics.

Definition of manual testing

Manual testing is a software examination and utilization from the end-user perspective to assess its quality, detect existing defects, and uncover possible weaknesses. In contrast to automated testing that utilizes a software program to conduct tests, manual testing reckons upon human's actions.

Manual testing process

As a rule, to provide professional and comprehensive testing, most manual testers organize their workflow in the following way:

Requirement studying

Manual testers must get acquainted with the requirements to ensure a successful QA process. Thus, they can understand the software's objective, operation principles, main functions, and possible vulnerabilities. Testers can make up excellent test cases and identify more defects using this information. However, project specifications and user stories are not always available. In this case, manual testers should be more inventive and check other information resources to realize how an application should work.

Drawing up of test cases

As soon as manual testers have gotten acquainted with project requirements, they start to draw up test cases using various test design techniques. Test cases are used as instructions for manual testers, as they contain the consecutive steps for a particular function or scenario testing. They should be rather detailed. Therefore, other testers will also be able to carry out your tests, not asking clarification questions.
Some manual testers still use an Excel document to file their test cases. However, various case management tools are available; they can significantly simplify test case creation, storage, and management.

Useful articles:
How to write test cases
Manual Test Cases Writing Hacks

Test conduction and bug reporting

After all test cases have been made up, and the test environment has been prepared, manual testers go ahead with test conduction. If manual testers detect bugs during testing, they must log bug reports using a bug tracking system. The bug reports must also be detailed and contain reproduction steps, an expected result, an actual result, and relevant attachments (screenshots, screen recording, etc.). Therefore, a developer can get bug essence right and quickly reproduce it.

Analysis of testing results

As a rule, after testing completion, manual testers undertake an analysis to receive useful metrics that can help make the right consequences and plan future activities. It can take just a few clicks if you use one of the various test case management systems.

Types of Manual Testing

In our manual testing guide, we cannot leave out the manual testing types. Frankly speaking, there are countless manual testing types, but we will consider the most widely used.

Exploratory Testing

Generally, manual testers turn to exploratory testing when they want to understand the basic idea of the software or get acquainted with its new features. It has no guidelines, and testers explore systems following their own initiative and curiosity. Exploratory testing is carried out whenever a tester thinks it is required.

Usability Testing

There is no chance to effectively carry out usability testing without engaging people because the user-friendliness of software is evaluated on the ground of users behavior and emotions. During usability testing, manual testers make sure that an application is simple, convenient, and intuitive to avoid end-users confusion and frustration.

GUI testing

In the process of GUI testing manual testers check fonts, color palette, windows, buttons, and all other GUI elements and characteristics to guarantee that there are no errors or confusions and provide the best user experience.

Localization

Manual testers perform localization testing to ensure that an application meets various language and cultural requirements. Localization testing is particularly important if an application is targeted at a global user community.

Compatibility testing

Compatibility testing is conducted to ensure that an application works properly with other software, different hardware, network, etc. Compatibility testing is rather time-intensive. For this reason, most QA teams tend to automate this process. Still, if there is a need to carry it out manually, it is advised to define the top-priority browsers and platforms and concentrate on them.

Smoke Testing

Smoke testing verifies only the key functionality of the application to define whether it corresponds to its primary objective and does not contain critical bugs. Smoke testing is a kind of preliminary checking that is carried out before more critical, in-depth testing.

Regression Testing

Regression testing is executed before every release to check whether updates have not affected existing functionalities. Regression testing of large, sophisticated applications can be extremely time-consuming. For this reason, if there are opportunities, many companies try to automate it.

Main Manual Testing Techniques

Three manual testing techniques are differentiated. So, let's consider each of them one by one.

Black box testing

Black box testing is conducted in a way that resembles end-users interaction with a system. It doesn't require knowledge of the internal structure and coding skills. Conducting black-box testing, QA engineers interact with UI, test functionality, and verify whether outputs coincide with expected results.

White box testing

White box testing technique is usually utilized by developers to review code and dataflows and detect possible hidden bugs. This technique requires good coding skills.

Grey Box testing

This manual testing technique is a balanced combination of either black box testing or white box testing. Grey box testing covers all aspects and deals with everything but in a more superficial manner.

Common Myths of Manual Testing

A manual tester profession appeared quite a long time ago, and a considerable number of myths have grown around it. Right now, we will debunk the most common myths.

Manual testing is easy

It is probably the most widely spread myth. According to mainstream thinking, everybody can do testing. Let me disagree. Manual testing is a sort of art. A special mindset and skills are required to conduct testing as it is not just a random clicking, as some people may think. A manual tester must be extremely detail-oriented, obsessed with product quality, think extraordinary, know different software development aspects, and many others. Not everyone can be a professional manual tester.

Manual testing is out-of-date

This misbelief is becoming very popular these days. Some people think that manual testing has become out-of-date, and automated testing will completely replace it very soon. We must reassure you that manual testing is the base of the QA process. Automated testing would never displace it. Manual testing helps make a stable product, and only after that, automated testing gets into a game. Automated testing is impossible without previous manual testing.

Manque programmers choose the testing profession

It's worth saying that not every programmer can become a good tester while many manual testers successfully master programming to understand all aspects of software operation and move to a new level.

Testers get down to work only on the post-development stage

There is an opinion that manual testers idle most of the time and get to work in the end. It is a blunder. If some companies indeed use this approach, the project has severe problems. The involvement of a QA team at a final stage is a big risk for product quality and schedule. Testers must get to work as early as possible to have enough time to study documentation, analyze risks, draw up and conduct test cases.

Importance of Manual Testing

As was mentioned earlier, some people have doubts about the importance of manual testing. Nevertheless, any professional QA engineer can easily refute this statement. There are many reasons why manual testing is indispensable, so let's consider some of them.

Certain tests must be conducted only manually

User experience plays an important role in quality assurance. In some situations, just a human can properly conduct certain tasks and provide an appropriate assessment. For instance, it is impossible to imagine the conduction of usability testing or localization testing without human involvement. Compatibility and interactions with mobile devices can't be checked with automated testing as well. Application performance during wi-fi turning off and turning on, simultaneous running with other apps, incoming calls or messages, etc. can be checked only by a human too.

Automated testing only supports human testers

Automated testing cannot substitute manual testing but can be and even must be effectively used in combination with it. While automated tests save time with repetitive tasks, manual testing efforts can concentrate on more creative work.

Most bugs are hidden in unexpected places

Even keeping up with the previously prepared test cases, manual testers can still uncover bugs in places they have not expected to find. That is very important. Testers detect many bugs while they are checking something else. In its turn, automated testing can't spot defects that it hasn't been programmed to search.

Automation is not budget-friendly for small-scale projects

Automation requires not only considerable initial investments but also high maintenance expenses. For large scale projects, automation can be indeed beneficial in a long-term perspective. However, for small projects, automation is still a waste of either time or money.

Automation often falls behind sprints

Due to continuous script updating, it is difficult for automated testing to keep up with sprints. Only perfectly managed automation can manage to do this.

Manual testers assess product from the end-users perspective

Manual testers usually act as end-users, so they not only ensure correct work of application and help define new ways of product improvement with their pieces of advice and suggestions.

Repetition and variability are of great importance

Repetition and variability are two characteristics of effective testing. Automated testing is appropriate for the repetitive testing process, but it's not enough. Manual testing, in its turn, can provide needed variability.

Top Manual Testing Tools List

Manual testers have at their disposal various useful tools that considerably simplify the testing process. Here you can find the most widely used.

Test Case Management Tools (TestRail, Zephyr, Practitest, TestLink)

These tools are handy and used by manual testers to easily create, manage, update, and store test cases.

Related article: Best Test Management Tools

Bug Tracking Tools (Jira, Trello, Bugzilla, Mantis)

Bug tracking solutions are indispensable helpers of manual testers. With their help, QA engineers create, modify, store and manage bug reports as well as get valuable statistics.

Related Article: Best Bug Tracking Tools

Cross Browser Testing Tools (BrowserStack, Sauce Labs, Browser Shots)

These tools help manual testers quickly verify whether the UI elements are rendered correctly across multiple browsers and how the code responds in each browser.

Related Article: Best Cross Browser Testing Tools

Screen Capture Tools (CloudApp, Loom, Skitch)

Manual testers often have to attach images and videos to bug reports to describe an issue more precisely. These tools help to create needed images and screen recordings quickly.

Conclusion

Manual testing involves much effort and can also take much time. Nevertheless, it is essential to ensure a high level of software quality. Automated testing provides fantastic opportunities, but despite its advantages, automated scripts can test just what they have been written to test, while manual testing is more adaptable and creative. A human tester can always find things that an automated test cannot. To gain the best results and guarantee the top-notch quality, QA teams must use a combination of either manual or automated testing, as they are two sides of the same coin.

A Guide to automated testing for microservices

DeviQA — Wed, 22 Apr 2020 08:47:01 +0000

Today a lot of applications use microservice architecture, but what should we know and how we should test this type of software? In this article, we will consider basic theoretical concepts about CDC testing and testing tools that will be helpful.

Microservices architecture - is a collection of small services and each service responsible for certain functionality and can be written by using different programming languages, approaches, and solutions. One of the main goals of microservices-based application it's define strong contracts between the various microservices.

Consumer-driven contract testing (CDC) - is a technique that allows us to test each application in isolation. After executing consumer tests we get generated pact files that will be used for getting verification results during executing provider tests. As a result, we will have artifacts that will be placed in Pact-broker.

Let’s familiarize ourselves with the main concepts from the official sources.
Pact - is a code-first consumer-driven contract testing tool, and is generally used by developers and testers who code. In general, a contract is between a consumer (for example, a client that wants to receive some data) and a provider (for example, an API on a server that provides the data the client needs).
Pact Broker - is an application for sharing for consumer driven contracts and verification results. It is optimized for use with "pacts" (contracts created by the Pact framework), but can be used for any type of contract that can be serialized to JSON. The Pact Broker is an open source tool that requires you to deploy, administer and host it yourself but also they provide plug-and-play option at pactflow.io.
Mocha - is a feature-rich JavaScript test framework running on Node.js and in the browser, making asynchronous testing simple and fun. Mocha allows you to use any assertion library.
Chai - is a BDD / TDD assertion library for node and the browser that can be delightfully paired with any javascript testing framework.
Axios - promise based HTTP client for the browser and node.js.

Simple example of CDC testing.

Here is a code snippet from package.json where you can see all packages that will be used.

{
"@pact-foundation/pact": "^9.9.3",
"axios": "^0.19.0",
"chai": "^3.5.0",
"mocha": "^5.1.1"
}

Step 1. Consumer tests.

There are two different specs that will be describing TODO API - “JSONPlaceholder” and USER API - “REQ | RES” . The first example you can see below. Second spec the same except the request and response objects.

todoClient.js


const getUserById = (id, baseURL) => {
   return axios.request({
       method: "GET",
       baseURL,
       url: `/users/${id}`,
       headers: { Accept: "application/json" },
   })
};

todo.spec.js


describe("The Todo API", () => {
 const mockServerURL = 'http://localhost:1234';

 // Create the Pact object to represent your provider
 const provider = new Pact({
   port: 1234,
   log: path.resolve(process.cwd(), "logs", "mockserver-integration.log"),
   dir: path.resolve(process.cwd(), "pacts"),
   consumer: "Consumer",
   provider: "Todo Provider",
   spec: 2,
   logLevel: "error",
   pactfileWriteMode: "merge"
 });

 // this is the response you expect from your Provider
 const EXPECTED_BODY = {
   userId: 1,
   id: 1,
   title: 'delectus aut autem',
   completed: false
 };

 // Start the mock server
 before(() => provider.setup());

 // Write Pact when all tests done
 after(() => provider.finalize());

 // verify with Pact, and reset expectations
 afterEach(() => provider.verify());

 describe("Get todo by id", () => {
   before(async () => {
     const interaction = {
       state: "Get todo",
       uponReceiving: "A request for one todo",
       withRequest: {
         method: "GET",
         path: "/todos/1",
         headers: {
           Accept: "application/json",
         },
       },
       willRespondWith: {
         status: 200,
         headers: {
           "Content-Type": "application/json; charset=utf-8",
         },
         body: EXPECTED_BODY,
       },
     };
     // add interactions to the Mock Server
     await provider.addInteraction(interaction)
   });

   it("GET /todos/1", async () => {
     let { data } = await getTodoById(1, mockServerURL);
     expect(data).to.eql(EXPECTED_BODY);
   });
 })
});

After running tests we will receive the following pact files:
consumer-user_provider.json
consumer-todo_provider.json:

{
 "consumer": {
   "name": "Consumer"
 },
 "provider": {
   "name": "Todo Provider"
 },
 "interactions": [
   {
     "description": "A request for one todo",
     "providerState": "Get todo",
     "request": {
       "method": "GET",
       "path": "/todos/1",
       "headers": {
         "Accept": "application/json"
       }
     },
     "response": {
       "status": 200,
       "headers": {
         "Content-Type": "application/json; charset=utf-8"
       },
       "body": {
         "userId": 1,
         "id": 1,
         "title": "delectus aut autem",
         "completed": false
       }
     }
   }
 ],
 "metadata": {
   "pactSpecification": {
     "version": "2.0.0"
   }
 }
}

Step 2. Publishing pact files. In this example we will use pactflow.io (Developer FREE plan).

publish.pacts.js


const gitHash = require('child_process')
    .execSync('git rev-parse --short HEAD')
    .toString().trim();

const opts = {
   pactFilesOrDirs: ['./pacts/'],
   pactBroker: 'https://yourCompanyName.pact.dius.com.au',
   pactBrokerToken: 'yourPactBrokerToken',
   tags: ['prod', 'test'],
   consumerVersion: gitHash
};

pact
   .publishPacts(opts)
   .then(() => {
       console.log('Pact contract publishing complete!');
    })
   .catch(e => {
       console.log('Pact contract publishing failed: ', e)
   });

For getting pactBrokerToken you need to proceed to /settings/api-tokens and copy Read/write token (CI)

After running this script we will receive the following results:

Step 3. Pacts Verification


provider.spec.js
const opts = {
   logLevel: "INFO",
   providerVersion: "1.0.0",
   pactBrokerUrl: 'https://yourCompanyName.pact.dius.com.au',
   pactBrokerToken: 'yourPactBrokerToken',
   publishVerificationResult: true
};

describe("Pacts Verification", () => {
   it("Validates the expectations of Todo provider", () => {
       return new Verifier({
           ...opts,
           provider: 'Todo Provider',
           providerBaseUrl: 'https://jsonplaceholder.typicode.com',
       }).verifyProvider();
   });

   it('Validates the expectations of User provider ', () => {
       return new Verifier({
           ...opts,
           provider: 'User Provider',
           providerBaseUrl: 'https://reqres.in/api',
       }).verifyProvider();
   });
});

After running tests we will receive the following results:

VIEW PACT:

Let’s change the title in the expected result from step 1 for simulating an error.

Previous object:


 const EXPECTED_BODY = {
   userId: 1,
   id: 1,
   title: 'delectus aut autem',
   completed: false
 };

New object:


 const EXPECTED_BODY = {
   userId: 1,
   id: 1,
   title: 'Our new title that will trigger the error',
   completed: false
 };

Result after running tests:

Console output:

Conclusions.

Testing microservices it's a very difficult process that requires a good knowledge of different testing tools for achieving needed goals. In this article, we considered basic concepts and simple example of writing CDC tests using Pact. You can deeply familiarize yourself with this powerful testing tool on the official site and use all provided features for testing your microservices.