DEV Community: Ruchira Shukla

How To Run JUnit Selenium Tests using TestNG

Ruchira Shukla — Fri, 05 Aug 2022 09:49:23 +0000

Both JUnit and TestNG are popular unit testing frameworks that have been widely accepted by Java developers. JUnit was introduced as an open-source unit testing framework for Java way back in 1997. In fact, JUnit is one of the widely used test automation frameworks for test automation. TestNG is another Java-based test automation framework that is not only open-source but also offers awesome features that are best suited for large-scale web automation testing. TestNG was created for a range of testing categories, including (but not limited to) unit testing, functional testing, end-to-end testing, and integration testing.

Like me, if you are well-versed with both the test automation frameworks, you might come across scenarios where you would want to run JUnit tests using TestNG. The major push for this cross-pollination (to put it in simple terms) is the extensive set of features provided by the TestNG framework. TestNG supports broader categories of tests, making it easy for testers to migrate the JUnit test cases to the TestNG framework. The advantage of this approach is avoiding complete rewrite of the test scenarios that are using the JUnit framework.

In this blog, we look at how you can run test cases using JUnit and TestNG in Selenium without any re-work in writing the tests.

Hey! Test on Selenium Testing Grid Cloud of 3000+ Desktop & Mobile Browsers.

Introduction to JUnit

Java came up with its open-source unit testing framework called JUnit, following the practice of extreme programming. Though JUnit is primarily used for unit testing, automated testing with JUnit and Selenium can be extensively used for cross browser testing (or browser compatibility testing).

On the other hand, the TestNG (Test Next Generation) framework has advanced features that let you perform various categories of tests (i.e., unit testing, integration testing, functional testing, etc.).

How to install JUnit

The latest version of JUnit is JUnit 5. Unlike JUnit 4, JUnit 5 is composed of modules from three different projects — JUnit Platform, JUnit Jupiter, and JUnit Vintage.

JUnit Platform — Provides the base for launching testing frameworks
JUnit Jupiter — Provides a TestEngine for running Jupiter-based tests on the platform
JUnit Vintage — Provides a test engine for running Unit 3 and JUnit 4 tests

In case you are using JUnit 4 tests and intrigued to know about running with the JUnit 5 framework, do have a look at our blog that helps you execute JUnit 4 tests with JUnit 5 framework.

To get started with the JUnit framework, follow the below-mentioned steps:

To install JUnit in your project, navigate to the JUnit 5 website → Docs → User Guide and download the dependencies.
To use JUnit, include the jar files in your project ClassPath.
To use JUnit in Eclipse, include JUnit libraries in your Java project. For example, Right-click on the Java project → Properties → Java Build Path → Add Libraries → Select JUnit → Choose library version (JUnit 4/JUnit5).

You can check our getting started with JUnit blog to take the first steps in using the framework for Selenium test automation.

How to create and execute test cases in JUnit

JUnit provides a handful of annotations to create and execute test cases. Therefore, it is important to have a good understanding of JUnit Annotations in Selenium if you are using the framework for Selenium automation testing.

Now that we have covered the essentials of JUnit, let’s get our hands dirty with a JUnit test example that demonstrates Selenium test automation:

Test Scenario

Navigate to https://todomvc.com/examples/react/#/
Enter the text “JUnit Parallel Test sample” in the textbox
Validate that the exact text appears in the list below the textbox

Implementation

To implement the above test case, we will write JUnitDemo test class:

package Test;
    import org.openqa.selenium.By;
    import org.openqa.selenium.Keys;
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.chrome.ChromeDriver;
    import org.junit.After;
    import org.junit.Assert;
    import org.junit.Before;
    import org.junit.Test;

    public class JUnitDemo {
       WebDriver driver;

       @Before
       public void setUp() throws Exception {
           System.setProperty("webdriver.chrome.driver", "YOUR DRIVER EXE PATH");
           driver = new ChromeDriver();

       }

       @Test
       public void testJunitDemo() throws Exception {
           try { // Change it to production page
               int total_elements = 0;
               driver.get("https://todomvc.com/examples/react/#/");
               driver.findElement(By.className("new-todo")).sendKeys("JUnit Test sample");
               driver.findElement(By.className("new-todo")).sendKeys(Keys.ENTER);
               total_elements = driver.findElements(By.xpath("//ul[@class='todo-list']/li//label")).size();
               Assert.assertEquals(1, total_elements);

           } catch (Exception e) {
               System.out.println(e.getMessage());
           }
       }

       @After
       public void tearDown() throws Exception {
           driver.quit();
       }

    }

Code Walkthrough

@Before and @After annotations specify that the method code will be executed before and after each test method. An assert is not raised if the expected result matches the actual result.
The test method testJUnitDemo contains the Selenium script to perform the test steps. If you are using the JUnit framework, you should leverage parameterized testing with JUnit to make the most out of Selenium and JUnit as a complete package!

How to run JUnit Test

1- To run the above code in Eclipse, right-click on the test class and select Run As → JUnit Test.

2- To run the test case outside Eclipse, you can create a TestRunner class.

3- You can include more than one test using runClasses testclass1,[testclass2,…].

    import org.junit.runner.JUnitCore;
    import org.junit.runner.Result;
    import org.junit.runner.notification.Failure;
    public class TestRunner {
      public static void main(String[] args) {
         Result result = JUnitCore.runClasses(JUnitDemo.class);
         for (Failure failure : result.getFailures()) {
            System.out.println(failure.toString());
         }
         System.out.println(result.wasSuccessful());
      }
    }

4- To run in the command line, follow the syntax-

    $ java org.junit.runner.JUnitCore TestClass1 [TestClass2 ...]

You can refer to our detailed JUnit Tutorial on the LambdaTest learning hub to gain insights into the other offerings of the JUnit framework.

Watch this video to learn how TestNG has become one of the most robust test automation frameworks and all you need to know to get started with TestNG in Selenium.

Introduction to TestNG

TestNG is also a very prevalent Java testing framework that is widely used by the developer and tester community. The framework is extensively used for test creation with an immense focus on features such as grouping, dependency, prioritization, annotations, and more. At the time of writing this blog, the latest version of TestNG is version 7.3.0.

With TestNG certification, you can challenge your skills in performing automated testing with TestNG and take your career to the next level.

Here’s a short glimpse of the TestNG certification from LambdaTest:

Automate Cypress testing and perform browser automation testing with LambdaTest Now!

How to install TestNG

Head over to the official website of TestNG and copy the dependencies (Maven/Gradle) to your project. In case you are using the Eclipse IDE, you can refer to our comprehensive guide on how to install TestNG in Eclipse.

Once you have installed Eclipse on your system, you have to create a project and download the corresponding dependencies. You can check out our blog to help you create a TestNG project In Eclipse for Selenium test automation.

How To Create and Execute Test Cases in TestNG

To demonstrate the usage of the TestNG framework for web automation testing, we use the same test scenario that we had demonstrated in the earlier section. The browser and platform combination also remains unchanged.

Test Scenario

Navigate to https://todomvc.com/examples/react/#/.
Enter the text “TestNG Sample Test” in the textbox.
Validate that the exact text appears in the list below the textbox.

Implementation

  package Test;

    import static org.testng.Assert.assertTrue;

    import java.net.MalformedURLException;
    import java.net.URL;

    import org.junit.Assert;
    import org.openqa.selenium.By;
    import org.openqa.selenium.Keys;
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.chrome.ChromeDriver;
    import org.testng.annotations.AfterClass;
    import org.testng.annotations.AfterMethod;
    import org.testng.annotations.BeforeClass;
    import org.testng.annotations.BeforeMethod;
    import org.testng.annotations.Test;

    public class TestNGDemo {
       public WebDriver driver;

       @BeforeClass
       public void setUp() throws Exception {
           System.setProperty("webdriver.chrome.driver", "YOUR CHROME DRIVER EXE PATH");
           driver = new ChromeDriver();
       }

       @BeforeMethod
       public void beforeMethod() {
           System.out.println("Before Method Setup");// for illustration
       }

       @Test
       public void testNGDemoTest() throws Exception {
           try { // Change it to production page
               int total_elements = 0;
               driver.get("https://todomvc.com/examples/react/#/");
               driver.findElement(By.className("new-todo")).sendKeys("TestNG Sample Test");
               driver.findElement(By.className("new-todo")).sendKeys(Keys.ENTER);
               total_elements = driver.findElements(By.xpath("//ul[@class='todo-list']/li//label")).size();
               Assert.assertEquals(1, total_elements);

           } catch (Exception e) {
               System.out.println(e.getMessage());
           }
       }

       @AfterMethod
       public void afterMethod() {
           System.out.println("Äfter Method");
       }

       @AfterClass
       public void tearDown() throws Exception {
           driver.quit(); // for illustration
       }

    }

Code WalkThrough

@BeforeClass and @AfterClass annotation specify that the method code will be executed before and after the class is invoked. Generally, we specify configuration methods, initialize and close driver instances, etc., in these annotations.
@BeforeMethod and @AfterMethod annotation is used to invoke the method before and after execution of each of the test methods.
@test annotation denotes the test methods same as in JUnit.
assertEquals() method passes if the expected result matches with the actual result.

Watch this video to learn about the TestNG Annotations and how they help provide better structure and readability to the code.

How to run TestNG test cases

TestNG test cases can be run via testng.xml or through any build tool (e.g., Maven, Ant, or Gradle). testng.xml gives more flexibility for test case execution as compared to the JUnit framework.

   <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
    <suite name="TestNGTestCaseDemo">
     <test name=""TestNGSeleniumDemo >
       <classes>
          <class name="Test.TestNGSeleniumDemo/>
       </classes>
     </test>
    </suite>

You can run the XML file in Eclipse or by triggering the following command from the terminal:

   $ java org.testng.TestNG testing.xml

Make sure that the TestNG jar file path is set to Java ClassPath before running the tests from the command line.

If you are intrigued to learn more about the other essentials of the TestNG framework, head over to the detailed TestNG tutorial on LambdaTest learning hub.

This JUnit certification establishes testing standards for those who wish to advance their careers in Selenium automation testing with JUnit.

Here’s a short glimpse of the JUnit certification from LambdaTest:

How To Run JUnit Tests using TestNG in Selenium

Before jumping on ‘How,’ let us first understand ‘why’ we need to run test cases using JUnit and TestNG in Selenium test automation. Here is the simplest answer to the above question
‘Provision of extensive capabilities of TestNG over JUnit.’ TestNG was released much later than JUnit but offers numerous features like improved manageable test execution capabilities, better annotations to provide flexibility, and more.

Here is a brief rundown of the TestNG vs JUnit comparison:

1- Annotations: TestNG offers more annotations to make automation test case management more flexible when compared to JUnit. Although JUnit 5 has introduced a few new annotations to achieve more flexibility in test case management, we still need to see how it performs compared to JUnit.

2- Grouping of Test Cases: TestNG supports the grouping of test cases if you wish to run only a set of test cases. This can be achieved with the help of groups parameters in @test annotation. The same tag is specified in testng.xml with the name of the groups you wish to run.

3- Parallel test case execution: TestNG supports parallel execution of test cases using multiple threads and classes using the “parallel” keyword in testng.xml

   junit.jupiter.execution.parallel.enabled = true
    junit.jupiter.execution.parallel.mode.default = concurrent

Here concurrent mode specifies parallel threads to be run for each test method. We can also set it to ‘default’ if we want a single thread for all methods. Test methods in such test classes are only executed concurrently if the @execution(CONCURRENT) annotation is present on the test class or method. Parallel test execution in TestNG is preferred since it helps you achieve test results at a much faster pace.

4- Test case dependency: The dependsOnMethods parameter in @test annotation is used to define the dependent method (or a group of methods). This is mostly used where one test method depends on the output of another test method(s). However, dependent tests is not a good practice when it comes to Selenium automation testing. Hence, it is recommended to follow Selenium best practices to make the most out of the test automation framework.

You can achieve dependency injection in JUnit 5, where the test methods are permitted to have parameters.

As seen until now, TestNG has many advantages over JUnit 4 (which is still in extensive use). Many of us would like to harness the capabilities of TestNG so that tests can be partially or fully migrated to TestNG. Now the question is, How do you run test cases using JUnit and TestNG in Selenium?

You might have ’N’ number of unit tests written in JUnit, which are integrated with the CI/CD pipeline. The test framework, the CI/CD pipeline, reporting, and everything are in place and running well before each release. Re-writing those test cases would lead to a significant increase in the costs. So what is the ideal solution to make the most out of JUnit, TestNG, and the corresponding CI/CD tool used in the project?

Well, TestNG solves this problem by recognizing a number of JUnit test examples through the JUnit tag in the XML file. To put it in simple words, TestNG has built-in support for running JUnit test cases. We will discuss this in detail with the same example with the help of LambdaTest’s cloud Selenium Grid.

This JUnit Tutorial for beginners and professionals will help you learn how to use JUnit framework with Selenium and Java for performing Selenium automation testing.

Parallel Test Execution of JUnit Test Cases in TestNG Using Remote Selenium Grid

Parallel testing in Selenium helps you achieve better test coverage at an expedited pace. Though parallel test execution is not a key feature of JUnit, it can be achieved using certain methods, which we further discuss in this section.

To make the execution of JUnit test cases more efficient, we can execute them in TestNG using a cloud-based remote Selenium Grid. Executing test cases using JUnit and TestNG in Selenium can be achieved in the same way as discussed earlier. However, a cloud-based Selenium Grid would ensure secure, fast, and stable test execution.

For demonstrating how to run JUnit tests using TestNG on a cloud Selenium Grid, we consider the following test scenario:

Test Scenario

Navigate to https://todomvc.com/examples/react/#/.
Enter the text “JUnit Parallel Test sample” in the textbox.
Validate that the exact text appears in the list below the textbox.

For demonstrating the potential of parallel testing with JUnit and TestNG using Selenium cloud grid, we run the same test on multiple browser and platform combinations:

Browser	Version	Platform
Chrome	70.0	WIN10
Safari	14.0.2	macOS Big Sur
Firefox	76.0	WIN8

Let us first write the JUnit code to support the parallel execution of threads for each of the browser-platform combinations. To run a JUnit test on different browsers simultaneously, you will need to create a helper class that extends parameterized class (org.junit.runners.Parameterized) and implements RunnerScheduler class.

To demonstrate this, we are using LambdaTest cloud-based Selenium Grid that offers Selenium Automation Testing With JUnit across 2000+ browsers and operating systems.

   package Test;
    import java.util.concurrent.ExecutorService;
    import java.util.concurrent.Executors;
    import java.util.concurrent.TimeUnit;
    import org.junit.runners.Parameterized;
    import org.junit.runners.model.RunnerScheduler;
    public class Parallelized extends Parameterized {
       private static class ThreadPoolScheduler implements RunnerScheduler {
           private ExecutorService executor;
           public ThreadPoolScheduler() {
               String threads = System.getProperty("junit.parallel.threads", "15");
               int numThreads = Integer.parseInt(threads);
               executor = Executors.newFixedThreadPool(numThreads);
           }

           public void finished() {
               executor.shutdown();
               try {
                   executor.awaitTermination(10, TimeUnit.MINUTES);
               } catch (InterruptedException exc) {
                   throw new RuntimeException(exc);
               }
           }

           public void schedule(Runnable childStatement) {
               executor.submit(childStatement);
           }
       }
       public Parallelized(Class<?> klass) throws Throwable {
           super(klass);
           setScheduler(new ThreadPoolScheduler());
       }
    }

Now we will write the script to implement the above-mentioned test scenario. The Parallelized class will be passed as a parameter in @RunWith annotation. To execute the test case on LamdaTest, you have to provide our username and access key, which is available in the LambdaTest profile section.

    package Test;
    import org.openqa.selenium.By;
    import org.openqa.selenium.Platform;
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.remote.DesiredCapabilities;
    import org.openqa.selenium.remote.RemoteWebDriver;
    import org.junit.After;
    import org.junit.Assert;
    import org.junit.Before;
    import org.junit.Test;
    import org.junit.jupiter.api.RepeatedTest;
    import org.junit.runner.RunWith;
    import org.junit.runners.Parameterized;
    import java.net.MalformedURLException;
    import java.net.URL;
    import java.util.ArrayList;
    import java.util.LinkedList;
    import java.util.List;

    [@RunWith](http://twitter.com/RunWith)(Parallelized.class)
    public class JunitParallelExecutionDemo {
       public static String username = "YOUR_USER_NAME;
       public static String accessKey = "YOUR_ACCESS_KEY";
       public String gridURL = "[@hub](http://twitter.com/hub).lambdatest.com/wd/hub";
       public String platform;
       public String browserName;
       public String browserVersion;
       public RemoteWebDriver driver = null;
       boolean status = false;

    [@Parameterized](http://twitter.com/Parameterized).Parameters
       public static LinkedList<String[]> getEnvironments() throws Exception {
           LinkedList<String[]> env = new LinkedList<String[]>();
           env.add(new String[] { "WIN10", "chrome", "70.0" });
           env.add(new String[] { "macOS Big Sur", "safari", "14.0.2" });
           env.add(new String[] { "WIN8", "firefox", "62.0" });
           return env;
       }

    public JunitParallelExecutionDemo(String platform, String browserName, String browserVersion) {
           this.platform = platform;
           this.browserName = browserName;
           this.browserVersion = browserVersion;
       }

    [@Before](http://twitter.com/Before)
       public void setUp() throws Exception {
           DesiredCapabilities capabilities = new DesiredCapabilities();
           capabilities.setCapability("browserName", browserName);
           capabilities.setCapability("version", browserVersion);
           capabilities.setCapability("platform", platform);
           capabilities.setCapability("build", "JUnitRepeatedTestSample");
           capabilities.setCapability("name", "JUnitRepeatedTestSample");
           capabilities.setCapability("network", true); // To enable network logs
           capabilities.setCapability("visual", true); // To enable step by step screenshot
           capabilities.setCapability("video", true); // To enable video recording
           capabilities.setCapability("console", true); // To capture console logs
           try {
               driver = new RemoteWebDriver(new URL("https://" + username + ":" + accessKey + gridURL), capabilities);
           } catch (MalformedURLException e) {
               System.out.println("Invalid grid URL");
           } catch (Exception e) {
               System.out.println(e.getMessage());
           }
       }
       [@Test](http://twitter.com/Test)
       [@RepeatedTest](http://twitter.com/RepeatedTest)(value=2)
       public void testParallel() throws Exception {
           try { // Change it to production page
               int total_elements=0;
               driver.get("[https://todomvc.com/examples/react/#/](https://todomvc.com/examples/react/#/)");
               driver.findElement(By.className("new-todo")).sendKeys("Lambdatest Cross Browser Testing");;
               total_elements = driver.findElements(By.xpath("//ul[[@class](http://twitter.com/class)='todo-list']/li")).size();
               Assert.assertEquals(1, total_elements);

    } catch (Exception e) {
               System.out.println(e.getMessage());
           }
       }

    [@After](http://twitter.com/After)
       public void tearDown() throws Exception {
               driver.quit();
           }

    }

Now the above test case can be run using JUnit and TestNG in Selenium, the same way as we discussed earlier. We will provide junit=”true” in the suite tag, and we are done!

    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE suite SYSTEM "[http://testng.org/testng-1.0.dtd](http://testng.org/testng-1.0.dtd)" >
    <suite name = "Parallel Tests" junit="true">
     <test name="JunitParallelExecutionDemo">
       <classes>
          <class name="Test.JunitParallelExecutionDemo"/>
       </classes>
    </test>
    </suite>

As we can see from the below execution log for the above test case ran on three different browsers-platforms combination-

As we can see in the screenshot, the test has run on the given browsers in parallel. We can also see the execution video by selecting the particular execution and click on the video on the right side. This would help to find the issue in case there is any failure.

Hey! Are you using Playwright for automation testing? Run your Playwright test scripts instantly on 50+ browser/OS combinations using the LambdaTest cloud!

Conclusion

To summarize, we have seen the two testing frameworks -JUnit and TestNG and how they are different. Also, what are the capabilities of TestNG, and why would we need to run test cases using JUnit and TestNG in Selenium test automation. We have also explained how to do that with the help of a JUnit test example. This is how we can utilize existing test cases with the enhanced abilities of TestNG.

I hope this blog will help you run test cases using JUnit and TestNG in Selenium and find ways to scale the automation test suite using their extensive capabilities.

Happy Testing!

Complete JUnit 5 Mockito Tutorial For Unit Testing

Ruchira Shukla — Thu, 04 Aug 2022 13:11:00 +0000

Mockito is a unit testing framework for Java that simplifies the task of automation testing. It makes unit testing highly effective with clean tests, thanks to dependency injection and compile-time checks. In addition, Mockito helps improve the test independence of components being developed by helping you create objects that have no dependencies on a test-specific configuration. The most popular way of using Mockito is within the JUnit framework to help you write better tests.

For starters, mocking in unit testing is used for isolating the AUT (Application Under Test) from external dependencies. Mocking must be leveraged when implementation of external dependencies is not yet completed.

In this JUnit 5 Mockito tutorial, we will see how Mockito can be used with the JUnit unit testing framework for mocking in JUnit 5.

Let’s get started!

Getting started with Mockito and JUnit 5

Mockito is an open-source test automation framework that internally uses Java Reflection API to create mock objects. Mock objects are dummy objects used for actual implementation. The main purpose of using a dummy object is to simplify the development of a test by mocking external dependencies and using them in the code.

As per my experience, you should mock objects in unit tests when the real object has a non-deterministic behavior or the real object is a callback function or the real object is yet to be implemented.

In this JUnit 5 Mockito tutorial, we will use Mockito with JUnit for performing Selenium automation testing. In case you are starting with JUnit, look at this article on how to setup JUnit environment for your first test?. However, you can also go through the Selenium JUnit tutorial to check out the basics of JUnit, from writing your first test to running tests in Selenium with examples.

Mockito Installation

When you use Mockito in your unit tests, you will need to download the jar file and place it in a path that your build system can find. Mockito is available in two versions: mockito-core (which contains only the core of Mockito, and mockito-all (which contains all modules).

The preferred way of installing Mockito is to declare a dependency on mockito-core with a build system of choice. The second best way is to download the artifacts using a manual approach and add them to the classpath. You can also add dependencies to your existing Maven or Gradle project.

Add the following dependencies in your pom.xml:

    <dependency> <groupId>org.mockito</groupId> <artifactId>mockito-core</artifactId> <version>4.1.0</version> <scope>test</scope> </dependency>

When using Gradle, this can be achieved by declaring the following dependency to your build.gradle:

  compile("org.mockito:mockito-core:4.1.0")

The latest version of Mockito when writing this article is 4.1.0.

Annotations in Mockito

Like JUnit annotations, Mockito annotations are used to specify the behavior of the test code. It allows the users to focus more on their logic while still testing the code very effectively. This section of the JUnit 5 Mockito tutorial mainly focuses on Mockito annotations and how to use them in Selenium.

1. @Mock annotation in Mockito

Mock annotation is used to create a mock object.

    [@Mock](http://twitter.com/Mock)
    ToDoService serviceMock;

The @Mock annotation is always used with @RunWith, a class-level annotation. We will see in detail how both annotations are used to create and use a mock object in the next segment.

For now, let’s see the following syntactic example:

    [@RunWith](http://twitter.com/RunWith)(MockitoJUnitRunner.class) 
    public class DemoMock { 
    ..... 
    }

2. @spy annotation in Mockito

Spy annotation is used to create a real object and spy on that real object. This would help to call all the object methods while still tracking every interaction that is being mocked.

Below is the simple implementation of @spy annotation:

    [@Spy](http://twitter.com/Spy)
    List<String> myList = new ArrayList<String>();

    [@Test](http://twitter.com/Test)
    public void usingSpyAnnotation() {
       myList.add("Hello, This is LambdaTest");
       Mockito.verify(spyList).add("Hello, This is LambdaTest");
       assertEquals(1, spyList.size());
    }

3. @Captor annotation in Mockito

Captor annotation is used to create an ArgumentCaptor instance to capture method argument values for further assertions.

Here is a simple implementation of @Captor annotation that captures MyMap’s key and values:

    [@Mock](http://twitter.com/Mock)
    HashMap<String, Integer> MyMap;
    [@Captor](http://twitter.com/Captor)
    ArgumentCaptor<String> keyCaptor;
    [@Captor](http://twitter.com/Captor)
    ArgumentCaptor<Integer> valueCaptor;
    [@Test](http://twitter.com/Test)
    public void ArgumentCaptorTest()
    {
       hashMap.put("A", 10);
       Mockito.verify(MyMap).put(keyCaptor.capture(), valueCaptor.capture());
       assertEquals("A", keyCaptor.getValue());
       assertEquals(new Integer(10), valueCaptor.getValue());
    }

4. @InjectMocks annotation in Mockito

InjectMocks annotation is used to mock a class with all its dependencies. This is quite useful to test the behavior completely.

In the below example, we will use @InjectMock Countries into Continent:

    [@Mock](http://twitter.com/Mock)
    Map<String, String> Countries;

    [@InjectMocks](http://twitter.com/InjectMocks)
    MyDictionary dic = new Continent();

    [@Test](http://twitter.com/Test)
    public void UseInjectMocksAnnotation() {
       Mockito.when(Countries.get("India")).thenReturn("asia");

       assertEquals("asia", dic.getContinent("India"));
    }

Now that we have seen different Mockito annotations, let’s deep-dive into creating Mocks in the Mockito framework.

However, if you are intrigued to know more about JUnit Annotations in Selenium, you can go through the following video from the LambdaTest YouTube Channel and stay updated with more such videos on the JUnit tutorial with Selenium, Selenium Testing, Cypress Testing, and more.

How to create Mocks in Mockito?

In this section of the JUnit 5 Mockito tutorial, we will see different ways to create Mocks in the Mockito framework. In Mockito, we can create mock objects in two ways:

Using @Mock annotation
Using Mock() method

Using @Mock annotation to create Mocks

@Mock annotation minimizes repetitive mock objects and makes more readable test code and verification errors. It is available in the org.mockito package. Here is the code snippet of @mock annotation:

    [@Mock](http://twitter.com/Mock)
    ToDoService serviceMock;

Using @Mock method to create Mocks

The Mockito.mock() method allows us to create a mock object of classes and interfaces. Here is the code snippet for a simple class mock:

    MyList listMock = mock(MyList.class);
    when(listMock.add(anyString())).thenReturn(false);

In the above code snippet, MyList is a class for which we have created a mock object by passing that as a parameter in the mock() method. The second line sets an expectation. When the add() method of MyList class is called, it has to return false.

To understand this more clearly, let’s take an example of an eCommerce application. Let’s assume a scenario where you have to test the checkout feature. There is a class Order which has a method checkout() that finalizes the order and processes the payment. We are deliberately overlooking other conditions for the sake of simplicity.

So the Order class would look something like this:

    public class Order {

           public Order() {
           }

           public String checkout(PaymentProviderService payment) {
               try {
                   return payment.processPayment().equalsIgnoreCase("Approved") ? "Success" : "Failure";
               }
               catch(Exception e) {
                   return "Exception occurred at payment provider service when trying to checkout";
               }
           }
       }

In the above example, the checkout() method uses PaymentProviderService class for payment processing. The success or failure of checkout is based on the method processPayment().

So if we want to write test cases for checkout functionality, we also need to create an instance of the PaymentProviderService class. But we don’t want to depend on this class to start writing our unit tests. In this case, we would create a mock object of the PaymentProviderService class and continue writing tests for the Order class.

Here is the code snippet:

    public class OrderTests {

           [@Test](http://twitter.com/Test)
           public void checkoutOrderSuccessOnPaymentSuccess() {

               Order order = new Order();
               PaymentService ppMock = mock(PaymentService.class);
               when(ppMock.processPayment()).thenReturn("Approved");

               Assert.assertEquals(
                   "Success",
                   order.checkout(ppMock)
               );
           }

Code Walkthrough

I have created a test case for the Order class in the above code, which tests the successful order placement scenario.

    PaymentProviderService ppMock = mock(PaymentProviderService.class);

The above line creates a mock object of the PaymentProviderService class. Now we can use this object to call the processPayment() method. We can write the expected behavior using when…thenReturn as:

 when(ppMock.processPayment()).thenReturn("Approved");

After that, we can assert the expected and actual outputs and perform the desired function. Likewise, we have created 4 unit tests for the Order class. The output of those tests on running them on IDE is shown below:

Are you using Playwright for automation testing? Run your Playwright test scripts instantly on 50+ browser/OS combinations using the LambdaTest cloud. Sign up for free!

How to use Mockito with JUnit 5 extensions?

This section of the JUnit 5 Mockito tutorial is devoted to the usage of Mockito with JUnit 5. It shows how to add a mock object to a test case and verify its behavior with JUnit 5. JUnit 5 has an extension model that supports Mockito out-of-the-box. The JUnit 5 extension model allows users to provide custom annotations and behaviors for test doubles.

To start with, JUnit Jupiter dependencies need to be added apart from mockito-core dependencies. So following dependencies have to be added in pom.xml(for Maven):

    <dependency>
       <groupId>org.mockito</groupId>
       <artifactId>mockito-junit-jupiter</artifactId>
       <version>4.1.0</version>
       <scope>test</scope>
    </dependency>

Here is how the dependencies can be added for Gradle:

    dependencies{
    testImplementation('org.mockito:mockito-core:4.1.0')

The extension eliminates the need for the MockitoAnnotations.openMocks() method call. So the above example can be modified as:

    [@ExtendWith](http://twitter.com/ExtendWith)(MockitoExtension.class)
    public class OrderTestsUsingExtension {

               [@Test](http://twitter.com/Test)
               public void checkoutOrderSuccessOnPaymentSuccess() {

                   Order order = new Order();
                   PaymentProviderService ppMock = mock(PaymentProviderService.class);
                   when(ppMock.processPayment()).thenReturn("Approved");

                   Assert.assertEquals(
                       "Success",
                       order.checkout(ppMock)
                   );
               }
    }

It’s simple, isn’t it? This makes our code more readable and clean. Also, it can detect incorrect stubbing.

The next section of this JUnit 5 Mockito tutorial, will showcase parallel testing in Selenium using Mockito and JUnit.

This JUnit certification establishes testing standards for those who wish to advance their careers in Selenium automation testing with JUnit.

Here’s a short glimpse of the JUnit certification from LambdaTest:

How to perform parallel testing using Mockito and JUnit 5 on cloud Selenium Grid

JUnit 5 has the much-awaited capability of executing tests parallely, which significantly reduces the execution time. This is a life savior for testers as we have a huge no. of test cases in the real world. Leveraging this capability using an online Selenium Grid like LambdaTest would be more efficient.

Selenium testing tools like LambdaTest, allow you to perform cross browser testing on more than 3000 online browsers and operating system combinations.

Here is a quick video tutorial on the real-time browser testing.

In this section of the JUnit 5 Mockito tutorial, we will take a simple example to demonstrate parallel testing with Mockito and JUnit 5:

Problem Statement

Navigate to https://www.lambdatest.com.
Validate the page title is “Most Powerful Cross Browser Testing Tool Online | LambdaTest.”
Enter username and password.
Click on the login button.
Validate successful login by “Welcome — LambdaTest” message.

We will be executing this test case on the following browsers, versions, and platform combinations using LambdaTest remote Selenium Grid:

Browser	Version	Platform
Chrome	70.0	WIN10
Safari	14.0.2	macOS Big Sur
Firefox	76.0	WIN10

package MockitoDemo.MockitoDemo;
    import org.junit.jupiter.api.AfterAll;
    import org.junit.jupiter.api.AfterEach;
    import org.junit.jupiter.api.Assertions;
    import org.junit.jupiter.api.BeforeAll;
    import org.junit.jupiter.api.BeforeEach;
    import org.junit.jupiter.api.Test;
    import org.junit.jupiter.api.parallel.Execution;
    import org.junit.jupiter.api.parallel.ExecutionMode;
    import org.junit.jupiter.params.ParameterizedTest;
    import org.junit.jupiter.params.provider.Arguments;
    import org.junit.jupiter.params.provider.MethodSource;
    import org.openqa.selenium.By;
    import org.openqa.selenium.WebElement;
    import org.openqa.selenium.remote.DesiredCapabilities;
    import org.openqa.selenium.remote.RemoteWebDriver;
    import org.openqa.selenium.support.ui.ExpectedConditions;
    import org.openqa.selenium.support.ui.WebDriverWait;
    import java.util.stream.Stream;
    import java.net.MalformedURLException;
    import java.net.URL;
    import java.util.concurrent.TimeUnit;
    import static org.junit.jupiter.params.provider.Arguments.arguments;
    @Execution(ExecutionMode.CONCURRENT)
    public class CrossbrowserDemo {
    String username = "YOUR_USER_NAME";
           String accesskey = "YOUR_ACCESS_KEY";

          static RemoteWebDriver driver = null;
          String gridURL = "@hub.lambdatest.com/wd/hub";
          String urlToTest = "https://www.lambdatest.com/";

          @BeforeAll
          public static void start() {
              System.out.println("=======Starting junit 5 tests in LambdaTest  Grid========");
          }

          @BeforeEach
          public void setup(){
              System.out.println("=======Setting up drivers and browser========");
          }

          public void browser_setup(String browser) {
              System.out.println("Setting up the drivers and browsers");
              DesiredCapabilities capabilities = new DesiredCapabilities();

              if(browser.equalsIgnoreCase("Chrome")) {
                  capabilities.setCapability("browserName", "chrome");    //To specify the browser
                  capabilities.setCapability("version", "70.0");        //To specify the browser version
                  capabilities.setCapability("platform", "win10");        // To specify the OS
                  capabilities.setCapability("build", "Running_Junit5Tests_In_Grid_Chrome");               //To identify the test
                  capabilities.setCapability("name", "JUnit5Tests_Chrome");
                  capabilities.setCapability("network", true);        // To enable network logs
                  capabilities.setCapability("visual", true);            // To enable step by step screenshot
                  capabilities.setCapability("video", true);            // To enable video recording
                  capabilities.setCapability("console", true);            // To capture console logs
              }
              if(browser.equalsIgnoreCase("Firefox")) {
                  capabilities.setCapability("browserName", "Firefox");  //To specify the browser
                  capabilities.setCapability("version", "76.0");    //To specify the browser version
                  capabilities.setCapability("platform", "win10");   // To specify the OS
                  capabilities.setCapability("build", "Running_Junit5Tests_In_Grid_Firefox");    //To identify the test
                  capabilities.setCapability("name", "JUnit5Tests_Firefox");
                  capabilities.setCapability("network", true);      // To enable network logs
                  capabilities.setCapability("visual", true);       // To enable step by step screenshot
                  capabilities.setCapability("video", true);        // To enable video recording
                  capabilities.setCapability("console", true);      // To capture console logs

              }
              if(browser.equalsIgnoreCase("Safari")) {
                  capabilities.setCapability("browserName", "Safari");  //To specify the browser
                  capabilities.setCapability("version", "14.0.2");    //To specify the browser version
                  capabilities.setCapability("platform", "macOS Big Sur");   // To specify the OS
                  capabilities.setCapability("build", "Running_Junit5Tests_In_Grid_Safari");    //To identify the test
                  capabilities.setCapability("name", "JUnit5Tests_Firefox");
                  capabilities.setCapability("network", true);      // To enable network logs
                  capabilities.setCapability("visual", true);       // To enable step by step screenshot
                  capabilities.setCapability("video", true);        // To enable video recording
                  capabilities.setCapability("console", true);      // To capture console logs

              }
              try {
                  driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities);
              } catch (MalformedURLException e) {
                  System.out.println("Invalid grid URL");
              } catch (Exception e) {
                  System.out.println(e.getMessage());
              }

          }

          @ParameterizedTest
          @MethodSource("browser")
          public void launchAndVerifyTitle_Test(String browser) {
              browser_setup(browser);
              String methodName = Thread.currentThread()
                      .getStackTrace()[1]
                      .getMethodName();
              driver.get(urlToTest);
              driver.manage().window().maximize();
              driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);
              String actualTitle = driver.getTitle();
              System.out.println("The page title is "+actualTitle);
              String expectedTitle ="Most Powerful Cross Browser Testing Tool Online | LambdaTest";
              System.out.println("Verifying the title of the webpage started");
              Assertions.assertEquals(expectedTitle, actualTitle);
              System.out.println("The webpage has been launched and the title of the webpage has been veriified successfully");
              System.out.println("********Execution of "+methodName+" has ended********");
          }

          @ParameterizedTest
          @MethodSource("browser")
          public void login_Test(String browser) {
              browser_setup(browser);

              String methodName = Thread.currentThread()
                      .getStackTrace()[1]
                      .getMethodName();
              driver.get(urlToTest);
              driver.manage().window().maximize();
              driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);
              WebElement login = driver.findElement(By.xpath("//a[text()='Login']"));
              login.click();
              WebElement username = driver.findElement(By.xpath("//input[@name='email']"));
              WebElement password = driver.findElement(By.xpath("//input[@name='password']"));
              WebDriverWait wait = new WebDriverWait(driver,20);
              wait.until(ExpectedConditions.visibilityOf(username));
              username.clear();
              username.sendKeys("ruchirashukla89@gmail.com");
              password.clear();
              password.sendKeys("ruchira@89");
              WebElement loginButton = driver.findElement(By.xpath("//button[text()='Login']"));
              loginButton.click();
              driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);
              String actual = driver.getTitle();
              String expected = "Welcome - LambdaTest";
              Assertions.assertEquals(expected, actual);
              System.out.println("The user has been successfully logged in");
              System.out.println("********Execution of "+methodName+" has ended********");
          }

          @ParameterizedTest
          @MethodSource("browser")
          public void logo_Test(String browser) {
              browser_setup(browser);
              String methodName = Thread.currentThread()
                      .getStackTrace()[1]
                      .getMethodName();
              System.out.println("********Execution of "+methodName+" has been started********");
              System.out.println("Launching LambdaTest website started..");
              driver.get(urlToTest);
              driver.manage().window().maximize();
              driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);
              System.out.println("Verifying of webpage logo started..");

              WebElement logo = driver.findElement(By.xpath("//*[@id=\"header\"]/nav/div/div/div[1]/div/a/img"));
              boolean is_logo_present = logo.isDisplayed();
              if(is_logo_present) {
                  System.out.println("The logo of LambdaTest is displayed");
              }
              else {
                  Assertions.assertFalse(is_logo_present,"Logo is not present");
              }
              System.out.println("********Execution of "+methodName+" has ended********");
          }

          @AfterEach
          public void tearDown() {
              System.out.println("Quitting the browsers has started");
              driver.quit();
              System.out.println("Quitting the browsers has ended");
          }

          @AfterAll
          public static void end() {
              System.out.println("Tests ended");
          }

          static Stream<Arguments> browser() {
              return Stream.of(
                      arguments("Chrome"),
                      arguments("Firefox"),
                      arguments("Safari")
              );
          }

    }

Code Walkthrough

Three tests will run in parallel in the above code on the mentioned browser OS combinations.

The @execution(ExecutionMode.CONCURRENT) annotation defines the parallel mode in JUnit 5. The LambdaTest remote Selenium Grid initiates the execution based on the arguments defined in the capabilities.

The @ParameterizedTest annotation defines that the annotated method is parameterized where the @MethodSource annotation is used to provide the method name, which is used as a source of parameter values.

At last, the browser() method, which is the method source, has the browser names as the arguments.

After execution, we will get logs as below:

As we can see, three tests are executed in parallel on three different browsers where they took around 60 seconds to execute.

We can also get the video of each run to find the issue if a test fails.

So, if you’re curious to know more about performing parallel testing with JUnit 5 and Selenium, this video will help you get started. This video will guide you through different parallel parameters in JUnit testing, including methods — execute test methods in separate threads and classes — to execute test classes in separate threads.

Test on Selenium Automation Testing Grid Cloud of 3000+ Desktop & Mobile Browsers.

Conclusion

In this JUnit 5 Mockito tutorial, we have gone through the basics of the Mocking and Mockito framework. Also, what are different annotations in Mockito and their use cases. In JUnit 5, the creation of mocks using Mockito has been made more simple by extension. We have also discussed the parallel execution of test cases in JUnit5 using LambdaTest cloud Selenium Grid. Though Mockito is highly utilized in unit testing, we can also leverage its extensive capabilities in integrating those unit tests for Selenium automation testing.

How have you used Mockito in your testing? Let us know in the comments below.

Happy Testing!

15 Most Important Mobile App Testing Scenarios

Ruchira Shukla — Wed, 03 Aug 2022 11:39:13 +0000

Mobile app testing is critical in the mobile development lifecycle. Developers and app testers should think of different mobile app testing scenarios before making them live. This is particularly important given the number of apps released daily. According to Statista, mobile app downloads worldwide have increased to 218 billion in 2020.

This has led to a demand for delivering high-quality mobile applications to stand out in the crowd and get maximum traffic. In this blog, we will discuss the different mobile app testing scenarios. Continuously upgrading Mobile OS, new devices, and localization challenges make a app tester’s job more challenging.

To cater to this, it is crucial to create intelligent and efficient mobile app test scenarios which uncover the issues which can impact the user experience.

Hey! This Playwright testing tutorial will guide you through the setup of the Playwright test framework, which will enable you to write end-to-end testing for your future projects. Click on the link to learn more!

What is a Test Case

A test case is a more granular level of testing where the app tester performs various operations on the app to ensure that the given functionality works fine. This helps in understanding, tracking, and resolving quality issues. In short, test cases test the given scenario.

What is a Test Scenario

A Test scenario is a broader term and is often tied up with the user stories in the agile model. Test scenario helps engineers to test the mobile app with real-world use cases. Testers similarly execute scenarios as the end-user would use the app.

Why Create Mobile App Testing Scenarios?

Test scenarios are created to:

It assures complete test coverage.
All the stakeholders approve the scenarios. Therefore every person involved is on the same page.
They help determine the end-to-end use case of the Application and test everything accordingly.

When not to create a Mobile App Testing Scenarios?

Test scenarios are not helpful in the below-mentioned cases.

When the application is complicated or unstable or has a time crunch.
When you have a Scrum or Kanban methodology.
Test Scenario should not be created to fix a new bug fix or during regression testing.

15 Important Mobile App Testing Scenarios

Functional Mobile Testing Scenario

Along with the usual test scenarios which tests the basic functionality, there are some mobile app testing scenarios which one should consider while performing functional testing.

Ensure that mobile works in multitask mode when needed. For example, while using the app, if the user gets a call, he should perform the expected actions.
Test that the AUT is not consuming other apps’ memory and not stopping the activities of other apps.
Page scroll, swipe is working as expected.
Check that the app’s automatic startup is working as expected.
Check that auto-logout is working as expected.
Navigation between various modules should work as expected.
Test that the app’s social networking options such as sharing, posting, login are working correctly.

UI Mobile Testing Scenario

UI testing can be tricky for mobile apps as one has to check for the component placement on a comparatively smaller screen. As all the web applications tend to be responsive now, one has to make sure that the layout, placement, and functionality are working as expected. Along with that, a few other mobile app testing scenarios must be noted.

Validate that text is not cut off and must be fitted on the screen.
Validate that all popups, alert messages, and error messages are meaningful and correct.
Validate the carousel, navigation through menu, and swipe functions.
The page should not take more than five seconds to load, if it does, it should have a progress bar that allows the user to comprehend the scenario.
Check for spelling & grammatical errors.
Check for image sizes, image rendering, and company logo.
When the screen allows editing without saving, it must show a message to save details to the user when navigating to another screen without saving.
If there is lazy loading, check if the content loads without a lag when the user scrolls up.
Validate that data does not ghost off when moving the app to the background while filling a form.

Compatibility Mobile Testing Scenario

Check for the interrupt conditions like incoming calls, messages, or notifications while using the app.
The app should go into the background if there is an incoming call. After the call disconnects, it should again resume as it was before.
Validate that the phone’s essential functions like alarm, storing messages, sending/receiving messages are all working as expected while using the app.

Hey, Also learn what Selenium is and its benefits for automated cross browser testing.

Performance Mobile Testing Scenario

Performance of the app is critical as the purpose of convenient access would be lost if the app does not perform well.

Validate the performance of the application by changing the network to 2G,3G, 4G, 5G, and wifi.
Validate the CPU consumption while using the app.
Validate how many parallel users would crash the app to benchmark performance.
Perform the load testing on individual API calls and validate how the application behaves under a certain load.

Usability Mobile Testing Scenario

Check for the font size, button size, and content format so that it is accessible effortlessly.
Inform the client while downloading a lot of data which might not be favorable for the performance of an app.
Validate that the app should have easy navigations which do not confuse users.

Security Mobile Testing Scenario

Validate the flow for security vulnerabilities that involve the user’s personally identifiable data, such as mobile number, email, etc.
Validate that the data is secured and encrypted, sent to and back from the server over the network.
Validate that no confidential data is stored without encryption and visible/accessible.

Battery & Data Mobile Testing Scenario

Testing the app against data usage and power consumption should also be considered, as it should not drain the battery while running.

Recoverability Mobile Testing Scenario

Validate that the app resumes the last operation in case of a hard reboot.
Test what happens in case of a crash between transactions or any R/W function.

Career & Network Connectivity Testing

Test how well the app connects with the network and mobile data career online.

Installation & Resource usage Testing

Test what resources are required to install the app and if the installation process is user-friendly and quick. Also, test that there is no crash during installation and it’s smooth.

Localization and Time Zone Testing

If an app supports different locals, the testers should also test the app for localization settings. The labels, functional flows, and other features should work as expected in different languages. Similarly, all the features should work as expected while operating in different time zones.

Certification Mobile Testing Scenario

Ensure that the app adheres to the guidelines of the Android or iOS phones platforms when installed.

Restrict/Allow Permission Testing

Verify that the app adheres to the user permission settings for accessing the phone’s features such as contacts, camera, gallery, etc.

Multi-Touch and Single Touch Testing

Thoroughly test the multi-touch features such as pinch to zoom or pinch to shrink, etc., and create plenty of test cases for this for every applicable screen.

Mobile Testing Strategy

We have already discussed test scenarios for mobile app testing. We know that mobile devices have different OS and versions. The question is how to test the app for such a huge number of devices.

The effective mobile testing strategy should aim for maximum coverage with a reasonable device investment. We can use virtual testing platforms of emulators and simulators or use real device cloud which is nothing but real devices hosted on the cloud.

There are different types of mobile virtual testing platforms:

iOS and Android EmulatorOnline — Emulates different mobile devices.
Browser Emulator Online — Emulates different browsers.
OS Emulators — Provided by respective OS designers; Android by Google, Windows Phone by Microsoft, and iOS emulator by Apple.

Using Real Device Cloud

App testers should select the real devices based on the current market research data on the usage of devices. The devices which are most popular among users should be selected for testing.

This also holds good for the OS versions. LambdaTest allows you to test apps on real devices using real device cloud.

Looking to perform Android and iOS app testing on Real Device Cloud, check out our video below –

Automation Testing

Test engineers should also consider automating the functional/regression scenarios as much as possible. This will make testing more effective and less time-consuming. There are many automation tools available for mobile automation. For example — Appium, Calabash, Robotium, UIAutomator, etc.

Cloud-Based Automation Testing Solution for Mobile Testing

Tools that use emulators or simulators for mobile testing are available in two categories- Cloud-based and on-premise. Cloud based tools are gaining popularity because of their ease of use and scalability.

LambdaTest is a cross browser testing platform that offers both real device cloud and virtual platform testing using emulators and simulators.

Here’s a short glimpse of LambdaTest mobile app testing features:

LambdaTest allows you to upload the app in a single click.
It offers both emulators/simulators and real devices.
You can test your websites and webapps on mobile devices.
Perform responsive test and compatibility test on different devices.
Completely eliminate the need for an on-premise device lab.
Test fast and ship with confidence
Eliminate false positives
Integrate with different communication and CICD tools
Test native app features and debug in real time using full page screenshot, videos and logs.
Perform both automated and manual testing

Now, Automate Cypress testing and perform browser automation testing with LambdaTest.Try for free!

Summary

Now perform free cross browser & live testing. Mobile app testing is different from web application testing, and we have already addressed different scenarios that should be considered while testing mobile apps. It can be challenging to design an effective mobile testing strategy due to numerous devices. The app must ensure platform and browser compatibility. I hope this blog helps you to create a mobile testing strategy effectively.

Top 9 Challenges In Automation Testing [2022]

Ruchira Shukla — Tue, 02 Aug 2022 11:15:46 +0000

Automation Testing has become a necessity in the world of DevOps and Agile. Effective automation testing can be an awesome productivity booster for the testing team and an overall system quality enhancer in the long run. However, the most difficult element of starting with test automation is making sure it is used correctly.

Those who have attempted to set up test automation will attest to the fact that it is not a cakewalk.
As per the State of Test Automation 2020–2021 Survey by Kobiton, 25.9% of the organizations find choosing the right tool as the biggest challenge, followed by setting the right test automation approach, and so on.

In this article on the automation testing challenges, we will discuss the top 9 key challenges in automation testing faced by developers and organizations and how to overcome the test automation challenges.

Hey! You can now test on Selenium Automation Testing Grid Cloud of 3000+ Desktop & Mobile Browsers.

Why is automation testing important?

The most important reason to utilize automated testing is to obtain a high-quality product with exceptional performance that offers several advantages. It results in a significant reduction in maintenance costs and a significant increase in software performance with faster Time to Market (TTM). In terms of budget, the importance of automation testing lies in charge reductions because the human is no longer required for anything other than the test’s maintenance. Automated testing aims to make testing as simple as feasible with scripts.

If you are new to automation testing or want to migrate from manual testing to automation testing, this article on how to migrate from manual testing to automation testing will help you get started.

From the below stats, test automation was cited by 42.5% of organizations as an important aspect of their quality assurance process.

Key challenges in automation testing

An effective continuous testing plan is one of the important components in a software development organization. Continuous testing necessitates the use of end-to-end automated testing, which allows for a better understanding of application quality early in the development cycle.

It sounds fantastic, and it certainly is. However, as with any other development process, moving to an automated testing strategy frequently means dealing with common mistakes that can make this period unpleasant.

What obstacles will a DevOps team face when attempting to execute a successful automated testing strategy? Let’s discuss them in detail.

Run Selenium, Cypress, and Appium tests on LambdaTest to scale with the demand of your website and web apps.

1. Finding the right framework and tool

Finding the right automation tool is a very crucial challenge in automation testing for the success of automation. There are a lot of tools available, both licensed as well as open-source, which makes it difficult to choose. The choice of tool depends upon the kind of application and the extent of automation testing. Before selecting a particular tool, the test engineer should evaluate the pros and cons of each tool and consider based on the basis of the following points:

Ease of use
Scalability
Maintainability
Integration with third-party tools like Github, Jira, Jenkins, Maven/Gradle, etc.

No matter what testing methodology we use, if we do not have a tool that matches the business requirement, automation tends to fail eventually. There are several top reasons for test automation failure.

Researching tools and determining the best-suited one is a difficult task in and of itself. Likewise, this is a challenge in itself to research tools and find out the most appropriate one. Nevertheless, responses from the above stats indicate that Selenium is the preferred and one of the most popular test automation frameworks among all.

A key part of this automation testing stack is the grid of devices used. Testing can never be comprehensive unless the tests are run on real devices to mimic real user conditions. Unfortunately, setting up a real device lab is expensive, cumbersome, and requires constant updations. Picking a real device cloud could be a great option.

Choosing the cloud-based infrastructure platforms like LambdaTest gives you access to more than 3000 real devices running on real operating systems to perform automation testing. In addition, this cross browser testing platform allows users to test public or locally hosted web apps on various browsers, operating systems, and devices.

LambdaTest provides developers and testers with a single integrated ecosystem in which they can securely perform all their tests. This can help you achieve higher levels of efficiency and test coverage while reducing the costs associated with setting up and maintaining test environments.

This certification is for anyone who wants to stand out among Selenium automation testing professionals as they advance their careers.

Here’s a short glimpse of the Selenium 101 certification from LambdaTest:

You can subscribe to the LambdaTest YouTube Channel and stay updated with the latest videos around Selenium testing, Cypress testing, CI/CD, and more.

Execute & analyze Cypress test scripts online. Deploy quality builds faster with 40+ browser versions on cypress cloud.

2. Effective communication & collaboration in team

Lack of communication and collaboration among the team is one of the key challenges in automation testing. Since automation testing requires significant investment and gives long-term ROI (Return On Investment) instead of instant gratification, its success is very much dependent on effective communication among all stakeholders.

Wondering how to calculate your ROI? This article on how to calculate your ROI with test automation will help you make smart decisions.

Business analysts, developers, and project managers should discuss the test cases that need to be automated and the necessary functionality. So planning a strategy that is in line with all the respective stakeholders is a challenge. Also, to get a green light from management on the strategic timeline, making necessary changes in the product is not an easy task.

To overcome this, the automation team should try to make things transparent among all identified risks at the initial stage and communicate them as soon as possible.

3. Identifying the test automation strategy

Without a proper test automation strategy, the efficiency of technology will be lost. This is like a body without a brain. Choosing the right strategy is another key challenge in automation testing that is instrumental in the success of automation and in showing business values. So what is a test automation strategy?

Take a look at the below stats, where the majority of respondents stated that the primary strategic motivator for pursuing test automation was a desire to increase overall application quality.

Simply putting a test automation strategy is a microcosm of the larger test strategy. Automation is a part of the overall testing process and should be considered thoughtfully. This is another obstacle or one of the key challenges in automation testing as test engineers would find it difficult to chalk out a plan with so many dimensions to consider.

To deduce an effective strategy first, one should understand what is the need for strategy? A strategy should typically address the following:

What are the risks, capabilities, and functionalities, and how to inform the stakeholders, i.e., to arrive at a stable and repeatable communication process?
Setting our goals and plans.
Conceptualize the overall thought process and proof of that concept.
Auditing tool where you can go back and check your schedule and what has actually been done.

So ideally, to start a test automation strategy, one should take into account the following:

Define and identify high business value tests.
Understand your technology, tools, and resources.
Identify your risks.
Define your DevSecOps.
Consider your testing environment.
Consider your test data management.
Consider agile tools.

4. Setting realistic expectations of automation

A lot has been said about test automation. But unfortunately, organizations tend to believe that automation would bring the magic that would solve all their issues which is not true.

The expectations with test automation should be realistic and communicated clearly to higher management. In addition, what and how much should be automated to reap maximum benefit should be well documented. As already discussed, a well-defined automation strategy can play a vital role in setting up realistic expectations.

5. High initial investment

Automation needs a high initial investment in terms of tools, skilled resources, infrastructure, etc. Sometimes it might require licensing costs on top of operational costs. Moreover, even for open-source tools like Selenium, one must take into account the cost incurred in training the resources and setting up the grid. This is a major challenge in automation testing as it is sometimes difficult to convince the management to invest in it.

From the below stats, based on the overall quality assurance budget, the majority of organizations claimed that they spend between 30% and 49% of their overall QA budget on test automation.

To prove to the management that the financial benefits of automation are substantial, one should calculate the cost incurred in the production defects v/s cost involved in setting up automation.

Don’t forget about the intangible benefits, such as the opportunity to update and adjust old workflows and procedures to boost efficiency by eliminating repetition or redistributing staff.

6. Inadequate testing infrastructure

Another major challenge in automation testing is a lack of infrastructure to enable proper test coverage and execution speed. For example, consider the scenario in which an application must be tested on a variety of operating systems and browsers. It’s critical to execute test scripts in parallel to run tests against many configurations in an acceptable amount of time. And in order to achieve this, the testing infrastructure must be extremely robust in order to support parallel testing.

While a company that is just getting started with test automation may not be well prepared to develop an internal testing infrastructure, there are other choices, such as a cloud-based testing infrastructure.

LambdaTest provides a scalable infrastructure for startups and enterprises to fulfill their test automation needs. In addition, it eliminates the need to manage the in-house infrastructure so that the organizations can focus more on quality.

7. Problems with data reliance

One of the most challenging aspects of test automation is data management. When the test script is executed, the data must be in a specific state; otherwise, it can cause serious issues in several scenarios. What happens, for example, if the test script is executed simultaneously in several test environments and configurations?

Will it fail if the same data is used in multiple instances of the script? Any change to the data in one test execution instance can cause another instance to fail. What if the data of a script is built up by the execution of another test script?

Developing self-contained and independent test scripts is the most effective technique to deal with these data management difficulties. All test scripts should be written in such a way that they create and clean all of the data needed for successful execution. In this case, the team must eliminate the possibility of test failure owing to data-related concerns.

8. Finding the right skills

From designing a test automation framework to writing test scripts, dilemma and coding skills are required. Most test automation tools are really only effective when manned by individuals who can design and maintain automation frameworks, build solutions, and test scripts accurately. They must also be able to deal with unexpected technical problems.

Since automation frameworks differ in terms of features and operability, they must be familiar with the design and implementation protocols of the framework. Therefore, it would be wrong to assume that manual testers can work on automation tools or build a solution without proper training. However, investing in training and hiring new skilled resources is also challenging. It also demands time and is not feasible for organizations that want to scale up quickly.

9. Stakeholders and end-users expectations

No matter how good your tools or resources are, you must constantly keep in mind that testing cannot be done in a vacuum. It has one goal: to improve your product.

While test automation is useful for checking for flaws automatically, humans are still required to finish the non-automated testing phases. And these tests provide plenty of useful data for making the best possible decisions when it comes to developing the system — not for the benefit of developers but the benefit of end-users.

As a result, you must make certain that your testing team, management, and other stakeholders all agree on and understand the anticipated outcomes of your automation strategy.

Hey! Run your Playwright test scripts instantly on 50+ browser and OS combinations using the LambdaTest cloud. Execute Playwright Automation testing in parallel and cut down your test execution time by multiple folds.

Conclusion

Test automation is a continuous development and enhancement process. In this article on key challenges in automation testing, we have seen how by focusing more on fast value rather than fast tests, one can ensure the right approach to automation. Tests that run fast but miss the defects do not add any value. So the approach should be reducing the bugs in the production to reduce the costs that incur to resolve them at a later stage.

Automation is challenging as well as expensive. But the result can be a better quality product in front of the customer and a faster release cycle.

How To Use @RepeatedTest Annotation In JUnit 5

Ruchira Shukla — Thu, 28 Jul 2022 10:32:00 +0000

The JUnit framework is a popular test automation framework for unit testing in Java. The latest version of JUnit, i.e., JUnit 5 (also known as Jupiter), offers a number of improvements over JUnit 4. However, the community of JUnit users are often divided on this new release, with some favoring the stability of JUnit 4 and others preferring the new features being introduced with each release of JUnit 5. If you have been using JUnit 4, you can check the JUnit4 vs JUnit 5 comparison to understand the difference between them.

While JUnit 5 has introduced a lot of new and useful annotations, @RepeatedTest in JUnit 5 is one of the most powerful JUnit 5 annotations in the list. As an automation tester, we often get into a situation where we want to repeat a test case execution. If we have a flexible automation framework, we can pass such information through configuration in specific cases. Having said that, it is always good to have a feature that exclusively supports this requirement and can be instrumental while performing Selenium automation testing.

What is RepeatedTest in JUnit 5?

RepeatedTest, as the name suggests, is a test that is repeated during the test execution a certain no. of times. Let’s not confuse this with the parameterized test in JUnit. A parameterized test repeatedly executes with different input data, while a repeated test repeatedly executes with the same input data.

Are you looking for a Decimal to Gray Code covertor? It is a is a free online converter that converts any Decimal number base into gray code format.

What is the use of RepeatedTest ?

As a QA engineer, the first question that comes to mind is “Why do we need to repeat a test multiple times?”. First, there can be many scenarios where we need to do so. For example, a specific link on a web page results in an environment error, but this error disappears after two clicks. This is a known issue and you do not want your test case to fail for such a known issue.

Another scenario can be automating an application and clicking a link or navigating from one page to another. It always takes different load times, which sometimes results in timeout exceptions for the test case. In this case, we might want to execute the test case multiple times to get an average load time. And then pass that average load time as the threshold for the timeout.

Basically, most of the time, repetition is required when there is some randomness in the test case.

So, if you’re curious about the world of JUnit 5 annotations, this video will help you get started. You’ll learn JUnit 5 annotations in Selenium, how to write a JUnit test with Selenium, how to execute JUnit tests on a cloud Selenium Grid, and much more.

However, you can follow the LambdaTest YouTube Channel and stay updated with more such videos on the JUnit tutorial with Selenium.

How to use JUnit 5 @RepeatedTest annotation

JUnit Jupiter @RepeatedTest annotation is used to repeat the test case for specified no. of times. Each invocation of the test case behaves like a regular @test method, so it has support for the same lifecycle callbacks and extensions in JUnit 5.

The following example declares myTest() to repeat execution five times:

    [@RepeatedTest](http://twitter.com/RepeatedTest)(5)
    void myTest() {
    // ...
    }

There are certain rules associated with the method @RepeatedTest in JUnit 5.

The @RepeatedTest in JUnit 5 can not be private and static.
The return type for @RepeatedTest in JUnit 5 must be void.

Let’s see how it is used in the test case with a basic example-

Test Case

Launch the URL- https://www.amazon.com/
Click on the navigation menu item- “BestSellers.”
Validate that the page opened has the heading-“Amazon BestSellers.”
Repeat the same test case 5 times.

package Test;

    import java.net.MalformedURLException;
    import java.net.URL;

    import org.junit.After;
    import org.junit.Before;
    import org.junit.jupiter.api.AfterEach;
    import org.junit.jupiter.api.BeforeEach;
    import org.junit.jupiter.api.RepeatedTest;
    import org.openqa.selenium.By;
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.chrome.ChromeDriver;
    import org.junit.jupiter.api.Test;
    import junit.framework.Assert;

    public class RepeatedTestDemo {
       public WebDriver driver = null;

    [@BeforeEach](http://twitter.com/BeforeEach)
       public void setup() throws Exception {
           System.setProperty("webdriver.chrome.driver",
                   "DRIVER_EXE_PATH");
           driver = new ChromeDriver();
       }

    [@RepeatedTest](http://twitter.com/RepeatedTest)(value = 5)
       public void repeatedTestDemo() throws Exception {
           try {

               driver.get("[https://www.amazon.in/](https://www.amazon.in/)");
               driver.findElement(By.xpath("//a[text()='Best Sellers']")).click();
               String actual = driver.findElement(By.xpath("//div[[@id](http://twitter.com/id)='zg_banner_text']/div[1]")).getText();
               Assert.assertEquals("Pass", "Amazon Bestsellers", actual);
           } catch (Exception e) {
               System.out.println(e.getMessage());
           }
       }

    [@AfterEach](http://twitter.com/AfterEach)
       public void tearDown() throws Exception {
           driver.quit(); // for illustration
       }

    }

Following is the output of the above code:

As we can see, by default, JUnit displays the repetition number along with the total number of repetitions. We can also modify the display name, which we will discuss later. For now, let us do a code walk-through.

As it is quite evident that @RepeatedTest in JUnit 5 is treated as @test annotation by JUnit 5, the lifecycle is also the same. For example, @BeforeEach and @AfterEach methods will be invoked before and after each invocation of the @RepeatedTest method in JUnit 5.

Are you looking for a Decimal to BCD converter? In computing and electronic systems, a binary-coded decimal (BCD) is a method of representing each decimal digit by its binary sequence. The Decimal to BCD Converter converts an integer to a binary-coded decimal (BCD).

How to configure the custom test display name using @RepeatedTest annotation in JUnit 5

@RepeatedTest in JUnit 5 supports DisplayName placeholder to give a custom display name to the test. It returns the value provided in the annotation @DisplayName for the test method. If there is no @DisplayName annotation given, it returns the method name itself.

Syntax

    @RepeatedTest(name={displayName})

The annotation supports two display name patterns.

SHORT_DISPLAY_NAME
LONG_DISPLAY_NAME

SHORT_DISPLAY_NAME is the default placeholder. When used, JUnit displays the following pattern-

repetition {currentRepetition} of {totalRepetitions}

LONG_DISPLAY_NAME when used, JUnit display the following pattern-

{displayName}:: repetition {currentRepetition} of {totalRepetitions}

Where the placeholders-

{currentRepetition}: is the current repetition count

{totalRepetitions}: is the total number of repetitions

Customized display names in JUnit 5

We can also provide a string value and the supported placeholders to make it more meaningful. Let’s modify the above code to demonstrate the custom display name.

package Test;

    import java.net.MalformedURLException;
    import java.net.URL;
    import org.junit.jupiter.api.AfterEach;
    import org.junit.jupiter.api.BeforeEach;
    import org.junit.jupiter.api.DisplayName;
    import org.junit.jupiter.api.RepeatedTest;
    import org.openqa.selenium.By;
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.chrome.ChromeDriver;

    public class RepeatedTestDemo {
       public WebDriver driver = null;

    [@BeforeEach](http://twitter.com/BeforeEach)
       public void setup() throws Exception {
           System.setProperty("webdriver.chrome.driver",
                   "YOUR_DRIVER_EXE_PATH");
           driver = new ChromeDriver();
       }

    [@DisplayName](http://twitter.com/DisplayName)("Link Verification")
       [@RepeatedTest](http://twitter.com/RepeatedTest)(value = 3, name = "Testcase: {displayName} _ The current repetition count is  {currentRepetition} out of Total repetition count {totalRepetitions}")

    public void repeatedTestDemo() throws Exception {
           try {

    driver.get("[https://www.amazon.in/](https://www.amazon.in/)");
               driver.findElement(By.xpath("//a[text()='Best Sellers']")).click();
               String actual = driver.findElement(By.xpath("//div[[@id](http://twitter.com/id)='zg_banner_text']/div[1]")).getText();
               Assert.assertEquals("Pass", "Amazon Bestsellers", actual);
           } catch (Exception e) {
               System.out.println(e.getMessage());
           }
       }

    [@AfterEach](http://twitter.com/AfterEach)
       public void tearDown() throws Exception {
           driver.quit(); // for illustration
       }

    }

In the above example, the displayName is passed as a parameter, and it would replace the value of @displayName annotation at the run time. Therefore, the output of the above code would be something as shown below:

This JUnit certification establishes testing standards for those who wish to advance their careers in Selenium automation testing with JUnit.

Here’s a short glimpse of the JUnit certification from LambdaTest:

How to use RepeatedTest: RepetitionInfo Interface in JUnit 5

RepetitionInfo interface defines methods to retrieve the information about the current repetition and the total no. of repetitions. Here’s how we can use it.

RepetitionInfo.getCurrentRepetition() returns current repetition.
RepetitionInfo.getTotalRepetitions() returns total no. of repetitions.

Here is A Definitive Guide To Regression Testing. Do Check it out.

How to use RepeatedTest : RepeatedTest Interface in JUnit 5

RepeatedTest interface defines methods to get attributes of the annotation @RepeatedTest of the test method. Here’s how we can use it-

RepeatedTest.name() returns the name value passed as a parameter in @RepeatedTest Annotation.
RepeatedTest.name() returns the name value passed as a parameter in @RepeatedTest Annotation.

However, the best way to harness the real potential of the JUnit framework for test automation is to shift it from a local Selenium Grid to a cloud-based Selenium Grid so that it can be executed in any browser anywhere in the world. When combined with LambdaTest, the JUnit framework allows developers to execute automated acceptance tests on a cloud Selenium Grid, which provides true remote testing capabilities.

LambdaTest is a cloud-based automated cross browser testing platform that allows you to easily run Selenium tests on more than 3,000 real browsers and operating systems in the cloud. You can run tests on any online browser or operating system with just one command, or create powerful suites that contain thousands of your own custom-built test steps.

Selenium-Jupiter allows you to run Selenium tests against a Grid of Remote Web Browsers (RWB) on different browser and platform combinations. LambdaTest helps you run parallel testing in Selenium with increased efficiency and speed, by making use of cloud-based automation.

Conclusion

In this Selenium JUnit 5 tutorial, we have covered the annotation @RepeatedTest in JUnit 5 along with its use case. As we discussed, the annotation is particularly useful when we want to ascertain a scenario. In addition, the advantage of having an exclusive functionality for such a case is wonderful. So, what’s your use case for this annotation? Let us discuss this in the comments.

Happy Testing!

TestNG vs JUnit: Which testing framework should you choose?

Ruchira Shukla — Wed, 27 Jul 2022 13:21:00 +0000

A test automation framework is a set of components that facilitates the execution of tests along with reporting the results of the test execution. However, the discovery of the right test automation framework can be super-challenging since there are so many options at your perusal. Picture this — When performing Selenium using Java, you have to choose from a gruelling list of 10 Java testing frameworks.

As far as test automation frameworks in Selenium Java are concerned, JUnit and TestNG are preferred in comparison to the other frameworks in the list. Some of the pertinent questions are:

How does JUnit stack up against TestNG in the TestNG vs JUnit comparison?
Is it possible to run JUnit tests with TestNG (and vice-versa)?
What annotations are present in JUnit and TestNG?

A side-by-side comparison of TestNG vs JUnit will give a clear indication of how the two prominent test automation frameworks stack up against each other. Unearthing the difference between JUnit and TestNG frameworks in Selenium WebDriver will help in deciding the best-suited framework for automation tests.

In this blog, we do a thorough TestNG vs JUnit comparison — learnings of which will help in arriving at a wise decision for choosing the ideal test automation framework for your project. In this TestNG vs JUnit comparison, we have used JUnit 5 (the latest version of the JUnit framework) for showcasing example implementation.

Introduction to JUnit &TestNG Test Automation Frameworks

Before we get started with the TestNG vs JUnit comparison, let’s look at some of the basic essentials of the two frameworks. JUnit is an open-source unit testing framework for Java that was first introduced in 1997. At the time of writing this blog, JUnit 5 is the latest version of the JUnit framework. As the Selenium framework supports Java; many QA engineers prefer using the JUnit framework for web automation testing. In case you are getting started with JUnit, you could check out our earlier blog that helps in setting up JUnit Environment for the first test.

TestNG is also a popular open-source testing framework for Java. TestNG was created in 2007 with a goal to cover a wider range of test categories — Unit Testing, Functional Testing, End-To-End Testing, Integration Testing, and more. As of writing this blog, the latest version of TestNG is 7.3.0. You can check out our introductory blog on how to create a TestNG project in Eclipse for realizing web automation testing with TestNG.

It is necessary to know the difference between JUnit and TestNG so that you can make an informed decision when it comes to choosing the ideal test automation framework. This is the primary reason why we came up with a head-on TestNG vs JUnit comparison.

Did you know? JSON is a common format for storing data, but it can be hard to read. Prettify JSON will indent the JSON file, make sure stuff is lined up in a similar manner, and will put closing parenthesis in the same space every time.

TestNG vs JUnit — Comparison of the best test automation frameworks

Both TestNG and JUnit are amongst the top automation frameworks in Java. Though they are hugely popular, the fact is that both frameworks have their own share of pros and cons. We would keep this discussion for a later blog since the TestNG vs JUnit can be a good reference to choose the framework for your automation project.

For the TestNG vs JUnit comparison, we have made use of the JUnit 5 (latest version of JUnit framework) instead of JUnit 4 framework. If you are still using the JUnit 4 framework with Selenium WebDriver, you can still execute JUnit 4 tests with JUnit 5 framework.

Shown below is the detailed difference between JUnit and TestNG frameworks in Selenium WebDriver:

Watch this video to learn how TestNG has become one of the most robust test automation frameworks and all you need to know to get started with TestNG in Selenium.

Test Suites in JUnit and TestNG

For starters, a test suite is a collection of test cases that lets you run the test cases simultaneously. It’s a logical grouping that can be treated as a single pack while execution.

JUnit 5 was re-designed to overcome the limitations of the previous JUnit versions (including JUnit 4). The concept of test suites was introduced in JUnit 5 since Junit 4 (and earlier versions of JUnit) did not provide the feature to create test suites.

Test suites in JUnit 5 are realized using the @ RunWith and @ Suite classes

    [@RunWith](http://twitter.com/RunWith)(Suite.class)

    [@Suite](http://twitter.com/Suite).SuiteClasses({
       JUnitTestSuiteDemo1.class,
        JUnitTestSuitDemo2.class
    })

In TestNG, the test suite is defined in an xml file (e.g. testng.xml) The suite tag is defined in the file as shown below:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "[http://testng.org/testng-1.0.dtd](http://testng.org/testng-1.0.dtd)" >
<suite name="DemoTestSuite">
  <test name="DemoTest" >
    <classes>
       <class name="demo.demo.TestSuiteDemo/>
    </classes>
  </test>
</suite>

With TestNG certification, you can challenge your skills in performing automated testing with TestNG and take your career to the next level.

Here’s a short glimpse of the TestNG certification from LambdaTest:

Annotations in JUnit and TestNG

Annotations in the test automation framework provide additional information about the class or the test method. For a quick recap, check out our detailed guide on TestNG annotations for Selenium automation.

Support for annotation can be considered as one of the vital points of TestNG vs JUnit comparison. Both JUnit and TestNG are annotation-based frameworks.

Most of the annotations in TestNG and JUnit offer the same functionality with a slight change in the naming nomenclature. The difference between JUnit and TestNG from an annotation standpoint is that TestNG has some additional annotations in comparison to JUnit.

Here is the TestNG vs JUnit comparison as far as annotations are concerned:

SCRIPTION	TESTNG	JUNIT 5
Marks the method as test method	@ Test	@ Test
The annotated method is executed before the first test method of the class	@ BeforeClass	@ BeforeAll
The annotated method is executed after all the test methods of the current class have been executed.	@ AfterClass	@ AfterAll
The annotated method is executed before each test method	@ BeforeMethod	@ BeforeEach
The annotated method is executed after each test method	@ AfterMethod	@AfterEach
The annotated method is executed before the suit.	@ BeforeSuite	NA
The annotated method is executed after suit.	@ AfterSuite	NA
The annotated method is executed before the test.	@ BeforeTest	NA
The annotated method is executed before the test.	@ AfterTest	NA
The annotated method is executed before the first test method of any of these groups.	@ BeforeGroups	NA
The annotated method is executed after the first test method of any of these groups.	@ AfterGroups	NA
Ignore Test	@ Test(Enable=false)	@ Disabled (In JUnit4 it is @ignore)
Expected exception	@ Test(expectedException=Arithmetic
Exception.class)	@test(expected=Arithmetic
Exception.class)
Timeout	@ Test(timeout = 1000)	@ Timeout

Check out our earlier blog on JUnit Annotations in Selenium WebDriver to get detailed insights on annotations in the said framework.

Watch this video to learn about the TestNG Annotations and how they help provide better structure and readability to the code.

TestNG vs JUnit: Test case Management in TestNG and JUnit frameworks

Managing test case execution is a lot easier in TestNG in comparison to JUnit. In TestNG, QA engineers can group tests, ignore tests, parameterize tests, and write dependent tests in an efficient manner.

Let’s look at each of those points in this section of TestNG vs JUnit comparison:

Grouping Test Cases

In TestNG, you can group the test cases by simply providing parameters in the @ Test annotation as shown below:

    @Test(groups={“groupname1”,<”group2">..,<”groupN”>})

You can check out our detailed blog on how to group test cases in TestNG for more information on grouping tests in TestNG. The particular group(s) can be executed by providing the name(s) in the XML file under and tags as shown below:

<suite name="MyTestSuite">  
  <groups>  
     <run>  
        <include name="Dashboard"/>  
     </run>  
  </groups>  
 <test name="LoginTest">  
    <classes>  
        <class name="CheckUserLogin"/>  
        </classes>  
    </test>  
    <test name="UserHomePageTest">  
   <classes>  
      <class name="CheckUserHomePage"/>  
   </classes>  
   </test>  
</suite>

Here, the tests under group “Dashboard” will be executed. JUnit does not provide a straightforward way for grouping the test cases.

Ignore Test

There are scenarios where you might need to run a select set of tests from a huge test suite. The test cases that need not be executed have to be ignored. Ignoring tests in JUnit and TestNG is specifically useful when only a particular feature has to be tested.

TestNG and JUnit automation frameworks provide the feature where tests can be ignored. In TestNG, we can provide a parameter in @ Test annotation as shown below:

    [@Test](http://twitter.com/Test)(enabled=false)
    public void TestIgnoreDemo()
    {  
        /* Implementation goes here */
    }

In Junit 5, the @ ignore annotation is used for ignoring test cases:

    [@Ignore](http://twitter.com/Ignore)
    public void TestIgnoreDemo()
    {
        /* Implementation goes here */
    }

Parameterization

Parameterization in the test suite lets you run a certain test against different input values. Improved code reusability and readability are the major benefits of parameterization in automation testing. Parameterization can turn out to be a make or break deal in the TestNG vs JUnit comparison.

In TestNG, passing parameters to a testcase is straightforward. It uses @ Parameter annotation with the parameter to the given test method. The @ Parameters annotation should be used in case you want to pass more than one parameter to the method.

    public class Demo {
        [@Test](http://twitter.com/Test)
        [@Parameters](http://twitter.com/Parameters)({"browser"})
        public void demoTest(String browser)
        {
            System.out.println("The browser Is : " + browser);
        }
    }

The value of variable ‘browser’ is declared in the xml file (e.g. testng.xml) as shown below:

    <suite name="ParameterizedDemo">
      <test name="test1">
          <parameter name="browser" value="chrome"/>
          <parameter name="browser" value="IE"/>
          <classes>
              <class name="Demo" />
          </classes>
      </test>
    </suite>

That’s not all. Data Providers in TestNG can also be used for realizing parameterized tests in TestNG.

In JUnit, parameter(s) can be passed using the @ParameterizedTest annotation:

    [@ParamerizedTest](http://twitter.com/ParamerizedTest)
    [@ValueSource](http://twitter.com/ValueSource)(strings = { "Chrome", "IE", "Safari" })
    void DemoParameterize(String browser) {
        void  setBrowser("The browser is:"+browser));
    }

On executing the above code, the test DemoParameterize will be triggered separately with the input parameters as Chrome, IE, and Safari. You can check out our detailed blog on Parameterization in JUnit with Selenium Driver to gather insights on how to use parameterized tests in JUnit.

Dependent Test

Autonomous test design is considered as one of the best practices for Selenium test automation. However, there are cases where dependency between automation tests cannot be avoided at any cost. Test dependency refers to the ability of one test being dependent on another test method.

In TestNG, Dependent methods can be defined with the help of @ DependsOnMethods annotation.

    public class DependentDemo {

        [@Test](http://twitter.com/Test) (dependsOnMethods = { "Launcher" })
        public void login() {
            System.out.println("Login to the portal");
        }

        [@Test](http://twitter.com/Test)
        public void Launcher() {
            System.out.println("Launch the application");
        }
      }

Here, the test method login() is dependent on the Launcher() method. Here is the execution output:

JUnit does not support method dependency, hence TestNG scores a brownie point in the test dependency TestNG vs JUnit comparison 🙂

This JUnit certification establishes testing standards for those who wish to advance their careers in Selenium automation testing with JUnit.

Here’s a short glimpse of the JUnit certification from LambdaTest:

Exception Handling

Exception handling in web automation testing refers to the ability of a test method to throw an exception when it encounters an error during the execution phase. Both JUnit and TestNG frameworks have support for exception handling.

In TestNG, exceptions are handled using the expectedException parameter in @ Test annotation

    [@Test](http://twitter.com/Test)(expectedExceptions = ArithmeticException.class)
    public void DivideByZero() {
      int i = 10/0;
    }In JUnit 5, the assertThrows API is used for throwing an exception

    [@Test](http://twitter.com/Test)
    public void whenDividerIsZero_thenDivideByZeroExceptionIsThrown() {
        Calculator calculator = new Calculator();
        assertThrows(DivideByZeroException.class, () -> calculator.divide(10, 0));
    }Timeouts

Timeout in tests means that the tests should fail if they take longer than expected time for executing a test. This is a critical feature in web automation testing as an alarm is raised if the tests are not executed within a stipulated time period.

In TestNG, the timeout parameter which specifies the maximum execution time is added to the @ Test annotation.

    [@Test](http://twitter.com/Test)(timeOut = 1000)
    public void timeoutDemo() {
        while (true);
    }

Here is how timeouts in Selenium are handled in the JUnit 5 framework:

    [@Test](http://twitter.com/Test)
    public void timeoutDemo() throws InterruptedException {
        Assertions.assertTimeout(Duration.ofMillis(1000), () -> Thread.sleep(10000));
    }

Customization of Test Names

There is no way to customize test names in the TestNG framework. In JUnit 5, we can provide a customized name to the tests with the help of @ DisplayName annotation. The same is shown below:

    [@ParameterizedTest](http://twitter.com/ParameterizedTest)
    [@ValueSource](http://twitter.com/ValueSource)(strings = { "Hello", "World" })
    [@DisplayName](http://twitter.com/DisplayName)("Test Method to check that the inputs are not nullable")
    void givenString_TestNullOrNot(String word) {
        assertNotNull(word);
    }

If you are looking for customized test names, JUnit should be the go-to test automation framework.

Nesting of Test Cases

The @ Nested annotation in JUnit Jupiter can be used to mark nested classes that have to be included in test case execution. @ Nested tests give the QA engineer more capabilities to express the relationship among several groups of tests.

    public class MyApp
    {
        [@BeforeAll](http://twitter.com/BeforeAll)
        static void launchMyApp()
        {
            //the code to launch MyApp
        }
        [@Test](http://twitter.com/Test)
        [@DisplayName](http://twitter.com/DisplayName)(“Testcase to create new user”)
        void signUp()
        {
            //the code to create new user 
        }

        [@Nested](http://twitter.com/Nested)
        [@DisplayName](http://twitter.com/DisplayName)(value=“Nested class for Login to MyApp”)
        class login
        {
            void verifyUser()
            {
                //the code to validate the user
            }
            void verifyNotNull()
            {
                //code to validate that username and password fields are not Null
            }
        }
        [@AfterAll](http://twitter.com/AfterAll)
        static void closeMyApp()
        {
            //the code to close MyApp
            //releasing all used objects goes here
        }
    }

Grouping tests in TestNG is a common way for realizing nested tests in TestNG. As far as nested tests are concerned, TestNG scores high in the TestNG vs JUnit battle.

Did you know? Minify JSON is a JavaScript library that removes whitespace and comments from blocks of JSON-like content, while preserving its syntax.

Parallel Test Execution in TestNG and JUnit frameworks

Parallel testing in Selenium is one of the preferred ways to perform tests in parallel across different input combinations. Using a cloud-based Selenium Grid instead of a local Selenium Grid helps to achieve more from parallelism offered by the respective framework.

Both TestNG and JUnit frameworks support parallel test execution. In testNG, testers can assign the thread count in xml file (e.g. testng.xml) and run the tests parallely as shown below:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "[http://testng.org/testng-1.0.dtd](http://testng.org/testng-1.0.dtd)" >
<suite name="DemoTestSuite">
  <test name="DemoTest"  parallel="methods" thread-count="2">
    <classes>
       <class name="demo.demo.TestSuiteDemo/>
    </classes>
  </test>
</suite>

Our blog on parallel testing in JUnit is a good resource to gain insights on how to practically use parallelism in JUnit for Selenium automation testing. In testNG, parallel execution can be achieved at method, test, class, and instance levels.

Parallel test execution in JUnit was introduced in JUnit 5, however it is still in the experimental mode. Here are some ways in which parallel testing can be achieved with JUnit 5:

junit.jupiter.parallel.process = true
junit.jupiter.execution.parallelism = true
junit.jupiter.execution.parallel.enabled = true
junit.jupiter. execution = “parallel”

The value for junit.jupiter.execution.parallel.enabled configuration can be set by providing necessary parameters in the Maven Surefire plugin. Alternatively, you can also set the parameter in unit-platform.properties or by providing system properties to JVM.

This is how the difference between JUnitand TestNG pans out when evaluated on parameters like test execution, parallelism, parameterization, and more.

Do check out Android Emulator Online- Test your web and mobile apps on Android Emulators online. Ensure your apps are compatible across latest and legacy Android operating systems, devices, and browsers. Start Testing Now !!

Parallel Test Execution using TestNG on LambdaTest Grid

The true potential offered by parallel testing in JUnit and TestNG can be harnessed by running the tests on a cloud-based Selenium Grid (instead of a local Selenium Grid). There are numerous benefits offered by Selenium testing on the cloud.

For demonstration of parallel testing, we will use the cloud-based Selenium Grid by LambdaTest. LambdaTest offers both Selenium Automation testing with JUnit and Selenium Automation testing with TestNG frameworks. The browser capabilities are generated using the Lambda Test capability generator. The combination of username and access key is used to access the LambdaTest Grid.

Here are the two test cases used for the demonstration of parallel testing with TestNG:

TestCase — 1

Go to Selenium Playground Simple Form Demo
Enter the message in the text field
Click on the Show Message button
Verify the message on the right panel

TestCase — 2

Go to the Selenium Playground Simple Form Demo
Enter the value of a in the first textbox.
Enter the value of b in the second textbox.
Click on the Get Total button.
Verify the sum on the right panel.

Implementation (using TestNG Framework)

    package Test; 
    import static org.testng.Assert.assertTrue;

    import java.net.MalformedURLException;
    import java.net.URL;
    import org.openqa.selenium.By;
    import org.openqa.selenium.remote.DesiredCapabilities;
    import org.openqa.selenium.remote.RemoteWebDriver;
    import org.testng.annotations.AfterClass;
    import org.testng.annotations.BeforeClass;
    import org.testng.annotations.Test;

    public class Lambdatest {
        public RemoteWebDriver driver = null;
        public static String username = "user-name";
        public static String accessKey = "access-key";
        @BeforeClass

        public void setup() throws Exception {

            DesiredCapabilities capabilities = new DesiredCapabilities();
            capabilities.setCapability("platform", "Windows 10");
            capabilities.setCapability("browserName", "Chrome");
             capabilities.setCapability("version", "88.0");
                capabilities.setCapability("resolution","1024x768");
                capabilities.setCapability("build", "Lambdatest");
                capabilities.setCapability("name", "Lambdatest");
                capabilities.setCapability("network", true); // To enable network logs
                capabilities.setCapability("visual", true); // To enable step by step screenshot
                capabilities.setCapability("console", true);
            capabilities.setCapability("user", username);
            capabilities.setCapability("accesskey", accessKey);
            try {       
                driver= new RemoteWebDriver(new URL("https://"+username+":"+accessKey+"@hub.lambdatest.com/wd/hub"), capabilities);            
            } catch (MalformedURLException e) {
                System.out.println("Invalid grid URL");
            }
        }

        @Test
        public void testAddition() throws Exception {
                    try {
                        driver.get("");
                        driver.findElement(By.id("sum1")).clear();
                        driver.findElement(By.id("sum1")).sendKeys("100");
                        driver.findElement(By.id("sum2")).clear();
                        driver.findElement(By.id("sum2")).sendKeys("200");    
                        driver.findElement(By.className("btn btn-dark selenium_btn")).click();
                        String strResult=driver.findElement(By.id("addmessage")).getText();
                        assertTrue(strResult.equals("300"),"Pass");
                    } catch (Exception e) {
                        System.out.println(e.getMessage());
                    }
        }
        @Test
        public void testUserMessage() throws Exception {
                    try {
                        String strMessage="Hello! Welcome to the Lambda Test";
                        String actualMsg="";
                        driver.get("");
                        driver.findElement(By.id("user-message")).clear();
                        driver.findElement(By.id("user-message")).sendKeys(strMessage);
                        driver.findElement(By.id("showInput")).click();    
                        actualMsg=driver.findElement(By.id("message")).getText();
                        assertTrue(actualMsg.equals(strMessage),"Pass") ;

                    } catch (Exception e) {
                        System.out.println(e.getMessage());
                    }
        }
        @AfterClass
        public void tearDown() throws Exception{
            driver.quit();
        }
    }

Here is the execution snapshot from LambdaTest:

LambdaTest also provides the video recording of the test case execution

We have not covered parallel testing with JUnit 5 here since, the feature is still in the experimental mode. In case you are using JUnit 4, do check out how to perform parallel testing with JUnit 4 and Selenium.

Conclusion

While JUnit 4 has several limitations in test case management, JUnit 5 tried to overcome those and also added new features in the process. Though there is a difference between JUnit and TestNG as far as annotations are concerned, the point is that both the frameworks are equally capable for usage in Selenium automation testing.

We have to wait and watch how JUnit 5 (and the subsequent versions of JUnit) evolve over a period of time. Overall, there is a thin line of difference between JUnit and TestNG and the choice for the test automation framework purely depends on the requirements of the project.

In the TestNG vs JUnit battle, my overall vote would go for TestNG since parallel testing in Selenium can be achieved with ease using the TestNG framework.

Irrespective of whether you choose JUnit or TestNG, the true potential of the corresponding framework can only be exploited by running the tests on a cloud-based Selenium Grid like LambdaTest. The major reason being parallel tests can be executed on different browser and OS combinations at blazing fast speeds.

Which framework do you prefer in the TestNG vs JUnit battle? Do leave your reasons in the comments section of this blog…

Happy Testing 🙂